Mutual Information to find delay coordinates

Eric Weeks - personal pages - research - time series analysis

My Adventures in Chaotic Time Series Analysis

weeks@physics.emory.edu

1. Meet the time series
2. Fourier Transforms
3. Mutual information to find delay coordinates (you are here)
4. Plotting attractors
5. Attractors in 3-D
6. Autocorrelation functions
7. Poincaré sections
8. 1-D Maps
9. More later. Perhaps fractal dimensions.

A0. Links and related information

For an explanation of what these pages are all about, select topic 1 above.

The method of "mutual information" is a way to determine useful delay coordinates for plotting attractors.

The idea of delay coordinates is simple. If you can only observe one variable from a system, X(t), and you want to reconstruct a higher-dimensional attractor, you can consider (X(t),X(t+T),...,X(t+nT)) to produce a (n+1)-dimensional coordinate which can be plotted. It is important to choose a good value for T, the delay. If the delay is too short, then X(t) is very similar to X(t+T) and when plotted all of the data stays near the line X(t)=X(t+T). If the delay is too long, then the coordinates are essentially independent and no information can be gained from the plot. This can be seen in the next section.

The method of mutual information for finding the delay T was proposed in an article by Andrew M. Fraser and Harry L. Swinney ("Independent coordinates for strange attractors from mutual information," Phys. Rev. A 33 (1986) 1134-1140). The idea is that a good choice for T is one that, given X(t), provides new information with measurement X(t+T). To quote from the article, mutual information I is the answer to the question, "Given a measurement of X(t), how many bits on the average can be predicted about X(t+T)?" We want I(T) to be small. For more information on information theory and this method, you should read the article. Information theory is interesting stuff.

Practically speaking, the way this method works is to calculate I(T) for a given time series. A graph of I(T) starts off very high (given a measurement X(t), we know as many bits as possible about X(t+0)=X(t)). As T is increased, I(T) decreases, then usually rises again. Fraser and Swinney suggest using the first minimum in I(T) to select T. Click here to get the program I wrote to calculate the mutual information. Or, go to Fraser's home page (above link) and get his original software, although I found it a little more difficult to use.

Below I have the graphs of I(T) for my time series.

Rössler attractor:

I did this one first as it's the test case that Fraser and Swinney use. Fortunately my graph resembles their graph, proving that I didn't make any mistakes writing the software. This graph has the x-axis shown in units of the natural period of the Rössler attractor. This period was determined from the Fourier transform of the data; see the previous section for a discussion. From this graph, the minimum is T=0.25 (times the period).

Lorenz attractor:

The first minimum occurs at T=0.16; the second slightly lower minimum occurs at T=0.62. In the next section you can see what the attractor looks like for these two choices.

Hénon attractor:

This graph looks much different, which is not suprising. For a simple map, the natural delay time is T=1 iteration. Given that the mutual information has no local minimum, but is only monotonically decreasing, this suggests that T=1 is a choice reasonably supported independently by the mutual information calculation.

Experimental data: periodic

The first minimum occurs at 0.9 seconds; the second at 2.1 seconds. Recall that the for the periodic time series, the period was 6.9 seconds. The second maximum occurs at 3.5 s, about half the period.

Experimental data: quasi-periodic-2

The first three minima occur at 2.5 s, 5.6 s, and 9.6 s. For this time series, the dominant period (as given by the Fourier transform) was 13 s. The first three maxima are at 4.8 s, 6.8 s (approximately half the period), and 12.3 s (close to one period).

Experimental data: quasi-periodic-3

The first minimum is at 2.0 s, the second at 4.8 s. The largest peak in the power spectrum corresponds to a period of 6.1 s.

Experimental data: chaotic (?)

The first minimum is at 2.2 s. The first maximum is at 3.5 s. The natural period is 6.5 s.

So far the analysis has given me several different times. From the Fourier transform I have certain characteristic periods. From the mutual information calculation I have the first minimum. Thus, a table:

System Period Mutual info 1st minimum Mutual info 1st maximum

Lorenz Attractor ? 0.16 0.22

Rössler Attractor 6.07 1.52 3.10

Hénon Map 2.2 "1" none

Expt-periodic 6.9 0.9 2.1

Expt-q-periodic-2 13.0 2.5 6.8

Expt-q-periodic-3 6.1 2.0 3.2

Expt-chaotic 6.5 2.2 3.5

System	Period	Mutual info 1st minimum	Mutual info 1st maximum

Lorenz Attractor	?	0.16	0.22
Rössler Attractor	6.07	1.52	3.10
Hénon Map	2.2	"1"	none
Expt-periodic	6.9	0.9	2.1
Expt-q-periodic-2	13.0	2.5	6.8
Expt-q-periodic-3	6.1	2.0	3.2
Expt-chaotic	6.5	2.2	3.5

Note that all times are in natural units (seconds for experimental data, iterations for Hénon map, the time step of the equations for the Rössler and Lorenz systems). Links in the table will take you to the appropriate location where the information was found.

Previous page: Fourier transforms
Next page: Plotting the attractors

Links...

Graphs were made with a program I wrote, psdraw. The program makes PostScript output which I then converted to GIF. Rotations & 3-D boxes done with rotate, another program I wrote. Both psdraw and rotate are public domain.
If you have any questions or comments, send me email; email address below.

Current address:
Eric R. Weeks
weeks@physics.emory.edu
Department of Physics
Emory University
Atlanta, GA 30322-2430