PHYSICAL REVIEW LETTERS Vouume 65 24 SEPTEMBER 1990 Nuwner 13 Lyapunoy Exponents from Observed Time Series Paul Bryant and Reggie Brown Institute for Nonlinear Science, University of California at San Diego La Jolla, California 92093-0402 Henry D. L. Abarbanel Department of Physics and Marine Physical Laboratory. Scripps Institution of Oceanography; University of California at San Diego, La Jolla, California 92093-0402 (ceived 11 June 1990) We examine the question of accurately determining Lyapunov exponents for a time series. We find that it i advantageous to use local mappings with higher-order Taylor series, rather than linear maps as done earlier. We demonstrate this procedure for the Tkeda map and the Lorenz system, We present methods for identifying spurious exponents by analyzing data-set singularities and by determining the Lyapunov direction vectors. The behavior of spurious exponents in the presence of noise is also investi- ‘gated, and found to be different from that of the true exponents PACS numbers: 05.45.40 Determining the Lyapunov exponents of a nonlinear system from measurements of a time series is an impor- tant challenge for any analysis of the dynamics. Positive exponents are generally regarded as equivalent to the presence of real dynamical chaos, and the Lyapunov ex- ponents are classifiers of the dynamics since they are characteristic of the attractor and independent of any given orbit or initial condition." Ifthe governing equa- tions are known, then there are reliable methods for determining all of the exponents. If one only has a time series, then the problem becomes much more difficult There are several reported efforts to provide algorithms for the determination of the Lyapunoy exponents from observations alone." Our own experience’ with these algorithms is that they are reliable only for the largest exponent and not for the others. The importance of Lyapunov exponents in the study of physical systems has led us to provide an improvement on these previous efforts and to address several other questions of impor- tance in the determination of Lyapunov experiments from data. In this Letter we report on the basic outline of our methods. We leave details and numerous other examples to our longer paper.* Earlier work and ours assume that a sealar time series xCigtne) x(n), 1,2... Np, has been observed. From this the phase space of the system has been recon- structed by the familiar time-delay method*="? to pro- duce data vectors in d dimensions: yl) xn), xln 1), xn T=) In discretized time one takes the dynamics to be a map of R# to itself’ which evolves the vectors y(n): y(n 4+T;)=F(y(n)), where the time delay 72 is indepen- dent of T. The product of the Jacobians of this map DF(y) =F (y )/Ay evaluated along an orbit contains the information required for the Lyapunov exponents. "? The first step in the analysis isto find the neighboring points of a given point in the data set. Our choice of neighbors is limited by the finite size of the data set, by stochastic noise, and most importantly by the fractal na- lure of the attractor. These limitations are the main source of difficulties in the analysis. Finding legitimate neighbors of a given point is one of the most critical tasks in obtaining accurate results. For this reason it is often advisable to maintain two different dimensions for converting the scalar data set into time-delay vectors. The first is a “local dimension” d which is equal to the umber of Lyapunov exponents that the calculation will produce and is the dimension of the Jacobian matrices. ‘The second is a “global dimension” de which is used in © 1990 The American Physical Society 1523VoLUME 65, NUMBER 13 PHYSICAL REVIEW LETTERS 24S PTEMBER 1990 the process of identifying neighbors. dc can be made larger than d in order to insure that we do not have any false neighbors entering the calculation as might be the case if the attractor is folded in such a way that it crosses itself. One way of choosing values for d and do is to start by computing a rough estimate of the fractal dimension di of the attractor. I an object of dimension dy is mapped in a very general way into a space of di- mension de it can typically have sei-intersections of di- mension 2d,—de. Thus we would like to make do larger than 2d,. It is not advisable to make d much areater than ds, since locally the attractor has very little extension into these additional directions. A good first choice isto try making d the next integer greater than or ‘equal to the fractal dimension dy. This choice will ways give at least one negative exponent for a chaotic system. One is then free to further increase d and see what effect this has on the results Earlier work attempts to find the required Jacobians bby making local linear maps of neighborhoods near the ‘orbit y(n) to neighborhoods at a subsequent time. We depart from these earlier works by making local polyno- rial maps, allowing for a more accurate determination of DF(y). While there have been previous uses of higher-order mappings in studies of dynamical systems (see, eg. Refs. 13 and 14) this is, we believe, the first application of this approach to the calculation of the full Lyapunov spectrum. The vector from the rth neighbor to an orbit point y(n) is denoted by 2°. Using a least- squares fit to the data we obtain a polynomial map from this vector at time 0 to the same vector on time step T2 later: 2/(n0)—+ 2"(n:T2), including all terms up to some specified order ray. We have examined the ef- fects of retaining terms up to fifth order in 2”(v:0). We use atleast twice the number of neighbors as parameters to be determined in order to insure reliable results. We then proceed to calculate the Lyapunov exponents using the OR decomposition technique discussed by Eckmann et al.™'*"® As we show with our methods, one can deter- mine the positive, zero, and often one or more negative ‘exponents. . In this Letter we report on results from the Ikeda!” map of the complex plane to itself, (n+ 1) =p Be(ndexplix ial [20/72 , where p=1.0, B=0.9, «=0.4, and 6.0, For these parameters we calculate (using the map) that 21 and &2 are 0.503 and —0.719, respectively. We also study the Lorenz system of three ordinary differential equations:'" aX Gyn. _ q_Pmatxa 1, EO= nn WOt re O— 20) @ as (ya - eA = bes, 1524 where we take 616, b=4, and r=45.92. For these pa- rameters the accepted values for the Lyapunov exponents are 1.50, 0.00, and —22.5, respectively. The large nega- tive exponent makes this system a particularly challeng- ing test for our time-series method, and also requires the use of very accurate data, as will be shown. The difficulty in determining the negative exponents from a time series comes primarily from the fact that the attractor is often very “thin” at many locations in the directions associated with certain negative exponents. Even when there is a reasonably large and accurate data set, this will often make curvature effects within a given neighborhood become significant, A linear analysis be- comes totally inaccurate when the displacement due to local data-set curvature is comparable to the thickness of the data set. Going to a higher-order approximation of the mapping can correct this. ‘The Ikeda map is an excellent example of a system for which the use of separate local and global dimensions is important. Examination of the two-dimensional time- delay representation Fig. 1 shows clearly the self- intersection effect which was discussed previously. Hav- ing determined the fractal dimension dy to be about 1.8 we would choose de to be at least 3 and preferably 4, and an appropriate value for d is 2. Using the incorrect values dg =d=2 in a third-order calculation we obtain 2y 0.565 and 22=—0.426, while if we use dg =4 and d=2 we obtain 4) =0.512 and 22 —0.736 which are ‘much closer to the correct values (0.503 and —0.719). If we keep d equal to dg but increase both of their values to 3, the overlap problem is reduced but we now obtain 3 exponents (0.554, ~0.262, und ~0.821) and so we are faced with the problem of deciding which, if any, are val- id exponents. We move now to the Lorenz system. In the case of data from the Lorenz equations we have used two slight- ly different settings for the evolution time lag. We display the results of both our calculations in Table 1 FIG, |. Reconstructed phase portrait of the Theda map in 42. ‘The lack of @ one-to-one projection of the attractor onto the [x(n),xn+ 1] plane is clear to the eyeVoLUME 65, NUMBER 13, PHYSICAL R EVIEW LETTERS 24 SEPTEMBER 1990 TABLE 1. In the top part of the table we display the Lyapunov exponents for the Lorenz system computed ftom ‘50000 data points evaluated with # sampling time r= 7>0.02 and a time delay T=Sr=0.1. The data have five digits of ac- curacy and are analyzed using d=de™3 for varying orders Nruy of the mapping, In the bottom part we display the Lyapunov exponents for the Lorenz system computed from 20000 date points evaluated with a sampling time #0.05 and time delay T=T>=2e—0.1. The data have 94+ digits of ae- ‘curacy and are analyzed using d=3 and de; 7 for mapping or- ers 1-5. Nr hy a ay t 1.4308 00057123 = 13.999 2 1.5027 0.086081, 19.498 3 isi 0.0069641 22925 4 15561 0.032219 =23.465 ' 1.549 0.09470 1431 2 1319 0.02647 20.26 5 150s 0.005695 2259 4 1.502 0.002847 5 0.000 387 AAs the reader will observe, the negative exponent is very difficult to obtain, and here we see it dramatically “snap- ping into place” as we increase the order of the calcula~ tion to 3 and above. Also note the improvement in accu- racy of the zero exponent in the lower part of Table 1 ‘The greater accuracy of the lower part of Table I is due primarily to the higher accuracy of the data set. In Table I1, we analyze the Lorenz equations with a local dimension d=4, which we know must generate at least one spurious exponent. When using a second-order fitting to our local map, the results are poor for all of the Lyapunov exponents. Increasing the polynomial fitting to third order we find that the last three exponents are very close to the true exponents, while the first is 10 times larger than the true value of the largest exponent In addition to obtaining the Lyapunov exponents, one can also obtain the direction vectors L, associated with these exponents, The L, are defined by the requirement that a small displacement along any one of these direc- tions followed forward or backward in time will expand (or contract on average at the rate given by the corre- TABLE II. Lyapunov exponents and thicknesses (Th) ofthe Lorenz system, through order Ny, =4, The ealeulation was d spurious exponent, For N'r4,=2 and above, the spurious expon small thickness value sponding exponent. Although they are different at every location on the attractor, their calculation requires knowledge of the orbit far into the past and future for a given point on the attractor. The details of their calcula- tion are to be found in our longer paper. They should be examined to see if two or more of them are nearly col- linear. This ean occur if a poor choice was made for the delay time (probably too small) or if nonlinear effects ae generating a spurious exponent. The spurious nature of A; in our Lorenz-system calculations can be rapidly identified by examining the local data thickness Thy in the L; direction, which is over S orders of magnitude smaller than the thickness Th» for L:. A valid positive exponent should not exhibit any significant “thinness,” and it is preferable for all exponents to have thickness levels above the intrinsic noise level of the data set. The thickness Th, is essentially the rms displacement of the data points within a local neighborhood in the L; direc- tion, with corrections for data-set curvature; more details ‘can be found elsewhere. Although we have shown that it is possible to include singular directions in the calculation and later identify the questionable exponents, the presence of relatively small amounts of noise makes this more difficult, This is illustrated in Fig. 2 for the Lorenz system. We have added Gaussian white noise to the data points with the indicated standard deviation. In Fig. 2(a) we have used d=3, while in 2(b) we used d=4, which gives one spuri- ‘ous exponent. In both cases we used a third-order ex- pansion for the local mappings. The spurious exponent in 2(b) drops rapidly as the added noise is increased, go- ing from +19 down to ~6. This behavior is in fact another way of identifying a spurious exponent in ex- tremely accurate data. However, the absence of such a drop does not guarantee that spurious exponents do not exist. We conclude by noting that a simple extension to higher order of earlier methods for determining the Lyapunov spectrum for a dynamical system from obser= vations alone works strikingly well when tested on famil- iar systems such as the Ikeda map and Lorenz attractor. We have also suggested and tested on these examples ‘ways to determine which of the exponents are valid and which are spurious. Finally, we explored the effect of numerical accuracy and external noise on the determina- attractor along the corresponding Lyapunov direction vectors for the jone with 20000 data points using d=4 and dy, =7 so that there is one went separates from the true ones and can be identified hy its extremely Naw he Th The 1 1936 0.467 0.802 1.083 0.162, 2 436 0.00256 L401 oad 0.001 16 3 18.08 89x10 1502 0.129 2710-# 4 26.96 25x10~" 1503 0.066 S6x10-* 1525VoLuME 65, NUMBER 13 10.0 80 60 oo FIG. 2. The effect of external noise on the determination of| Lyapunov exponents forthe Lorenz system. (a) Here local di- mension is d™=3, (b) Here d=4. The spurious exponent wanders from about +19 to nearly ~6 as the noise level is varied. Note that the exponents do not cross each other but prefer to switch roles as they become close. In the d=3 case the correct exponents are more robust against the addition of tion of these exponents, We found the calculation to be ‘quite sensitive to noise, which reflects the fact that fluc- tuations in phase-space points make determination of the required local Jacobians quite sensitive, Further exam= ples and extensive details on the methods outlined here will be found in our longer paper.* Also, the algorithms used here are available on request from the authors. We are most appreciative for productive conversations with Andy Fraser, Ed Lorenz, and Jorma Rissanen about the material covered in this Letter. This work was supported in part under Defense Advanced Research Projects Agency-University Research Initiative, URI Contract No, NO0014-86-K-0758, Some of the compu- tation reported in this paper was carried out at the NASA Ames Research Center's Numerical Aerodynam- 1526 PHYSICAL REVIEW LETTERS 24 SEPTEMBER 1990 ic Simulation Program under the auspices of the Joint Program in Nonlinear Science between the University of California and NASA Ames, ‘)Also at Institute for Nonlinear Science, V. 1 Oseledec, Tr. Mosk. Mat. ODS. 19, 17 (1968), 24. Eckmann and D. Ruelle, Rev. Mod. Phys. $7, 617 (1985). 34. Eckman, S. 0. Kamphorst, D Ruelle, and S. Ciliber- to, Phys. Rev. A 34, 4971 (1986). 4G. Beneutin, L.'Galgani, A. Giorgi, and JM. Streleyn, Meccanica 18, 9 (1980). SA, Wolf, J. B, Swift, H. L. Swinney, and J. A. Vastano, Physica (Amsterdam) 16D, 285 (1985). °M. Sano and Y. Sawada, Phys. Rev. Lett $5, 1082 (1985), DH, D.L. Abarhanel, R. Brown, and J. B. Kadike, Phys. Let. ‘4.138, 401 (1989); Phys. Rev. A 41, 1782 (1990) 8R. Brown, P. Bryant, and H. D. I. Abarbanel, “Computing the Lyapunov Spectrum of a Dynamical System from Ob- served Time Series,” Institute for Nonlinear Science fornia at San Diego report, 1990 (to be pub- Takens, in Dynamical Systems and Turbulence Warwick, 1980, edited by D. Rand and L. S. Young, Lecture Notes in. Mathematics Vol. 898 (Springer-Verlag, Berlin, 1981), p. 366, "R. Maa. in Dynamical Systems and Turbulence, Warnick, 1980 (Ref. 9), p. 230. NA. M. Fraser and H. L. Swinney, Phys. Rev. A 33, 1134 (1986); A. M. Fraser, IEEE Trans. Int. Theory 38, 245 (1989) TH. 'D. 1. Abarbanel and J. B. Kadike, “Information Theoretic Methods for Choosing the Minimum Embedding Di- mension for Strange Attractos,” University of California at ‘San Diego-Institute for Nonlinear Science report, 1989 (to be published), 134. D, Farmer and J. J. Sidorowich, in Evolution Learning and Cognition, edited by Y. C. Lee (World Scientie, Singa- pore, 1988). "SP, Bryant and C, Jeffries, Physica (Amsterdam) 28D, 196 (1987). 'SW. H. Press, B. Flannery, S. Teulkolshy, and W. Vetterling, Numerical Recipes (Cambridge Univ. Press, New York, 1986) "6.3, Dongarra, J. R. Bunch, C. B. Moler, and G. W. Stewart, Linpack Users’ Guide (Society for Industrial and Ap- plied Mathematis, Philadelphia, 1979). "°K Tkeda, Opt. Commun. 30, 257 (1979) SEN. Lorenz, J. Atmos. Sci. 20, 130 (1963).