Numerical Calculation of Lyapunov Exponents Marco Sandri, University of Verona, Italy ‘The Lyapunov characteristic exponents play a crucial role in the description of the behavior of dynamical systems. They measure the average rate of divergence or convergence of orbits starting, from nearby initial points. Therefore, they can be used to analyze the stability of limits sets and to check sensitive dependence on initial conditions, that is, the presence of chaotic attractors. This article shows how to use Mathematica to compute the Lyapunoy spectrum of a smooth dynami- cal system, A large number of fields of human knowledge, from physics to chemistry, from medicine to meteorology, from biology to economics, makes use of difference and differential equa- tions (linear or nonlineas, ordinary or partial) to describe and explain facts observed in the real world (see, for exam- ple, [Beltrami 1989]). Advances made in the last two decades in the analysis of nonlinear systems have led to new areas of application and research for dynamical system theory. ‘Among these developments isthe discovery that deterministi systems can exhibit a wide range of erratic behaviors (usually called *chaos"), in many respects indistinguishable from noise. Several approaches have developed for the study of such systems, both geometrical [Guckenheimer and Holmes, 1983] and statistical [Eckmann and Ruelle 1985]. The statis- tical approach, which is rooted in ergodic theory, seeks 10 characterize dynamical systems through concepts such as entropy, dimension, and Lyapunov characteristic exponents. Roughly speaking, dimensions express the number of excited degrees of freedom of the system and the degree of “geo: metric complexity” of the attractor, while entropies describes, the production of information in the system. Lyapunoy char- acteristic exponents (L.CEs) measure the separation in time of two orbits starting from arbitrary close initial points. All these quantities are used to quantify the erratic or chaotic, behavior of a system's dynamics. In this article, we develop a package for estimating the Lyapunov exponents of continuous and discrete differen- tiable dynamical systems, based on the algorithms presented in [Benettin etal. 1980] and [Eckmann and Ruelle 1985} Differentiable Dynamical Systems In this section, we briefly describe some basic notions of dynamical system theory that will be useful in the next sec- tions. For a more detailed treatment, see [Guckenheimer and Holmes 1983] Marco Soni sadiad economic a the Faculty of Economie and Commerce of ‘he Unversity of Verona, iy and smn orn the Calas Cente of hat rier a9 manager oft Scintfic Caleas Ar. His reach terete Include tne sves ans, neural mencorks, wavelets, and Synamical system eons. 78 TENATHEUATCA JOURNAL © 1896 Ue Faso eats ‘An n-dimensional continuous-time (autonomous) smooth dynamical system is defined by the differential equation Fu, ay where # =de/dt, x(t) e R" is the state vector at time t and F:U-+R® isa C” function (r2 1) on an open set UCR” (chat is, F has derivatives of order r which are continuous at ‘each point of U). Here, we will not explicitly consider non- autonomous systems #= Fle, t), because, for our purposes, it will be sufficient to treat tas an additional dependent vari- able with the trivial evolution equation = 1. In other words, we will rewrite every nonautonomous system as an autonomous system #= Fla, t), (at the expense of increasing the dimension by one). The space of dependent variables is often referred to as the phase space M of the system, which, in our discussion, will be R®. The function F is a called vector field. If itis linear, then the system (1) is linear. Moreover, we say that the vector field F generates the flow f : Ux RR", where f(z) = f(a, t) is a C” function defined for all xe Uand t € R, such that Fe=Fyiey forall ze Ute R. Given an initial state x9 € U, the solution of (1) is the func tion f'(ro) sR R® such that (xq) =o. The set (f"(ro) te R) is called the trajectory of the system through z. ‘Any Cv map F : U > RY onan open set UCR" defines an rnedimensional discrete-time (autonomous) smooth dynamical system by the state equation 21 = Sn), te Ny where 2 R'is the state ofthe system at time t and f maps 2,10 the next state 2,1. Starting with an initial condition zo, repeated applications (iterates) of f generate a discrete set of points (the orbit) f(z): © N}, where f'(z)=f 2° fiz). ‘One of the main goals of dynamical system theory is the study of steady-state behaviors, that is the quantitative and qualitative description of the asymptotic evolutions of sys tems as {+ s2, An important concept for this purpose is the notion of a limit set.‘A point p is an o-limie point of «if there are points f(x}, £42), .. on the trajectory of x such that f(z) — p as t; >>, ‘The o-limit set Oz) is the set of all the w-limit points of x. Moreover, an o-limit set © is attracting if there exists an open neighborhood U of Q(x) such that O(c) = for all xe U. The basin of attraction Bg of an attracting set Sis the union of all such neighborhoods U. In other words, Bg is the set of all initial conditions x that tend toward @ as t —> ~. (We make no distinction between attracting limit sets and attractors, even if they are quite different; from a practical point of view, there is little harm in ignoring the difference.) Of course, for a scientist, only attracting limit sets are of interest because non-attracting limit sers cannot be observed in real systems and simulations. As we will show, LCEs are a useful tool to investigate the stability of limit sets and to. classify systems on the basis of their asymptotic evolution, ‘A dynamical system may have only one attracting limit set, or it may have several, each one with a different basin of attraction. In this case, the initial condition determines which limie set will be approached. There are four foundamental types of limic sets, corresponding to as many types of solu- tions of differential (or difference) equations. 1. Fixed points. A fixed point is a point x € M such that Flo) =2 for allt 2. Periodic motions. A periodic motion is a solution f* such that /'(z)= f*¥(2) for some fixed constant T'> 0 (the period) and allt. The limit set corresponding to a periodic solution is the closed curve traced out by /'(2) over one Period, which is topologically equivalent to a circle $, 3. Quasiperiodic motions. A quasiperiodic solution of a dynamical system is a function f : R-> R® chat can be represented in the form f(t) = H(t, -., ant), where His periodic of period 2x in each argument, and the real num: bers «15 + ay form a finite set of base frequencies (see [Parker and Chua 1989, 13-18)). A quasiperiodic solu- tion with q base frequencies is called g-periodic. The limit set of a g-periodic solution is a diffeomorphic copy of a g-dimensional torus TY = $x «x $1, where each $" rep- resents one of the base frequencies. 4. Chaotic motions. Avoiding a formal definition, we say that chaotic dynamics are characterized by three properties: (a) they are bounded random-like steady-state trajectories dis tinet from the previous kinds of motions (b) they converge to a set in the phase space, called a strange attractor, which is not a simple manifold like a point, circle, or torus, but hhas a complex (fractal) geometrical structure with a frac- tional Hausdorff dimension (Falconer 1985]; (c) they exhibit sensitive dependence to initial conditions, that is, chaotic trajectories locally diverge away from each other and small changes in starting conditions build up expo- nentially fast into large changes in evolution, Lyapunov Exponents Lyapunov exponents provide a quantitative measure of the divergence or convergence of nearby trajectories for a ‘dynamical system. If we consider a small bypersphere of ini tial conditions in the phase space, for sufficiently short time scales, the effect of the dynamics will be to distort this set into a hyperellipsoid, stretched along some directions and contracted along others, The asymptotic rate of expansion of the largest axis, corresponding to the most unstable direction of the flow, is measured by the largest LCE 2, In general, if we sort the axes and LCEs in decreasing order by magni- tude (e} 2 «+ 2 8, and 2; > +» 24), each A, quantifies the average exponential rate of expansion or contraction for the th axis &, ‘More formally, consider two nearby points, and zy + to, in the phase space M, where 1g isa small perturbation of the initial poine 2» (see Figure 1). After a time t, their images under the flow will be f'(29) and /'(xp + up) and the pertur- bation , will become y= flr + uo) = f'\z0) = Dag (20) “uo, (2) ‘where the last term is obtained by linearizing f*, Therefore the average exponential rate of divergence or convergence of the two trajectories is defined by Azo) = fim tin Heel — sim ttn iDeg (0) wll (3) a ee lua where [lull denotes the length of a vector u. If A(x, u) > 0, then one has exponential divergence of nearby orbits. It can be shown that, under very weak smoothness conditions on the dynamical system, the limit (3) exists and is finite for almost all points xq € 4M, and, for almost all tangent vectors ‘ug, it is equal to the largest LCE A; [Oseledec 1968], Definition (3) refers to LCEs of vectors, also called LCEs of order 1. A natural and, for our purposes, useful general- ization is to define LCEs of order p, 1

yO) J; kos at aL /. Threadly > yO + K1/2) J; Kae @t ACF /. Threadly ~> y0 + 12/2) 1; kos ot AL /- Toreadly -> yO + 43) 15 yo (eh + 2K2 +243 + KO/6] 80 THE MUTHEMATICALOURNAL © 196 eon besos ‘Topaogial Dynamics of LE aasdoot Simension the aactor spscrum 1 Fixed point ~3 2 apes 7 Hyperion T> 3 Chior on? 3eDea ___Hypercuos@® 0-3 eet Fixed point aan Period Mocion oon ot (w= Thor Qedens t (wo2}ehuos peg Qe dons keleDeN “TABLE 1. LOE specrum of continuous tine atactors rom Kein and Bier 1940, Wis TntYorEQ(F_List, OPhiLList, x_List, Phi List, xO_List, Phi0.List, th, dt] = Moduletin, f, y, yO, yt, engthls0]; Flatten[Join(F, 0Phi]]; y= Flatten(Joinlr, Pill: yO = Flstten(Join(xo, Phio}}; yt = Nest{ RiSteplf, y, #, Ndt]]8, MIyOL, Round(NCe1/at]) J; First(t], Restie)}e @ Partition(yt, 0 We also define functions to compute the Jacobian matrix of F and the Euclidean norm of a vector. vis JocobianPatrin{funs.List, vars.List] <= uter(D, funs, vars) Waele Mormfs] :* SqrtCkx} Choosing a step size of 0.02 and 7'=20, we compute the 3x3 mateix ®p(29) = Day fa): lane T= 205 ine stepsize = 0.025 vie 9 = Lengthts0]s wathe = Arrayla, 05 babe Phi = Meraytby dn, 0BD5 ism OPAL = Phi. TransposeLJacobiantatrda(FOs], 0s ioe PhiO = Tdentstynatria; oa AT, PhiTD = ‘ntarEq(FL, OPhi, x, Phi, 10, Paid, {T, stepsized]; ats aT cows {47.7334, 15.016, 53.42}ihe PRAT // MotriaForm outers uw 2 2 3.1698 10" 4.9580 1? 5.1210 u 2 2 5.9083 10" -8.¢52 10"? o,o0008 10 1 2 2.64816 10 “4.13294 10° 4.84979. 10? In the previous section, we mentioned that 2(x0, to) = Ay for almost al intial conditions 1 and for almost all tangent vectors up. Therefore, we can easily obtain an estimate of the largest LCE by directly applying the definition (3): be u = TabhefRandon{], {02 iih= LogtRorn(PhiT.u}]/T oa 1.46874 The calculation of the entire LCE spectrum is more prob- lematic. Oseledee [1968] shows that one can obtain the entire spectrum by computing the eigenvalues of the matrix, 5 (29)-y{z0), where } (20) isthe adjoint matrix of y(zo), and using the fact that they behave like e274, ..., ¢27*, Unfortunately, for large , (2) is anill-conditioned matrix, ‘because its columns tend to line up with the eigenvector asso: ciated to the largest eigenvalue. Consequently, the small rel ative errors in the largest eigenvalue might contaminate the smaller ones, giving unreliable estimates. To circumvent these problems, we adopt the algorithm discussed in [Benettin etal. 1980], which relies on the ealeu- lation of the order-p LCEs defined in equation 4 and the repeated application of the Gram-Schmidt orthonormaliza tion procedure. Given a set (uy stg} of p linearly independent vectors in RY, the Gram-Schmide procedure generates an orthonormal set (045 «+ Up] of vectors which spans the same subspace spanned by (ty, ..-slp). The vectors v, are given by w= uy = wifi w = uw=(auhr, ou = wr/|iwl, (8) Wp = Up— Supp riders te = wp/llelly where (,-)is the scalar product of vectors. It is easy to show thatthe volume ofthe parallelepiped spanned by (1, ,} ley Following the algorithm of [Benettin etal. 1980], we start by choosing an initial condition zp and an n xn matrix Uo=[us, «5 ud]. Using the Gram-Schmide procedure, we calculate the cortesponding matrix of orthonormal vectors Vo= [vfs ev] and integrate the variational equation (7) from {to, Vo) for a short interval 7, to obtain 2y = f"(zg) and Vol fuivotih= lll Ty 5 [upp es Ue] = Dy f7(Uo) = @rlto)« [uf ..., us Again, we calculate the orthonormalized version of U; and integrate the equation from [z;, V3] for T seconds to obtain 22) and Uy. We repeat this integration-orthonormalization procedure K times. During the k-th step, the p-dimensional volume Vol? defined in (4) increases by a factor of || wt || -- up|], where (wf, sm] is the set of orthogonal vectors calculated from Uy using (8). The definition (4) then implies (a0, th) = tin 1 Stn( pol ‘ AP (#0, Uo) = izn etn ot -~- Ip). Subtracting 27"! from 22 and using (5), we obtain the p-th LCE of order 1: ay fim 2S lato “This rlation suggests an easy way to calulate the Lyapunov spectrum, For a suitable value of 7, continue to ealeulate ‘the quantities FF Dwi oo BEV Ialuhlis an, until they show convergence (for example, using the rela tivelabsolute convergence test proposed in [Parker and Chia 1989, 302}) or until a maximum (sufficiently large) iteration count is reached. This procedure can be easily implemented with Mathe- ‘matica. Te function GranSchuidt is defined in the package Linear kigebra“Orthogonshization". baits <€ Lineartigebra*Orthogonaization” wath T= 0.45 wets PRIT = Pais lois = 8005, 2a Dot {0T, PhiT} = IntvartqQ FEC, OPRi, x, Phi, af, Pai, {T, stepsize} X= GranSchnidt(PhiT, Kornalized -> False); norms = Rep(torm, W; 5 = Appendts, norms]; hit = W/norns, Ws isis Leas « Rest[Feldlist(Plus, 0, Logls]II/(T RangelKd); ‘The three LCEs are mass LastDhees) cuane {4.48085, -0.0001195, ~72.4654) vouwes sue 81‘To show the convergence of the LCES, we use the Canver- sguncePlot command defined inthe package UE. me «LCE ts ConvergencePlot(lces] : 7 $0 as a a seeps -IQURE 2. Conese ot fh youn specu ote aren model ‘We can obtain the same results using the LCEs¢ command: uss LOESCLF, 49, 20, 50), 0.1, 800, 20, 0.02] cum 461. 48045, -0.0001195, -22.4654), 2.06558) ‘This function returns the LCE specteum and the Lyapunov dimension of the atractor. The convergence plot of the LCEs can be displayed by setting the option LCsFlat to rus. ‘An interesting example of a hyperchaotic continuous-time attractor (“Réssler hyperchaos") with two positive LCEs is siven by Inso= eossderE Ca Yay Zap WDD 2 Gy, x60. you, 344 z, 0.05 v= 0.5 2 (See [Rossler 1991].) The LCE spectrum is vas Leesrossler = LcEsC{rossler, {-10, -14, 0.3, 28}, 0.1, 2000, 1, 0.02] we €(0.182433, 0.006138, -0.00407495, -24.0851}, 3.00596) A 3D projection of the four-dimensional attractor of this dynamical system can be obtained with the PhaseSpacet command. isms PhaseSpaceC[rossler, £10, -14, 0.3, 29, 100, 20, 0.02, (1,2,31] 82 THE MATHEMATICA JOURNAL © 1985 ile Fnan itone FIGURE 3. A pojnton oh ou inarsoa Ress part ata The last argument {1,2,3} in the PhaseSpacet command speci- fies thatthe first three variables of the model (3, y, and 2) are to be plotted on the 2, y, 2 axes, respectively Finally, we can also make use of the algorithm in [Benettin et al. 1980] for estimating the LCEs of a nonautonomous system, for example the Duffing, model: ih duffingL, yy tH wih= LEESCLdutting, (0.8, 0.7, 0.2b, 0.05, 1000, 10, 0.02, LeEsPlot -> True] comsae 40. 0794484, 0., 0.159148}, 2.49730} Ys Ol y= 3+ 10 Cosle], D Lees | 020 40 «60001000 Sueps aun 4. Cnn pte be apr pec foe Dig del In this cas, there is a spurious Lyapunov exponent which converges to zero. It corresponds to the additional trivial evolution equation # = 1LCEs of Discrete Systems ‘The discrete-time case is much simpler because, by the chain rule, DaSi\to) = If Mxg)) I fleo)) Seas where J(za) = D-f(2)| nay: To avoid the above-mentioned numerical problems in the calculation of Df'z), and to show a different approach to the numerical calculation of the whole LCE spectrum, here we prefer to follow the simpler algorithm of [Eckman and Ruelle 1985]. We star by using the classical QR decomposition to write J(z) as Jtz) = QiRiy ‘where Q; is an orthogonal matrix and Ris upper triangular, ‘Then, for F=2, 3,..5f, we define UPON Qe 9) and decompose Ji’ = Qi. Clearly, Def'(z) = QuRy ~ Ry. It is possible to show that the diagonal elements u! of the uupper-triangular matrix product Y"" = R, Ry satisfy a “o 1 lim + inv =m ane Let us take, for example, the fourdimensional discrete time dynamical system wane Otte, WH = 85 = 22 = 0.05 4, x, ys 2d We can estimate the Lyapunov spectrum in the following way. First, define the starting condition and the system dimension: Hiss 10 = (0.1, 0, 0, 0; vis 9 = LengthCa0]; ‘Then, compute the QR decomposition of the Jacobian Jtx), after a 100-step transient: ose x AreayLa, as iss JEp.] s= dacobiamatrix[6Ee], 1] J. Theeadlx -> y) ats at = WestlG, HOO), 100]; i= {Gy P= QRDeconpositionLsCxe]}; Finally, repeat K times the QR decomposition procedure (9): wane K = 10005 wane 590; vans Dol it = Ghat]; {q, = Weconposition(JLst].Transposel 4]; 5 = Append{s, Teble(r(Li,i1], 44, eB], 41; wish Lees © Rest{Foldtist(Plus, 0, ReLLogls]]J1/Range(X]; ‘The LOEs are Ini6= LastChees) cots. {0. 16985, 0.160085, 0.156975, -3.48704) Ings ConvergencePlot ces) 0200 400-600 ~—«00—«1000 seeps FIGURE 8, Conerpne pat ofthe Lapua spectrum oe hypa hima, Again, we can obtain the same results using the LCESD com- mand defined in L¢E.m: nts Leeshyper2 = LeESDEG, {0.1, 0, 0, 0}, 1000, 100] cua 4016985, 0.160085, 0.156375, -3.48004), 3.13966) ‘The presence of three positive LCEs points out that the orbits of the system lie on a so-called “hyper’-chaotic attractor” [klein and Baier 1991), Lyapunov Dimension Kaplan and Yorke (1979, 228] have suggested an interesting conjecture that relates the fractal dimension of the attractor to the Lyapunov spectrum: where the LCEs are ordered in the usual way a8 hy 2s. 2 and where j isthe largest integer such that 2 +--+, > 0. In particular, Kaplan and Yorke suggest that Dis a lower bound of the capacity dimension, that is, D, < De. For more detail, see [Farmer etal. 1983] sith LyppunorBinensionlx} Modulel CL, su, jh, 1 = Sortls, Greater]; sunk = Rest{Felatist{Ptus, 0, 11) (j= LastlPosition(son, -7Positivell; First ~ sua UTIL 1 For the Réssler hyperchaos example, the Lyapunov dimen- sion is given by uss LyapunovDinension( First [Leesrossler]) aor 9.00596 voumes sue 49For the hyper’-chaotic attractor, Dy is vets yapunvDinension First Deeshyper2} st 3.1396 ‘Acknowledgments Lam very geateful to Prof. Giancarlo Benettin for his illumi- nating lectures on ergodic theory and many helpful discus- sions. Special thanks to Gianna Zuliani and Elena Alberti for their kind suggestions, and Giovanni Tondini for placing, ‘Mathematica at my disposal. Lam also grateful to Troels, Petersen and an anonymous reviewer for useful revisions to the paper and associated Mathematica program, References Beltrami, E. 1989. Mathematics for Dynamic Modeling. Boston: Academic Press. Benettn, Ga, L. Galganiy A. Giorgi, and J.M. Streleyn, 1980. Lyapunov characteristic exponents for smooth dynamical systems and for Hamiltonian systems: A method for computing all of them, Part 2: Numerical application. Meccanica 15:21-30. Fekmann, JP, and D. Ruelle, 1985, Ergodic theory of chaos and strange attractors, Reviews of Modern Physics $7 (3, Part I): 617-656. Falconer, K.J. 1985. The Geometry of Fractal Sets. Cam- bridge University Press. Farmer, J.D., E. Ort, and J.A. Yorke. 1983. The dimension of chaotic attractors. Physica 7D:153-180. Geist, KU, Parlit, and W. Lauterborn, 1990. Comparison of different methods for computing Lyapunov exponents. Progress of Theoretical Physics 83(5): 875-893. Guckenheimer, J., and P. Holmes. 1983. Nonlinear Oscilla- tions, Dynamical Systems and Bifrcation of Vector Felds. Berlin: Springer Verlag. (94 THE MATHEMATICA JOURNAL © 196 le Fearn Abies Haken, H, 1983. At least one Lyapunov exponent vanishes if the trajectory of an attractor does not contain a fixed point. Physics Letters 944(2}: 71-72. Kaplan, J., and J.A. Yorke. 1979. Functional differential ‘equations and the approximation of fixed points. In Suom- ‘merschool and Conference on Functional Differential Equations and Approximation of Fixed Points, University ‘of Bonn, 1978, edited by H.-O. Peitgen and H.-O. ‘Walther. Lectures Notes in Mathematics 730. Springer Verlag. Klein, M., and G. Baier, 1991. Hierachies of dynamical sys- tems. In A Chaotic Hierarchy, edited by G. Baier and M. Klein, Singapore: World Scientific. Maeder, R. 1991. Programming in Mathematica, 2nd ed. ‘Addison-Wesley. Oseledee, V.I, 1968. A multiplicative ergodic theorem: Lyapunov characteristic numbers for dynamical systems. Trans. Moscow Math. Soc. 19:197-231. Parkes, TS., and L.O. Chua. 1989. Practical Numerical ‘Algorithms for Chaotic Systems. New York: Springer- Verlag. Réssler, O.E. 1991. The chaotic hierarchy. In A Chaotic Hierarchy, edited by G. Baier and M. Klein. Singapore: World Scientific. Wiggins, $. 1988. Global Bifurcations and Chaos: Analytical Methods. New York: Springer-Verlag.