LECTURE 1: INTRODUCTION

1. Introduction
2. Maps
NONLINEAR DYNAMICS AND CHAOS
3. Flows
Patrick E McSharry 4. Fractals and Attractors

Systems Analysis, Modelling & Prediction Group 5. Bifurcations

6. Quantifying Chaos
www.eng.ox.ac.uk/samp
7. Nonlinear Time Series Analysis
[email protected]
8. Nonlinear Modelling and Forecasting
Tel: +44 20 8123 1574
9. Real-World Applications
10. Weather Forecasting
Trinity Term 2007, Weeks 3 and 4
11. Biomedical Models
Mondays, Wednesdays & Fridays 09:00 - 11:00
12. Time Series Analysis Workshop
Seminar Room 2
Mathematical Institute
University of Oxford

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.1 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.2

Suggested Reading Relevant journal articles

Strogatz, S. H., Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Yorke, J. and Li, T. Y., Period Three Implies Chaos, American Mathematical Monthly
Chemistry, and Engineering, Reading, MA: Addison-Wesley (1994) 82:985-992 (1975)
Eubank, S., and D. Farmer, An introduction to chaos and randomness. In Jen, E. (Ed.), 1989 May, R., Simple mathematical models with very complicated dynamics. Nature 261:
Lectures in Complex Systems. Santa Fe Institute Studies in the Sciences of Complexity, 459-467 (1976)
Lecture Vol. II, pp. 75-190. Reading, MA: Addison-Wesley, (1990) Packard, N. and Crutchfield, J. and Farmer, J. D. and Shaw, R., Geometry from a time
Ott, E. and Sauer, T. and Yorke, J. Coping with Chaos, J. A. John Wiley & Sons, New York series, Phys. Rev. Lett. 45: 712-716 (1980)
(1984) Crutchfield, J. P, N. H. Packard, J. D. Farmer, and R. S. Shaw. (1986) Chaos, Scientific
Ott, E., Chaos in Dynamical Systems, Cambridge: Cambridge University Press (1993) American 255:46-57 (1996)
Schuster, H.G., Deterministic Chaos: An Introduction, VCH, (1995).
Wiggins, S., Introduction to Applied Nonlinear Dynamical Systems and Chaos, Springer
(1990)
Kantz, H. and Schreiber, T., Nonlinear Time Series Analysis, Cambridge Univ. Press (1997)
Abarbanel, H.D.I, Analysis of Observed Chaotic Data, Springer, (1996)

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.3 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.4
Sources of information Suggested special topics

Journals: Look for evidence of low-dimensionality in real time series

Physica D Investigate the predictability of a real time series using nonlinear methods
Physical Review Letters
Available time series:
Physical Review E electronic circuits
Physics Letters A lasers
International Journal of Bifurcation and Chaos sunspot record
Online discussion groups: electricity data (demand, price, grid frequency)
sci.nonlinear weather data (temperature, precipitation, wind speed)
www.jiscmail.ac.uk/lists/allstat.html economic data (GDP, inflation, interest rates, unemployment)
www.jiscmail.ac.uk/lists/timeseries.html financial data (stockmarket tick data, S&P500, Dow Jones, FTSE100)
Websites: biomedical signals (ECG, EEG, blood pressure, respiration, blood gases)
www.societyforchaostheory.org (Society for chaos theory) Construct and investigate a nonlinear model of a particular system
www.physionet.org (MIT-Harvard biomedical database and tools)
Compare nonlinear methods for forecasting, (e.g. RBFs vs. local linear)
www.comdig.org (Complexity Digest)
Develop new methods for classification of health/disease using biomedical signals
TISEAN package
Matlab software

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.5 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.6

Special topic instructions Linear analysis

Definition of linearity:
Should be one week of work L(ax) = aL(x)
Marking scheme:- L(x + y) = L(x) + L(y)
content 20 pts Principle of superposition: If x and y are solutions, then z = ax + by is also a solution
presentation 5 pts Advantages of linear models:
Two copies of final report required Often have analytical solutions
A large body of historical knowledge for helping with model specification and estimation
Less parameters - smaller chance of overfitting
Can employ Fourier spectral analysis and associated techniques
Disadvantages of linear models:
Real-world systems are usually nonlinear
Linearity is a first order approximation and neglects higher orders
Stanislaw Ulam: nonlinear science is like non-elephant zoology
In practice, while underlying dynamics may be nonlinear, observed data may only provide
sufficient resolution for linear models
Need relevant null hypothesis tests for nonlinearity (surrogate data)

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.7 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.8
Normal distributions Dynamical systems I
Variables Linear Nonlinear
n=1 Growth, decay, equilibrium
Mean x0 and variance σ 2 : Exponential growth Fixed points
RC circuit Bifurcations
−(x − x0 )2
» –
1 Radioactive decay Overdamped systems, relaxational dynamics
p(x) = √ exp
2πσ 2 2σ 2 Logistic equation for single species
n=2 Oscillations
All higher order moments are given in terms of x0 and σ
Linear oscillator Pendulum
Easily manipulated: if x ∼ N (0, σx2 ) and y ∼ N (0, σy2 ), then x + y ∼ N (0, σx2 + σy2 ) Mass and spring Anharmonic oscillator
Central limit theorem: sum of a large number of IID random variables (with finite mean and RLC circuit Limit cycles
variance) is normally distributed 2-body problem Biological oscillators
For linear models: Predator-prey cycles
Normal distributions are preserved by principle of superposition Nonlinear electronics (van Der Pol)
Normally distributed forecast errors: Maximum likelihood gives least squares n≥3 Chaos
Useful for calculating prediction intervals Civil engineering Strange attractors (Lorenz)
Electrical engineering 3-body problem (Poincaré)
Problems for nonlinear systems:
Chemical kinetics
Use of normal distributions neglects possibility of asymmetric distributions
Iterated maps (Feigenbaum)
Fat tailed distributions imply larger probability of worse case scenarios (risk
management) Fractals (Mandelbrot)
Forced nonlinear oscillators (Levison, Smale)

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.9 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.10

Dynamical systems II Napolean’s Army’s Russian Campaign

Variables Linear Nonlinear
nÀ1 Collective phenomena
Coupled harmonic oscillators Coupled nonlinear oscillators
Solid-state physics Lasers, nonlinear optics
Molecular dynamics Nonequilibrium statistical mechanics
Equilibrium statistical mechanics Nonlinear solid-state physics
Heart cell synchronisation
Neural networks
Economics
Continuum Waves and patterns Spatio-temporal complexity
Elasticity Nonlinear waves (shocks, solitons)
Wave equations Plasmas
Electromagnetism (Maxwell) Earthquakes
Quantum mechanics (Schrödinger, Heisenberg) General relativity (Einstein)
Heat and diffusion Quantum field theory
Acoustics Reaction-diffusion, biological waves
Viscous fluids Fibrillation
Epilepsy Map drawn by the French engineer Charles Joseph Minard in 1861 to show the tremendous
Turbulent fluids losses of Napolean’s army during his Russian Campaign of 1812
Life

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.11 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.12
Happiness time evolution Happiness versus GDP

From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)
From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)
Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.13 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.14

Time series from nonlinear systems Mathematical characterisation

Time series: variables are recorded as a function of time

A
Steady state: a constant solution of a mathematical equation
e.g. Homeostasis: relative constancy of the internal environment with respect to variables
B such as blood sugar, blood gases, blood pressure and pH. Control mechanisms constrain
variables to narrow limits. e.g. Following a hemorrhage, reflex mechanisms quickly restore
blood pressure to equilibrium values.
C Oscillations: periodic solutions of mathematical equations
e.g. Heartbeat, respiration, sleep-wake cycles and reproduction

D
Irregular activity: intrinsic fluctuations, can even be present when external parameters are
relatively constant. Two distinct mathematical descriptions: noise and chaos
Noise:
E Variability cannot be linked with any underlying stationary or periodic process
e.g. Fluctuating environment: eating, exercise, rest and posture affects heart rate, blood
pressure, blood-sugar levels and insulin levels
F
e.g. Respiratory Sinus Arrhythmia (RSA): heart rate increases during inspiration
Chaos:
G Irregularity that arises in a deterministic system
Chaos can exist without influence of external noise

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.15 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.16
Noise versus Chaos An exponential distribution, but not a Poisson process

Inter-event time sequences (e.g. heartbeat, neurons firing) 1.5

Believed to be a random process ti+1 1

Simplest model for such a random process is a Poisson process: 0.5

Probability of event to occur in a very short time increment dt is Rdt

0
The probability R is independent of the previous history 0 0.1 0.2
ln2/3
0.3 0.4 0.5
ti
0.6 0.7 0.8 0.9 1

Probability of two or more events occurring during dt is negligible 1.4

(Rt)k −Rt 1.2

Probability of k events in time interval t is Pk (t) = k!
e (Poisson distribution) 1

Probability that interval between event and (k + st

1) following event is ti
0.8

0.6
R(Rt)k −Rt
pk (t) = k!
e (PDF of Poisson process) 0.4

0.2
Average time between events is 1/R and variance is 1/R2 0
0 10 20 30 40 50 60 70 80 90 100
i
Nonlinear map, ti+1 = −(1/R) ln |1 − 2 exp(−Rti )| also gives p(t) = Re−Rt
4

Observation of an exponential probability density is not sufficient to identify a Poisson 3.5

process! 3

2.5

Use recurrence plots (e.g. ti+1 versus ti ) to identify structural equations p(t) 2

1.5

0.5

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
t
Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.17 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.18

Determinism and Predictability Poincaré

After Newton people believed in a deterministic, and hence, predictable Universe After tackling the 3-body problem Poincarè identified the phenomenon of sensitive
“Given for one instant an intelligence which could comprehend all the forces by which nature dependence on initial conditions (SDIC), this provided a definition of “chaos”
is animated and the respective situation of the beings who compose it—an intelligence and “If we knew exactly the laws of nature and the situation of the universe at the initial moment,
sufficiently vast to submit these data to analysis—it would embrace in the same formula the we could predict exactly the situation of that same universe at a succeeding moment. But
movements of the greatest bodies of the universe and those of the lightest atom; for it, even if it were the case that the natural laws had no longer any secret for us, we could still
nothing would be uncertain and the future, as the past, would be present before its eyes.” only know the initial situation approximately. If that enabled us to predict the succeeding
[P.S. Laplace, 1814] situation with the same approximation, that is all we require, and we should say that the
“Laplacian dream” excludes stochastic laws of physics phenomenon had been predicted, that is is governed by laws. But it is not always so; it may
Laplace acknowledged that we would never achieve the “intelligence” required—a tacit happen that small differences in the initial conditions produce very great ones in the final
appreciation that deterministic systems might not, in practice, be predictable phenomena. A small error in the former will produce an enormous error in the latter.
Prediction becomes impossible, and we have the fortuitous phenomenon.”[H. Poincaré,
1903]
deterministic 6= predictable

Laplace saw probabilities as a way to describe our ignorance of a deterministic system

Analytic expediency means most mathematics revolves around linear systems

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.19 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.20
Lorenz’s butterfly effect Sensitive dependence on initial condition

20 50

45
15
40

35
10

30

5
25

z
20
x(t) 0

15

−5 10

5
−10
0
−20 −15 −10 −5 0 5 10 15 20
x
−15
Perfect model and perfect knowledge of observational uncertainty
Predictability varies with position
−20
0 1 2 3 4 5 6 7 8
time [secs]

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.21 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.22

Chaos Two faces of Chaos

Advent of digital computer allowed numerical investigation of nonlinear equations The word chaos refers to disorder and extreme confusion
Lorenz found SDIC in a numerical model of the atmosphere and constructed the “Lorenz To a scientist, it implies “deterministic disorder”
system” to illustrate the effect in a simple system [1963] This might suggest that a chaotic system should be unpredictable
Yorke and Li coined the word “chaos” in 1975 On the contrary, a chaotic deterministic system is, in principle, perfectly predictable
May demonstrates chaos in the one-dimensional Logistic map in 1976 The sensitive dependence of the system dynamics to the initial conditions (SDIC) implies
Chaos becomes trendy, “Chaos” is published by James Gleick, 1987 that, in reality, any error in specifying the initial condition will lead to an erroneous prediction
Claims of chaos in the brain, heart, economy, stockmarket, ... Laplace suggests using probabilistic predictions to overcome the problems, of diverging
Investigations of nonlinear dynamical systems, claims of chaos are played down! trajectories, posed by chaotic systems
Chaos is sometimes used as a scapegoat: meteorologists blame chaos for inaccurate
predictions when it is often model inadequacy that is at fault
Important research: separating model inadequacy (structural and parametrical errors) from
effects of observational uncertainty

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.23 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.24
Deterministic versus Stochastic Deterministic versus Stochastic

stochastic system Deterministic:

Everything is described by a single point in state space. This description
xt+1 = axt + εt εt : N (0, 1) completely determines the future.
Stochastic: Knowledge of present states does not determine the evolution of future states.
deterministic system
yt+1 = ayt + σzt Can you tell if a system is stochastic or deterministic?
Should a system be modelled as stochastic or deterministic?
zt+1 = 4zt (1 − zt )
High dimensional deterministic chaotic system might be modelled as a stochastic system
State space trajectory of an autonomous, deterministic system never crosses itself
Sources of forecast uncertainty:
Uncertainty in the initial condition
Model inadequacy (parametrical and structural)

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.25 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.26

Introduction to Dynamical Systems Properties of dynamical systems

A state is an array of numbers that provides sufficient information to describe the future Determinism: trajectories should not diverge when going forward in time
evolution of the system. Invertibility: A dynamical system is invertible if each state x(t) has a unique predecessor
If m numbers are required, then these form an m-dimensional state vector x. x(t − 1). This implies that trajectories should never merge.
The collection of these state vectors defines an m-dimensional state space. Thus continuous deterministic flows are always invertible!
The rule for evolving from one state to another may be expressed as a discrete map or a Maps derived from flows (Poincaré maps) are also invertible
continuous flow:
Reversibility: if the dynamical system obtained by the transformation t → −t is equivalent to
map xt+1 = F (xt ) the original one
flow ẋ(t) = f (x(t))
Invariance under coordinate transforms: an invariant of a dynamical system represents a
Fixed point of a map: x0 = F (x0 ) fundamental property of that dynamical system, e.g. dimensions and Lyapunov exponents
Fixed point of a flow: ẋ0 = f (x0 ) = 0 System invariant offer a means of summarising the behaviour of a particular system: (e.g.
Non-autonomous system: ẋ = f (x, t) health and disease)

Autonomous system: ẋ = f (x)

If the non-autonomous nature is due to periodic terms it can be made autonomous
Dissipative flow: ∇ · f < 0 implies contracting state space volume
Hamiltonian systems are non-dissipative or conservative, preserving state space volume
(Liouville theorem)

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.27 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.28
Simple Pendulum Fixed Points

An example of a simple two-dimensional dynamical system Consider the fixed point of a flow f (x0 ) = 0
From Newtons’s second law, knowledge of the forces, position and velocity are sufficient to Let x(t) = x0 + ε(t)
determine future motion ε̇ = f (x0 + ε)
Pendulum (constrained to move in the plane)
ε̇ = f (x0 ) + Dx f (x0 )ε + O(||ε||2 )
Dynamics fully specified by the displacement angle θ(t) and the angular velocity θ̇(t)
State vector given by x(t) = [θ(t), θ̇(t)] ε̇ ≈ Dx f (x0 )ε = Jε

Let m be the mass of the pendulum J is Jacobian matrix of partial derivatives

g is the acceleration due to gravity The solution is
l is the length of the pendulum ε(t) = eJt ε0

Tangential restoring force due to gravity: −mg sin θ Let λi be (distinct) eigenvalues of J
Tangential force due to angular acceleration: mlθ̈
P −1 JP = Λ
In the absence of friction, dynamics are governed by
where Λii = λi and Λij = 0 if i 6= j
d
θ = θ̇ Let ε = P y so
dt
d g y = eΛt y0
θ̇ = − sin θ
dt l
Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.29 Nonlinear dynamics and chaos °
c 2007 Patrick McSharry – p.30

Classification of fixed points

<(λi ) 6= 0 for all i: hyperbolic fixed point

Otherwise non-hyperbolic fixed point
Hyperbolic fixed points:
sinks: all <(λi ) < 0
sources: all <(λi ) > 0
saddle point: some positive and some negative
If =(λi ) = 0: node
If =(λi ) 6= 0: focus
Non-hyperbolic fixed points:
if <(λi ) > 0 for some i: unstable
some <(λi ) < 0 some = 0: neutrally stable
all <(λi ) = 0: centre

Nonlinear dynamics and chaos °

c 2007 Patrick McSharry – p.31