MA261: Mathematical Modelling and Numerical Analysis,

Term 2 Week 1-10 2020

Andreas Dedner ([email protected]), office C2.3

Lectures:
Monday 1-2, Tuesday 3-4, Friday 3-4
Support Classes and Assessments discussed tomorrow
but start next week (week 2)
Office hours: do not really have any
talk to me after lecture or send me an email
MA261: Mathematical Modelling and Numerical Analysis
General remarks:
I no lecture capture
I lecture mixture of slides and blackboard
I lecture notes and slides used in lecture provided on Moodle
I lecture notes will be updated periodically
I Aim: material from each week uploaded end of each week
I examples in lecture use Python, some scripts will be uploaded
Assessment:
I exam: in summer on theory
I coursework: mixture of theory and practical work
I practical work can be done in MATLab or Python
I some templates might be provided if required
(but possibly only in MATLab, but the Python scripts from
the lecture could serve as templates as well)
I for the practical part submitting in pairs is allowed but theory
part needs to be submitted individually
I deadline for coursework is Thursday at 12 weeks 3,5,7, and 9
(all worth the same 25% of the coursework’s 50%)
Assignments
Presenting results: plots are often better then tables but don’t put
too much into one plot and think about the form of your plots e.g.
I a y-axis between 10−20 and 1020 and a more or less straight
line is not very enlightning - reduce the range to 10−8 and 102
for example.
I for comparing the error plots between two methods the axis
should have the same range e.g. axis([0 1 1e-6 10])
I consider using a log scale on either or both axes
Relate results to lecture, e.g., mention a convergence proofs...
Avoid code duplication - if you are plotting results for different test
problem reuse code as much as possible
Always include m codes. Not every modification of the template is
required, a short remark that you changed the definition of the
function ‘f(y)‘ for example is good enough.
Look at what the printer did with your output - can the plots be
read without microscope...
Screenshots of code are hard to read! Screenshots of graphs even
more so...
Mathematical Modelling

Possible Objectives
I Developing scientific understanding
through quantitative expression of current knowledge of a
system (as well as displaying what we know, this may also
show up what we do not know)
I test the effect of changes in a system
I aid decision making (managers, planners, politicians)
Mathematical Modelling

Types of models
I Static models: soap bubble
I Discrete models: change of population from year to year
I Dynamic models: ODEs
I Stochastic models: include some form of randomness in for
example a given dynamic model
I Spatial and temporal models: PDEs for fluid flow, acoustics
Mathematical Modelling

Many real-world systems described as evolution of state variables

starting from given initial configurations and can be modelled by
ordinary differential equations (ODE):
I Mechanics: motion of masses subjected to forces
I Physics: radioactive decay of materials
I Statistical systems: queues, games, multi-stage processes
I Chemistry/Biochemistry: chemical reactions
I Biology: epidemic models for diseases in populations
I Ecology: dynamics of populations of predator and prey species
I Finance: stock markets
I Sociology: behaviour of crowds
Mathematical Modelling

Can also distinguish between types of models depending on level of

understanding on which the model is based, e.g., ”spatial”
I herd
I individual animal
I organs
I cells
I molecules
”temporal”/”spatial”:
I weather over next hours / very local over airport
I weather over the weekend / over the Midlands
I will we have snow during the winter / in the UK
I how is the climate changing / whole planet
Mathematical Modelling, Analysis, Numerical Analysis
Many steps and expertise required to cycle through:
1. Modeling: the problem is formulated in mathematical terms
2. Analysis: the mathematical model is analysed: e.g.
well-posedness (existence and uniqueness) and investigate
sensitivity to errors in the data Also determine importance of
different terms allowing to reduce complexity of model
3. Discretization: the problem is approximated by (a sequence
of) finite dimensional problems.
4. Numerical analysis: the discretization is studied:
I show that important properties are maintained e.g., positivity
of some density
I again analyse well-posedness
I most importantly: show that error between solution to finite
dimensional problem and mathematical model goes to zero
5. Implementation: the finite dimensional problems are solved
using a computer program.
Numerical Analysis:
construct methods for finding solution x of problem which are
I not exactly solvable
(e.g. ordinary or partial differential equations)
I too time consuming to solve exactly
(e.g. linear systems of equations)
Idea: find computable (finite dimensional) approximations xn
(n=dimension or n=iteration)
construction and implementation of methods for solving finite
dimensional problems
mathematical study of method for approximating x
Aim: establish convergence xn → x for n → ∞
I establish condition of data for convergence
I establish minimum size of n for |xn − x| < TOL
I study properties of different approximation methods for x
Many Sources of Errors
1. Modelling: the problem is formulated in mathematical terms,
e.g., as differential equation Modelling and data error
2. Discretization: the problem is approximated by a series of
finite dimensional problems: Discretization error
3. Implementation: the finite dimensional problems are solved
using a computer program: Rounding error
Many Sources of Errors
1. Modelling: the problem is formulated in mathematical terms,
e.g., as differential equation Modelling and data error
2. Discretization: the problem is approximated by a series of
finite dimensional problems: Discretization error
3. Implementation: the finite dimensional problems are solved
using a computer program: Rounding error
Numerical analysis: study with respect to well-posedness,
robustness, and the convergence of the numerical solution
I Modelling: construction of mathematical formulation for x.
Should know something about properties of x e.g. x > 0:
Question: does xn > 0 hold?
I Data: data needed for formulating problem for x
Should know something about properties of x e.g. data noisy
Question: if data is discontinuous or noisy does method work?
I Complexity: what are the requirements of my applications
Question: how accurate do I need the approximation to be?
Question: is there enough time to compute xn for a given n?
Leads to many schemes for solving same problem
MA261: influence of errors

u 0 (t) = (c − u(t))2 , u(0) = 1, c > 0.

Consider that instead of c = 1 we have c̃ = 1 + ε, |ε|  1. Then
the solution is
1 + t c̃ε
u(t) =
1 + tε
MA261: influence of errors

u 0 (t) = (c − u(t))2 , u(0) = 1, c > 0.

Consider that instead of c = 1 we have c̃ = 1 + ε, |ε|  1. Then
the solution is
1 + t c̃ε
u(t) =
1 + tε
ε = 0 : u ≡ 1.
ε > 0 : 1 + tε > 0 and lim u(t) = 1 + ε (stable).
t→∞
1
ε < 0 : 1 + tε = 0 (t = t0 = ε > 0), lim u(t) = ∞ (not stable).
t→t0

u(t)
c
u(t)

1 1
c
0
0 t t0
MA228 Numerical Analysis: Problems with error in data

Consider
    
1.2969 0.8648 x1 0.86419999
= =: b.
0.2161 0.1441 x2 0.14400001
   
x1 0.9911
with solution = .
x2 −0.4870
 
0.8642
Due to some error, we solve with right hand side b̄ =
0.1440
which is a relative error less than 10−7 .
MA228 Numerical Analysis: Problems with error in data

Consider
    
1.2969 0.8648 x1 0.86419999
= =: b.
0.2161 0.1441 x2 0.14400001
   
x1 0.9911
with solution = .
x2 −0.4870
 
0.8642
Due to some error, we solve with right hand side b̄ =
0.1440
−7 .
which is a relative error less than 10   
x̄1 2
The solution to the new problem is = , which
x̄2 −2
is a relative error of more than 100%.
MA261: making predictions...
Example: predicting population based on data 1910 − 1990 to
estimate population in 2000, 2010, 1900.
Polynomial interpolation (9 points fitted with 8th order polynomial)
MA261: making predictions...
Example: predicting population based on data 1910 − 1990 to
estimate population in 2000, 2010, 1900.
Polynomial interpolation (9 points fitted with 8th order polynomial)
MA228 Numerical Analysis: making predictions...
Example: predicting population based on data 1910 − 1990 to
estimate population in 2000, 2010, 1900.
Polynomial interpolation

Idea: fit l(x) = ax + b or e(x) = exp(ax + b) to given data

MA228 Numerical Analysis:
Used also for data approximation
Approximation of measurements by quadratic function
MA261: Introduction
Where is the stuff covered here used?
Historically that would include
I computing the size of some object
I computing planet and star orbits
I cartography
I finding values for complex functions sin, log, ...
Many more modern examples (big change due to computers)
I global climate prediction
I hazard mapping (flooding, earthquakes,...)
I risk prediction (finance)
I computing properties of workpieces and materials
(engineering)
I medical imaging and biology
Understanding processes, making predictions, fitting data
MA261: Introduction

You use it everyday (probably are carrying it around with you):

I MP3 player (compression)
I digital photography (compressing, sharpening, de-noising)
I weather forcast
MA261: Introduction

You use it everyday (probably are carrying it around with you):

I MP3 player (compression)
I digital photography (compressing, sharpening, de-noising)
I weather forcast
You have already seen some numerical analysis results
√  
I square root approximation x = a: xn = 12 xn−1 + a
xn−1
I Gauss elimination for solving Ax = b
1 1
I approximation of the tangent x = f 0 (a): xn = f (a+ n )−f
2n
(a− n )

I Riemann sum x = 01 f (s) ds: xn = n1 ni=1 f ( ni )

R P

I proof of intermediate value theorem

f continuous and f (a)f (b) < 0 then f (x) = 0 for some
x ∈ [a, b] (x is constructed by nested intervals).
MA228 Numerical Analysis: varying schemes and problems
Z1 Z1
1 3√
dx = ln(2) x dx = 1
x +1 2
0 0

Four composite quadratures using trapezoidal, Simpson, 2-point

Gauss, Richardson extrapolation (Romberg method)

1
Error as function of subinterval size h: 0.1

0.01
0.01
0.0001

1e-06
0.001
error

error
1e-08

0.0001
1e-10

1e-12
Trapezoidal Rule 1e-05 Trapezoidal Rule
Simpson rule Simpson rule
1e-14
2 point Gauss 2 point Gauss
Romberg Romberg
1e-16 1e-06
0.001 0.01 0.1 1 0.001 0.01 0.1 1
h h
MA228 Numerical Analysis: varying schemes and problems
Z1 Z1
1 3√
dx = ln(2) x dx = 1
x +1 2
0 0

Four composite quadratures using trapezoidal, Simpson, 2-point

Gauss, Richardson extrapolation (Romberg method)

1
Error as function of subinterval size h: 0.1

0.01
0.01
0.0001

1e-06
0.001
error

error
1e-08

0.0001
1e-10

1e-12
Trapezoidal Rule 1e-05 Trapezoidal Rule
Simpson rule Simpson rule
1e-14
2 point Gauss 2 point Gauss
Romberg Romberg
1e-16 1e-06
0.001 0.01 0.1 1 0.001 0.01 0.1 1
h h

Error vs. the number of function evaluations:

1 0.1
Trapezoidal Rule Trapezoidal Rule
Simpson rule Simpson rule
0.01
2 point Gauss 2 point Gauss
Romberg 0.01 Romberg
0.0001

1e-06
0.001
error

error

1e-08

0.0001
1e-10

1e-12
1e-05
1e-14

1e-16 1e-06
1 10 100 1000 1 10 100 1000
Function evaluations Function evaluations
Solving an ODE

y 0 (t) = f (t, y (t)) , t ∈ [0, T ] , y (0) = y0

Assumption: data f and exact solution y are smooth (in C k ).
Question: How to approximate solution?
Solving an ODE

y 0 (t) = f (t, y (t)) , t ∈ [0, T ] , y (0) = y0

Assumption: data f and exact solution y are smooth (in C k ).
Question: How to approximate solution?
Ansatz: for points in time tn = n h (h > 0) find values yn with

yn ≈ y (tn )

I only finite many points in time, number depending on h.

I Question: what happens when the discretization parameter h.
is reduced (more points are used)?
Solving an ODE

y 0 (t) = f (t, y (t)) , t ∈ [0, T ] , y (0) = y0

Assumption: data f and exact solution y are smooth (in C k ).
Question: How to approximate solution?
Ansatz: for points in time tn = n h (h > 0) find values yn with

yn ≈ y (tn )

I only finite many points in time, number depending on h.

I Question: what happens when the discretization parameter h.
is reduced (more points are used)?
d
Still have to cope with dt y ...
Ansatz: Taylor series expansion of y
Solving an ODE
Theorem (Taylor Theorem (Version 1))
Let y ∈ C m (a, b) and x0 ∈ (a, b) be given. Then there exist a
function ωm : R → R with lim ωm (x) = 0, so that
x→x0

y (x) = Pm (x) + ωm (x)(x − x0 )m ,

where
m
X 1 (k)
Pm (x) = y (x0 )(x − x0 )k ,
k!
k=0

is the m-th order Taylor polynomial which is of degree m.

Solving an ODE
Theorem (Taylor Theorem (Version 1))
Let y ∈ C m (a, b) and x0 ∈ (a, b) be given. Then there exist a
function ωm : R → R with lim ωm (x) = 0, so that
x→x0

y (x) = Pm (x) + ωm (x)(x − x0 )m ,

where
m
X 1 (k)
Pm (x) = y (x0 )(x − x0 )k ,
k!
k=0

is the m-th order Taylor polynomial which is of degree m.

With m = 1, x0 = tn , x = tn+1 :
y (tn+1 ) = y (tn ) + y 0 (tn )(tn+1 − tn ) + ω1 (tn+1 )(tn+1 − tn )
= y (tn ) + f (tn , y (tn ))(tn+1 − tn ) + ω1 (tn+1 )(tn+1 − tn )
= y (tn ) + f (tn , y (tn ))h + ω1 (tn+1 )h
Explicit or Forward Euler Method: yn+1 = yn + hf (tn , yn )
Solving an ODE (final part of course)
Explicit or Forward Euler Method: yn+1 = yn + hf (tn , yn )
(note that y0 is given).

y1 = y0 + hf (t0 , y0 ), y2 = y1 + hf (t1 , y1 ), y3 = y2 + hf (t2 , y2 ), . . .

Example: f (t, y ) = −50(y − cos(t)) − sin(t)

Solving an ODE (final part of course)
Explicit or Forward Euler Method: yn+1 = yn + hf (tn , yn )
(note that y0 is given).

y1 = y0 + hf (t0 , y0 ), y2 = y1 + hf (t1 , y1 ), y3 = y2 + hf (t2 , y2 ), . . .

Example: f (t, y ) = −50(y − cos(t)) − sin(t)

h = 0.2
1200
forward Euler
exact
1000

800

600
y(t)

400

200

-200

-400
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
t
Solving an ODE (final part of course)
Explicit or Forward Euler Method: yn+1 = yn + hf (tn , yn )
(note that y0 is given).

y1 = y0 + hf (t0 , y0 ), y2 = y1 + hf (t1 , y1 ), y3 = y2 + hf (t2 , y2 ), . . .

Example: f (t, y ) = −50(y − cos(t)) − sin(t)

h = 0.1
40
forward Euler
exact
30

20

10

0
y(t)

-10

-20

-30

-40

-50

-60
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
t
Solving an ODE (final part of course)
Explicit or Forward Euler Method: yn+1 = yn + hf (tn , yn )
(note that y0 is given).

y1 = y0 + hf (t0 , y0 ), y2 = y1 + hf (t1 , y1 ), y3 = y2 + hf (t2 , y2 ), . . .

Example: f (t, y ) = −50(y − cos(t)) − sin(t)

h = 0.05
1.4
forward Euler
exact
1.2

0.8
y(t)

0.6

0.4

0.2

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
t
Solving an ODE (final part of course)
Explicit or Forward Euler Method: yn+1 = yn + hf (tn , yn )
(note that y0 is given).

y1 = y0 + hf (t0 , y0 ), y2 = y1 + hf (t1 , y1 ), y3 = y2 + hf (t2 , y2 ), . . .

Example: f (t, y ) = −50(y − cos(t)) − sin(t)

h = 0.025
1
forward Euler
exact
0.9

0.8

0.7

0.6
y(t)

0.5

0.4

0.3

0.2

0.1

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
t
Solving an ODE
Theorem (Taylor Theorem (Version 1))
Let y ∈ C m (a, b) and x0 ∈ (a, b) be given. Then there exist a
function ωm : R → R with lim ωm (x) = 0, so that
x→x0

y (x) = Pm (x) + ωm (x)(x − x0 )m ,

where
m
X 1 (k)
Pm (x) = y (x0 )(x − x0 )k ,
k!
k=0

is the m-th order Taylor polynomial which is of degree m.

With m = 1, x0 = tn+1 , x = tn :
y (tn ) = y (tn+1 ) + y 0 (tn+1 )(tn − tn+1 ) + ω1 (tn )(tn − tn+1 )
= y (tn+1 ) + f (tn+1 , y (tn+1 ))(tn − tn+1 ) + ω1 (tn )(tn − tn+1 )
= y (tn+1 ) − f (tn+1 , y (tn+1 ))h − ω1 (tn )h
Implicit or Backward Euler Method: yn+1 − hf (tn+1 , yn+1 ) = yn
Solving an ODE (final part of course)
Implicit or Backward Euler Method: yn+1 − hf (tn+1 , yn+1 ) = yn
(note that y0 is given).
y1 − hf (t1 , y1 ) = y0 , y2 − hf (t2 , y2 ) = y1 , y3 − hf (t3 , y3 ) = y2 , . . .
Problem: find root of Fn (y ) = y − hf (tn+1 , y ) − yn (discussed in
lecture)
Example: f (t, y ) = −50(y − cos(t)) − sin(t)
Solving an ODE (final part of course)
Implicit or Backward Euler Method: yn+1 − hf (tn+1 , yn+1 ) = yn
(note that y0 is given).
y1 − hf (t1 , y1 ) = y0 , y2 − hf (t2 , y2 ) = y1 , y3 − hf (t3 , y3 ) = y2 , . . .
Problem: find root of Fn (y ) = y − hf (tn+1 , y ) − yn (discussed in
lecture)
Example: f (t, y ) = −50(y − cos(t)) − sin(t)
h = 0.2
2
forward Euler
backward Euler
exact
1.5

1
y(t)

0.5

-0.5

-1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Solving an ODE (final part of course)
Implicit or Backward Euler Method: yn+1 − hf (tn+1 , yn+1 ) = yn
(note that y0 is given).
y1 − hf (t1 , y1 ) = y0 , y2 − hf (t2 , y2 ) = y1 , y3 − hf (t3 , y3 ) = y2 , . . .
Problem: find root of Fn (y ) = y − hf (tn+1 , y ) − yn (discussed in
lecture)
Example: f (t, y ) = −50(y − cos(t)) − sin(t)
h = 0.1
2
forward Euler
backward Euler
exact
1.5

1
y(t)

0.5

-0.5

-1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Solving an ODE (final part of course)
Implicit or Backward Euler Method: yn+1 − hf (tn+1 , yn+1 ) = yn
(note that y0 is given).
y1 − hf (t1 , y1 ) = y0 , y2 − hf (t2 , y2 ) = y1 , y3 − hf (t3 , y3 ) = y2 , . . .
Problem: find root of Fn (y ) = y − hf (tn+1 , y ) − yn (discussed in
lecture)
Example: f (t, y ) = −50(y − cos(t)) − sin(t)
h = 0.05
2
forward Euler
backward Euler
exact
1.5

1
y(t)

0.5

-0.5

-1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Solving an ODE (final part of course)
Implicit or Backward Euler Method: yn+1 − hf (tn+1 , yn+1 ) = yn
(note that y0 is given).
y1 − hf (t1 , y1 ) = y0 , y2 − hf (t2 , y2 ) = y1 , y3 − hf (t3 , y3 ) = y2 , . . .
Problem: find root of Fn (y ) = y − hf (tn+1 , y ) − yn (discussed in
lecture)
Example: f (t, y ) = −50(y − cos(t)) − sin(t)
h = 0.025
2
forward Euler
backward Euler
exact
1.5

1
y(t)

0.5

-0.5

-1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Solving an ODE
Theorem (Convergence result (next week))
If the exact solution Y satisfies Y ∈ C 2 (0, T ) then

max |y (tn ) − yn | ≤ Ch
n=0,...,N

Question: Can we find method to compute yn so that

max |Y (tn ) − yn | ≤ Chp for some p > 1?

n=0,...,N

later in lecture
1

0.1

0.01
approximation error

0.001

0.0001

1e-05

1e-06

1e-07

1e-08 forward Euler

backward Euler
higher order
1e-09
0.0001 0.001 0.01 0.1 1
h
Forward Euler method for mass-spring problem
T = 2π (and T = 200π), h = 2−i 200
π

i N yN |y (T ) − yN |
0 101 [1.207662 1.227752] 0.308212
1 201 [1.101395 1.105955] 0.146654
2 401 [1.050037 1.051122] 0.071534
3 801 [1.024848 1.025113] 0.035328
4 1601 [1.012381 1.012446] 0.017555
5 3201 [1.006179 1.006196] 0.008751

i N yN |y (T ) − yN |
0 10001 [-2.00707197e+07 5.08075873e+08] 508472148.289626
1 20001 [14884.248122 22777.185506] 27207.824258
2 40001 [131.598722 145.95242 ] 195.108253
3 80001 [11.637561 11.9422 ] 15.260715
4 160001 [3.422772 3.444954] 3.442038
5 320001 [1.851583 1.854576] 1.206438
Forward Euler method for mass-spring problem

2 10

0 0

−2 −10

−2 0 2 −10 0 10
Backward Euler method for mass-spring problem

1 1

0 0

−1 −1

−1 0 1 −1 0 1

i N yn |y (T ) − yN |
0 101 [0.814386 0.827934] 0.2531
1 201 [0.904188 0.907932] 0.132877
2 401 [0.951364 0.952347] 0.06809
3 801 [0.975503 0.975755] 0.034467
4 1601 [0.987707 0.987771] 0.01734
5 3201 [0.993842 0.993859] 0.008697
Observation

For fixed T the approximation seems to converge to zero for h → 0

with both forward and backward Euler method

For large T need very small h to get good approximation for y (T )

we have the choice of our satellite spiriling out into space or
dropping onto earth...

Can we derive a method that observes d 0

dt H(y (t), y (t)) = 0, i.e.,
H(yn+1 ) = H(yn )
Symplectic Euler for mass-spring problem (later in lecture)
γ
yn+1,1 = yn,1 + hyn,2 , yn+1,2 = yn,2 − h yn+1,1 .
µ
(Forward Euler: yn+1,1 = yn,1 + hyn,2 , yn+1,2 = yn,2 − h µγ yn,1 ).

1 1
0 0
−1 −1

−1 0 1 −1 0 1
i N yn |y (T ) − yN |
0 101 [1.001066 0.998932] 0.001509
1 201 [1.000262 0.999737] 0.000371
2 401 [1.000065 0.999935] 9.2e-05
3 801 [1.000016 0.999984] 2.3e-05
4 1601 [1.000004 0.999996] 6e-06
5 3201 [1.000001 0.999999] 1e-06
Observation

For fixed T the approximation seems to converge to zero for h → 0

with forward, backward, and symplectic Euler method

For forward/backward Euler and large T need very small h to get

good approximation for y (T )

Symplectic method gives better results with 100 points then

forward/backward on 3000 but
specific for mass-spring type systems, i.e., not universal