Numerical Solution of Ordinary Differential Equations

Engineering Computation

Ordinary Differential Equations

Most fundamental laws of Science are based on models that explain variations in physical properties and states of systems described by differential equations. Several examples of laws appear in C&C PT 7.1 and are applied in Ch. 28 --

dv F Newton's 2nd law: = dt m dT Fourier's heat law: q = k ' dx dc Fick's diffusion law j = D dx
Faraday's law:
Engineering Computation

velocity(v), force (F), and mass (m) heat flux (q), temperature (T), and thermal conductivity (k ') mass flux (j), concentration (c) and diffusion coefficient (D) voltage drop (V), inductance (L) and current (i)
2

di Vi = L dt

Differential Equation Basics

ODE's Ordinary Differential Equations Only one independent variable, i.e., x as in y(x):

d2 y dx
2

+ 2 y = g(x)

1-dimensional problem in space x time-dynamics problem

dy = f (y, t) dt

PDE's Partial Differential Equations More than one independent variable, i.e., x and y as in T(x,y): 2T x 2u t
2 2

2T y
2

=0 =0

Laplace Equation Wave Equation u = u(x,t)

3

= c2

2u x
2

Engineering Computation

Auxiliary Conditions

Auxiliary Conditions Because we are integrating an indefinite integral, we need additional information to obtain a unique solution. Note that an nth order equations generally requires n auxiliary conditions.

Initial Value Problem (IV) Information at a single value of the independent variable, typically at the beginning of the interval: y(0) = yo Boundary Value Problem (BV) Information at more that one value of the independent variable, typically at both ends of the interval: y(0) = yo and y(1) = yf
Engineering Computation 4

Outline of our Study of ODE's

I. Single-Step Methods for IV Problems (C&C Ch. 25) a. Euler b. Heun and Midpoint/Improved Polygon c. General Runge-Kutta d. Adaptive step-size control II. Stiff ODEs (C&C 26.1) III. Multi-Step Methods for IV Problems (C&C 26.2) a. Non-Self-Starting Heun b. Newton-Cotes c. Adams IV. Boundary Value (BV) Problems (C&C Ch. 27.1 & 27.2) a. Solution Methods 1. Shooting Method 2. Finite Difference Method b. Eigenvalue Problems { [A] [I ] } {x} = {0}
Engineering Computation 5

ODE's: One-step methods

One-Step Methods for IV Problems Consider the generic first-order initial value (IV) ODE problem:

dy = f (x, y) dx

where (xo,yo) are given and we wish to find y = y(x) The independent variable x may be space x, or time t.

Engineering Computation

ODE's: One-step methods We can solve higher-order IV ODE's by transforming to a set of 1st-order ODE's,

dy + + 5y = 0 2 dx dx dy dz + z + 5y = 0 Let z = & substitute dx dx Now solve a SYSTEM of two linear, first order ordinary differential equations: dy =z dx and dz = z 5y dx

d2 y

Engineering Computation

ODE's: first order IV problem - One-step methods

dy = f (x, y) dx

where (xo,yo) are given and we wish to find y = y(x).

The basic approach to numerical solution is stepwise: Start with (xo,yo) => (x1,y1) => (x2,y2) => etc. Next Value = Previous Value + slope step size yi+1 = yi + i h

h = xi+1 xi = step size Key to the various one-step methods is how the slope is obtained. This slope represents a weighted average of the slope over the entire interval and may not be the tangent at (xi, yi)

Engineering Computation

Using an estimate of the slope to guess f(xi+1 )

Predicted True

xi

x i+1

Engineering Computation

Euler's Method

Given (xi,yi), need to determine (xi+1,yi+1) xi+1 = xi + h Estimate the slope as i = f(xi,yi) yi+1 = yi + f(xi,yi) h The slope at the beginning of the step is applied across the entire interval yi+1 = yi + i h

Analysis Local Error for the Euler Method Taylor series of true solution: yi+1 = yi + yi' h + yi" h2/2 +

where yi' = f(xi,yi) = fi Euler rule: Truncation error: (local) Global Error:
Engineering Computation

h2 i+1 = yi " = O (h 2 ) E a = yi+1 y 2 Ea = G(n,h) O (h2) = O (h)

10

i+1 = yi + fi h y

Euler's Method: Graphic example

y y' = - 2x3 + 12x2 - 20x + 8.5 y(xo) = a Euler's method solution y(xi+1) = y(xi) + y'(xi)h y1 True solution

a xo x1 h Matlab demo
Engineering Computation 11

Error Analysis
1. Truncation Error (Truncating Taylor Series) [large step size large errors] 2. Rounding Error (Machine precision) [very small step size roundoff errors] Two kinds of Truncation Error:

Local error within one step due to application of the numerical method Propagation error due to previous local errors.
Global Truncation Error = Local + Propagation Generally if the local truncation error is O (hn+1) then, as with numerical quadrature formulas, the global truncation error is O(hn). (Proof is more difficult.) General Error Notes: 1. For stable systems, error is reduced by decreasing h 2. If method is O (hn) globally, then it is exact for (n-1)th order polynomial in x
Engineering Computation 12

Improved One-Step Methods Heun's Method (simple predictor-corrector)

Predict:

yi+1 o = yi + f(xi,yi)h
f (x i , yi ) + f (x i+1 , yi+1 ) f = 2
o

Estimate the avg. slope as: Correct: xi+1 = xi + h

yi+1 = yi + f h

Notes: 1. Local Error O (h3) and Global Error O (h2) 2. Corrector step can be iterated using a as stop criterion 3. If derivative is only f(x) and not f(x,y), then the predictor has no effect and f (x i ) + f (x i +1 ) yi +1 = yi + h 2

which is the Trapezoid Rule.

Engineering Computation 13

Euler's Method: Graphic example

y' = - 2x3 + 12x2 - 20x + 8.5 y(xo) = a Heun's method solution y(xi+1) = y(xi) + h [y'(xi)+y'(xi+1)]/2 True solution y1

a xo x1 h Matlab demo
Engineering Computation 14

Midpoint Method (Improved Polygon, Modified Euler)

Predict yi+1/2 with a half step:

yi+1/ 2 = yi + f (x i , yi ) h 2 h = xi + 2

Estimate the slope:

slope = f(xi+1/2,yi+1/2) with x i+1/2

Correct yi+1:

yi+1 = yi + f(xi+1/2,yi+1/2) h

Note: Local Error O (h3) and Global Error O (h2) Error same order as Heun.

Matlab demo
Engineering Computation 15

General Runge-Kutta (RK) Methods Employing single-step yi+1 = yi + (xi ,yi, h) h

Increment function or slope, (xi ,yi,h), is a weighted average: = a1k1i + a2k2i + + ajkji + + ankni (nth order method) where: aj = weighting factors (that sum to unity) kji = slope at point xji such that xi < xji < xi+1 k1i = f (xi ,yi) k2i = f ( xi + p1h, yi + q11k1ih) k3i = f ( xi + p2h, yi + q21k1ih + q22k2ih) kni = f ( xi + pn-1h, yi + qn-1,1k1ih + qn-1,2k2ih + . . . + qn-1,n-1kn-1,ih)
Engineering Computation 16

Runge-Kutta (RK) Methods First-order Runge-Kutta Method: Euler's Method k1 = f(xi ,yi) a1=1 yi+1 = yi + k1 h Has a global truncation error, O (h) Second-order Runge-Kutta Methods: Assume a2 = 1/2; Heun's w/ Single Corrector: no iteration k1 = f(xi ,yi) 1 1 yi+1 = yi + 2 k1 + 2 k 2 h k = f( x + h, y + hk ) 2 i i 1

Assume a2 = 1; Midpoint (Improved Polygon) yi+1 = yi + k2 h

1 hk k 2 = f xi + 1 h, y + i 1 2 2

same k1

Assume a2 = 2/3; Ralston's Method (minimum bound on Et) 2 k h k = f x + 3 h, y + 3 hk yi+1 = yi + 1 k + 2 i 4 i 4 1 same k1 1 3 3 2

All have global truncation error, O (h2)

Engineering Computation 17

Classical Fourth-order Runge-Kutta Method Classical Fourth-order Runge-Kutta Method Employing single-step: yi+1 = yi + (1/6) [ k1 + 2k2 + 2k3 + k4 ] h

where

k1 = f(xi , yi) k2 = f(xi + h/2 , yi + k1h/2) k3 = f(xi + h/2 , yi + k2h/2) k4 = f(xi + h , yi + k3h)

Euler step Midpoint Better midpoint Full step

1. Global truncation error, O (h4) 2. If the derivative, f, is a function of only x, this reduces to Simpson's 1/3 Rule. yi+1 = yi + h (k1 + 2k2 + 2k3 + k4)/6 where k2 = k3
Matlab demo
Engineering Computation 18

Examples of Frequently Encountered, Simple ODE's 1. Exponential growth unconstrained growth of biological organisms, positive feedback electrical systems, and chemical reactions generating their own catalyst)

dy =y dt

with solution

y(t) = y0 e t

2. Exponential decay discharge of a capacitor, decomposition of material in a river, wash-out of chemicals in a reactor, and radioactive decay

dy = - y dt
Engineering Computation

with solution

y(t) = y0 e - t
19

Classical Fourth-order Runge-Kutta Method -- Example Numerical Solution of the simple differential equation
y = + 2.77259 y with y(0) = 1.00; Solution is y = exp( +2.773 x) = 16x
Step sizes vary so that all methods use the same number of functions evaluations to progress from x = 0 to x = 1.

x 0.000 0.125 0.250 0.375 0.500 0.625 0.750 0.875 1.000

Exact Solution 1.000 1.414 2.000 2.828 4.000 5.657 8.000 11.31 16.00 h=

Euler 1.000 1.347 1.813 2.442 3.288 4.427 5.962 8.028 10.81 0.125

Heun w/o iter 1.000

4th-order RungeKutta 1.000

h * ki for R-K

1.933 3.738 7.227 13.97 0.25 15.56 0.5 3.945

1.386 2.347 3.013 5.564 5.469 9.260 11.89 21.95

20

Engineering Computation

Classical Fourth-order Runge-Kutta Method Example (cont.)

x 0.0000 0.0625 0.1250 0.1875 0.2500 0.3125 0.3750 0.4375 0.5000 0.5625 0.6250 0.6875 0.7500 0.8125 0.8750 0.9375 1.0000 Exact Solution 1.0000 1.1892 1.4142 1.6818 2.0000 2.3784 2.8284 3.3636 4.0000 4.7568 5.6569 6.7272 8.0000 9.5137 11.314 13.454 16.000 Euler 1.0000 1.1733 1.3766 1.6151 1.8950 2.2234 2.6087 3.0608 3.5911 4.2134 4.9436 5.8002 6.8053 7.9846 9.3683 10.992 12.896 Heun w/o iter 1.0000 1.4066 1.9786 2.7832 3.9149 5.5068 7.7460 10.8958 15.326 15.952 7.9820 3.9940 1.9985 4th-order RungeKutta h * ki for R-K 0.6931 0.9334 1.0166 1.3978 1.3853 1.8653 2.0317 2.7935 2.7684 3.7279 4.0604 5.5829 5.5327 7.4502 8.1147 11.157 Matlab demo
21

h=
Engineering Computation

0.0625

0.125

0.25

Classical Fourth-order Runge-Kutta Method In each case, all three RK methods used the same number of function evaluations to move from 0.00 to 1.00. Which was able to provide the more accurate estimate of y(1)?

Engineering Computation

22

Higher-Order ODEs and Systems of Equations (C&C 25.4, p.711) An nth order ODE can be converted into a system of n, coupled 1storder ODEs. Systems of first order ODEs are solved just as one solves a single ODE. Consider the 4th-order ODE:

f(x) = y'''' + a(x) y''' + b(x) y'' + c(x) y' + d(x) y Let: y''' = v3; y'' = v2; and y' = v1 Write this 4th order ODE as a system of four coupled 1st order ODEs:
v1 y v v d 1 2 = v3 dx v 2 v f (x) a(x)v b(x)v c(x)v d(x)y 3 2 1 3
Engineering Computation 23

Higher-Order ODEs and Systems of Equations (C&C 25.4, P. 711)

v1 dy / dx dv / dx v 2 1 = dv 2 / dx v3 dv / dx f (x) a(x)v b(x)v c(x)v d(x)y 3 2 1 3

Given the initial conditions @ x = 0: y(0); v1 = y'(0); v2 = y''(0); v3 = y'''(0), a numerical scheme can be used to integrate this system forward in time.

Engineering Computation

24

Example: Harmonically Driven Oscillator Example: Single-degree-of-freedom, undamped, harmonically driven oscillator (see also C&C 28.4, p. 797)

ODE is 2nd order: m m = mass k = spring constant P(t) = forcing function = P sin(t) i.e., harmonically driven d2x dt
2

+ kx = P(t)

or

d2x

P(t) + x = 2 m dt
2

k = = circular frequency m 2 T= = natural period

Engineering Computation

25

Example: Harmonically Driven Oscillator

d2x dt 2 Initial conditions: Analytical solution:

x(t) = P

+ 2 x =

P(t) m dx (0) = 0 dt

x(0) = 0 and

m( )

sin(t) +

P m( )
2 2

sin( t)

= (homogeneous soln.) + (particular soln.)

Engineering Computation

26

Example: Harmonically Driven Oscillator

d2x dt 2

+ 2 x =

P(t) m

Recast the 2nd-order ODE as two 1st-order ODEs): dx = v = f (t, x, v) dt dv P sin( t) = 2 x = g(t, x, v) dt m with x(0) = 0 with v(0) = 0

Engineering Computation

27

Example: Harmonically Driven Oscillator Euler Method

d2x dt 2

+ 2 x =

P(t) m

dx = v = f (t, x, v) dt dv P sin( t) = 2 x = g(t, x, v) dt m

We can solve these two 1st-order ODE's sequentially. Let m = 0.25, k = 42, P = 20, = 8, Thus g = 80 sin(8t) - 162x.
t x

Solve from t = 0 until t = 1.

v

For example, by the Euler method, with h = 0.01:

t0 = 0 t1 = t0 + h = 0.01 t2 = t1 + h = 0.02 ... tn = tn-1 + h
Engineering Computation

x0 = 0 x1 = x0 + f(t0,x0,v0)h = 0 x2 = x1 + f(t1,x1,v1)h = 0.0639 ...

xn = xn-1 + f(tn-1,xn-1,vn-1)h

v0 = 0 v1 = v0 + g(t0,x0,v0)h = 6.39 v2 = v1 + g(t1,x1,v1)h = 12.75 ... demo vn = vn-1 + g(tn-1,xn-1,vn-1Matlab )h

28

Adaptive Step-size Control (C&C 25.5, p. 710) Goal: with little additional effort estimate (bound) magnitude of local truncation error at each step so that step size can be reduced/increased if local error increases/decreases.

1. Repeat analysis at each time step with step length h and h/2. Compare results to estimate local error. (C&C 25.5.1) Use Richardson extrapolation to obtain higher order result. 2. Use a matched pair of Runge-Kutta formulas of order r and r+1 which use common values of ki, and yield estimate or bound local truncation error. C&C 25.5.2 discuss 4th-5th-order Runge-Kutta-Fehlberg pair.
Engineering Computation 29

A 2nd 3rd order RK pair yielding local truncation error estimate:

2nd-order Midpoint Method O(h2): k2 = f(xi + 1/2h, yi + 1/2hk1) k1 = f(xi ,yi); yi+1 = yi + k2 h Third-order RK due to Kutta O(h3) (C&C 25.3.2) k3 = f(xi + h, yi +h(2k2 k1) ) yi+1* = yi + h(k1 +4k2+ k3) Estimate of truncation error for midpoint formula is Ei = yi+1 yi+1* = h(k1 - 2k2+ k3) This is a central difference estimate of (const.) h3 y''' which describes the local truncation error for midpoint method. Use more accurate values yi+1*, but Ei provides estimate of the local error that can be used for step-size control.
Engineering Computation 30

Stiff Differential Equations (C&C 26.1, p. 719)

A stiff system of ODEs is one involving rapidly changing components together with slowly changing ones. In many cases, the rapidly varying components die away quickly, after which the solution is dominated by the slow ones. Even simple first-order ODEs can be stiff. C&C gives the example:

dy = 1000 y + 3000 2000e t dt

With initial conditions y(0) = 0 and with solution

y (t ) = 3 0.998e 1000t 2.002e t

Fast term
Engineering Computation

Slow term
31

Stiff Differential Equations

In the solution

y (t ) = 3 0.998e 1000t 2.002e t

the middle term damps out very quickly and after it does, the slow solution closely follows the path:

y (t ) 3 2e t
We would need a very small time step h = t to capture the behavior of the rapid transient and to preserve a stable and accurate solution, and this would then make it very laborious to compute the slowly evolving solution.
Matlab demo
Engineering Computation 32

Stiff Differential Equations

A stability analysis of this equation provides important insights. For a solution to be stable means that errors at any stage of computation are not amplified but are attenuated as computations proceed. To analyze for stability, we consider the homogeneous part of the ODE

dy = ay dt
Euler's method yields

with solution

y (t ) = y0 e

at

Assume some small error exists in the initial condition y0 or in an early stage of the solution. Then we see that after n steps,

dyi yi +1 = yi + h = yi ayi h = yi (1 ah) dt

yn = y0 (1 ah)n
Engineering Computation

Bounded iff |(1-ah) | < 1, i.e., h < 2/a

Matlab demo
33

Stiff Differential Equations

Eulers method is known as an explicit method because the derivative is taken at the known point i. An alternative is to use an implicit approach, in which the derivative is evaluated at a future step i+1. The simplest method of this type is the backward Euler method that yields

dyi +1 yi yi +1 = yi + h = yi ayi +1h yi +1 = 1 + ah dt

Because 1/(1+ah) will remain bounded for any (positive) value of h, this method is said to be unconditionally stable. Implicit methods always entail more effort than explicit methods, especially for nonlinear equations or sets of ODEs, so the stability is gained at a price. Moreover, accuracy still governs the step size.
Engineering Computation 34

Systems of Stiff Differential Equations What to do?

Need to be aware of the potential for encountering stiff ODEs. Use special codes for solving stiff differential equations which are generally implicit multistep methods. For example, MATLAB has some methods specifically designed to solve stiff ODEs, e.g., ode23S.

Engineering Computation

35

Predictor-Corrector Multi-Step Methods for ODE's Utilize valuable information at previous points. For increased accuracy, use a predictor that has truncation of same order as corrector. In addition iterate corrector to minimize truncation error and improve stability. Non-Self Starting (NSS) Heun Method O (h3)-Predictor yi+1o = yi-1m + f(xi,yim) 2h Higher order predictor than the self-starting Heun. This is an open integration formula (midpoint).

(h3)-Corrector

j
i +1

m j1 f (x , y ) + f (x , y m i i i +1 i +1 ) = yi + h 2

Notes: 1. Corrector is applied iteratively for j=1, ..., m 2. yim is fixed (from previous step iterations) 3. This is a closed integration formula (Trapezoid)
Engineering Computation 36

Multi-step methods Two general schemes solving ODE's: 1. Newton-Cotes Formulas n 1 n +1 y h c f O(h ) open predictor + + k ik in k =0 yi+1 = n 1 y n +1 h c f O(h ) closed corrector + + k ik +1 in +1 k =0 with ck from Table 21.4, and ck from Table 21.2 of C&C.

2. Adams Formulas (Generally more stable) n 1 open predictor n +1 h k fi k + O(h ) Adams - Bashforth k =0 yi+1 = yi + n 1 closed corrector h f n +1 k i k +1 + O(h ) Adams - Moulton k =0
with k and k determined from Tables 26.1 and 26.2, respectively.
Engineering Computation 37

Popular Multi-Step Methods

1. Heun non-self starting Newton-Cotes with n=1 2. Milne's Newton-Cotes with n=3 (but use Hammings corrector for better stability properties) 3. 4th-Order Adams Predictor 4th-Order Adams-Bashforth Corrector 4th-Order Adams-Moulton
If predictor and corrector are of same order, we can obtain estimates of the truncation error during computation.

Engineering Computation

38

Improving Accuracy and Efficiency (use of modifiers) 1. provide criterion for step size adjustment (Adaptive Methods) 2. employ modifiers determined from error analysis

For Non-self-Starting (NSS) Heun: Final Corrector Modifier 1 m i0+1 E c = yi +1 y 5

m 0 y y m i +1 i +1 yim y i +1 +1 5

Predictor Modifier (excluding 1st step) 4 m 4 m 0 0 0 i i+1 y i+1 + yi y i0 E p = yi y y 5 5 0 i0 = unmodified y i+1 with y from previous step

m yim = unmodified yi+1 from previous step

Engineering Computation

39

General Predictor-Corrector Schemes Errors cited are local errors. Global errors order h smaller. Predictors: Euler:

i +1 = yi + h(xi, yi) + O(h2) y

Midpoint (same as Non-Self-Starting Heun): i +1 = yi-1 + 2h(xi, yi) + O(h3) y Adams-Bashforth 2nd-Order: i +1 = yi + h/2{3(xi, yi) (xi-1, yi-1)} + O(h3) y Adams-Bashforth 4th-Order:

i +1 y

= yi + h/24{55(xi, yi) 59(xi-1, yi-1) +37(xi-2, yi-2) 9(xi-3, yi-3) } + O(h5)

Hamming (and Milne's Method): i +1 = yi-3 + 4h/3{2(xi, yi) (xi-1, yi-1) +(xi-2, yi-2)} + O(h5) y
Engineering Computation 40

Correctors: i +1 refine value of yi+1 given a y more stable numerically no spurious solutions which go out of control Adams-Moulton 2nd-Order closed (Non-self starting Heun):
i +1 ) +(xi, yi)} + O(h3) yi+1 = yi + h/2{(xi+1, y

Adams-Moulton 4th-Order closed (relatively stable):

yi+1 = yi + h/24{9(xi+1,y i +1) 19(xi, yi) 5(xi-1, yi-1) + (xi-2, yi-2) } + O(h5)

Milne's (N.C. 3-point, Simpson's 1/3 Rule, not always stable!!!):

i +1) + 4(xi, yi) +(xi-1, yi-1)} + O(h5) yi+1 = yi-1 + h/3{(xi+1,y

Hamming (modified Milne, stable):

yi+1 = 9/8 yi 1/8 yi-2 + 3h/8{(xi+1,y i +1 ) + 2(xi, yi) +(xi-1, yi-1)} + O(h5)

Engineering Computation

41

Predictor-Corrector solution Example: Undamped, harmonically driven oscillator

m m = mass k = spring constant

d2x dt
2

+ kx = P(t) k = = circular frequency m 2 T= = natural period dx (0) = 0 dt

P m( )
2 2

P(t) = forcing function = P sin( t) = driving frequency Initial conditions: Analytical solution:
x(t) =
Engineering Computation

x(0) = 0 and
P

m( )

sin(t) +

sin( t)
42

Predictor-Corrector solution: Undamped, harmonic oscillator

d2x dt
2

+ kx = P(t)

Recast 2nd-order ODE as two 1st-order ODE's: dx = v = f (t, x, v) with x(0) = 0 dt dv P sin( t) = 2 x = g(t, x, v) dt m Let m = 0.25, k = 42, P = 20, = 8, thus:
g = 80 sin(8t) - 162x

with v(0) = 0

Use 4th-ord. Adams Method to solve from t=0 until t=1 w/ h=0.1

Engineering Computation

43

Predictor-Corrector solution: Undamped, harmonic oscillator

d2x dt
2

+ kx = P(t)

First, we need start-up values. Use 4th-order RK to get 4 points:

t 0.0 0.1 0.2 0.3 x(t) 0.0000 0.1038 0.5324 0.8692 y(t) 0.0000 2.6234 5.1401 0.4949

Engineering Computation

44

Predictor-Corrector solution: Undamped, harmonic oscillator Second, the Adams-Bashforth 4th-order predictor is:
yi +1 = yi + h {55f (x i , yi ) 59 f (x i 1 , yi 1 ) + 37 f (x i 2 , yi 2 ) 9 f (x i 3 , yi 3 )} + O(h 5 ) 24

Example :

We apply this predictor to both x and v: x(0.4) = x(0.3) + [55(0.4949) - 59(5.1401) + 37(2.6234) - 9(0.0)] = = 0.1235 v(0.4) = v(0.3) + (0.1/24) [55 g(0.3, 0.8692, 0.4949) 59 g(0.2, 0.5324, 5.1401) + 37 g(0.1, 0.1038, 2.6234) 9 g(0,0,0)] = 11.2492 where dv/dt = g(t, x, v) = 80 sin 8t 162 x
Engineering Computation 45

Predictor-Corrector solution: Undamped, harmonic oscillator Third, the Adams-Moulton 4th-order corrector is:
yi +1 = yi + h {9 f (x i +1 , yi +1 ) + 19 f (x i , yi ) 5f (x i 1 , yi 1 ) + f (x i 2 , yi 2 )} + O(h 5 ) 24

We apply the corrector to both x and v to obtain the 1st iteration: x(0.4) = x(0.3) + (0.1/24) [9 (-11.2492) + 19(0.4949) 5(5.1401) + (2.6234)] = 0.8169 v(0.4) = v(0.3) + (0.1/24) [9 g(0.4, 0.1235, -11.2492) + 19 g(0.3,...) ...] We then can iterate until convergence: x(0.4) = 0.8169, 0.842862, 0.843847, 0.843834,... v(0.4) = -6.7435, -10.84973, -11.00371, -11.00949,...
Do the Second and Third tasks for each successive step.
Engineering Computation 46

= -6.7435

where dv/dt = g(t, x, v) = 80 sin 8t 162 x

Numerical Methods for ODE's Advantages of Runge-Kutta [such as O(h4) formula] simple to apply self-starting (single-step) easy to change step size always stable (all h) can use matched pairs of different order to estimate local truncation error Advantages of Predictor-Correctors [such as Adams-Moulton Formula of O(h4)]

without iteration is twice as efficient as Runge-Kutta 4th-Order local truncation error is easily estimated from difference so we can adjust step sizes and apply modifiers For same step sizes, error terms are reasonably similar.
Engineering Computation 47

Summary

Single-step Euler Heun Mid-point RK4

Multi-step (predictor-corrector) Non-self-starting Heun Milne (Newton-Cotes n=3) Adams (Adams-Bashford, Adams-Moulton)

Stiffness - stability Adaptive methods local error estimates modifiers Half step RK Fehlberg

Engineering Computation

48

Outline of our Study of ODE's I. Single-Step Methods for I.V. Problems (C&C Ch. 25) a. Euler b. Heun and Improved Polygon c. General Runge-Kutta d. Adaptive step-size control II. Stiff ODEs (C&C 26.1) III. Multi-Step Methods for I.V. Problems (C&C 26.2) a. Non-Self-Starting Heun b. Newton-Cotes c. Adams IV. Boundary Value Problems (C&C Ch. 27.1 & 27.2) a. Solution Methods 1. Shooting Method 2. Finite Difference Method b. Eigenvalue Problems { [A] l [I ] } {x} = {0}
Engineering Computation 49

ODE's Boundary Value Problems Recall: Unique solution to an nth-order ODE requires n given conditions Conditions for a 2nd-order ODE (requires 2 given conditions): Initial value problem y(xo) = yo y'(xo) = y'o Boundary value problem y(xo) = yo y(xf) = yf Beam Equation:

Engineering Computation

50

Beam Equation Sailboat mast deflection problem

d4v dz
4

f (z) EI

(Euler-Bernoulli Law of Bending)

at the base of mast: v(0) = 0; v'(0) = 0; at the top of mast:

v''(L) = 0; v'''(L) = 0

where:

E = I = f(z) = v(x) = z = E I v'' = E I v''' =

Modulus of Elasticity (material property) Moment of Inertia (geometry of material) wind pressure at height z deflection (displacement) from vertical height moment shear (applied force)
51

Engineering Computation

ODE's Boundary Value Problem

(C&C 27.1)

d 2T = h (T Ta ) 2 dx
TL Ta TR

Solve for the steady-state temperature distribution in the rod For a given TL and TR < TL

Engineering Computation

52

ODE's Boundary Value Problem Shooting Method d2 y Solve = = f (x, y, y ') given y(xo) = yo and y(xf) = yf 2 dx

1. Convert 2nd-Order ODE to two 1st-Order ODE's dy dz =z = f (x, y, z) dx dx 2. Given initial condition, y(xo) = yo, estimate (best guess) the initial condition z(xo) = zo(1) 3. Solve ODE using the assumed initial values from x = xo to x = xf using stepping methods. Because we estimated the initial conditions for z(xo); will find y(xf) yf

Engineering Computation

53

ODE's Boundary Value Problem Shooting Method

4. Re-estimate initial condition z(xo) = zo(2), and again solve the assumed initial value ODE from x = xo to x = xf. Because we estimated the I.C. z(xo), y(xf) yf. 5. Interpolate or extrapolate to find the "correct" z(xo) = zo Given yf, (yf(1), z0(1) ), and (yf (2), z0(2)):
zo = zo
(1)

zo yf

(2) (2)

zo yf

(1) (1)

yf yf (1)

[This equation is not in C&C in general form]

If ODE is linear, zo is the correct solution If ODE in nonlinear, iterate until yf(h) yf
Engineering Computation 54

Boundary Value Problem: Classical Shooting Method Example: Single-degree-of-freedom, undamped, harmonically driven oscillator (see also C&C 28.4, p. 797)

ODE is 2nd order: m d2x dt 2 + kx = P(t) or d2x dt 2 + 2 x = P(t) m

k = = circular frequency m k = spring constant 2 P(t) = forcing function = P sin( t) T = = natural period Now, we are given boundary conditions: m = mass x(0) = 0 and x(1) = 0.5 (that is, the initial condition v = dx/dt at t = 0 is unknown, v0 )
Engineering Computation 55

Boundary Value Problem: Classical Shooting Method Use the Shooting Method by employing a 4th-order RK

d2x dt 2

+ 2 x =

P(t) m

Recast the 2nd-order ODE as two 1st-order ODEs: dx = v = f (t, x, v) dt dv P sin( t) = 2 x = g(t, x, v) dt m Let m = 0.25, k = 42, P = 20, = 8, and thus g(t, x, v) = 80 sin(8t) - 162x.
Engineering Computation 56

with x(0) = 0 with v(0) = v0

Boundary Value Problem: Classical Shooting Method

Use 4th-ord. RK Method to solve from t=0 until t=1 w/ h=0.1. 1. Guess v(0) = 1.0 2. Find x(1) = 0.91687 (Note: not equal to 0.5) 3. Guess v(0) = 100 4. Find x(1) = 0.09332 5. Because our ODE is linear, we can always interpolate yields the required initial value. 100 1.0 v(0) = (0.5 0.91687) = 51.113 0.09332 0.91687 6. With v(0) = 51.113, check for x(1) = 0.5000

OK

Engineering Computation

57

Finite Difference Method

The shooting method is inefficient for higher-order equations with several boundary conditions. Finite Difference method has advantage of being direct (not iterative) for linear problems, but requires the solution of simultaneous algebraic equations.
0 xo 1 2 3 ... i-1 i i+1 ... n
n+1

x = h

xf

Engineering Computation

58

Finite Difference Method

Approach: 1. Divide domain to obtain n interior discrete points (usually evenly spaced @ h). 2. Write a finite difference expression for the ODE at each interior point. 3. Use known values of y at x = xo and x = xf
4. Set up n linear equations with n unknowns. System is banded and often symmetric, so solve with an efficient method.

Note: If higher-order FDD equations and/or centered differences are used, you may need to employ imaginary or "phantom" points outside of domain to express B.C.'s (together with appropriate FD versions of B.C.'s).

Engineering Computation

59

Finite Difference Method

Sailboat mast deflection problem

d4v dz
4

f (z) EI

(Euler-Bernoulli Law of Bending)

at the base of mast: v(0) = 0; v'(0) = 0; at the top of mast:

where: E = I = f(z) = v(x) = z = E I v'' = E I v''' = v''(L) = 0; v'''(L) = 0

Modulus of Elasticity (material property) Moment of Inertia (geometry of material) wind pressure at height z deflection (displacement) from vertical height moment shear (applied force)
60

Engineering Computation

Finite Difference Method

f (z) = v = Sailboat mast deflection problem: 4 EI dz

at the base (z = 0): v(0) = 0; v'(0) = 0; at the top (z = L):

d4 v

v''(L) = 0; v'''(L) = 0

v '''' =

d4v dx
4

vi 2 4vi 1 + 6vi 4vi+1 + vi+ 2 h 2h 3

4

[dist. load,f(z)]/EI (shear)/EI (bending moment)/EI (slope)

61

v ''' =

d3v dx 3 d2v dx 2

vi 2 + 2vi 1 2vi +1 + vi + 2 vi 1 2vi + vi+1 h2

v '' =

dv vi1 vi+1 v' = dx 2h

Engineering Computation

Finite Difference Method

Sailboat mast deflection problem:

at the base: v(0) = 0; v'(0) = 0; at the top: Imaginary Node

d4v

f (z) = 4 EI dz

v''(L) = 0; v'''(L) = 0

Mast with 11 Nodes 3 4 5 6 7 8

9 10

Imaginary Nodes

Use equation:

d 4v dz 4

f (z) EI Use v''(L) = 0 to get in terms of v(9) & (10) Use v'''(L) = 0 to get in terms of other v's

Use v'(0) to get in terms of v(1)

v(0) given

Engineering Computation

62

Finite Difference Method

@ pt. i:

f (z) = 4 EI dz vi-2 4 vi-1 + 6 vi 4 vi+1 + vi+2 = h4 f(xi)/EI

d4v

Using h = L/10, write the FD equations at points 1, 2, 9 and 10: @i=1 @i=2 @i=9 @ i = 10 v-1 4 v0 + 6 v1 4 v2 + v3 = h4 f(x1)/EI v0 4 v1 + 6 v2 4 v3 + v4 = h4 f(x2)/EI v7 4 v8 + 6 v9 4 v10 + v11 = h4 f(x9)/EI v8 4 v9 + 6 v10 4 v11 + v12 = h4 f(x10)/EI We now need to eliminate v1, v0, v11, and v12 with the four boundary conditions
Engineering Computation 63

(1) (2) (3) (4)

Finite Difference Method

d4v dz 4

f (z) EI

Eliminating v1, v0, v11, and v12 with the four Boundary conditions v0 = 0
v 1 v1 =0 2h
v9 2v10 + v11 h2 =0

v(0) = 0: v'(0) = 0: v''(L) = 0: v'''(L) = 0:

v-1 = v1 v11 = 2v10 + v9

=0

v8 + 2v9 2v11 + v12 2h 3

v12 = v8 - 4v9 + 4v10

64

Engineering Computation

Finite Difference Method

d4v dz 4

f (z) EI

We substitute these into equations (1) to (4) to eliminate the unknowns at the imaginary points v-1 = v1 v12 = v8 - 4v9 + 4v10 (1a) (2a) (3a) (4a)

v0 = 0 v11 = 2v10 + v9 @i=1 @i=2 @i=9 @ i = 10

7 v1 4 v2 + v3 = h4f(x1) / EI 4 v1 + 6 v2 4 v3 + v4 = h4 f(x2)/ EI v7 4 v8 + 5 v9 2 v10 = h4f(x9) / EI v8 4 v9 + 2 v10 = h4f (x10)/ EI

Engineering Computation

65

Finite Difference Method

7 -4 1 v1 4 6 -4 1 v 2 1 4 6 4 1 v3 v 1 -4 6 -4 1 4 4 1 -4 6 -4 1 v5 h f = 1 -4 6 -4 1 v6 EI v7 1 -4 6 -4 1 1 -4 6 -4 1 v8 1 -4 5 -2 v9 1 -4 2 v10

f (x1 ) f (x ) 2 f (x 3 ) f (x 4 ) f (x 5 ) f (x ) 6 f (x 7 ) f (x8 ) f (x 9 ) f (x10 )

Engineering Computation

66

Finite Difference Method

Once we have solved for all the vi, we can obtain secondary results such as bending moments and shear forces by substituting the finite-divided-difference operators and the values of the vi into such equations as: M = EI v'' V = EI v''' For more refined results, we can use a smaller h and more segments.

Engineering Computation

67

Merits of Different Numerical Methods for ODE Boundary Value Problems

Shooting method Conceptually simple and easy. Inefficient for higher-order systems w/ many boundary conditions. May not converge for nonlinear problems. Can blow up for bad guess of initial conditions. Finite Difference method Stable Direct (not iterative) for linear problems. Requires solution of simultaneous algebraic eqns. More complex.
FD better suited for eigenvalue problems.
Engineering Computation 68

Engineering Applications of Eigenvalue Problems

Natural periods and modes of vibration of structures and dynamical systems. (C&C Ex. 27.4, p. 761) Application: determination of the natural frequencies of a system Example: A mass-spring system with three identical (frictionless) masses connected by three springs with different spring constants.
3k
M x1

2k
M

x2

k
M

x3

The displacement of each spring is measured relative to its own local coordinate system with an origin at the spring's equilibrium position. d 2 x1 m 2 = 3kx1 2k(x1 x 2 ) dt d2x2 m 2 = 2k(x 2 x1 ) k(x 2 x 3 ) dt d2 x3 m 2 = k(x 3 x 2 )
Engineering Computation

dt

69

Engineering Applications of Eigenvalue Problems

If one assumes that x has the form: xi = ai cos t ; d2x/dt2 = - 2 ai cos t

Then, with = m 2 / k , the governing equations become: m2 a1 = 5a1 + 2a 2 k m2 a 2 = 2a1 3a 2 + a 3 k m2 a3 = a 2 a3 k ==> 5a1 2a 2 = a1

==> 2a1 + 3a 2 a 3 = a 2 ==> a 2 + a 3 = a 3

Engineering Computation

70

Engineering Applications of Eigenvalue Problems

5a1 2a 2 = a1 2a1 + 32a 2 a 3 = a 2 a1 + a 2 = a 3

5 2 0 a1 a1 ==> 2 3 1 a 2 = a 2 0 1 1 a 3 a 3 [A] {x} = {x}

or

0 a1 5 2 2 3 1 a2 = 0 0 a 1 1 3

[A I ]

{x} = 0

Engineering Computation

71

Engineering Applications of Eigenvalue Problems

For a non-trivial solution: det [A I ] = 0 find

0 5 2 det 2 3 1 = 0 0 1 1

The determinant yields a cubic equation for : (5 ) [(3 )(1 ) (1)(1)] 2 [(1)(0) (2)(1 )] = 0 (5 ) [2 4 + 2] 2 [ 2 2 ] = 0 6 18 + 9 2 3 = 0 The three solutions of the cubic equation are the three eigenvalues: 1 = 6.29 Fast oscillation
Engineering Computation

2 = 2.29

3 = 0.42 Slow oscillation

72

Engineering Applications of Eigenvalue Problems

The corresponding eigenvectors {ai} are found by solving: 2 0 a1i 5 i 2 3 i 1 a 2i = 0 0 a 1 1 i 3i

Because this is a homogeneous equation, we can only find the relative values of the ai's. For 1 = 6.29

2 0 a1i 5 6.29 2 3 6.29 1 a 2i = 0 0 a 1 1 6.29 3i a2 = 0.645 a1 a3 = 0.122 a1

Engineering Computation

73

Engineering Applications of Eigenvalue Problems

a2 = 0.645 a1

a3 = 0.122 a1

Only the relative values of the ai's are significant. Thus setting a1 = 1.00 we have:

Fast for = 6.29 1.000 {a}1 = 0.645 0.122

for 2 = 2.29

Slow
for 2 = 0.42

{a}2

0.74 = 1.000 0.77

{a}2

0.25 = 0.55 1.000

Engineering Computation

74

Engineering Applications of Eigenvalue Problems

Thus, there are 3 possible natural frequencies. k Recall that the frequency is defined as = m [A] {x} = {x} [A I ] {x} = 0 det [A I ] = 0 ==> find If [A] is n x n, there are n eigenvalues i and n eigenvectors {x}i If [A] is symmetric, the eigenvectors are orthogonal: {x}iT {x}j = 0 = 1
Engineering Computation

if j i if j = i
75

POWER METHOD for Eigenvector Analysis (An iterative approach for solving for eigenvalues & eigenvectors)

Assume that a vector {y} can be expressed as a linear combination of the eigenvectors: {y} = b1{x1} + b 2{x 2 } + . . . + b n {x n } = Multiplying the above equation by [A] yields: [A] {y} =

b {x }
i i i =1 i i

b [A]{x } = b {x }
i i i i =1 i =1

Ordering the i so that

| 1| > | 2| > | 3| > . . . > | n|

Then [A] {y} = 1 b1 {x1} +

Engineering Computation

i i bi {x i } where <1 1 1 i=2

n
76

POWER METHOD for Eigenvector Analysis Multiplying the equation by [A] again, we have: 2 n i 2 2 [A] {y} = 1 b1 {x1} + bi { x i } 1 i 2 = Repeating this process m times: m n [A]m {y} = 1m b1 {x1} + bi i {x i } 1 i =2 i Since ==> 0 for all i 1 as m ==> , 1 then [A]m {y} 1m b1{x1}

Since we can only know the relative values of the elements of {x}, we may normalize {x}. If {x} is normalized such that the largest element is equal to {1}, then is the first eigenvalue, 1
Engineering Computation 77

POWER METHOD Numerical Example

3 7 9 x1 x1 9 4 3 x = x 2 2 x x 9 3 8 3 3
Find the maximum eigenvector. Initial guess: {y}T = {1 1 1}

3 7 9 1 19 [A]{y} = 9 4 3 1 = 16 ==> 20 1 20 9 3 8
Approx. eigenvalue

0.95 0.80 1.00

eigenvector

Engineering Computation

78

POWER METHOD Numerical Example

2nd iteration

3 7 [A]2 {y} = 9 4 9 3 Approximate error, a: (ei, j ei1, j max *100% = ei, j j

9 0.95 0.92084 3 0.80 = 18.95 0.77836 1.00 8 1.00

0.92084 0.95 *100% = 4.2% 0.92084

3 7 9 0.92084 0.92420 3rd iteration 3 [A] {y} = 9 4 3 0.77836 = 18.623 0.77331 1.00 1.00 9 3 8
0.92084 0.95 a = *100% = 0.75% 0.92084
Engineering Computation 79

POWER METHOD Numerical Example

After 10 iterations:

= 18.622

0.92251 {x} = 0.77301 1.00

a = 0.00033%

Engineering Computation

80

Engineering Applications of Eigenvalue Problems

1. Natural periods & modes of vibration of dynamic systems and structures. Boundary-value problem from separation of variables: y(x, t) 2y m a = force m(x) = k(x) 2 2 t x If one assumes y(x,t) = exp(it) u(x) then displacement function u(x) satisfies ODE: 2 u(x) = [k(x)/m(x)] d2u/dx2 which may be written: a(x) d2u/dx2 + 2 u(x) = 0 With a FD approximation of u,xx = d2u/dx2 this becomes:
[A] {ui} = 2 {ui}
Engineering Computation 81

Engineering Applications of Eigenvalue Problems

2. Buckling loads and modes of structures (C&C 27.2.3, p. 762) Deflection of vertically loaded beam with horizontally constrained end: EI y'' Py = 0; y(0) = 0; y(L) = 0. Is there deflection? 3. Directions and values of principal stresses. 4. Directions and values of principal moments of inertia. 5. Condition numbers linear systems of equations. 6. Stability criteria for numerical solution of PDEs. 7. Other problems in which "principal values" are sought.

Engineering Computation

82

Engineering Applications of Eigenvalue Problems

Matrix Form of Eigenvalue Problem

[A] {x} = {x}

( [A] [I] ) {x} = 0

but want {x} 0

Computation of Eigenvalues and Eigenvectors

1. Power and Inverse Power method for largest and smallest eigenvalues. (C&C 27.2.5, p. 767) 2. Direct numerical algorithms: Jacobi, Given, Householder, and QR factorization. (C&C 27.2.6, p. 770) 3. Use of characteristic polynomial for toy problems. (C&C 27.2.4, p. 765)
Engineering Computation 83

Engineering Applications of Eigenvalue Problems

Power Method for the largest Eigenvalue

Compute xt+1 = A xt / || xt || for larger and larger t to estimate largest (A). Always works for Symmetric A.

Engineering Computation

84

Eigenvalue Problems: Power Method

Inverse Power Method for the smallest Eigenvalue

Compute xt+1 = B xt / || xt || for larger and larger t to estimate largest (B). Here B = A-1. Eigenvalues of B are inverse of those of A. BUT WE DO NOT COMPUTE A-1 ! (Dont do it.) Instead use LU decomposition of A.

Engineering Computation

85

Engineering Applications of Eigenvalue Problems

Shifting method for any Eigenvalue:

Suppose we want to compute eignvector whose eigenvalue is about . Let B = (A I)-1 and compute xt+1 = B xt / || xt || for larger and larger t to estimate largest [ (A I) -1]. This will compute the eignvalue nearest to !
But we DO NOT COMPUTE INVERSE. Use LU decomposition.

If A has eigenvector-value pairs (ei, i),

(A I) ei = A ei I eI = i ei ei = ( i ) ei

Thus shifted matrix has same eignvectors ei as A with shifted eigenvalues ( i ). Hence (A I)-1 has eigenvalues ( i )-1.
Engineering Computation 86