MATH3800 Notes-Eric Hua

Roots of Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Modeling Change with Difference Equation . . . . . . . . . . . . . . 8
1.2 Approximating Change with Difference Equation . . . . . . . . . . 10
1.3 Solutions to Dynamical Systems . . . . . . . . . . . . . . . . . . . . 13
1.4 Systems of Difference Equations . . . . . . . . . . . . . . . . . . . . 16
2.2 Modelling Using Proportionality . . . . . . . . . . . . . . . . . . . . 21
3.1 Fitting Models to Data Graphically . . . . . . . . . . . . . . . . . . 23
3.2 Analytic Methods of Model Fitting . . . . . . . . . . . . . . . . . . 25
3.3 Applying the Least-Squares Criterion . . . . . . . . . . . . . . . . . 26
4.1 One-Term Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.3 Smoothing: Low-Order Polynomial Models . . . . . . . . . . . . . 38
4.4 Cubic Spline Models . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Numerical Differentiation and Integration . . . . . . . . . . . . . . . . 48
Gaussian Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2 Generating Random Numbers . . . . . . . . . . . . . . . . . . . . . 57
5.3 Simulating Probabilistic Behaviour . . . . . . . . . . . . . . . . . . 58
Traffic Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Simulating Blackjack with MATLAB . . . . . . . . . . . . . . . . . . . 60
6.2 Modelling Component and System Reliability . . . . . . . . . . . . 61
6.1 Probabilistic Modeling with Discrete Systems . . . . . . . . . . . . 64
Matrix Computations . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
11.1 Population Growth . . . . . . . . . . . . . . . . . . . . . . . . . . 75
11.4 Graphical Solutions of Autonomous Differential Equations . . . . 80
12.1 Graphical Solutions of Autonomous Systems of First-Order Differ-
ential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
12.2 A Competitive Hunter Model . . . . . . . . . . . . . . . . . . . . 85
12.3 A Predator-Prey Model . . . . . . . . . . . . . . . . . . . . . . . 90
11.5 Numerical Approximation Methods . . . . . . . . . . . . . . . . . 93
2 MATH3800 NOTES-ERIC HUA

MATH3800 Notes - By Eric Hua

MATH3800 NOTES-ERIC HUA 3

Roots of Equations
Numerical methods: are mathematical methods that are used to approximate
the solution of complicated problems. Numerical methods are very useful be-
cause they are suitable for the use with computers.

Finding roots: Solve f (x) = 0 for x, when an explicit analytical solution is

impossible.

• Bisection Method.

• Fixed-point Iteration.

• Newton’s Method.

Bisection Method: To solve f (x) = 0 for x, we want to construct a sequence

x0 , x1 , x2 ,..., which converges to the root.
1. Chose two initial values a and b such that f (a)f (b) < 0.
a+b
2. x0 = .
2
3. x1 = a+x 2
0
if f (a)f (x0 ) < 0, or x1 = x02+b if f (b)f (x0 ) < 0.
4. And so on.

Example 1. Approximate the root of the equation f (x) = x3 − x + 1 = 0 by

bisection method.

Solution: f (−2) = −5 and f (0) = 1.

f (−2) = −5 and f (−1) = 1.
f (−1.5) = −0.875 and f (−1) = 1.
f (−1.25) = ....
4 MATH3800 NOTES-ERIC HUA

Fixed point method: If

g(a) = a,

then a is called a fixed point of the function g(x). An fixed point a is called
stable if the solutions that start near a stay near or approach a; if the solutions
that start near a moves away from it, then a is unstable.

Theorem 1. (Stability Theorem). Let a be a fixed point of g(x).

1. If |g 0 (a)| < 1, then a is stable;

2. If |g 0 (a)| > 1, then a is unstable.

Example 2. Find zeros of f (x) = x3 + 5x − 5 by fixed-point method.

To find zeros of f (x) = x3 + 5x − 5, i.e.,

5 − x3
x3 + 5x − 5 = 0, x = .
5

Let
5 − x3
g(x) = ,
5

then any fixed point of g(x), i.e., x = g(x), will be a zero of f (x).
Note that f (0.5) < 0, f (1) > 0, we start from x0 = 0.75 to approximate the
fixed point of g(x). Since

3
g 0 (x) = − x2 , |g 0 (x)| < 1 f or x ∈ (0.5, 1),
5

The recursive relation

5 − x3n
xn+1 = g(xn ), xn+1 =
5

will converge to the fixed point.

MATH3800 NOTES-ERIC HUA 5

%% Matlab m-file for fixed point iteration to find the root of f(x)=x^3-5x-5 in (0,1)
%% Set initial guess
x(1)=0.75;
for n=1:17
z(n+1)=(5-z(n)^3)/5;
x(n)=z(n+1);
end
plot(x,”+r”);
xlabel(‘Iteration n’);
ylabel(‘x(n)’);

n x(n)
0 0.75
1 0.915625
2 0.846473651
3 0.878697339
4 0.864309973
5 0.870866605
6 0.867905448
7 0.869248332
8 0.868640468
9 0.868915854
10 0.868791141
11 0.868847629
12 0.868822045
13 0.868833633
14 0.868828384
15 0.868830761
16 0.868829685
17 0.868830172

x(n)
1
0.8
0.6
0.4
0.2
0
0 2 4 6 8 10 12 14 16 18
6 MATH3800 NOTES-ERIC HUA

Newton’s method: To approximate solutions of the equation f (x) = 0, start

from x0 , we have approximate solutions

f (xn )
xn+1 = xn −
f 0 (xn )

provided we have started with a good value for x0 , this will produce approximate
solutions to any degree of accuracy.

The following graph shows that with bad initial value, Newton’s method fails:
This is likely the case when f 0 (x0 ) is close to 0, we can see that x2 is even outside
the domain.

√
6
Example 3. Find the roots 2 by Newton’s method, correct to 8 decimals.
√
Solution: Let f (x) = x6 − 2. Then 6 2 is a solution of the equation f (x) = 0.
Note that f (2) = 62 > 0 and f (0) = −2 < 0 . This tells us that the root is
between 0 and 2. So we chose x0 = 1 for our initial guess.

x6n − 2 5x6n + 2
xn+1 = xn − = .
6x5n 6x5n
With our initial guess of x0 = 1, we can produce the following values:
MATH3800 NOTES-ERIC HUA 7

x0 1
x1 1.16666667
x2 1.12644368
x3 1.12249707
x4 1.12246205
x5 1.12246205

Example 4. Approximate the root of the polynomial f (x) = x3 − x + 1 = 0 by

Newton’s method with x0 = −1, correct to 6 decimals.

Solution: Note that f (−2) = −5 and f (0) = 1. This tells us that the root is
between -2 and 0. So we chose x0 = −1 for our initial guess.

x3n − xn + 1 2x3n − 1
xn+1 = xn − = .
3x2n − 1 3x2n − 1
With our initial guess of x0 = −1 , we can produce the following values:

x0 -1
x1 -1.500000
x2 -1.347826
x3 -1.325200
x4 -1.324718
x5 -1.324717
x6 -1.324717
x7 -1.324717

Notice how the values for xn become closer and closer to the same value. This
means that we have found the approximate solution to six decimal places. In
fact, this was obtained after only five relatively painless steps.
8 MATH3800 NOTES-ERIC HUA

Mathematical Modelling

• Mathematical model: mathematical representations of the behavior of real

devices or objects.

• By mathematical model, we can formulate and solve problems.

• How:
Simplif ication /
Real world data Model
O

V erif ication Analysis


Predictions/explanations o Mathematical conclusions
Interpretation

• Some check lists:

– Identify the need for the model.

– List the data we are seeking.
– Identify the available relevant data.
– Identify the circumstances.
– How should we look at this model? Identify the governing principles.
– Identify the equations that will be used, the calculations that will be
made, and the answers that will result.
– Validate the model: consistent with its principles and assumptions?
– Verify the model: useful to initial problem?

1.1 Modeling Change with Difference Equation

Definition 1. Two variables x and y are proportional if

y = kx

for some nonzero constant k. We write y ∝ x, y is proportional to x.

• Change = future value - present value.

• A sequence is a function, domain={all non-negative integers}, range={subset

of real numbers}.

• A dynamical system (DTDS) is a relationship among terms in a sequence.

A numerical solution is a table of values satisfying the dynamical system.
MATH3800 NOTES-ERIC HUA 9

• Consider a sequence of numbers A = {a0 , a1 , a2 , ...}.

∆a0 = a1 − a0 , ∆a1 = a2 − a1 , ..., ∆an = an+1 − an , ...

are called the first differences, and ∆an is called the nth first difference.

Example 5. You deposit $ 100 into a bank account, that pays 1% interest an-
nually. Model the balance after each year.

Solution: Let an be the balance after n-th year. Then we have the following
the sequence
A = {100, 101, 102.01, 103.0301, ....}.
The first differences are:

1, 1.01, 1.0201, ..., ∆an = 0.01an .

The general model is given by the following difference equation:

an+1 = an + 0.01an , i.e., an+1 = 1.01an .

The solution of the difference equation is

an = 1.01n a0 .

Remark. The balance an is proportional to 1.01n , i.e., an ∝ 1.01n .

10 MATH3800 NOTES-ERIC HUA

1.2 Approximating Change with Difference Equa-

tion
• Discrete-time Dynamical System (DTDS): The depositing of interest into
a bank account.

• Continuous Dynamical System: Change of temperature of a cup of coffee.

Example 6. (Yeast culture 1): Here we have data for yeast culture. From dot
plot, we can approximate the change in population as linear relation to population
size. Find the model.

The solution pn can be modeled by exponential function.

Example 7. (Yeast culture 2): With more data, the model may be changed.
MATH3800 NOTES-ERIC HUA 11

The solution pn :

∆pn = −0.0008p2n + 0.5161pn + 6.4112,

pn+1 = pn − 0.0008p2n + 0.5161pn + 6.4112,

pn+1 = −0.0008p2n + 1.5161pn + 6.4112.

Numerical solutions from the model:

12 MATH3800 NOTES-ERIC HUA

Another method is given by the text book, page 12.

MATH3800 NOTES-ERIC HUA 13

1.3 Solutions to Dynamical Systems

The Method of Conjecture

1. Observe a pattern.

2. Conjecture the solution of the dynamical system.

3. Test the conjecture by substitution.

4. Conclusion.

Example 8. Consider the dynamical system

an+1 = 1.03an ,

which can model account balance with annual interest rate 3%. Predict the solu-
tion.

Solution: If the initial is $100, then the dynamical system is:

100, 1.03(100), 1.03 ∗ 1.03(100), ...

Conjecture: an = 1.03n a0 .
Test: an+1 = 1.03n+1 a0 = 1.03 ∗ 1.03n a0 = 1.03an , same as the dynamical
system.
Conclusion: an = 1.03n a0 .

Example 9. Find the general solution of the basic exponential discrete-time

dynamical system: bn+1 = rbn .

Solution:
bn = r n b0 .

Equilibrium (or fixed point):

Definition 2. A point a is called an equilibrium (or fixed point) of the discrete-

time dynamical system
an+1 = f (an )
if
f (a) = a,
where f (x) is called the updating function of the DTDS.
14 MATH3800 NOTES-ERIC HUA

Remark. At any equilibrium, f (x) neither increases nor decreases, remains the
same.

Graphic approach: The intersections of the updating function and the line y = x.

Stability of equilibrium: Consider the DTDS

an+1 = f (an )

with an equilibrium a.

1. If |f 0 (a)| < 1, then a is stable;

2. If |f 0 (a)| > 1, then a is unstable;

3. If |f 0 (a)| = 1, then use Cobwebbing method to classify.

Example 10. an+1 = 4a3n . Find and classify all equilibria.

Solution: Note that f (x) = 4x3 .

Step 1: Construct the equation a = 4(a)3 ;
Step 2: Solve the equation, we obtain 0, 1/2, −1/2.

Step 3: f (x) = 4x3 . f 0 (x) = 12x2 . Thus f 0 (0) = 0, |f 0 (± 21 )| = 3 > 1.

By Stability Thm, it is stable at 0, unstable at 1/2, −1/2.

Linear dynamical systems

Theorem 2. Consider the dynamical system of the form

an+1 = ran + b,

where r and b are constants.

• If r = 1: the solution is
an = a0 + nb.

• If r 6= 1: the solution is

b
an = rn (a0 − a) + a, a = ,
1−r
where a is the equilibrium.

– If |r| > 1: a is unstable.

– If |r| < 1: a is stable.
MATH3800 NOTES-ERIC HUA 15

Solution:
a1 = ra0 + b,
a2 = ra1 + b = r(ra0 + b) + b = r2 a0 + (r + 1)b,
a3 = r3 a0 + (r2 + r + 1)b,
(1 − rn )b
an = rn a0 + (rn−1 + · · · + r2 + r + 1)b = rn a0 + = rn a0 + (1 − rn )a =
1−r
rn (a0 − a) + a.
Note that f (x) = rx + b. Set

b
a = f (a), a = ra + b, a = .
1−r
Since f 0 (a) = r,

• if |r| > 1: |f 0 (a)| > 1, a is unstable.

• If |r| < 1: |f 0 (a)| < 1, a is stable.

Example 11. By ct we denote the amount (in mg) of caffeine at time t (in
hours). On average, our body eliminates 13% per hour. Assume that at the end
of the same time interval we consume 94.8 extra mg of caffeine.

• Construct the model.

• Solve the model.

• What is the long-term behaviour?

Solution:
1. The model will be:
ct+1 = 0.87ct + 94.8.
2. The solution is
94.8 94.8
ct = 0.87t (c0 − )+ .
0.13 0.13
3. The equilibrium is
b 94.8
a= = =729.
˙
1−r 0.13
Since |r| = 0.87 < 1, a is stable, which is long-term behaviour.
16 MATH3800 NOTES-ERIC HUA

1.4 Systems of Difference Equations

Example 12. MATH3800 Tutoring Ottawa has two branches in Ottawa: Andrew
Park, and Billings Bridge, with market portion a0 and b0 respectively. Accord-
ing to market statistics, due to variety of tutors, after each year, 40% of AP’s
customers will switch to BB’s, while 30% of BB’s customers will switch to AP’s.
(a) Construct the dynamical system model.
(b) Find the equilibrium values.
(c) Analyse the data. Predict the long term market share of the two branches.

Solution:
(a) Let an and bn be the number of customers at the branch Andrew Park
and the branch Billings Bridge respectively, at the end of year n. Then

an+1 = 0.6an + 0.3bn

bn+1 = 0.4an + 0.7bn

(b) Set up system:

a = 0.6a + 0.3b
b = 0.4a + 0.7b

a = b = 0, and a = 3000, b = 4000.

(c) We will show that the equilibrium values a = 3000, b = 4000 are stable
later. The system is insensitive to initial values. We predict that 3/7 customers
will end up in Andre Park, 4/7 in BB.

Dynamics in a Discrete Competition System. Consider fish population

oscillations in Ontario Rideau Lake. Let an be the population of pike fish at the
end of year n, and Let bn be the population of bass fish at the end of year n.
The model is based on the following assumptions:
MATH3800 NOTES-ERIC HUA 17

1. In the absence of other species, each individual species will grow uncon-
strained, and the population change is proportional to the population size.

2. The effect of the presence of the second species is to diminish the growth
of the other species. The decrease is proportional to the product of both
populations.

The Model:

an+1 − an = k1 an − k3 an bn
bn+1 − bn = k2 bn − k4 an bn ,

or

an+1 = (1 + k1 )an − k3 an bn
bn+1 = (1 + k2 )bn − k4 an bn ,

where k1 , k2 , k3 , k4 are positive constants.

• (1 + k1 ) is pike population growth rate if no bass fish.

• (1 + k2 ) is bass population growth rate if no pike fish.

• k3 : the pike population decrease rate by the presence of bass fish.

• k4 : the bass population decrease rate by the presence of pike fish.

18 MATH3800 NOTES-ERIC HUA

Example 13. Consider the system

an+1 = 1.2an − 0.001an bn

bn+1 = 1.3bn − 0.002an bn ,

Find the equilibrium values and analyse sensitivity to initial values. When initial
values are a0 = b0 = 10, who will win?

Solution:

a = 1.2a − 0.001ab
b = 1.3b − 0.002ab,

(a, b) = (0, 0), or (150, 200).

Dynamics of a discrete-time predator-prey system


xn+1 = axn − bxn yn ,
yn+1 = cyn + dxn yn ,
MATH3800 NOTES-ERIC HUA 19

where xn represents prey population densities at time n, and yn represents preda-

tor population densities at time n, and a, b, c, d are positive parameters.

• Here a > 1 represents the natural growth rate of the prey in the absence
of predators.

• b represents the consumption rate of the predators (where bxn is the average
number of prey eaten per predator in time n), i.e., the decrease rate of the
prey by the presence of predator.

• 0 < c < 1 is the survival rate of the predator (decrease rate) in the absence
of its prey source.

• d > 0 is the growth rate of the predator population due to the consumption
of prey.

The amount of prey consumed by the predator at each time step depends on not
only on the numbers of predators present, but also the numbers of prey present.
When the prey density is higher, it is easier for the predators to find prey and thus
more prey will be consumed. Likewise, the growth rate the predators depends
not only on the number of predators present (more predators should produce
more offspring overall), but also on the number of prey present (a large food
source often correlates with higher reproductive rates and/or larger litter or
clutch sizes).
The system has two fixed point (0, 0), and ( 1−c d
, a−1
b
).

Example 14. Consider the system

an+1 = 1.2an − 0.001an bn

bn+1 = 0.6bn + 0.002an bn ,

Find the equilibrium values and analyse sensitivity to initial values. When initial
values are a0 = b0 = 10, or a0 = b0 = 500. Compare results.

Solution:

a = 1.2a − 0.001ab
b = 0.6b + 0.002ab,

(a, b) = (0, 0), or (200, 200).

20 MATH3800 NOTES-ERIC HUA
MATH3800 NOTES-ERIC HUA 21

2.2 Modelling Using Proportionality

Kepler’s third law - shows the relationship between the period (days) of a
planet and mean distance to the sun (can be used for anything naturally orbiting
around any other thing). Formula:

T 2 = kR3 , or, T = cR3/2 ,

where:
T = period of the orbit, measured in days.
R = average distance of the object, measured in units of distance.
k, c = constants.

Example 15. T ∝ R3/2 , i.e., T = cR3/2 . Estimate c value.

Solution:
90466.8 − 0
c=
˙ = 0.41.
220869.1115 − 0

Example 16. The following is the data for Ontario muskie fish:
22 MATH3800 NOTES-ERIC HUA

(a) Is weight ∝ length?

(b) Is weight ∝ length3.325 ?

(a) No.

(b) Yes. W = 0.000089L3.325 .

MATH3800 NOTES-ERIC HUA 23

3.1 Fitting Models to Data Graphically

When we analyse data,

• Fitting a selected model type to the data.

• Chose the most appropriate model.

• Making predictions from data.

1. Visual model fitting with the original data

Example 17. (The best-fitting line) A waiter at a restaurant received tips ac-
cording to his working hours. The data are:

Model the data with the best-fitting line such that the sum of absolute
deviations is minimum.

2. Transforming the data

Sometimes we may not be easy to predict model visually. In this case, we

can try to transform data. Such as

• Apply log function to data.

• Apply exponential function to data.

• Apply power function to data, e.g., muskie fish length to weight.

24 MATH3800 NOTES-ERIC HUA
MATH3800 NOTES-ERIC HUA 25

3.2 Analytic Methods of Model Fitting

LEAST SQUARES CRITERION: Fitting the data set

{(x1 , y1 ), ..., (xm , ym )}

by the function y = f (x) such that

m
X
S= [yi − f (xi )]2
i=1

is minimum.

”Predictions from models are not always correct, the least squares criterion
proves minimized predictions based on this assumption.”
26 MATH3800 NOTES-ERIC HUA

3.3 Applying the Least-Squares Criterion

Here we apply the LS Criterion to estimate the parameters for several types of
curves.

1. Fitting a Straight Line (with the section 6.3)

Theorem 3. Fitting the data set

{(x1 , y1 ), ..., (xm , ym )}

by the function
y = f (x) = ax + b
such that m
X
S= [yi − f (xi )]2
i=1

is minimum. Then
Pm
xi y i − ( m xi )( m
P P
m i=1P i=1 i=1 yi )
a= ,
m m 2
Pm 2
i=1 ix − ( i=1 x i )
Pm
x2i )( i=1 yi ) − ( m xi yi )( m
P P P
( i=1 i=1 i=1 xi )
b= Pm 2 Pm 2
,
m i=1 xi − ( i=1 xi )
where a is called the slope, b is the intercept.

Solution: m
X
S= [yi − axi − b]2 .
i=1

To have optimality,
∂S ∂S
= 0, = 0.
∂a ∂b
We derive the system
m
X m
X
yi = a xi + mb
i=1 i=1
m
X m
X m
X
xi y i = a x2i +b xi .
i=1 i=1 i=1

By solving the system we get a and b.

How do we know where the model is good or bad?

The error sum of squares (SSE), which reflects variation about the regression
line, is given by
Xm
SSE = [yi − (axi + b)]2 .
i=1
MATH3800 NOTES-ERIC HUA 27

The total sum of squares (SST) measures the total amount of variation in
observed y values, and is defined as
m
X
SST = (yi − ȳ)2 .
i=1

Regression sum of squares (SSR) given by

SSR = SST − SSE.

SSR reflects the amount of variation in the y values explained by the linear
regression line y = ax + b when compared with the variation in the y values
about the line y = ȳ.
Coefficient of determination measures the proportion of the total variations
in y that can be explained by the linear model and it is defined as
SSE
R2 = 1 − ,
SST
pronounced as R squared.

1. R is called the sample correlation coefficient.

2. R2 ≤ 1.

3. The value of R2 does not depend on which of the two variables is labelled
x and which is labelled y.

4. The value of R2 is independent of the unit of x and y.

5. R2 is often interpreted as the proportion of response variation ”explained”

by the regressors in the model. Thus, R2 = 1 indicates that the fitted model
explains all variability in y, while R2 = 0 indicates no ’linear’ relationship.
An interior value such as R2 = 0.7 may be interpreted as follows: ”Seventy
percent of the variance in the response variable can be explained by the
explanatory variables.

Residuals: The residuals are the errors between the actual and predicted values

ri = yi − (axi + b).

1. One way to evaluate the reasonableness of the fit of the model is to look
at the plot of residuals versus the independent variable.

2. The residuals should be randomly distributed and contained in a reasonable

small band that corresponds with the accuracy of the data.

3. An extremely large residual should indicate that we need to take a further

look at the associated data point to discover the cause of the large residual.

4. A pattern or trend in the residuals indicates that some effect or aspect on

the data has not been captured in the model. The nature of the pattern
can provide hints as to how to refine our model.
28 MATH3800 NOTES-ERIC HUA

Example 18.

Solution: Here is the calculation according to the formula:

2. Fitting a Power Curve y = axn , where a is a parameter and n is a

constant.

Theorem 4. Fitting the data set

{(x1 , y1 ), ..., (xm , ym )}

by the power function

y = f (x) = axn
such that m
X
S= [yi − f (xi )]2
i=1

is minimum, where a is the parameter to be determined, n is a constant. Then

Pm n
xi y i
a = Pi=1
m 2n
.
i=1 xi

Example 19. Consider the data:

MATH3800 NOTES-ERIC HUA 29

We are going to fit the set with power function y = ax2 . Find a.

Solution: Here is the calculation according to the formula: y = 0.46003x2 .

3. Fitting a Power Curve y = axn , where both a and n are parameters.

Theorem 5. Fitting the data set

{(x1 , y1 ), ..., (xm , ym )}

by the general power function

y = f (x) = axn

such that m
X
S= [yi − f (xi )]2
i=1

is minimum, where both a and n are parameters. Then

m m xi ln yi − ( m ln xi )( m
P P P
i=1 lnP i=1 i=1 ln yi )
n= m m ,
m i=1 ln x2i − ( i=1 ln xi )2
P

( m 2
ln yi ) − ( m ln xi ln yi )( m
P P P P
i=1 ln xi )( i=1 i=1 i=1 ln xi )
ln a = Pm 2
Pm 2
.
m i=1 ln xi − ( i=1 ln xi )
30 MATH3800 NOTES-ERIC HUA

Proof.
ln y = n ln x + ln a.

Example 20. Consider the data:

We are going to fit the set with power function y = axn . Find a and n.

Solution: Here is the calculation according to the formula: y = 0.09956x2.7696 .

4. Fitting Exponential Curve y = c akx , where both a and k are pa-
rameters.

Note that
log y = log c + (k log a)x,
we need to transform y to log y. Then use Line of Best Fit.

Example 21. Consider the data:

MATH3800 NOTES-ERIC HUA 31

(a) Fit the data with y = c ekx .
Find c and k.
(b) Fit the data with y = c 10kx . Find c and k.
(c) Fit the data with y = c (dx ). Find c and d.

Solution:
(i) ln y = ln c + kx. So we transform data (x, y) → (x, ln y). Then

ln y = ax + b = kx + ln c.

(ii) log y = log c + kx. So we transform data (x, y) → (x, log y). Then

log y = ax + b = kx + log c.

(iii) ln y = ln c + (ln d)x. So we transform data (x, y) → (x, ln y). Then

ln y = ax + b = (ln d)x + ln c.
32 MATH3800 NOTES-ERIC HUA

5. More example from Section 6.3.

Example 22. Ponderosa pines: Diameter versus volume.

Data

Problem Identification: Predict the volume as a function of the diameter.

Scatterplot

The plot is concave up and increasing, which suggests a power function model
or exponential model.

Model Formulation: If we may assume that the height is proportional to the

diameter. Then
V ∝ a(d3 ) + b.
MATH3800 NOTES-ERIC HUA 33

If we assume that the pines have constant height. Then

V ∝ a(d2 ) + b.

Model Solutions:

Residuals Plot:

Remark:

• R2 values are close to 1, which indicates a strong linear relationship.

• For the model V = 0.152d2 , SSE is much bigger than other models, resid-
uals plot shows an apparent trend, we would probably reject the model.
34 MATH3800 NOTES-ERIC HUA

4.1 One-Term Models

When we construct empirical model, we always begin with DATA analysis:
1. Any existence of a trend? If yes, find a function that transform data into
a straight line. Most often transformations are:
y = a(xb ), y = a(bx ).

2. Are there data points lie obviously outside the trend? If yes, we may need
to discard them.
Sometimes it is hard to construct a good model. We may think about the fol-
lowing transformations:

Ladder of powers
——————
..
.
z2
√z
z
ln z
√1
z
1
z
1
z2
——————

How to use the ladder?

• If the data concave up, then go down the ladder (below z).
MATH3800 NOTES-ERIC HUA 35

• If the data concave down, then go up the ladder (above z).

Example 23. Consider the following data set, Harvesting of bluefish and blue
crabs versus time:

By plotting Bluefish data,

√
Now we try Ladder of powers: y → y
36 MATH3800 NOTES-ERIC HUA

Next we try Ladder of powers: y → log(y)

The line of best fit is:

log y = (0.1654)x + 0.7231.

Thus

y = 100.7231 100.1654x = 5.2857(1.4635)x .

Next we try Ladder of powers: y → √1

y

Similarly, by plotting crab data,

MATH3800 NOTES-ERIC HUA 37

We imply that √
y = 158.34 x.
38 MATH3800 NOTES-ERIC HUA

4.3 Smoothing: Low-Order Polynomial Models

Interpolation
The Lagrange interpolating polynomial is the polynomial that passes through
the given points. We try a polynomial
pn (x) = a0 + a1 x + ... + an xn
fitting data (x1 , y1 ), ..., (xn , yn ), (xn+1 , yn+1 ), i.e.,
pn (xj ) = yj , j = 1, ..., n + 1.
The method is called Lagrange polynomial interpolation:

n+1
X
pn (x) = Lj (x)yj ,
j=1

where
Lj (x) = Πn+1 n+1
i=1,i6=j (x − xi )/Πi=1,i6=j (xj − xi ).

Example 24. Given three points (1.3, 3.2), (1.7, 4.4), (2.1, 5.2). Then
(x − x2 )(x − x3 ) (x − 1.7)(x − 2.1)
L1 (x) = = = 3.125x2 −11.875x+11.15625,
(x1 − x2 )(x1 − x3 ) (1.3 − 1.7)(1.3 − 2.1)
(x − x1 )(x − x3 ) (x − 1.3)(x − 2.1)
L2 (x) = = = −6.25x2 +21.25x−17.0025,
(x2 − x1 )(x2 − x3 ) (1.7 − 1.3)(1.7 − 2.1)
(x − x1 )(x − x2 ) (x − 1.3)(x − 1.7)
L3 (x) = = = 3.125x2 −9.375x+6.90625.
(x3 − x1 )(x3 − x2 ) (2.1 − 1.3)(2.1 − 1.7)
Then
P2 (x) = L1 (x)(3.2) + L2 (x)(4.4) + L3 (x)(5.2)
= −1.25x2 + 6.75x − 3.4625.

Example 25. Given four points (0, 1), (1, −1), (2, −3), (3, 1). Then
(x − x2 )(x − x3 )(x − x4 ) (x − 1)(x − 2)(x − 3) 1
L1 (x) = = = − x3 +x2 −2x+1,
(x1 − x2 )(x1 − x3 )(x1 − x4 ) (0 − 1)(0 − 2)(0 − 3) 6
(x − 0)(x − 2)(x − 3) 1 5
L2 (x) = = x3 − x2 + 3x,
(1 − 0)(1 − 2)(1 − 3) 2 2
(x − 0)(x − 1)(x − 3) 1 3
L3 (x) = = − x3 + 2x2 − x,
(2 − 0)(2 − 1)(2 − 3) 2 2
(x − 0)(x − 1)(x − 2) 1 1 1
L4 (x) = = x3 − x2 + x.
(3 − 0)(3 − 1)(3 − 2) 6 3 3
Then
P3 (x) = L1 (x)(1) + L2 (x)(−1) + L3 (x)(−3) + L4 (x)(1)
= x3 − 3x2 + 1.
MATH3800 NOTES-ERIC HUA 39

Divided Differences
Divided differences is an algorithm, historically used for computing tables of
logarithms and trigonometric functions.[citation needed] Charles Babbage’s dif-
ference engine, an early mechanical calculator, was designed to use this algorithm
in its operation.
Divided differences is a recursive division process. The method can be used
to calculate the coefficients in the interpolation polynomial.
Consider the model
P (x) = a + bx + cx2 + · · · .
We have
P 0 (x) = b + 2cx + ...
P 00 (x) = 2c + ...
Recall the definition of the derivative
f (x + h) − f (x) ∆y
f 0 (x) = lim = lim .
h→0 x+h−x h→0 ∆x

Thus
0 ∆P 00∆P 0
P ≈ , P ≈ , ...
∆x ∆x
Definition 3. Given the data set {(x1 , y1 ), ..., (xm , ym )}.

The first differences are

∆ = yi+1 − yi , i = 1..., m − 1.

The second differences are

∆2 = (yi+2 − yi+1 ) − (yi+1 − yi ), i = 1..., m − 2.

The kth differences ∆k are computed by finding the difference between succes-
sive k − 1th differences.

The first divided differences (1DD) are

yi+1 − yi
f [xi , xi+1 ] = , i = 1..., m − 1.
xi+1 − xi
The second divided differences (2DD) are

f [xi+1 , xi+2 ] − f [xi , xi+1 ]

f [xi , xi+1 , xi+2 ] = , i = 1..., m − 2.
xi+2 − xi
The kth divided differences (kDD) are

f [xi+1 , ..., xi+k ] − f [xi , xi+1 , ..., xi+k−1 ]

f [xi , xi+1 , ..., xi+k ] = , i = 1..., m − k.
xi+k − xi

Theorem 6. Given the data set {(x1 , y1 ), ..., (xm , ym )}.

40 MATH3800 NOTES-ERIC HUA

The kth degree polynomial model is given by

P (x) = f (x0 ) + f [x0 , x1 ](x − x0 ) + f [x0 , x1 , x2 ](x − x0 )(x − x1 )

+... + f [x0 , x1 , ..., xk ](x − x0 )(x − x1 )...(x − xk−1 ).

Example 26. Consider the following data:

P (x) = 1 + 2x + 7x(x − 1) + 3x(x − 1)(x − 3) = 3x3 − 5x2 + 4x + 1.

Error Estimates
If we use a polynomial pn (x) to approximate f (x) such that

pn (xj ) = f (xj ), j = 1, ..., n + 1,

then the error

f (n+1) (c)
en (x) = |f (x) − pn (x)| = |(x − x1 )...(x − xn+1 ) |,
(n + 1)!
where c is some (unknown) point in the interval [a, b] containing all xj .

Low-order polynomial is a technique to model data points. For

example, if we decided to use quadratic, it is not possible to force a quadratic
to pass through all data points. Thus it must be decided which quadratic best
ts the data. This process is called smoothing. This smoothing process requires
two decisions:
1. The order of the polynomial.
2. The coefficients of the polynomial.
Theorem 7. If
P2 (x) = a + bx + cx2
is the model of best-fit to the data set

{(x1 , y1 ), · · · , (xm , ym )},

MATH3800 NOTES-ERIC HUA 41

m
! m
! m
X X X
ma + xi b+ x2i c = yi
i=1 i=1 i=1
m
! m
! m
! m
X X X X
xi a+ x2i b + x3i c = xi yi
i=1 i=1 i=1 i=1
m
! m
! m
! m
X X X X
x2i a + x3i b + x4i c = x2i yi
i=1 i=1 i=1 i=1

Equivalently,
a + xb + x2 c = y
xa + x2 b + x3 c = xy
x2 a + x3 b + x4 c = x2 y

By Cramer’s Rule, we have

y x x2 1 y x2 1 x y
xy x2 x3 x xy x3 2
x x xy
x2 y x3 x 4 x2 x2 y x4 x2 x3 x2 y
a= , b= , c=
1 x x2 1 x x2 1 x x2
x x2 x3 x x2 x3 x x2 x3
x2 x3 x4 x2 x3 x4 x2 x3 x4
Proof. To minimize
m
X 2
yi − (a + bxi + cx2i ) ,

S=
i=1

∂S ∂S ∂S
= = = 0.
∂a ∂b ∂c

Example 27. Consider the data collected for the tape recorder problem

where ci are the counter reading, t = P2 (c) are the elapsed time. Then
P2 (c) = 0.14286 + 1.94226c + 0.00105c2 .
Solution:
8a + 3600b + 2040000c = 9128
3600a + 2040000b + 1296000000c = 5318900
2040000a + 1296000000b + 8.772 × 1011 c = 3435390000
42 MATH3800 NOTES-ERIC HUA

Solving this we get the solution.

Testing of the model: The deviations

The errors are relatively small.

A divided difference table for the tape recorder data

All 2DDs are almost same, so second order polynomial to the data is appropriate.

Example 28. Consider the following data, which degree of polynomial will be
good model?
MATH3800 NOTES-ERIC HUA 43

All 4DDs are same, so 4th order polynomial to the data is appropriate.
44 MATH3800 NOTES-ERIC HUA

4.4 Cubic Spline Models

Consider the data set {(x1 , y1 ), (x2 , y2 ), (x3 , y3 )}.

Linear Splines: Using piecewise linear functions model the data as follows:


S1 (x) = a1 + b1 x, f or x ∈ [x1 , x2 )
S2 (x) = a2 + b2 x, f or x ∈ [x2 , x3 ]

• Unfortunately the function is not smooth at x = x2 .

• Usually it fails to capture the trend of data.

Example 29. Given the data set (xi , y(xi )) = (1, 1), (2, 4), (3, 2). The spline
model is:


S1 (x) = −2 + 3x, f or x ∈ [1, 2)
S2 (x) = 8 − 2x, f or x ∈ [2, 3]
MATH3800 NOTES-ERIC HUA 45

Cubic Splines (Natural Spline): Consider the data set {(x1 , y1 ), (x2 , y2 ), (x3 , y3 )}
(3 points can be extended to N points). The Cubic Spline Model constructs 2
third-order piecewise continuous cubic polynomials (cubic splines) that connect
3 data points:

S1 (x) = a1 + b1 x + c1 x2 + d1 x3 , f or x ∈ [x1 , x2 )
S2 (x) = a2 + b2 x + c2 x2 + d2 x3 , f or x ∈ [x2 , x3 ]
Polynomial coefficients are chosen such that the resulting curve and its first
derivative are smooth at the points.

y1 = S1 (x1 ) : y1 = a1 + b1 x1 + c1 x21 + d1 x31

y2 = S1 (x2 ) : y2 = a1 + b1 x2 + c1 x22 + d1 x32
y2 = S2 (x2 ) : y2 = a2 + b2 x2 + c2 x22 + d2 x32
y3 = S2 (x3 ) : y3 = a2 + b2 x3 + c2 x23 + d2 x33
S10 (x2 ) = S20 (x2 ) : b1 + 2c1 x2 + 3d1 x22 = b2 + 2c2 x2 + 3d2 x22
S100 (x2 ) = S200 (x2 ) : 2c1 + 6d1 x2 = 2c2 + 6d2 x2
S100 (x1 ) = 0 : 2c1 + 6d1 x1 = 0
S200 (x3 ) = 0 : 2c2 + 6d2 x3 = 0.
Remark. If we replace the last two equations by S10 (x1 ) = f 0 (x1 ), S20 (x3 ) =
f 0 (x3 ), then the cubic spline is called clamped spline.
Example 30. Given the data set (xi , y(xi )) = (1, 5), (2, 8), (3, 25).
(1) Construct the natural spline model, i.e.,

S1 (x) = a1 + b1 x + c1 x2 + d1 x3 , f or x ∈ [1, 2)
S2 (x) = a2 + b2 x + c2 x2 + d2 x3 , f or x ∈ [2, 3]
(2) Use (1) to estimate y(1.5), y(2.5).
46 MATH3800 NOTES-ERIC HUA

Solution:

(1)

y1 = S1 (x1 ) : 5 = a1 + b 1 + c 1 + d 1 (1)
y2 = S1 (x2 ) : 8 = a1 + 2b1 + 4c1 + 8d1 (2)
y2 = S2 (x2 ) : 8 = a2 + 2b2 + 4c2 + 8d2 (3)
y3 = S2 (x3 ) : 25 = a2 + 3b2 + 9c2 + 27d2 (4)
S10 (x2 ) = S20 (x2 ) : b1 + 4c1 + 12d1 = b2 + 4c2 + 12d2 (5)
S100 (x2 ) = S200 (x2 ) : 2c1 + 12d1 = 2c2 + 12d2 (6)
S100 (x1 ) = 0 : 2c1 + 6d1 = 0 (7)
S200 (x3 ) = 0 : 2c2 + 18d2 = 0. (8)


S1 (x) = 2 + 10x − 10.5x2 + 3.5x3 ,
S2 (x) = 58 − 74x + 31.5x2 − 3.5x3 .

(2) S1 (1.5) = 5.188, S2 (2.5) = 15.188.

Summary.
MATH3800 NOTES-ERIC HUA 47
48 MATH3800 NOTES-ERIC HUA

Numerical Differentiation and Integration

Numerical differentiation

First Order Derivatives

We start from Taylor series:

f 00 (xi ) 2 f 000 (xi ) 3 f (n) (xi ) n

f (x) = f (xi ) + f 0 (xi )h + h + h + ... + h + ... h = x − xi .
2! 3! n!

• The first forward finite divided difference: Let x = xi+1 ,

0 f 00 (xi ) 2 f 000 (xi ) 3 f (n) (xi ) n

f (xi+1 ) = f (xi )+f (xi )h+ h + h +...+ h +... h = xi+1 −xi
2! 3! n!
Then
f (xi+1 ) − f (xi )
f 0 (xi ) ≈ , error = O(h).
h
• The first backward finite divided difference: Let x = xi−1 ,

0 f 00 (xi ) 2 f 000 (xi ) 3 f (n) (xi ) n

f (xi−1 ) = f (xi )−f (xi )h+ h − h +...+ h +... h = xi −xi−1
2! 3! n!
Then
f (xi ) − f (xi−1 )
f 0 (xi ) ≈ , error = O(h).
h
• The first centred finite divided difference:

f (xi+1 ) − f (xi−1 ) = 2f 0 (xi )h + O(h3 ) 2h = xi+1 − xi−1

Then
f (xi+1 ) − f (xi−1 )
f 0 (xi ) ≈ , error = O(h2 ).
2h
Example 31. Estimate the first derivative of

f (x) = −0.1x4 − 0.15x3 − 0.5x2 − 0.25x + 1.2

at xi = 0.5 with h = 0.25.

Solution: The exact value is:

f 0 (x) = −0.4x3 − 0.45x2 − x − 0.25, f 0 (0.5) = −0.9125.

When h = 0.25, xi = 0.5, xi+1 = 0.75, xi−1 = 0.25,

f (xi ) = 0.925, f (xi+1 ) = 0.6363, f (xi−1 ) = 1.1035.

• The first forward finite divided difference:

f 0 (xi ) ≈ −1.155.
MATH3800 NOTES-ERIC HUA 49

• The first backward finite divided difference:

f 0 (xi ) ≈ −0.714.
• The first centred finite divided difference:
f 0 (xi ) ≈ −0.934.
Lower approximation error comes from centered finite divided difference.

Second Order Derivatives

• The second forward finite divided difference: We start from Taylor series:
f 00 (xi ) 2 f 000 (xi ) 3 f (n) (xi ) n
f (xi+1 ) = f (xi )+f 0 (xi )h+ h + h +...+ h +... h = xi+1 −xi
2! 3! n!
f 00 (xi ) f 000 (xi ) f (n) (xi )
f (xi+2 ) = f (xi )+f 0 (xi )(2h)+ (2h)2 + (2h)3 +...+ (2h)n +... 2h = xi+2 −xi
2! 3! n!
Then
f (xi+2 ) − 2f (xi+1 ) + f (xi )
f 00 (xi ) ≈ , error = O(h).
h2
• The second backward finite divided difference: Similarly,
f (xi ) − 2f (xi−1 ) + f (xi−2 )
f 00 (xi ) ≈ , error = O(h).
h2
• The second centred finite divided difference: Similarly,
f (xi+1 ) − 2f (xi ) + f (xi−1 )
f 00 (xi ) ≈ , error = O(h2 ).
h2

Numerical integration Z b
b−a
Methods approximating definite integrals f (x)dx: Let h = n
, xi = a + ih,
a
i = 0, 1, ..., n.
• Midpoint rule: uses the midpoint of each subinterval. mid(n) denotes the
result obtained by using ”midpoint rule” with n subintervals:
n−1  
X xj + xj+1
midrule(a, b, n) = h f .
j=0
2
50 MATH3800 NOTES-ERIC HUA

• Trapezoid rule:
n−1
hX h
traprule(a, b, n) = [f (xj ) + f (xj+1 )] = {f (x0 )+2f (x1 )+...+2f (xn−1 )+f (xn )}.
2 j=0 2

• Simpson’s Rules: In Simpson’s Rule, we will use parabolas to approximate

each part of the curve. This proves to be very efficient since it’s generally
more accurate than the other numerical methods we’ve seen. In Simp-
son’s Rule, n must be even.

Z b n/2
h X
f (t)dt = simprule(a, b, n) = [f (t2j−2 ) + 4f (t2j−1 ) + f (t2j )]
a 3 j=1

h
= {f (t0 ) + 4f (t1 ) + 2f (t2 ) + 4f (t3 ) + ... + 2f (tn−2 ) + 4f (tn−1 ) + f (tn )},
3
where we have used Lagrange polynomial interpolation through points
(t0 , f (t0 )), (t1 , f (t1 )), (t2 , f (t2 )):
(t − t1 )(t − t2 ) (t − t0 )(t − t2 ) (t − t0 )(t − t1 )
f (t) ≈ f (t0 )+ f (t1 )+ f (t2 )
(t0 − t1 )(t0 − t2 ) (t1 − t0 )(t1 − t2 ) (t2 − t0 )(t2 − t1 )
(t − t1 )(t − t2 ) (t − t0 )(t − t2 ) (t − t0 )(t − t1 )
= 2
f (t0 ) + 2
f (t1 ) + f (t2 ).
2h −h 2h2
The area of the small piece is
Z t2
f (t)f t.
t0
MATH3800 NOTES-ERIC HUA 51

For example,

h
simprule(a, b, 4) = {f (t0 ) + 4f (t1 ) + 2f (t2 ) + 4f (t3 ) + f (t4 )},
3

h
simprule(a, b, 6) = {f (t0 ) + 4f (t1 ) + 2f (t2 ) + 4f (t3 ) + 2f (t4 ) + 4f (t5 ) + f (t6 )}.
3

Error = |true value - estimated value|

Maximum error:

K(b − a)3 K(b − a)3 M (b − a)5

|EM | ≤ , |ET | ≤ , |ES | ≤ .
24n2 12n2 180n4

Z 2
Example 32. Estimate I = sin(x2 )dx by using midrule(0,2,4), traprule(0,2,4),
0
simprule(0,2,2).

b−a
Solution: 1) h = n
= 0.5, x0 = 0, x1 = 0.5, x2 = 1, x3 = 1.5, x4 = 2.

midrule(0, 2, 4) = 0.5{f (0.25) + f (0.75) + f (1.25) + f (1.75)}

0.5
traprule(0, 2, 4) = {f (0) + 2f (0.5) + 2f (1) + 2f (1.5) + f (2)}
2

0.5
simprule(0, 2, 4) = {f (0) + 4f (0.5) + 2f (1) + 4f (1.5) + f (2)}
3
52 MATH3800 NOTES-ERIC HUA

Here is an M-file using the midpoint rule with n intervals:

function S=midrule(a,b,n)
h=(b-a)/n;
xx=zeros(1,n);
for i=1:n
xx(i)=a+h/2+(i-1)*h;
end
S=0;
for i=1:n
S=S+h*(sin(xx(i)∧2));
end

Here is an M-file using the trapezoid rule with n intervals:

function S=traprule(a,b,n)
h=(b-a)/n;
x=zeros(1,n+1);
for i=1:n+1
x(i)=a+h*(i-1);
end
S=0;
S=S+(h/2)*(sin(x(1)∧2));
for i=2:n
S=S+h*(sin(x(i)∧2));
end
S=S+(h/2)*(sin(x(n+1)∧2));
End

M-file using Simpson’s rule with 2n intervals:

function S=simprule(a,b,n)
h=(b-a)/(2*n);
x=zeros(1,2*n+1);
for i=1:2*n+1
x(i)=a+h*(i-1);
end
S=0;
S=S+(h/3)*(sin(x(1)∧2));
for i=1:n-1
S=S+(4*h/3)*(sin(x(2*i)∧2));
S=S+(2*h/3)*(sin(x(2*i+1)∧2));
end
S=S+(4*h/3)*(sin(x(2*n)∧2));
S=S+(h/3)*(sin(x(2*n+1)∧2));
end
MATH3800 NOTES-ERIC HUA 53

Gaussian Quadrature
Degree of Precision: If the integration rule has zero error when integrating
any polynomial of degree ≤ k, and if the error is non-zero for some polynomial
of degree k + 1, then we say that the rule has degree of precision = k.

For example,
Trapezoid rule has degree of precision = 1.
Z b
b−a b−a
f (x)dx = f (a) + f (b).
a 2 2
Simpsons rule has degree of precision = 3.
Z b
b−a 2(b − a) a + b b−a
f (x)dx = f (a) + f( )+ f (b).
a 6 3 2 6
Gaussian Quadrature of f (x):
Z b n
X
f (x)dx ≈ wi f (xi ),
a i=1

where wi are called weights, xi are nodes. The above formula is exactly equal
for any polynomial f (x) of degree less than or equal to 2n − 1.

• Trapezoid, Simpson’s are based on equally spaced nodes.

• Gaussian Quadrature frees ourselves from equally spaced nodes
• Combine selection of the nodes and selection of the weights into one quadra-
ture rule

Example 33. When n = 2, 2-point Gaussian formula is

Z 1
1 1
f (x)dx ≈ f (− √ ) + f ( √ ).
−1 3 3
Proof.
Z 1 By solving
1dx = 2 = w1 + w2 ,
−1
Z 1
xdx = 0 = w1 x1 + w2 x2 ,
Z−11
2
x2 dx = = w1 x21 + w2 x22 ,
3
Z−11
x3 dx = 0 = w1 x31 + w2 x32 ,
−1
we imply that w1 = w2 = 1, x1 = − √13 , x2 = √1 .
3

Remark. By transformations,
Z b
b−a 1
 
b−a
Z
a+b
f (x)dx = f s+ ds.
a 2 −1 2 2
54 MATH3800 NOTES-ERIC HUA
Z 1.2
Example 34. Estimate sin xdx.
0

Z 1.2 Z 1 Z 1
Solution. sin xdx = 0.6 sin(0.6x + 0.6)dx = 0.6 sin 0.6(x + 1)dx
0 −1 −1
1 1
≈ 0.6 sin 0.6(− √ + 1) + sin 0.6( √ + 1) ≈ 0.6373216.
3 3
Legendre’s polynomials

Polynomial solution Pn (x) of the Legendre’s equation

(1 − x2 )y 00 − 2xy 0 + n(n + 1)y = 0, −1 < x < 1.

For example,

P0 (x) = 1,
P1 (x) = x,
1
P2 (x) = (3x2 − 1),
2
1
P3 (x) = (5x3 − 3x),
2
1
P4 (x) = (35x4 − 30x2 + 3),
8
1
P5 (x) = (63x5 − 70x3 + 15x).
8

Legendre’s polynomial can be obtained from the following Rodrigues formula

1 dn  2 n

Pn (x) = (x − 1) .
2n n! dxn

Orthogonality Relations for Pn (x):

Z 1 
0, m 6= n;
Pm (x)Pn (x) dx = 2
−1 2n+1
, m = n.

Theorem 8.
Z b n
X
f (x)dx ≈ wi f (xi ),
a i=1

where xi are the roots of the Legendre’s polynomial of degree n,

1
x − xj
Z
wi = Li (x)2 dx, Li (x) = Πnj=1,j6=i .
−1 xi − xj
MATH3800 NOTES-ERIC HUA 55

function coefs = LegendrePoly(n)

if n == 0

coefs = 1;

elseif n == 1

coefs = [1 0];

else

P_nm1 = 1;

P_n = [1 0];

for i=1:(n-1);

P_np1 = ((2*i+1)*[P_n,0] - i*[0,0,P_nm1])/(i+1); % recurrence

[P_nm1,P_n] = deal(P_n,P_np1); % shift

end

coefs = P_np1;

end
56 MATH3800 NOTES-ERIC HUA
MATH3800 NOTES-ERIC HUA 57

5.2 Generating Random Numbers

From previous section we see that, an important part of simulation is the gen-
eration of random numbers. The following method was developed by Newman
etc.

Middle-Square Method:
1. Start with a four-digit number x0 , called the seed.
2. Square it to obtain an eight-digit number (add a leading zero if necessary).
3. Take the middle four digits as the next random number.

Continuing, we obtain a sequence that appears random over the integers from
0 to 9999. These integers can be scaled to any interval [a, b].

Example 35. Generate 5 numbers by Middle Square Method with the seed x0 =
2041.

Linear congruence generator: a linear rule for generating the next element
of a sequence from previous elements:

xn+1 = axn + b (mod c),

where a, b, c are constants.

Example 36. Generate the sequence with the model:

xn+1 = 7xn + 2 (mod 10).

Solution: If we take x0 = 3, then we get a sequence 3, 3, 3, 3, ...

If we take x0 = 5, then we get a sequence 5, 7, 1, 9, 5, 7, 1, 9, ...
58 MATH3800 NOTES-ERIC HUA

5.3 Simulating Probabilistic Behaviour

In this section we will simulate simple probabilistic behaviour.

Flipping a Fair Coin.

When we flip a coin, theoretically the probability of obtaining head or tail

1
is . Experimentally, in the long run, the ratio of the number of tails to the
2
number of flips approaches 1 : 2.
For any random number x ∈ [0, 1], define

head, f or x ∈ [0, 0.5]
f (x) =
tail, f or x ∈ (0.5, 1]
Monte Carlo Fair Coin Algorithm.
Input: Total number n of random flips of a fair coin to be generated in the
simulation.
Output: Number of tails we get when the coin is flipped n times.
Step 1: Initialize COUNTER=0.
Step 2: for i = 1, ..., n, generate a random number xi between 0 and 1.
Step 3: if 0.5 < xi ≤ 1, then i-th flip is a tail.
Step 4: Output the total number of tails.

Monte Carlo Roll of an un-fair Die Algorithm: The probabilities of getting

each face are as follows:

• P(Face 1)=0.1,

• P(Face 2)=0.1,

• P(Face 3)=0.1,

• P(Face 4)=0.1,

• P(Face 5)=0.2,

• P(Face 6)=0.4.

Input: Total number n of random rolls of a die to be generated in the simu-

lation.
Output: Number of each face we get.

Step 1: Initialize COUNTER1=0,..., COUNTER6=0.

Step 2: for i = 1, ..., n, generate a random number xi between 0 and 1.

Step 3: 1. If 0 ≤ xi ≤ 0.1, then ith roll is a face 1.

2. If 0.1 < xi ≤ 0.2, then ith roll is a face 2.
3. If 0.2 < xi ≤ 0.3, then ith roll is a face 3.
4. If 0.3 < xi ≤ 0.4, then ith roll is a face 4.
MATH3800 NOTES-ERIC HUA 59

5. If 0.4 < xi ≤ 0.6, then ith roll is a face 5.

6. If 0.6 < xi ≤ 1, then ith roll is a face 6.

Step 4: Output the total of getting each face.

Example 37. Generate some numbers and find the total of getting each face.
60 MATH3800 NOTES-ERIC HUA

Traffic Simulation
https://en.wikipedia.org/wiki/Traffic simulation

Simulating Blackjack with MATLAB

http://www.mathworks.com/company/newsletters/articles/simulating-blackjack-
with-matlab.html
MATH3800 NOTES-ERIC HUA 61

6.2 Modelling Component and System Reliabil-

ity
Reliability of a component or system is the probability that it will not failure
over a specified period of time t. Let f (t) be the failure rate over time t, F (t) be
the cumulative distribution function corresponding f (t), we define the reliability
by
R(t) = 1 − F (t).

• Series system is one that performs well as long as all of the components
are fully functional.

Rs (t) = R1 (t) · · · Rn (t).

• Parallel system is one that performs well as long as a single one of its
components remains functional.

Rs (t) = 1 − Πni=1 [1 − Ri (t)].

If n = 2, then Rs (t) = R1 (t) + R2 (t) − R1 (t)R2 (t).

Example 38. A system contains two components, A and B, connected in series

as shown in the following diagram:

The system will function only if both components function. The probability
that A functions is given by P(A) = 0.98, and the probability that B functions is
62 MATH3800 NOTES-ERIC HUA

given by P(B) = 0.95. Assume that A and B function independently. Find the
probability that the system functions.

Solution:

P (system f unctions) = P (A ∩ B) = P (A) × P (B) = 0.95 × 0.98 = 0.931

Example 39. A system contains two components, C and D, connected in parallel

as shown in the following diagram:

The system will function if either C or D functions. The probability that C

functions is 0.90, and the probability that D functions is 0.85. Assume C and D
function independently. Find the probability that the system functions.

Solution: Since the system will function so long as either of the two
components functions, it follows that

P (systemf unctions) = P (C ∪ D) = P (C) + P (D) − P (C ∩ D)

= P (C) + P (D) − P (C) × P (D) = 0.90 + 0.85 − 0.90 × 0.85 = 0.985.

Example 40. Most practical equipments and systems are combinations of series
and parallel components. For example,

Find the reliability of the combined network, i.e., RAC .

Solution: To solve this network, one merely uses series and parallel rela-
tionships to decompose and recombine the network step by step.

Rad = R1 · R2 = (0.9)(0.8) = 0.72

Rbd = R3 · R4 · R5 = (0.8)(0.8)(0.9) = 0.576
MATH3800 NOTES-ERIC HUA 63

but Rad and Rbd are in parallel; thus,

RS1 = Rabd = 1 − (1 − 0.72)(1 − 0.576) = 1 − (0.28)(0.424) = 0.88128.

Now the network has been decomposed to

RS1,6 = RS1 · R6 = (0.88128)(0.9) = 0.793152

Since the last system is in parallel, the total system reliability is

RAC = 1 − (1 − 0.793152)(1 − 0.9)1 − (0.206848)(0.1) = 0.9793152.

Thus, the reliability of the combined network is 0.9793152.

64 MATH3800 NOTES-ERIC HUA

6.1 Probabilistic Modeling with Discrete Systems

In this section we will study the systems of difference equations, where coeffi-
cients can vary in a probabilistic manner. A special case is Markov chain.

Markov chain
• A Markov chain is a process in which there are the same finite number of
states (or outcomes) that can be occupied at any given time.

• The states do not overlap and cover all possible outcomes.

• The system moves from one state to another, one for each time step. There
is a probability associated with this transition for each possible outcome.

• The sum of the probabilities for transitioning from one state to the next is 1.

Probability vector: a vector with non-negative entries that add up to one.

Example 41. (Two-state Markov chain) MATH3800 Tutoring Ottawa has two
branches in Ottawa: Andrew Park, and Billings Bridge, with market portion
a0 and b0 respectively. According to market statistics, due to variety of tutors,
after each year, 40% of AP’s customers will switch to BB’s, while 30% of BB’s
customers will switch to AP’s.
We can array the data as following, which is transition matrix:

Markov chain graph:

MATH3800 NOTES-ERIC HUA 65

Let
an = the number of customers at the branch Andrew Park , at the end of
year n.
bn = the number of customers at the branch Billings Bridge, at the end of
year n.
Then

an+1 = 0.6an + 0.3bn

bn+1 = 0.4an + 0.7bn

Numerical method:

3 4
an → , bn → .
7 7
Algebra method: Let  T be the transition matrix, i.e., xk+1 = T xk , k =
a
0, 1, 2, · · · where xk = k Then Steady-State Vector ~q (which gives long-term
bk
distribution) is a probability vector such that

T ~q = ~q, ⇒ (T − I)~q = ~0.

Now      
0.6 0.3 −0.4 0.3 1 −3/4
T = ,T − I = ∼ .
0.4 0.7 0.4 −0.3 0 0
By solving the system q1 − 3/4q2 = 0, q1 + q2 = 1, q1 = 3/7, q2 = 4/7.

Example 42. (Three-state Markov chain) Presidential voting tendencies are of

interest every 4 years.

Problem Identification: Can we find the long-term behavior of voters in a

presidential election?
Assumptions: The data for past decades are arranged in the following transi-
tion matrix:
66 MATH3800 NOTES-ERIC HUA

Markov chain graph:

Model Formulation: Let

Rn = percentage of voters to vote Republican in period n.
Dn = percentage of voters to vote Democratic in period n.
In = percentage of voters to vote Independent in period n.

Rn+1 = 0.75Rn + 0.20Dn + 0.40In

Dn+1 = 0.05Rn + 0.60Dn + 0.20In
In+1 = 0.20Rn + 0.20Dn + 0.40In .
MATH3800 NOTES-ERIC HUA 67

Algebra method:
     
0.75 0.2 0.4 −0.25 0.2 0.4 1 0 −20/9
T = 0.05 0.6 0.2 , T − I =  0.05 −0.4 0.2  ∼ 0 1 −7/9  .
0.2 0.2 0.4 0.2 0.2 −0.6 0 0 0

5 7 1
Thus q1 = , q2 = , q3 = .
9 36 4
68 MATH3800 NOTES-ERIC HUA

Matrix Computations
1. LU Decomposition.

To solve Ax = b, one way is to use row reduction, another way is to use LU-
decomposition.

1. Write A = LU , where L is an lower triangular matrix, the entries on the

main diagonal are 1 only, U is an upper triangular matrix.

2. Solve Ly = b.

3. Solve U x = y.

Example 43. Let

     
a11 a12 a13 1 0 0 u11 u12 u13
A = a21 a22 a23  , L = m21 1 0 , U =  0 u22 u23  .
a31 a32 a33 m31 m32 1 0 0 u33

Then
   
u11 u12 u13 a11 a12 a13
LU = m21 u11 m21 u12 + u22 m21 u13 + u23  = a21 a22 a23  .
m31 u11 m31 u12 + m32 u22 m31 u13 + m32 u23 + u33 a31 a32 a33
 
1 2 4
Example 44. Solve Ax = b by LU-decomposition, where A = 3 8 14 , b =
  2 6 13
3
13.
4
Solution. Step 1:
   
u11 u12 u13 1 2 4
m21 u11 m21 u12 + u22 m21 u13 + u23  = 3 8 14 .
m31 u11 m31 u12 + m32 u22 m31 u13 + m32 u23 + u33 2 6 13
   
1 0 0 1 2 4
L = 3 1 0 , U = 0 2 2 .
2 1 1 0 0 3 
3
Step 2: Solve Ly = b, y =  4 .
−6 
3
Step 3: Solve U x = y, x =  4 .
−2
MATH3800 NOTES-ERIC HUA 69

2. Cholesky Decomposition.

Positive definite matrix A: if xT Ax > 0 for any nonzero vector x. We denote

this by A > 0.
 
a11 a12
• A > 0 iff all of its principle minors are positive, i.e., a11 > 0, det >
a21 a22
0, ..., det(A) > 0.

• A > 0 iff all eigenvalues are positive.

• If A is diagonally dominant with positive diagonal entries, then A > 0.

Here a matrix A is diagonally dominant if
X
|aii | ≥ |aij |, all i.
j6=i

• If A is symmetric positive definite, then A has special LU-decomposition:

A = GGT ,

which is called Cholesky decomposition, where G is lower triangular matrix.

 
4 2 14
Example 45. Solve Ax = b by Cholesky decomposition, where A =  2 17 −5 , b =
  14 −5 83
14
−101 .
155

Solution.Step 1: A = GGT ,
    2 
g11 0 0 g11 g21 g31 g11 g11 g21 g11 g31
g21 g22 2 2
0   0 g22 g32  = g21 g11 g21 + g22 g21 g31 + g22 g32 
2 2 2
g31 g32 g33 0 0 g33 g31 g11 g31 g21 + g32 g22 g31 + g32 + g33
 2
  
g11 g11 g21 g11 g31 4 2 14
2 2
g21 g11 g21 + g22 g21 g31 + g22 g32  =  2 17 −5 .
2 2 2
g31 g11 g31 g21 + g32 g22 g31 + g32 + g33 14 −5 83
 
2 0 0
We imply that G = 1 4 0 .

7 −3 5 
7
Step 2: Solve Gy = b, y = −27.
5 
3
Step 3: Solve GT x = y, x = −6.
1
70 MATH3800 NOTES-ERIC HUA

3. Gauss Seidel Iteration

In numerical linear algebra, the GaussSeidel method, also known as the Lieb-
mann method or the method of successive displacement, is an iterative method
used to solve a linear system of equations.
To solve Ax = b,
1. Write A = L + U , where L is the lower triangular matrix, U is the strict
upper triangular matrix:
     
a11 a12 · · · a1n a11 0 ··· 0 0 a12 · · · a1n
 a21 a22 · · · a2n   a21 a22 ··· 0  0 · · · a2n 
+ 0

· · · · · · · · · · · ·  = · · ·
    = L+U.
··· ··· ···  · · · · · · · · · · · ·
an1 an2 · · · ann an1 an2 · · · ann 0 0 ··· 0

2. Solve the recursion x(n+1) = L−1 (b − U x(n) ).

 
16 3
Example 46. Solve Ax = b by Gauss Seidel Iteration, where A = ,
    7 −11
11 1
b= , x(0) = .
13 1
     
16 3 16 0 0 3
Solution: A = = L + U, where L = , U = . We
7 −11 7 −11 0 0
imply that
     
(n+1) −1 (n) 0.0625 0 11 0 3 (n)
x = L (b − U x ) = ( − x )
0.0398 −0.0909 13 0 0
   
(n+1) 0 −0.1875 (n) 0.6875
x = x +
0 −0.1194 −0.7439
MATH3800 NOTES-ERIC HUA 71

The actual solution is

      
−11 −11 −3 11 1 −160 0.8122
x=A b= = =
−197 −7 16 13 −197 131 −0.6650
72 MATH3800 NOTES-ERIC HUA

4. Least Squares Fit of a line to Data, Algebraic method.

For any vector ~x = [x1 x2 ... xn ]T , we define

l1 -norm: ||x||1 = p
|x1 | + ... + |xn |.
l2 -norm: ||x||2 = |x1 |2 + ... + |xn |2 .
l∞ -norm: ||x||∞ = max{|x1 |, ..., |xn |}.

For any matrix A, we define norms as follows:

||A||1 = max1≤j≤n {|a1j | + ... + |anj |}.
√
||A||2 = maxi σi , where σi is an eigenvalue of AT A, which represents the
largest singular value of matrix A .
||A||∞ = v
max1≤i≤n {|ai1 | + ... + |ain |}.
u∞ X ∞
uX
||A||F = t |aij |.
i=1 j=1

Least Squares Fit of a line to Data, Algebraic method: Suppose we

have n points (xi , yi) : i = 1, ..., n. We want to find the ”best” straight line
  c
y = c + dx = 1 x .
d
 
1 x1  
1 x2    y1
c ~  .. 
1. Let A =  .. .. , ~x = , b =  . . We hope to solve A~x = ~b. But
 
. .  d
yn
1 xn
usually not possible.
 P   P 
T T~ n P 2 ~x = P yi .
x i
2. Solve A A~x = A b, i.e., P
xi xi xi y i

3. The error E = ||A~x − ~b||2 is minimum.

Example 47. Find the least square fit of a line by Algebraic method to the points
(−2, 1), (0, 2), (2, 4).
   
1 −2   1
c ~  
Solution: A = 1 0 , ~x =
  ,b= 2 .
d
1 2 4
   
T 3 0 T~ 7
A A= , A b= .
0 8 6
 
T −1 T~ 1 56
~x = (A A) (A b) = .
24 18
7 3
y= + x.
3 4
MATH3800 NOTES-ERIC HUA 73

5. Least Squares Fit of quadratic to Data, Algebraic method: Suppose

we have n points (xi , yi ) : i = 1, ..., n. We want to find the ”best” quadratic
y = c0 + c1 x + c2 x 2 .

 
1 x1 x21  
y1
 
1 x2 x2  c1
2 .
1. Let A =  .. .. .. , ~x = c2 , ~b =  .. .
 
. . . c3

yn
1 xn x2n
 P   P 
xi P x2i
P
n P yi
T T~ 2 3
P P
2. Solve A A~x = A b, i.e., P xi P xi P xi ~x = P xi yi .
 
x2i x3i x4i x2i yi

3. The error E = ||A~x − ~b||2 is minimum.

Example 48. Find the least square fit of quadratic by Algebraic method to the
points (0, 6), (1, 0), (2, 0).

Solution: y = 6 − 9x + 3x2 .
74 MATH3800 NOTES-ERIC HUA

6. Estimate eigenvalues: The eigenvalues of the matrix A are the solutions

of the characteristic equation

det(A − λI) = 0.

Abel’s Impossibility Theorem. There is no general algebraic solution for

solving a polynomial equation of degree ≥ 5.

Method to estimate eigenvalue λ (Gershgorin circle theorem):

n
X
|aii − λ| ≤ |aij |, 1 ≤ i ≤ n.
j6=i,j=1

 
5 −2 2
Example 49. Estimate eigenvalues of A = 2 0 4.
4 2 7
Solution:
|5 − λ1 | ≤ | − 2| + |2|, 1 ≤ λ1 ≤ 9;
|0 − λ1 | ≤ |2| + |4|, −6 ≤ λ1 ≤ 6;
|7 − λ1 | ≤ |4| + |2|, 1 ≤ λ1 ≤ 13.
√
Actual eigenvalues are 4, 6 ± 17.
MATH3800 NOTES-ERIC HUA 75

11.1 Population Growth

Population Growth. To model populations, a popular assumption is: the
population growth is proportional to the size of the population:

dP dP
∝ P, or, = kP.
dt dt
By separating variables, we get the solution:

P (t) = P (t0 )ek(t−t0 ) .

If k is positive, then the population increases. If k is negative, then the pop-

ulation decreases.

Example 50. A bacteria culture growth at a rate proportional to its size. After 2
hours there are 40 bacteria and after 4 hours the count is 120. Find an expression
for the population after t hours.

Solution: We measure the time t in hours. Let P (t) be the population at

t hours, then we have
dP
= kP.
dt
The solution of the equation is

P (t) = P (0)ekt .

Note that P (2) = 40 and P (4) = 120, we obtain

40 = P (0)e2k , 120 = P (0)e4k .

These imply that

40
P (0) =
3
and
e2k = 3, or k = ln 3/2.
We thus have
40 t/2 40 √ t 40 (ln 3/2)t
P (t) = 3 = 3 = e .
3 3 3

Example 51. The half-life of Sodium-24 is 15 hours. Suppose you have 100
grams of Sodium-24. How many grams remaining after 27 minutes (keep three
decimals)?

Solution: Assume m(t) be the amount after t hours. Then

1
m(t) = m(0)( )t/H ,
2
76 MATH3800 NOTES-ERIC HUA

where m(0) = 100, H = 15 hours. Note that 27 minutes = 27/60=0.45 hours.

Thus
1 1
m(0.45) = 100( )0.45/15 = 100( )0.03 = 97.942g.
2 2

Limited Growth Model (logistic model). If the population is limited by

maximum M , we have a simple submodel
k = r(M − P ), r > 0,
where r is a constant. Then the equation becomes:
dP
= r(M − P )P.
dt
By separating variables,
P
ln = rM t + C.
M −P
M P (t0 )
P (t) = .
P (t0 ) + (M − P (t0 ))e−rM (t−t0 )
M M M − P (t0 ) rM t0
P (t) = = , d = e .
1 + [(M − P (t0 ))/P (t0 )]e−rM (t−t0 ) 1 + de−rM t P (t0 )
M
Let t∗ be the time such that P (t∗ ) = . Then
2
1 P0 C
t∗ = t0 − ln , or t∗ = − .
rM M − P0 rM
M
P (t) = −rM (t−t∗ )
.
1+e
MATH3800 NOTES-ERIC HUA 77
78 MATH3800 NOTES-ERIC HUA

Example 52. Yeast Culture: Given the data and scatterplot are as follows:

(1) Estimate M : M ≈ 665.

(2) Plot ln[P/(M − P )] against t.

MATH3800 NOTES-ERIC HUA 79

(3) Estimate r, t∗ .

Solution:

rM = 0.5307, r = 0.5307/665 = 0.000798045.

C −4.1636
t∗ = − =− = 7.845487093.
rM 0.5307
80 MATH3800 NOTES-ERIC HUA

11.4 Graphical Solutions of Autonomous Differ-

ential Equations
• y 0 = f (y) is called autonomous differential equation.

• The solutions of y 0 = f (y) = 0 are called equilibrium values (points)

or rest points.

• An equilibrium point is stable if solutions that begin near the equilibrium

approach the equilibrium; An equilibrium point is unstable if solutions
that begin near the equilibrium move away from the equilibrium.

• Phase line is the plot on the y axis with equilibrium values and intervals
where y 0 and y 00 are positive and negative.

Theorem 9. (Stability Theorem). Consider the autonomous equation

dy
= f (y)
dt
with an equilibrium y∗. If f 0 (y∗) < 0, then y∗ is stable; If f 0 (y∗) > 0, then y∗ is
unstable.

Example 53. Consider the equation

dy
= (y + 1)(y − 2).
dx
(1) Find the equilibrium values (points).
Solution: y = −1, 2.

(2) Identify intervals where y 0 and y 00 are positive and negative.

Solution:
y 0 = y 2 − y − 2,
y 00 = 2yy 0 − y 0 = (2y − 1)(y + 1)(y − 2).
MATH3800 NOTES-ERIC HUA 81

(3) Graphic solution curves.

(4) Study the stability of the equilibrium points.

Solution: y = −1 is stable, y = 2 is unstable.

Example 54. Newton’s Law of Cooling:

dT
= k(A − T ),
dt
where k is a positive constant, A is a constant, the temperature of the surrounding
area.
Use Newton’s Law of Cooling to determine the time of death of a healthy man.
• He died in his room some time before noon;
• At noon, his body temperature was found to be 70 degrees;
• His body cooled another 5 degrees in 1 hour after noon;
• The room temperature was a constant 60 degrees;
• Normal temperature of people’s body is 98.6 degree.

Solution: Let T (t) be the temperature of the body at time t. Taking noon
as t = 0, we have T (0) = 70. Note that Ts = 60, we have
dT
= −k(T − 60),
dt
this equation is separable. We obtain
dT
= −kdt, ⇒ T (t) = 60 + Ce−kt .
T − 60
From T (0) = 70 we get 70 = 60 + C, C = 10 and

T (t) = 60 + 10e−kt .
82 MATH3800 NOTES-ERIC HUA

At 1:00pm, his body temperature is 70 − 5 = 65. Hence T (1) = 65.

65 = 60 + 10e−k(1) , ⇒ 5 = 10e−k , ⇒ k = ln 2, ⇒ T (t) = 60 + 10e−t ln 2 .

Thus
98.6 = 60 + 10e−t ln 2 , ⇒ t = − ln 3.86/ ln 2 = −1.95,
which means 1 hour and 0.95(60) minutes before noon, or 1 hour and 57 minutes
before noon. Or the time of death is 10:03AM.

Example 55. Consider the logistic equation

dP
= r(M − P )P.
dt
Analyse the equilibria.

Solution: The equilibrium points are 0, M . Let

f (P ) = rP (M − P ).

Then f 0 (P ) = rM − 2rP = r(M − 2P ). f 0 (0) = rM > 0, f 0 (M ) = −rM < 0.

• If 0 < P < M , then dP/dt > 0 and thus, population grows (the point in
the graph moves to the right).

• If P < 0 or P > M (of course, N < 0 has no biological sense), then

population declines (the point in the graph moves to the left).

The arrows show that the equilibrium P = 0 is unstable, whereas the equilib-
rium P = M is stable. From the biological point of view, this means that after
small deviation of population numbers from P = 0 (e.g., immigration of a small
number of organisms), the population never returns back to this equilibrium.
Instead, population numbers increase until they reach the stable equilibrium
P = M . After any deviation from P = M the population returns back to this
stable equilibrium.
MATH3800 NOTES-ERIC HUA 83

12.1 Graphical Solutions of Autonomous Systems

of First-Order Differential Equations
Autonomous Systems of First-Order Differential Equations :
 dx
dt
= f (x, y)
dy
dt
= g(x, y).

• A solution is a pair x = x(t), y = y(t) satisfying the system.

• The solution curve whose coordinates are (x(t), y(t)) is called a trajectory,
path, orbit of the system.

• The xy-plane is referred to as phase plane.

• The point(x, y) satisfying both f (x, y) = 0 and g(x, y) = 0 is called equi-

librium point or rest point.

• An equilibrium point (x0 , y0 ) is

– stable if any trajectory that starts close to the point stays close to it
for all future time;
– asymptotically stable it it is stable and if any trajectory that starts
close to (x0 , y0 ) approaches that point as t → ∞;
– unstable if (x0 , y0 ) is not stable.

Properties:

1. There is at most one trajectory through any point in the phase plane.

2. A trajectory that starts at a point other than an equilibrium point can not
reach an equilibrium point in a finite amount of time.

3. No trajectory can cross itself unless it is a closed curve. If it is a closed

curve, it is a periodic solution.

4. A particle along a trajectory behaves in one of three possible ways: (a) the
particle approaches an equilibrium point; (b) the particle moves along or
approaches asymptotically a closed path; (c) either x(t) → ∞ or y(t) → ∞
as t → ∞.

Example 56. Consider the system:

 dx
dt
= −x + y
dy
dt
= −x − y.

(a) Verify that The solution of the system is x = e−t sin t, y = e−t cos t.
(b) Find the equilibrium points and study the stability of the system
84 MATH3800 NOTES-ERIC HUA

Solution:
(b) Solve the system 
−x + y = 0
−x − y = 0.
The equilibrium point is (x, y) = (0, 0).

Note that
x2 + y 2 = e−2t → 0 as t → ∞.

The equilibrium point is asymtotically stable.

MATH3800 NOTES-ERIC HUA 85

12.2 A Competitive Hunter Model

Consider fish species in Ontario Rideau Lake.

• Let x(t) be the pike-fish population.

• Let y(t) be the bass-fish population.

Then we have the following Competitive Hunter model:

dx/dt = (a − by)x,
dy/dt = (m − nx)y,

where a, b, m, n are positive constants.

Equilibrium points:

1. x0 (t) = 0 ⇔ y = ab .

2. y 0 (t) = 0 ⇔ x = m
n
.

Thus equilibria: (x, y) = (0, 0), ( m , a ) in the phase plane.

n b

m
• Along the vertical line x = and the x axis in the phase plane, the growth
n
in the bass population is zero;
a
• Along the horizontal line y = and the y axis in the phase plane, the
b
growth in the pike population is zero.

Graphical Analysis of the Model:

• Whenever x0 (t) > 0, the component x(t) of the trajectory is increasing and
the trajectory is moving toward the right; whenever x0 (t) < 0 is negative,
the trajectory is moving to the left.

• Whenever y 0 (t) > 0, the component y(t) of the trajectory is increasing and
the trajectory is moving upward; whenever y 0 (t) < 0 , the trajectory is
moving downward.
n a−m
• Along the diagonal y = x+ , the trajectories move toward the
m a
b0 0 b
equilibrium ( n , b ). Here y /x = y/x.
n a+m
• Along the diagonal y = − x + , the trajectories move away from
m a
b 0 0 b
the equilibrium ( n , b ). Here y /x = −y/x.
86 MATH3800 NOTES-ERIC HUA

Stability of equilibria: both (x, y) = (0, 0), ( m , a ) are unstable.

n b

Analytic Solution of the Model:

dy (m − nx)y
= ,
dx (a − by)x
 
a m 
− b dy = − n dx,
y x
a ln y − by = m ln x − nx + K.
MATH3800 NOTES-ERIC HUA 87

Example.

• Let x(t) be the pike-fish population.

• Let y(t) be the bass-fish population.

we have the following Competitive Hunter model:

x0 = dx/dt = (0.2 − 0.1y)x,

y 0 = dy/dt = (0.2 − 0.6x)y,

Equilibrium points:
Solve the system

(0.2 − 0.1y)x = 0
(0.2 − 0.6x)y = 0.

Thus equilibria: (x, y) = (0, 0), ( 31 , 2) in the phase plane.

Graphical Analysis of the Model:

1
• Along the vertical line x = and the x axis in the phase plane, y 0 = 0, the
3
bass population has no change;

1
• On the left of the vertical line x = , y 0 > 0, the bass population increases;
3
1
• On the right of the vertical line x = , y 0 < 0, the bass population de-
3
creases;

• Along the horizontal line y = 2 and the y axis in the phase plane, x0 = 0,
thus the pike population has no change.

• Above the horizontal line y = 2, x0 < 0, the pike population decreases.

88 MATH3800 NOTES-ERIC HUA

• Below the horizontal line y = 2, x0 > 0, the pike population increases.

• Along the diagonal y = 6x, the trajectories move toward the equilibrium
(1/3, 2), for example, at the point (1/6, 1), x0 = 0.1/6, y 0 = 0.1.
• Along the diagonal y = −6x + 4, the trajectories move away from the
equilibrium (1/3, 2) .

Stability of equilibria: both (x, y) = (0, 0), (1/3, 2) are unstable.

Can two species be mutually co-existent?

Analytic Solution of the Model:

MATH3800 NOTES-ERIC HUA 89

dy (0.2 − 0.6x)y
= ,
dx (0.2 − 0.1y)x
0.2 ln y − 0.1y = 0.2 ln x − 0.6x + c, 2 ln y − y = 2 ln x − 6x + c.
90 MATH3800 NOTES-ERIC HUA

12.3 A Predator-Prey Model

Consider fish species in Ontario Rideau Lake. It’s reasonable to assume that:

• The predator species (such as pike, walleye, bass, muskie) is totally depen-
dent on the prey species as their only food supply.

• The prey species (such as sunfish, perch) has an unlimited food supply and
no threat to their growth other than the specific predator.

Lotka-Volterra Model . The mass action approach to modelling trophic inter-

actions was pioneered, independently, by the American physical chemist Lotka
(1925) and Italian mathematician Volterra (1926). These authors argued that
consumer and resource populations could be treated like particles interacting in
a homogeneously mixed gas or liquid.

Principle of Mass Action: The (reaction) rate of encounter between con-

sumers and resources would be proportional to the product of their masses.

• x(t) = resource population (bacteria, prey)n.

• y(t) = consumer population (amoebas, predator) .

The dynamics of the interaction between them is then described by the differen-
tial equations (system):

dx/dt = (a − by)x,
dy/dt = (nx − m)y, n = cb,

where a, b, m, n are positive constants, a represents the per-capita rate of change

of the resource in the absence of consumers, b the consumption rate of the con-
sumer, c the constant of conversion of resources into consumer offspring, and
m the per-capita mortality rate of consumers in the absence of resources. The
terms in the equations have the following meaning:

1. ax = the growth rate of the resource population in the absence of predators.

Thus, in the absence of predators, the resource population grows according
to the equation dx/dt = ax, i.e., x(t) = x(0)eat , exponential growth.

2. byx = the rate of consumption of resources, or their death rate due to

attack by predators.

3. cbxy = the rate of production of predator offspring, which is directly related

to the number of prey consumed.

4. my = the death rate of consumers in the absence of food (prey). Thus,

in the absence of prey, predators die according to dy/dt = −my, i.e.,
y(t) = y(0)e−mt , exponential decay.
MATH3800 NOTES-ERIC HUA 91

Equilibrium points: (x, y) = (0, 0), ( m , a ).

n b

Graphical Analysis of the Model:

1. Find the nullcline of x, i.e., the set of points where dx/dt = 0, or equiva-
lently f (x, y) = 0.

2. Find the nullcline of y, i.e., the set of points where dy/dt = 0, or equiva-
lently g(x, y) = 0.

3. Draw these two sets in the phase plane: xy-plane.

4. The intersection of the nullclines are equilibrium points (or steady states,
or rest points).

5. On each of the nullclines, draw the direction arrows.

6. In each of the regions in space in between the nullclines, draw the direction
arrows.

Stability of equilibria: The equilibrium (x, y) = (0, 0) is unstable.

Sketch trajectories (solution curves):

For a given initial condition, draw a solution into the phase plane. Then plot
the two components of the solution as a function of time.

Analytic Solution of the Model:

dy (nx − m)y
= ,
dx (a − by)x
 
a  m
− b dy = n − dx,
y x
92 MATH3800 NOTES-ERIC HUA

a ln y − by = nx − m ln x + K.

Example 57. Find the equilibrium points, study stabilities, and sketch trajecto-
ries (solution curves).

dx/dt = x(0.33 − 0.2y),

dy/dt = y(0.1x − 0.5).
MATH3800 NOTES-ERIC HUA 93

11.5 Numerical Approximation Methods

Euler’s method: Consider the initial value problem
dy
= f (x, y), y(x0 ) = y0 , x0 ≤ x ≤ b.
dx
If h = ∆x is the step size, then

yn+1 = yn + hf (xn , yn ), xn+1 = xn + h.

Example 58. Let y(x) be the solution of the initial value problem y 0 = x + y,
y(0) = 0.

(a) Use Euler’s method with step size 0.1 to estimate y(0.4);
(b) Verify that the exact solution is y = −x − 1 + ex .
(c) Calculate the error in (a).

Solution:
(a)
i xi yi
0 0 0
1 0.1 0
2 0.2 0.01
3 0.3 0.031
4 0.4 0.0641

(b) The actual value at 0.4 is y(0.4) = 0.091825.

(c) Absolute Error=|0.091825 − 0.0641| = 0.027725.

Remark. It is not true that by taking h sufficiently small one can obtain any
desired level of precision.
94 MATH3800 NOTES-ERIC HUA

Euler’s Method to a system:

dx/dt = f (x, y),

dy/dt = g(x, y).

Let h = ∆t. 
 tn+1 = tn + h,
xn+1 = xn + hf (xn , yn ),
yn+1 = yn + hg(xn , yn ).


Example 59. Consider the system

dx/dt = (3 − y)x,
dy/dt = (x − 2)y,

with t0 = 0, h = 0.1, x0 = 1, y0 = 2. Find t3 , x3 , y3 .

i ti xi yi
0 0 1 2
1 0.1 1.1 1.8
2 0.2 1.232 1.638
3 0.3 1.3997984 1.5122016
MATH3800 NOTES-ERIC HUA 95

Improved Euler’s Method. Use Euler’s method to get ’predicted’ value (pre-
dictor), then use improved formula to get ’corrected’ value (corrector):
p
yn+1 = ync + hf (xn , ync )
c 1  p
= ync + h f (xn , ync ) + f (xn+1 , yn+1

yn+1 )
2
Example 60. Let y(x) be the solution of the initial value problem y 0 = x + y,
y(0) = 0. Use step size 0.1 to estimate y(0.4) and calculate the error.

Solution:
i xi yip yic
0 0 0 0
1 0.1 0 0.005
2 0.2 0.0155 0.021025
3 0.3 0.043128 0.049233
4 0.4 0.084156 0.090902

The actual value at 0.4 is y(0.4) = 0.091825.

Absolute Error=|0.091825 − 0.090902| = 0.000923.
96 MATH3800 NOTES-ERIC HUA

Runge-Kutta Method

Euler’s Method gave us one possible approach for solving differential equations
numerically. The problem with Euler’s Method is that you have to use a small
interval size to get a reasonably accurate result. That is, it’s not very efficient.
The Runge-Kutta Method of 4th order (RK4), Multi-step Predictor-corrector
Method , produces a better result in fewer steps.
We wish to approximate the solution to a first order differential equation
given by

dy
= f (x, y), y(xn ) = yn .
dx

We will use the following slope approximations to estimate yn+1 = y(xn + h) at

xn+1 = xn + h with step size h.

k1 = hf (xn , yn )
1 1
k2 = hf (xn + h, yn + k1 )
2 2
1 1
k3 = hf (xn + h, yn + k2 )
2 2
k4 = hf (xn + h, yn + k3 )

1
yn+1 = yn + (k1 + 2k2 + 2k3 + k4 ).
6

Each of these estimates can be described verbally.

• k1 is the increment based on the slope at the beginning of the interval,

using y (Euler’s method);

• k2 is the increment based on the slope at the midpoint of the interval, using
y and k1 ;

• k3 is the increment based on the slope at the midpoint of the interval, using
y and k2 ;

• k4 is the increment based on the slope at the end of the interval, using y
and k3 .
MATH3800 NOTES-ERIC HUA 97

Example 61. Use Runge-Kutta method of order 4 to solve the following equa-
tion, using the step size h = 0.1 for 0 ≤ x ≤ 1:
dy
= (5x2 − y)e−(x+y) , y(0) = 1.
dx

Solution:
Step 1:
k1 = hf (x0 , y0 ) = 0.1f (0, 1) = −0.03678794411.
1 1
k2 = hf (x0 + h, y0 + k1 ) = 0.1f (0.05, 0.98160602794) = −0.03454223937.
2 2
1 1
k3 = hf (x0 + h, y0 + k2 ) = 0.1f (0.05, 0.98272888031) = −0.03454345267.
2 2
k4 = hf (x0 + h, y0 + k3 ) = 0.1f (0.1, 0.96545654732) = −0.03154393258.
We take those 4 values and substitute them into the Runge-Kutta RK4 formula:
1
y1 = y(x0 + h) = y(0.1) = y(0) + (k1 + 2k2 + 2k3 + k4 )
6
1
= 1+ (−0.03678794411−2×0.03454223937−2×0.03454223937−0.03154393258)
6
98 MATH3800 NOTES-ERIC HUA

= 0.9655827899
Steps 2-10: Repeat Step 1 with initial values (x1 , y1 ) = (0.1, 0.9655827899).

Example 62. Use Runge-Kutta method of order 4 to solve the following equa-
tion, using the step size h = 0.5 for 0 ≤ t ≤ 2:

dy
= y − t2 + 1, y(0) = 0.5.
dt
Given that the exact solution is
1
y(t) = t2 + 2t + 1 − et ,
2
estimate the error.

Solution:
Step 1:
k1 = hf (t0 , y0 ) = 0.5f (0, 0.5).
1 1
k2 = hf (t0 + h, y0 + k1 ) = 0.5f (0.25, 0.5 + k1 /2).
2 2
1 1
k3 = hf (t0 + h, y0 + k2 ) = 0.5f (0.25, 0.5 + k2 /2).
2 2
k4 = hf (t0 + h, y0 + k3 ) = 0.5f (0.5, 0.5 + k3 ).
We take those 4 values and substitute them into the Runge-Kutta RK4 formula:

1
y1 = y(t0 + h) = y(0.5) = y(0) + (k1 + 2k2 + 2k3 + k4 )
6
Steps 2-4: Repeat Step 1 with initial values (t1 , y1 ).
MATH3800 NOTES-ERIC HUA 99

tn Exact solution y(tn ) Numerical solution yn Error |y(tn ) − yn |

0.0 0.5 0.5 0
0.5 1.425639364649936 1.425130208333333 0.000509156316603
1.0 2.640859085770477 2.639602661132812 0.001256424637665
1.5 4.009155464830968 4.006818970044454 0.002336494786515
2.0 5.305471950534675 5.301605229265987 0.003866721268688

If we take step size h = 0.2, then the solution is:

tn Exact solution y(tn ) Numerical solution yn Error |y(tn ) − yn |

0.0 0.5 0.5 0
0.2 0.829298620919915 0.829293333333333 0.000005287586582
0.4 1.214087651179365 1.214076210666667 0.000011440512698
0.6 1.648940599804746 1.648922017041600 0.000018582763146
0.8 2.127229535753766 2.127202684947944 0.000026850805823
1.0 2.640859085770477 2.640822692728752 0.000036393041726
1.2 3.179941538631726 3.179894170232231 0.000047368399496
1.4 3.732400016577663 3.732340072854980 0.000059943722683
1.6 4.283483787802442 4.283409498318406 0.000074289484036
1.8 4.815176267793527 4.815085694579435 0.000090573214092
2.0 5.305471950534674 5.305363000692655 0.000108949842019

All this can be done by using Matlab:

dy
Fourth-order Runge-Kutta Method proof: = f (x, y):
dx

dy
= f (x, y), y 0 (xn ) = f (xn , yn ).
dx

y 00 = fx (x, y) + fy (x, y)y 0 , y 00 (xn ) = fx (xn , yn ) + fy (xn , yn )f (xn , yn ).

y 000 = ...... y (4) = ......

100 MATH3800 NOTES-ERIC HUA

By Taylor’s formula,

y 00 (xn ) 2 y 000 (xn ) 3 y (4) (xn ) 4

y(xn+1 ) = y(xn )+y 0 (xn )h+ h+ h+ h +O(h5 ), h = xn+1 −xn .
2! 3! 4!
dy
General Runge-Kutta Method for = f (x, y):
dx

• 1st order RK method, Euler’s method: y(xn+1 ) = y(xn )+k1 , k1 = hf (xn , yn ).

• 2nd order RK method, Heun’s method:

1
y(xn+1 ) = y(xn ) + (k1 + k2 ), k1 = hf (xn , yn ), k2 = hf (xn + h, yn + k1 )).
2
• Generally,
s
X
yn+1 = yn + bi ki ,
i=1

k1 = hf (xn , yn ),
k2 = hf (xn + c2 h, yn + a21 k1 ), !
i−1
X
ki = hf xn + ci h, yn + aij kj ,
j=1
s−1
!
X
ks = hf xn + ci h, yn + aij kj ,
j=1

i−1
X
aij = ci for i = 2, . . . , s
j=1

c1 a11 a12 . . . a1s

c2 a21 a22 . . . a2s
.. .. .. . . . ..
. . . .
cs as1 as2 . . . ass

b1 b2 ... bs

• The Euler method is first order.

0 0
1

• Heun’s method is a second-order method with two stages. It is also known

as the explicit trapezoid rule, improved Euler’s method, or modified Euler’s
method.
0 0 0
1 1 0
1/2 1/2
MATH3800 NOTES-ERIC HUA 101

• Kutta’s third-order method

0 0 0 0
1/2 1/2 0 0
1 −1 2 0
1/6 2/3 1/6

• Classic fourth-order method: The ”original” RungeKutta method.

0 0 0 0 0
1/2 1/2 0 0 0
1/2 0 1/2 0 0
1 0 0 1 0
1/6 1/3 1/3 1/6