Calculus and Applications - Part II

Vahid Shahrezaei

2021-03-24
2
Contents

Welcome 5

Part I: Fourier Transforms 9

1 Fourier Transforms 9
1.1 Fourier’s integral formula . . . . . . . . . . . . . . . . . . . . . . 9
1.2 Fourier cosine and sine transforms . . . . . . . . . . . . . . . . . 13

2 Properties of Fourier Transforms 15

2.1 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Convolution theorem for Fourier transforms . . . . . . . . . . . . 18
2.3 Energy theorem for Fourier transforms . . . . . . . . . . . . . . . 19
2.4 The Dirac delta-function . . . . . . . . . . . . . . . . . . . . . . . 20

Part II: Ordinary Differential Equations 25

3 Introduction to ordinary differential equations 25
3.1 Particular and General Solutions . . . . . . . . . . . . . . . . . . 27

4 First and second order ODEs 29

4.1 First order ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.2 Second Order ODEs . . . . . . . . . . . . . . . . . . . . . . . . . 31

5 Linear ODEs 35
5.1 General solution of the non-homogeneous linear ODE . . . . . . 37
5.2 First order linear ODEs with constant coefficients . . . . . . . . . 39
5.3 Second order linear ODEs with constant coefficients . . . . . . . 41
5.4 𝑘th order Linear ODEs with constant coefficients . . . . . . . . . 45
5.5 Euler-Cauchy equation . . . . . . . . . . . . . . . . . . . . . . . . 46
5.6 Using Fourier Transforms to solve linear ODEs . . . . . . . . . . 47

6 Introduction to Systems of ODEs 49

3
4 CONTENTS

6.1 Systems of first order ODEs . . . . . . . . . . . . . . . . . . . . . 51

6.2 Systems of linear 1st order ODEs with constant coefficients . . . 52

7 Qualitative analysis of ODEs 61

7.1 asymptotic behaviour . . . . . . . . . . . . . . . . . . . . . . . . 61
7.2 Phase plane analysis . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.3 General system of linear ODEs in 2 dimension . . . . . . . . . . . 66
7.4 Phase plane analysis for 2D nonlinear systems . . . . . . . . . . . 75
7.5 Extension of phase plane analysis to higher dimensional systems 77

8 Introduction to Bifurcations 79
8.1 Bifurcations in linear systems . . . . . . . . . . . . . . . . . . . . 79
8.2 Qualitative behavior of non-linear 1D systems . . . . . . . . . . . 80

Part III: Introduction to Multivariate Calculus 93

9 Partial Differentiation 93
9.1 Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
9.2 Limit and continuity . . . . . . . . . . . . . . . . . . . . . . . . . 95
9.3 Partial and Total Differentiation . . . . . . . . . . . . . . . . . . 95
9.4 Total differentiation of a function of several variables . . . . . . . 97
9.5 Chain rule for functions of several variables . . . . . . . . . . . . 97
9.6 Implicit functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
9.7 Taylor expansion of multivariate functions . . . . . . . . . . . . . 100

10 Applications of Multivariate Calculus 103

10.1 Change of Coordinates . . . . . . . . . . . . . . . . . . . . . . . . 103
10.2 Partial Differential Equations (PDEs) . . . . . . . . . . . . . . . 106
10.3 Exact ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
10.4 Sketching functions of two variables . . . . . . . . . . . . . . . . 110
Welcome

These are lecture notes for the second part of Calculus and Applications first year
module at the Department of Mathematics, Imperial College London. The notes
are split into three parts on Fourier Transform, Ordinary Differential Equations
and Introduction to Multivariate Calculus. Please refer to course Blackboard
for additional materials recommended text books for further reading.
These lecture notes are adobted from existing courses in our department. Part
I of the course is based on the old M2AA2 course (Andrew Walton) and Part II
and III are based on the old M1M2 course (Frank Berkshire, Mauricio Barahona,
Andrew Parry). Some examples and ideas from the old mechanics course M1A1
is included as well.
These notes are produced with accessibility in mind and I hope it meets your
requirements. You can experiment with the controls in the toolbar at the top of
the html version of the notes. You can search for a word, adjust typeface, font
size, font and background color. You can also download a copy of these notes
in differnt formats, if you wish for offline use. I hope you enjoy this course and
let me know if you have any comments or questions by email.
© Vahid Shahrezaei (2021) These notes are provided for the personal study of
students taking this module. The distribution of copies in part or whole is not
permitted.

5
6 CONTENTS
Part I: Fourier Transforms

7
Chapter 1

Fourier Transforms

Last term, we saw that Fourier series allows us to represent a given function,
defined over a finite range of the independent variable, in terms of sine and co-
sine waves of different amplitudes and frequencies. Fourier Transforms are the
natural extension of Fourier series for functions defined over ℝ. A key reason for
studying Fourier transforms (and series) is that we can use these ideas to help
us solve differential equations as seen in this course regarding ordinary differen-
tial equations and more extensively next year in relation to partial differential
equations. There are also many other applications for Fourier transforms in
science and engineering, particularly in the context of signal processing.

1.1 Fourier’s integral formula

We can represent a function 𝑓(𝑥) defined over the interval [−𝐿, 𝐿] using the
Fourier series

∞
1 𝑛𝜋𝑥 𝑛𝜋𝑥
𝑓(𝑥) = 𝑎0 + ∑{𝑎𝑛 cos ( ) + 𝑏𝑛 sin ( )}.
2 𝑛=1
𝐿 𝐿
where the corresponding Fourier coefficients are given by
𝐿
1 𝑛𝜋𝑥
𝑎𝑛 = ∫ 𝑓(𝑥) cos ( ) 𝑑𝑥, 𝑛 = 0, 1, 2, ⋯ ,
𝐿 −𝐿 𝐿
𝐿
1 𝑛𝜋𝑥
𝑏𝑛 = ∫ 𝑓(𝑥) sin ( ) 𝑑𝑥, 𝑛 = 1, 2, ⋯ .
𝐿 −𝐿 𝐿
Expressed in the exponential form the Fourier series can be represented as
∞
𝑓(𝑥) = ∑ 𝑐𝑛 𝑒𝑖𝑛𝜋𝑥/𝐿 , |𝑥| < 𝐿,
𝑛=−∞

9
10 CHAPTER 1. FOURIER TRANSFORMS

𝐿
1
𝑐𝑛 = ∫ 𝑓(𝑥)𝑒−𝑖𝑛𝜋𝑥/𝐿 𝑑𝑥, 𝑛 = 0, ±1, ±2, ⋯ .
2𝐿 −𝐿

By defining angular frequency as 𝜔𝑛 = 𝑛𝜋/𝐿 and frequency difference as

𝛿𝜔 = 𝜔𝑛+1 − 𝜔𝑛 ,

we can rewrite the Fourier series in the new notation as

𝐿
1 ∞
𝑓(𝑥) = ∑ [∫ 𝑓(𝑠)𝑒−𝑖𝜔𝑛 𝑠 𝑑𝑠] 𝑒−𝑖𝜔𝑛 𝑥 𝛿𝜔.
2𝜋 𝑛=−∞ −𝐿

This result can be extended for a function 𝑓(𝑥) defined on ℝ by taking the limit
of 𝐿 → ∞ from the Fourier series. Using the angular frequency notation from
above and replacing sum with integral using the Riemann sum, noting that
𝛿𝜔 → 0 as 𝐿 → ∞ , we obtain
∞ ∞
1
𝑓(𝑥) = ∫ {∫ 𝑓(𝑠)𝑒−𝑖𝜔𝑠 𝑑𝑠} 𝑒𝑖𝜔𝑥 𝑑𝜔.
2𝜋 −∞ −∞

We therefore have shown that for a function 𝑓(𝑥) defined over −∞ < 𝑥 < ∞
we have the following
∞
1 ̂ 𝑖𝜔𝑥
𝑓(𝑥) = ∫ 𝑓(𝜔)𝑒 𝑑𝜔,
2𝜋 −∞

where
∞
̂
𝑓(𝜔) =∫ 𝑓(𝑥)𝑒−𝑖𝜔𝑥 𝑑𝑥.
−∞

̂
The function 𝑓(𝜔) (also denoted as ℱ{𝑓(𝑥)}) is known as Fourier transform
of 𝑓(𝑥), which is analogous to the Fourier coefficients in a Fourier series. The
̂
relation above between 𝑓(𝑥) and 𝑓(𝜔) is also known as inverse Fourier transform.
Note that some books use slightly different definitions of Fourier transform with
different normalisation. In order to evaluate the integrals above, a necessary
condition is that 𝑓(𝑥) and its transform decay at ±∞. Using the Dirac delta
function this restriction can be overcome as seen later.
Proof of Fourier’s integral formula
In the previous section in a non-rigorous way we arrived at the result

∞ ∞
1
𝑓(𝑥) = ∫ {∫ 𝑓(𝑠)𝑒−𝑖𝜔𝑠 𝑑𝑠} 𝑒𝑖𝜔𝑥 𝑑𝜔.
2𝜋 −∞ −∞

To prove this more formally, we need to assume that 𝑓(𝑥) is such that
1.1. FOURIER’S INTEGRAL FORMULA 11

∞
∫ |𝑓(𝑥)| 𝑑𝑥
−∞

converges. We will also assume that 𝑓(𝑥) and 𝑓 ′ (𝑥) are continuous for all 𝑥 (this
can be relaxed as discussed at the end). We start by writing the RHS above in
the form
𝐿 ∞
1
lim ∫ {∫ 𝑓(𝑠)𝑒−𝑖𝜔(𝑠−𝑥) 𝑑𝑠} 𝑑𝜔 =
𝐿→∞ 2𝜋
−𝐿 −∞
𝐿 ∞ ∞
1
lim ∫ {∫ 𝑓(𝑠) cos [𝜔(𝑠 − 𝑥)] 𝑑𝑠 − 𝑖 ∫ 𝑓(𝑠) sin [𝜔(𝑠 − 𝑥)] 𝑑𝑠} 𝑑𝜔.
𝐿→∞ 2𝜋
−𝐿 −∞ −∞

The first integral in curly brackets is even about 𝜔 = 0, while the second is
odd. Also, because of the absolute convergence of the inner integral, we can
interchange the order of integration. Therefore, the expression simplifies to

𝐿 ∞
1
lim∫ {∫ 𝑓(𝑠) cos [𝜔(𝑠 − 𝑥)] 𝑑𝑠} 𝑑𝜔
𝐿→∞ 𝜋
0 −∞
∞ 𝐿
1
= lim ∫ 𝑓(𝑠) {∫ cos [𝜔(𝑠 − 𝑥)] 𝑑𝜔} 𝑑𝑠
𝐿→∞ 𝜋
−∞ 0
∞
1 sin [𝐿(𝑠 − 𝑥)]
= lim ∫ 𝑓(𝑠) 𝑑𝑠
𝐿→∞ 𝜋 𝑠−𝑥
−∞
∞
1 sin (𝐿𝑢)
= lim ∫ 𝑓(𝑥 + 𝑢) 𝑑𝑢,
𝐿→∞ 𝜋 𝑢
−∞

using the substitution 𝑢 = 𝑠 − 𝑥. We now split the integral into two parts in
the following form

∞ ∞
1 𝑓(𝑥 + 𝑢) − 𝑓(𝑥) sin (𝐿𝑢)
lim {∫ sin (𝐿𝑢) 𝑑𝑢 + 𝑓(𝑥) ∫ 𝑑𝑢} .
𝐿→∞ 𝜋 𝑢 𝑢
−∞ −∞

The first integral tends to zero as 𝐿 → 0 using Riemann-Lebesgue Lemma (seen

last term in this course). We then use the substitution 𝑝 = 𝐿𝑢 in the second
integral to leave
∞
𝑓(𝑥) sin (𝑝)
lim ∫ 𝑑𝑝 = 𝑓(𝑥),
𝐿→∞ 𝜋 𝑝
−∞
∞
using the fact that ∫−∞ (sin 𝑝)/𝑝 𝑑𝑝 = 𝜋 (seen last term and also in the problem
sheet 1 this term).
We have therefore proved Fourier’s integral formula. As remarked earlier, we
have assumed here that 𝑓(𝑥) is continuous at all 𝑥. If there is a discontinuity at
12 CHAPTER 1. FOURIER TRANSFORMS

𝑥0 (with finite left and right hand derivatives there), the LHS of the formula is
replaced by [𝑓(𝑥0 +) + 𝑓(𝑥0 −)]/2 (analogous to the Fourier series convergence
we investigated earlier).

Example 1.1. Find the Fourier transform of the rectangular wave

1, if |𝑥| < 𝑑,
𝑓(𝑥) = {
0, if |𝑥| > 𝑑.

Using the Fourier transform formula we have

𝑑 𝑑
̂ 𝑒−𝑖𝜔𝑥 1 2
𝑓(𝜔) = ∫ 1.𝑒−𝑖𝑤𝑥 𝑑𝑥 = [ ] = − (𝑒−𝑖𝜔𝑑 − 𝑒𝑖𝜔𝑑 ) = sin 𝜔𝑑.
−𝑑 −𝑖𝜔 −𝑑 𝑖𝜔 𝜔

̂
See Figure 1.1 for a graph of 𝑓(𝜔) for different values of 𝑑. Note that as 𝑑 gets
larger, 𝑓 ̂ becomes more concentrated in the vicinity of 𝜔 = 0. This is a general
property of Fourier transforms and its inverse and relates to uncertainty princi-
ple. A function which is more localised around zero has a wider inverse Fourier
transform. See unseen question 1 for the derivation and further discussion of
the uncertainty principle.
8
6
4
2
0
−2

−6 −4 −2 0 2 4 6

Figure 1.1: Graph of the Fourier transform for 𝑑 = 1 (green) and 𝑑 = 4 (red)
1.2. FOURIER COSINE AND SINE TRANSFORMS 13

1.2 Fourier cosine and sine transforms

We can exploit the symmetry to define transforms over the range [0, ∞). First,
if we suppose that 𝑓(𝑥) is even about 𝑥 = 0, we have
∞ ∞
̂
𝑓(𝜔) =∫ 𝑓(𝑥)𝑒−𝑖𝜔𝑥 𝑑𝑥 = ∫ 𝑓(𝑥)(cos 𝜔𝑥 − 𝑖 sin 𝜔𝑥) 𝑑𝑥
−∞ −∞
∞
= 2∫ 𝑓(𝑥) cos 𝜔𝑥 𝑑𝑥.
0

We define Fourier cosine transform of 𝑓(𝑥) to be

∞
𝑓𝑐̂ (𝜔) = ∫ 𝑓(𝑥) cos 𝜔𝑥 𝑑𝑥.
0

̂
Thus, for an even function 𝑓(𝑥) we have 𝑓(𝜔) = 2𝑓𝑐̂ (𝜔).
Using the inversion formula for the regular transform and exploiting the evenness
of 𝑓𝑐̂ (𝜔), we can obtain the inversion formula for the Fourier cosine transform:
∞ ∞
1 ̂ 2
𝑓(𝑥) = ∫ 𝑓(𝜔)𝑒𝑖𝜔𝑥
𝑑𝜔 = ∫ 𝑓𝑐̂ (𝜔) cos 𝜔𝑥 𝑑𝜔.
2𝜋 −∞ 𝜋 0

In a similar way, by considering 𝑓(𝑥) to be odd about 𝑥 = 0, we can define

a Fourier sine transform and derive the corresponding inversion formula. We
obtain the pair of expressions:
∞
𝑓𝑠̂ (𝜔) = ∫ 𝑓(𝑥) sin 𝜔𝑥 𝑑𝑥
0
∞
2
𝑓(𝑥) = ∫ 𝑓𝑠̂ (𝜔) sin 𝜔𝑥 𝑑𝑤.
𝜋 0

̂
For an odd function 𝑓(𝑥), we have 𝑓(𝜔) = −2𝑖𝑓𝑠̂ (𝜔).

Example 1.2. Find the Fourier cosine transform of the rectangular wave

1, if |𝑥| < 𝑑,
𝑓(𝑥) = {
0, if |𝑥| > 𝑑.

We can use the definition of Fourier cosine transform directly noting that the
function is even. But also, as we have already obtained the Fourier transform
of this function in the last example, we simply have:
1 ̂ 1
𝑓𝑐̂ (𝜔) = 𝑓(𝜔) = sin 𝜔𝑑.
2 𝜔
14 CHAPTER 1. FOURIER TRANSFORMS
Chapter 2

Properties of Fourier
Transforms

In the following we present some important properties of Fourier transforms.

These results will be helpful in deriving Fourier and inverse Fourier transform
of different functions. After discussing some basic properties, we will discuss,
convolution theorem and energy theorem. Finally, we introduc Dirac delta
function.

2.1 Basic Properties

• (i) The Fourier and inverse Fourier transforms are linear, and so
̂ + 𝑏𝑔(𝜔),
ℱ{𝑎𝑓(𝑥) + 𝑏𝑔(𝑥)} = 𝑎𝑓(𝜔) ̂
̂ + 𝑏𝑔(𝜔)}
ℱ−1 {𝑎𝑓(𝜔) ̂ = 𝑎𝑓(𝑥) + 𝑏𝑔(𝑥),
where 𝑎 and 𝑏 are constants and ℱ−1 denotes the inverse Fourier trans-
form.
-(ii) If 𝑎 > 0:

1 ̂𝜔
ℱ{𝑓(𝑎𝑥)} =
𝑓( ).
𝑎 𝑎
Proof Starting on the LHS, and making the substitution 𝑠 = 𝑎𝑥:
∞ ∞
1 1 𝜔
ℱ{𝑓(𝑎𝑥)} = ∫ 𝑓(𝑎𝑥)𝑒−𝑖𝜔𝑥 𝑑𝑥 = ∫ 𝑓(𝑠)𝑒−𝑖(𝜔/𝑎)𝑠 𝑑𝑠 = 𝑓(̂ ).
−∞ 𝑎 −∞ 𝑎 𝑎

-(iii) In a similar way we can establish that

̂
ℱ{𝑓(−𝑥)} = 𝑓(−𝜔).

15
16 CHAPTER 2. PROPERTIES OF FOURIER TRANSFORMS

-(iv) The transform of a shifted function can be calculated as follows (using

𝑠 = 𝑥 − 𝑥0 ):
∞ ∞
ℱ{𝑓(𝑥 − 𝑥0 )} = ∫ 𝑓(𝑥 − 𝑥0 )𝑒−𝑖𝜔𝑥 𝑑𝑥 = ∫ ̂
𝑓(𝑠)𝑒−𝑖𝜔(𝑠+𝑥0 ) 𝑑𝑠 = 𝑒−𝑖𝜔𝑥0 𝑓(𝜔).
−∞ −∞

-(v) A similar result, but this time involving a shift in transform space:
∞
ℱ{𝑒𝑖𝜔0 𝑥 𝑓(𝑥)} = ∫ ̂ − 𝜔 ).
𝑓(𝑥)𝑒−𝑖(𝜔−𝜔0 )𝑥 𝑑𝑥 = 𝑓(𝜔 0
−∞

-(vi) Symmetry formula The following result is very useful. Suppose the Fourier
̂
transform of 𝑓(𝑥) is 𝑓(𝜔); change the variable 𝜔 to 𝑥; then

̂
ℱ{𝑓(𝑥)} = 2𝜋𝑓(−𝜔).
Proof Starting with the inversion formula and changing variables from 𝜔 to 𝑠,
we have ∞ ∞
1 ̂ 𝑖𝜔𝑥 1 ̂ 𝑖𝑠𝑥
𝑓(𝑥) = ∫ 𝑓(𝜔)𝑒 𝑑𝜔 = ∫ 𝑓(𝑠)𝑒 𝑑𝑠.
2𝜋 −∞ 2𝜋 −∞
If we now let 𝑥 = −𝜔 and then 𝑠 = 𝑥, we get:
∞ ∞
1 ̂ −𝑖𝜔𝑠 1 ̂ −𝑖𝜔𝑥 1 ̂
𝑓(−𝜔) = ∫ 𝑓(𝑠)𝑒 𝑑𝑠 = ∫ 𝑓(𝑥)𝑒 𝑑𝑥 = ℱ{𝑓(𝑥)},
2𝜋 −∞ 2𝜋 −∞ 2𝜋
as required.
The following results are particularly useful when applying Fourier transforms
to differential equations (as seen later this term and next year in the context of
partial differential equations).
-(vii)
𝑑𝑛 𝑓 ̂
ℱ{ } = (𝑖𝜔)𝑛 𝑓(𝜔).
𝑑𝑥𝑛

Proof This can be established by integration by parts. We assume that all

derivatives of 𝑓 tend to zero as 𝑥 → ±∞.

∞
ℱ{𝑑𝑛 𝑓/𝑑𝑥𝑛 } = ∫ (𝑑𝑛 𝑓/𝑑𝑥𝑛 )𝑒−𝑖𝜔𝑥 𝑑𝑥
−∞
∞
∞
= [(𝑑𝑛−1 𝑓/𝑑𝑥𝑛−1 )𝑒−𝑖𝜔𝑥 ]−∞ + 𝑖𝜔 ∫ (𝑑𝑛−1 𝑓/𝑑𝑥𝑛−1 )𝑒−𝑖𝜔𝑥 𝑑𝑥
−∞
= 𝑖𝜔ℱ{𝑑𝑛−1 𝑓/𝑑𝑥𝑛−1 }
=⋯
̂
= (𝑖𝜔)𝑛 𝑓(𝜔).
2.1. BASIC PROPERTIES 17

-(viii)

ℱ{𝑥𝑓(𝑥)} = 𝑖𝑓 ′̂ (𝜔).

Proof Considering the LHS:

∞ ∞
𝑑
∫ 𝑓(𝑥)𝑥𝑒−𝑖𝜔𝑥 𝑑𝑥 = ∫ 𝑓(𝑥) (𝑖𝑒−𝑖𝜔𝑥 ) 𝑑𝑥
−∞ −∞ 𝑑𝜔
∞
𝑑
=𝑖 ∫ 𝑓(𝑥)𝑒−𝑖𝜔𝑥 𝑑𝑥
𝑑𝜔 −∞
𝑑 ̂
= 𝑖 𝑓(𝜔).
𝑑𝜔

-(ix)

(a) ℱ𝑐 {𝑓 ′ (𝑥)} = −𝑓(0) + 𝜔𝑓𝑠̂ (𝜔),

(b) ℱ {𝑓 ′ (𝑥)} = −𝜔𝑓 ̂ (𝜔),
𝑠 𝑐

(c) ℱ𝑐 {𝑓 (𝑥)} = −𝑓 (0) − 𝜔2 𝑓𝑐̂ (𝜔),

″ ′

(d) ℱ {𝑓 ″ (𝑥)} = 𝜔𝑓(0) − 𝜔2 𝑓 ̂ (𝜔).

𝑠 𝑠

Proof We prove (a) and (c) and leave the others as exercises. For (a) we have,
integrating by parts:
∞
ℱ𝑐 {𝑓 ′ (𝑥)} = ∫ 𝑓 ′ (𝑥) cos 𝜔𝑥 𝑑𝑥
0
∞
∞
= [𝑓(𝑥) cos 𝜔𝑥]0 + 𝜔 ∫ 𝑓(𝑥) sin 𝜔𝑥 𝑑𝑥
0

= −𝑓(0) + 𝜔𝑓𝑠̂ (𝜔).

And we prove (c) with the use of (b):

∞
ℱ𝑐 {𝑓 ″ (𝑥)} = ∫ 𝑓 ″ (𝑥) cos 𝜔𝑥 𝑑𝑥
0
∞
∞
= [𝑓 ′ (𝑥) cos 𝜔𝑥]0 + 𝜔 ∫ 𝑓 ′ (𝑥) sin 𝜔𝑥 𝑑𝑥
0
= −𝑓 ′ (0) + 𝜔ℱ𝑠 {𝑓 ′ (𝑥)}
= −𝑓 ′ (0) − 𝜔2 𝑓 ̂ (𝜔). 𝑐

-(x) If 𝑓(𝑥) is a complex-valued function and [𝑓(𝑥)]∗ is its complex conjugate,

then
̂
ℱ{[𝑓(𝑥)]∗ } = [𝑓(−𝜔)] ∗
.
18 CHAPTER 2. PROPERTIES OF FOURIER TRANSFORMS

Proof We have that ∞

̂
𝑓(−𝜔) =∫ 𝑓(𝑥)𝑒𝑖𝜔𝑥 𝑑𝑥
−∞
and so by taking complex conjugate from both sides, it follows that
∞
̂
[𝑓(−𝜔)]∗
= ∫ [𝑓(𝑋)]∗ 𝑒−𝑖𝜔𝑥 𝑑𝑥 = ℱ{[𝑓(𝑥)]∗ }.
−∞

2.2 Convolution theorem for Fourier transforms

We define the convolution of two functions 𝑓(𝑥) and 𝑔(𝑥), defined over (−∞, ∞),
as ∞
𝑓(𝑥) ∗ 𝑔(𝑥) = ∫ 𝑓(𝑥 − 𝑢)𝑔(𝑢) 𝑑𝑢.
−∞
An important result is the so-called convolution theorem:

Theorem 2.1 (Convolution theorem). Suppose 𝑓(𝑥) and 𝑔(𝑥) are two functions
̂
defined over ℝ with Fourier transforms given as 𝑓(𝜔) and 𝑔(𝜔),
̂ we have:
̂ 𝑔(𝜔).
ℱ{𝑓 ∗ 𝑔} = 𝑓(𝜔) ̂

Proof We start on the LHS, change the order of integration and then use the
substitution 𝑠 = 𝑥 − 𝑢 at fixed 𝑢:
∞ ∞
∫ {∫ 𝑓(𝑥 − 𝑢)𝑔(𝑢) 𝑑𝑢} 𝑒−𝑖𝜔𝑥 𝑑𝑥
𝑥=−∞ 𝑢=−∞
∞ ∞
=∫ 𝑔(𝑢) {∫ 𝑓(𝑥 − 𝑢)𝑒−𝑖𝜔𝑥 𝑑𝑥} 𝑑𝑢
𝑢=−∞ 𝑥=−∞
∞ ∞
=∫ 𝑔(𝑢) {∫ 𝑓(𝑠)𝑒−𝑖𝜔(𝑠+𝑢) 𝑑𝑠} 𝑑𝑢
𝑢=−∞ 𝑠=−∞
∞ ∞
= (∫ 𝑔(𝑢)𝑒−𝑖𝜔𝑢 𝑑𝑢) (∫ 𝑓(𝑠)𝑒−𝑖𝜔𝑠 𝑑𝑠) = 𝑔(𝜔) ̂
̂ 𝑓(𝜔),
−∞ −∞

as required.
The convolution theorem suggests that convolution is commutative. This can
also be shown easily from the definition by using a change of variable in the
integration.
A similar convolution theorem holds for the inverse functions.
1 ̂
ℱ{𝑓(𝑥)𝑔(𝑥)} = 𝑓(𝜔) ∗ 𝑔(𝜔).
̂
2𝜋
2.3. ENERGY THEOREM FOR FOURIER TRANSFORMS 19

The proof of this result, using Dirac delta function is discussed as a quiz in the
lectures and using symmetry formula is seen in the problem sheet.

Example 2.1. Find the inverse Fourier transform of the function

1
.
(4 + 𝜔2 )(9 + 𝜔2 )

By setting
̂
𝑓(𝜔) = 1/(4 + 𝜔2 ), 𝑔(𝜔)
̂ = 1/(9 + 𝜔2 ),
2𝑎
we have (from the quiz in the lectures) that ℱ{𝑒−𝑎|𝑥| } = 𝑎2 +𝜔2 for 𝑎 > 0.
Therefore

𝑓(𝑥) = (1/4)𝑒−2|𝑥| , 𝑔(𝑥) = (1/6)𝑒−3|𝑥| .

Thus, by the convolution theorem:

1
ℱ−1 { } = 𝑓(𝑥) ∗ 𝑔(𝑥)
(4 + 𝜔2 )(9 + 𝜔2 )
∞
1
= ∫ 𝑒−2|𝑥−𝑢| 𝑒−3|𝑢| 𝑑𝑢
24 −∞
=⋯
1 −2|𝑥| 1
= 𝑒 − 𝑒−3|𝑥| .
20 30
Note that there are other ways to compute the inverse, for example, we could
decompose the original function into partial fractions and invert term-by-term.

2.3 Energy theorem for Fourier transforms

This is the analogous result to Parseval’s theorem for Fourier series.

Theorem 2.2 (Energy theorem). Suppose 𝑓(𝑥) s real valued function defined
̂
over ℝ with Fourier transform given as 𝑓(𝜔), we have:
∞ ∞
1 ̂
2 2
∫ ∣𝑓(𝜔)∣ 𝑑𝜔 = ∫ [𝑓(𝑥)] 𝑑𝑥.
2𝜋 −∞ −∞

Proof Properties (iii) and (x) of the Fourier transforms give

̂
ℱ{[𝑓(−𝑥)]∗ } = [𝑓(𝜔)] ∗
.

Since we are assuming 𝑓 to be real, this simplifies to

̂
ℱ{𝑓(−𝑥)} = [𝑓(𝜔)] ∗
.
20 CHAPTER 2. PROPERTIES OF FOURIER TRANSFORMS

If we now use the convolution theorem with 𝑔(𝜔)

̂ ̂
= [𝑓(𝜔)] ∗
, we have
2
̂
ℱ{𝑓(𝑥) ∗ 𝑓(−𝑥)} = 𝑓(𝜔)[ ̂
𝑓(𝜔)] ∗ ̂
= ∣𝑓(𝜔)∣ .

Using the definition of convolution and the inverse transform we have

∞ ∞
1 ̂
2
𝑓(𝑥) ∗ 𝑓(−𝑥) = ∫ 𝑓(𝑢 + 𝑥)𝑓(𝑢)𝑑𝑢 = ∫ ∣𝑓(𝜔)∣ 𝑒𝑖𝜔𝑥 𝑑𝜔.
−∞ 2𝜋 −∞

In particular, setting 𝑥 = 0, we obtain the required result:

∞ ∞
1 ̂
2
∫ [𝑓(𝑢)]2 𝑑𝑢 = ∫ ∣𝑓(𝜔)∣ 𝑑𝜔.
−∞ 2𝜋 −∞

2.4 The Dirac delta-function

Before we define the Dirac delta-function, we need to be aware of the following
theorem.

Theorem 2.3 (Mean-value theorem for integrals). If 𝑔(𝑥) is continuous on

[𝑎, 𝑏] then
𝑏
∫ 𝑔(𝑥) 𝑑𝑥 = (𝑏 − 𝑎)𝑔(𝑥),
̄
𝑎

for at least one 𝑥̄ with 𝑎 ≤ 𝑥̄ ≤ 𝑏.

The proof follows from the regular mean-value theorem for 𝐺 say, by defining
𝑔 = 𝐺′ . Geometrically this means that the area under the curve is equivalent
to that of a rectangle with length equal to the interval of integration.
Definition of the Dirac delta-function (impulse function)
Consider the following step-function:

𝑘/2, if |𝑥| < 1/𝑘,

𝑓𝑘 (𝑥) = {
0, if |𝑥| > 1/𝑘.

Clearly we can see that an important property of this function is that

∞
∫ 𝑓𝑘 (𝑥)𝑑𝑥 = 1.
−∞

As 𝑘 increases, 𝑓𝑘 (𝑥) gets taller and thinner (see Figure 2.1). We define the
Dirac delta function to be

𝛿(𝑥) = lim 𝑓𝑘 (𝑥),

𝑘→∞
2.4. THE DIRAC DELTA-FUNCTION 21

4
3
2
1
0

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

Figure 2.1: Graph of 𝑓𝑘 (𝑥) for 𝑘 = 1 (green), 𝑘 = 2 (red) and 𝑘 = 4 (blue).

although, of course, this limit doesn’t exist in the usual mathematical sense.
Effectively 𝛿(𝑥) is infinite at 𝑥 = 0 and zero at all other values of 𝑥. The key
property however, is that its integral (area under the curve) is one.
Sifting property of the delta function The delta function is most useful in
how it interacts with other functions. Consider
∞
∫ 𝑔(𝑥)𝛿(𝑥) 𝑑𝑥,
−∞

where 𝑔(𝑥) is a continuous function defined over (−∞, ∞). Using our definition
of the delta-function we can rewrite this as

∞ 1/𝑘
𝑘
lim ∫ 𝑔(𝑥)𝑓𝑘 (𝑥) 𝑑𝑥 = lim ∫ 𝑔(𝑥) 𝑑𝑥
𝑘→∞ −∞ 𝑘→∞ −1/𝑘 2
𝑘 1 1
= lim 𝑔(𝑥)̄ ( − (− )) ,
𝑘→∞ 2 𝑘 𝑘

for some 𝑥̄ in [−1/𝑘, 1/𝑘], using the mean-value theorem for integrals. Clearly,
as 𝑘 → ∞, we must have 𝑥̄ → 0. The expression above simplifies to
𝑘2
𝑔(0) = 𝑔(0).
2𝑘

We have therefore established that for any continuous function 𝑔:

∞
∫ 𝑔(𝑥)𝛿(𝑥) 𝑑𝑥 = 𝑔(0).
−∞
22 CHAPTER 2. PROPERTIES OF FOURIER TRANSFORMS

This result can easily be generalized to

∞
∫ 𝑔(𝑥)𝛿(𝑥 − 𝑎) 𝑑𝑥 = 𝑔(𝑎).
−∞

Example 2.2. Find the Fourier transform of 𝛿(𝑥).

We have using the sifting property

∞
ℱ{𝛿(𝑥)} = ∫ 𝛿(𝑥)𝑒−𝑖𝜔𝑥 𝑑𝑥 = 𝑒−𝑖𝜔0 = 1.
−∞

From this we can deduce that the inverse Fourier transform of 1 is 𝛿(𝑥). From
this last result, and using the inversion formula, we see that an alternative
representation of the delta function is
∞
1
𝛿(𝑥) = ∫ 𝑒±𝑖𝜔𝑥 𝑑𝜔,
2𝜋 −∞

with the ± arising from the observation that 𝛿(𝑥) is an even function of 𝑥 about
𝑥 = 0. If we are prepared to work in terms of delta-functions, we can now take
the Fourier transforms of functions that do not decay as 𝑥 → ±∞.

Example 2.3. Find the Fourier transform of cos 𝜔0 𝑥.

Using the definition of cos 𝜔0 𝑥 in terms of exponentials we have:

∞
1 𝑖𝜔0 𝑥
ℱ{cos 𝜔0 𝑥} = ∫ (𝑒 + 𝑒−𝑖𝜔0 𝑥 )𝑒−𝑖𝜔𝑥 𝑑𝑥
−∞ 2
∞ ∞
1 1
= ∫ 𝑒−𝑖(𝜔−𝜔0 )𝑥 𝑑𝑥 + ∫ 𝑒−𝑖(𝜔+𝜔0 )𝑥 𝑑𝑥
2 −∞ 2 −∞
= 𝜋𝛿(𝜔 − 𝜔0 ) + 𝜋𝛿(𝜔 + 𝜔0 ),

which is a two-spiked ‘function’.

We finish by recommending this video on a very intuitive visual introduction
to Fourier transform from the popular 3Blue1Brown YouTube channel in math-
ematics education. Do check it out and also the additional videos on related
topics such as uncertainty principle.
 Part II: Ordinary
Differential Equations

23
Chapter 3

Introduction to ordinary
differential equations

Differential equations are very important in science and engineering. In this

course, we focus on a specific class of differential equations called ordinary dif-
ferential equations (ODEs). Ordinary refers to dealing with functions of one in-
dependent variable. We initially focus on scaler functions of the form 𝑓 ∶ ℝ → ℝ.
Later in the course, we discuss systems of ODEs and consider vector functions
of the form 𝑓 ∶ ℝ → ℝ𝑛 . We assume 𝑓(𝑥) is differentiable upto order 𝑘. An ODE
is an equation for the function 𝑓(𝑥) that involves the function, its derivatives
and the independent variable. A general form for an ODE of order 𝑘 is
𝑑𝑓 𝑑𝑘 𝑓
𝐺(𝑥, 𝑓(𝑥), , ..., 𝑘 ) = 0,
𝑑𝑥 𝑑𝑥
where the highest derivative present in the equation 𝐺 is of the order 𝑘. The
degree of the ODE is the power of highest derivative (when fractional powers
have been removed). The ODE is called linear if G is a linear function of 𝑓(𝑥)
and its derivatives. This form for the ODE is the so called implicit form. In
an explicit form for an ODE, the highest order derivative is given as function of
the lower derivatives:
𝑑𝑘 𝑓 𝑑𝑓 𝑑𝑘−1 𝑓
𝑘
= 𝐹 (𝑥, 𝑓(𝑥), , ..., 𝑘−1 ) = 0.
𝑑𝑥 𝑑𝑥 𝑑𝑥

Example 3.1. Consider the following ODE for the function 𝑓(𝑥):
1
2 3
𝑑2 𝑓 𝑑𝑓
2
= 5 [1 + ( ) ]
𝑑𝑥 𝑑𝑥

This is a nonlinear explicit ODE of degree 3 and order 2.

25
26CHAPTER 3. INTRODUCTION TO ORDINARY DIFFERENTIAL EQUATIONS

Solving an ODE is the task of finding 𝑓(𝑥) such that the ODE is satisfied over
the domain of 𝑥 (e.g. ℝ).
ODEs appear naturally in many areas of sciences and humanities. In the follow-
ing we provide some examples.

Example 3.2 (Second Newton Law).

Mechanics: A very short introduction

Kinematics is a branch of mechanics that describes the motion of points (objects)
without considering the forces that cause them to move. In one dimension 𝑥(𝑡)
denotes the position of a particle at time 𝑡. Then 𝑑𝑥 𝑑𝑡 = 𝑥 ̇ = 𝑣 is defined as
2
the velocity of the particle and 𝑑𝑑𝑡2𝑥 = 𝑥̈ = 𝑎 is defined as acceleration. This
can be generalised to higher dimensions using vectors of location, velocity and
acceleration.
Dynamics is the branch of mechanics concerned with the study of forces and
their effects on motion. Isaac Newton came up with the fundamental physical
laws, which govern dynamics in physics:
-First law an object not acted upon by any force either remains at rest or
continues to move at a constant velocity
-Second law the vector sum of the forces 𝐹 on an object is equal to the mass
𝑚 of that object multiplied by its acceleration 𝑎: 𝐹 = 𝑚𝑎

𝑑2 𝑥 𝑑𝑥
𝑚 = 𝐹 (𝑡, 𝑥, )
𝑑𝑡2 𝑑𝑡
This is a second order ODE for the position of the object 𝑥(𝑡).
-Third law when one body exerts a force on a second body, the second body
exerts a force equal in magnitude and opposite in direction on the first body.
Mechanics used to be thaught until recently in our first year Mathematics course
as it provides many links to different areas of mathemaitcs. If you have any
doubts, look at this video, for a very cool counting problem for colliding particles
and its very unexpected solution and link to mathematics.

Example 3.3 (Population dynamics: Malthus (1798)).

Consider 𝑃 (𝑡) denotes the population of certain species at time 𝑡. Malthus

proposed the following simple ODE:
𝑑𝑃
= 𝑘𝑃 ,
𝑑𝑡
with 𝑘 > 0, this ODE results in an exponential increase in the population in
time.
3.1. PARTICULAR AND GENERAL SOLUTIONS 27

Example 3.4 (Population dynamics: Logistic Growth (Verhulst, 1845)).

Verhgulst proposed a modification to Malthus law, creating a carrying capacity

𝐶 for the population:
𝑑𝑃 𝑃
= 𝑘𝑃 (1 − ) .
𝑑𝑡 𝐶

Example 3.5 (Radius of curvature).

In geometry, given radius of curvature 𝑅(𝑥, 𝑦), we can find equation for the
curve 𝑦(𝑥) using the following ODE that definition for the radius of curvature.
3/2
𝑑𝑦 2
(1 + ( 𝑑𝑥 ) )
𝑅(𝑥, 𝑦) = 𝑑2 𝑦
.
𝑑𝑥2

3.1 Particular and General Solutions

𝑓𝑃 𝐼 (𝑥) is called a Particular Integral or Particular Solution of an ODE such
that
𝑑𝑓 𝑑𝑘 𝑓
𝐺(𝑥, 𝑓(𝑥), , ..., 𝑘 ) = 0,
𝑑𝑥 𝑑𝑥
is satisfied over the domain 𝑥 ∈ ℝ.
𝑓𝐺𝑆 is called a General Solution of an ODE of the order 𝑘, if

𝑓𝐺𝑆 = 𝑓𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 , ⋯ , 𝑐𝑘 )

is a general family of solutions that fulfil the ODE. The parameters {𝑐𝑖 }𝑘𝑖=1
are the constants of integration and are usually fixed by initial or boundary
conditions. In this course we, concern ourselves with methods that allow us
to obtain such solutions. Rigourous mathematical results on existance and
uniqueness of such solutions are discussed in the second year.

Example 3.6 (from kinematics). Object moving with constant speed 𝑣:

𝑑𝑥
= 𝑣.
𝑑𝑡

One particular solution is 𝑥𝑃 𝐼 = 𝑣𝑡 and another one is 𝑥𝑃 𝐼 = 𝑣𝑡+1. The general

solution is 𝑥𝐺𝑆 = 𝑣𝑡 + 𝑐1 , where 𝑐1 is the constant of integration.
If we are also told that 𝑥(𝑡 = 0) = 𝑥0 , we have 𝑥(𝑡) = 𝑣𝑡 + 𝑥0 , which is the
solution to the initial value problem.
28CHAPTER 3. INTRODUCTION TO ORDINARY DIFFERENTIAL EQUATIONS
Chapter 4

First and second order

ODEs

Not all ODEs are analytically solvable. In this section we discuss some types
of first and second order ODEs that are analytically solvable and see some
examples.

4.1 First order ODEs

A first order ODE has only the first derivative represented. The general implicit
form for a first order ODE for the function 𝑥(𝑡) is:
𝑑𝑥
𝐺(𝑡, 𝑥, ) = 0,
𝑑𝑡
and its explicit form is:
𝑑𝑥
= 𝐹 (𝑥, 𝑡).
𝑑𝑡
In the following we discuss some classes of first order ODEs and describe meth-
ods of obtaining a solution for them. These inculde separable and linear first
order ODEs. The other types of First Order ODEs that can be solved are based
on transformations or change of variables. We see two examples of this in the
following. Another important class of first order ODEs that can be solved are
Exact ODEs that will be discussed in part III of the course after introducing
partial and total differentiation.

4.1.1 Separable First Order ODEs

A separable first order ODE can be written in the following form:
𝑑𝑥
= 𝐹1 (𝑥)𝐹2 (𝑡).
𝑑𝑡

29
30 CHAPTER 4. FIRST AND SECOND ORDER ODES

Solution Rearranging and integrating both sides we get:

𝑑𝑥
∫ = ∫ 𝐹2 (𝑡)𝑑𝑡 + 𝑐1 .
𝐹1 (𝑥)

4.1.2 Linear First Order ODEs

First order linear ODEs have the following general form:

𝑑𝑦
+ 𝑝(𝑥)𝑦 = 𝑞(𝑥).
𝑑𝑥
Solution This is solved by finding an integrating factor (IF). We look for 𝐼(𝑥)
such that:
𝑑𝑦 𝑑[𝐼(𝑥)𝑦]
𝐼(𝑥) [ + 𝑝(𝑥)𝑦] = ,
𝑑𝑥 𝑑𝑥
Then, we have

𝑑[𝐼(𝑥)𝑦]
= 𝐼(𝑥)𝑞(𝑥),
𝑑𝑥
∫ 𝑑[𝐼(𝑥)𝑦] = ∫ 𝑞(𝑥)𝐼(𝑥) 𝑑𝑥 + 𝑐1 ,
1
𝑦(𝑥) = [∫ 𝑞(𝑥)𝐼(𝑥) 𝑑𝑥 + 𝑐1 ] .
𝐼(𝑥)

Integrating factors must fulfil:

𝑑(𝐼𝑦) 𝑑𝑦
=𝐼 + 𝐼𝑝𝑦,
𝑑𝑥 𝑑𝑥
𝑑𝑦 𝑑𝐼 𝑑𝑦
𝐼 +𝑦 =𝐼 + 𝐼𝑝𝑦,
𝑑𝑥 𝑑𝑥 𝑑𝑥
𝑑𝐼
∫ = ∫ 𝑝(𝑥) 𝑑𝑥 + 𝑐′ .
𝐼

So we have:
𝐼(𝑥) = 𝐴𝑒∫ 𝑝(𝑥) 𝑑𝑥 ,
where 𝐴 is a new arbitrary constant (of integration).
So, we have the following for the general solution:

𝑦(𝑥) = 𝑒− ∫ 𝑝(𝑥) 𝑑𝑥 [∫ 𝑒∫ 𝑝(𝑥) 𝑑𝑥 𝑞(𝑥) 𝑑𝑥 + 𝑐] ,

where 𝑐 = 𝑐1 /𝐴 is a new arbitrary constant of integration.

4.2. SECOND ORDER ODES 31

4.1.3 Dimensionally Homogeneous

The dimensionally homogeneous have the following general form:

𝑑𝑦 𝑦
= 𝐹 ( ).
𝑑𝑥 𝑥
Solution Let 𝑢 = 𝑦/𝑥 we obtain:

𝑑𝑦 𝑑𝑢
=𝑢+𝑥
𝑑𝑥 𝑑𝑥
The ODE in terms of 𝑢(𝑥), which is separabale is

𝑑𝑢
𝑢+𝑥 = 𝐹 (𝑢),
𝑑𝑥
Finding general solution 𝑢𝐺𝑆 (𝑥) for this ODE then we find the general solution
for the original ODE as 𝑦𝐺𝑆 (𝑥) = 𝑢𝐺𝑆 (𝑥)𝑥.

4.1.4 Bernoulli ODEs

There are other examples of transformations can turn specific ODEs into sepa-
rable or linear. Some such as Bernoulli are classic:
𝑑𝑦
+ 𝑝(𝑥)𝑦 = 𝑞(𝑥)𝑦𝑛 ,
𝑑𝑥
where 𝑛 ∈ ℝ.
Solution We use the change of variable 𝑢 = 𝑦1−𝑛 . We obtain:

𝑑𝑢 𝑑𝑦
= (1 − 𝑛)𝑦−𝑛 .
𝑑𝑥 𝑑𝑥
Writing the original ODE in terms of 𝑢 we have:

𝑑𝑢
+ (1 − 𝑛)𝑝(𝑥)𝑢 = (1 − 𝑛)𝑞(𝑥),
𝑑𝑥
which is a linear ODE for 𝑢(𝑥), so we obtain 𝑢𝐺𝑆 (𝑥) and then we have
1
𝑦𝐺𝑆 = 𝑢𝐺𝑆
1−𝑛
.

4.2 Second Order ODEs

The general implicit form is:

𝑑𝑦 𝑑2 𝑦
𝐺(𝑥, 𝑦, , ) = 0,
𝑑𝑥 𝑑𝑥2
32 CHAPTER 4. FIRST AND SECOND ORDER ODES

and the general explicit form is:

𝑑2 𝑦 𝑑𝑦
= 𝐹 (𝑥, 𝑦, ).
𝑑𝑥2 𝑑𝑥

The second order ODEs are common in Mechanics as Newton’s second law is
such ODE with independent variable as time 𝑡. They are difficult to solve for
general 𝐹 but there are some special cases that can be solved as described in
the following. Also, the linear case is discussed in the next chapter in detail.

4.2.1 𝐹 only depends on 𝑥

𝑑2 𝑦
= 𝐹 (𝑥)
𝑑𝑥2
𝑑𝑦 𝑑𝑢
Solution Let 𝑢 = 𝑑𝑥 then we have 𝑑𝑥 = 𝐹 (𝑥). A first integration gives us:

𝑢 = ∫ 𝐹 (𝑥)𝑑𝑥 + 𝑐1

A second integration then gives us 𝑦𝐺𝑆 :

𝑦𝐺𝑆 = ∫ [∫ 𝐹 (𝑥)𝑑𝑥] 𝑑𝑥 + 𝑐1 𝑥 + 𝑐2 .

𝑑𝑦
4.2.2 𝐹 only depends on 𝑥 and 𝑑𝑥
𝑑2 𝑦 𝑑𝑦
2
= 𝐹 (𝑥, )
𝑑𝑥 𝑑𝑥
𝑑𝑦
Solution Let 𝑢 = 𝑑𝑥 then we have 𝑑𝑢𝑑𝑥 = 𝐹 (𝑥, 𝑢), which is a first order ODE. If
we could obtain the general solution 𝑢𝐺𝑆 (𝑥; 𝑐1 ) then we have:

𝑦𝐺𝑆 (𝑥) = ∫ 𝑢𝐺𝑆 (𝑥; 𝑐1 ) 𝑑𝑥 + 𝑐2 .

4.2.3 𝐹 only depends on 𝑦

𝑑2 𝑦
= 𝐹 (𝑦)
𝑑𝑥2
𝑑𝑦 𝑑𝑢
Solution We let 𝑢 = 𝑑𝑥 then 𝑑𝑥 = 𝐹 (𝑦). Then we have:

𝑑𝑢 𝑑𝑢 𝑑𝑦 𝑑𝑢 𝑑 1 2
= =𝑢 = ( 𝑢 ) = 𝐹 (𝑦),
𝑑𝑥 𝑑𝑦 𝑑𝑥 𝑑𝑦 𝑑𝑦 2
which is a first order separable ODE for 𝑢(𝑦). We have:
1 2
𝑢 = ∫ 𝐹 (𝑦)𝑑𝑦 + 𝑐1 = 𝐺(𝑦) + 𝑐1 .
2
4.2. SECOND ORDER ODES 33

So we have:
𝑑𝑦
= 𝑢 = ±√2𝐺(𝑦) + 2𝑐1 ,
𝑑𝑥
which is a first order separable ODE for 𝑦(𝑥) and can be integrated to obtain
𝑦𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 ) as seen in the following example.

Example 4.1 (Mechanics Harmonic Oscillator). Hooke’s law states if 𝑥(𝑡) is

displacement relative to an ideal spring relaxed position, the spring force is:
2
𝐹 = −𝑘𝑥 Using second Newton Law we have: 𝑚𝑎 = 𝐹 ⟹ 𝑚 𝑑𝑑𝑡2𝑥 = −𝑘𝑥

𝑑𝑥
Let velocity to be 𝑢 = 𝑑𝑡 , then we have:
𝑑𝑢 𝑑 1 2 𝑘𝑥
𝑎= = [ 𝑢 ]=− .
𝑑𝑡 𝑑𝑥 2 𝑚
Integrating both sides we obtain:
𝑢2 𝑘 2
=− 𝑥 + 𝑐1 .
2 2𝑚
This equation gives us a constant of motion (𝐸 = 𝑐1 𝑚), which is known as total
energy, the sum of kinetic energy (1/2𝑚𝑢2 ) and potential energy (1/2𝑘𝑥2 ).
𝑑𝑥 2 𝑑𝑥
𝑢= 𝑑𝑡 = ±√ 2𝐸−𝑘𝑥
𝑚 ⟹ ∫ = ∫ 𝑑𝑡
2
±√ 2𝐸−𝑘𝑥
𝑚

Sticking with the postive sign on the LHS we have:

1 𝑑𝑥 𝑚 𝑘
∫ = √ sin−1 (√ 𝑥) = 𝑡 + 𝑐2
√2𝐸/𝑚 𝑘 2
√1 − 2𝐸 𝑥 𝑘 2𝐸

Rearranging the solution we obtain:

𝑥𝐺𝑆 = 𝐴 sin(𝜔𝑡 + 𝜙),

where, 𝜔 = √𝑘/𝑚 is the frequency of oscillations and 𝐴 = √2𝐸/𝑘 and 𝜙 =

√𝑘/𝑚𝑐2 are new constants of integration. We note that, if we had chosen to
use the minus sign above, we would have obtained the same family of solutions
but the constants of integrations would be differently defined.

𝑑𝑦
4.2.4 𝐹 only depends on 𝑦 and 𝑑𝑥
𝑑2 𝑦 𝑑𝑦
2
= 𝐹 (𝑦, )
𝑑𝑥 𝑑𝑥
𝑑𝑦 𝑑𝑢
let 𝑢 = 𝑑𝑥 ⟹ 𝑑𝑥 = 𝐹 (𝑦, 𝑢). So we have
𝑑𝑢 𝑑𝑢 𝑑𝑦 𝑑𝑢 𝑑 1 2
= =𝑢 = ( 𝑢 ).
𝑑𝑥 𝑑𝑦 𝑑𝑥 𝑑𝑦 𝑑𝑦 2
34 CHAPTER 4. FIRST AND SECOND ORDER ODES

Therefore we have the following first order ODE for 𝑢(𝑦) to solve

𝑑 1 2
( 𝑢 ) = 𝐹 (𝑦, 𝑢).
𝑑𝑦 2

Given 𝑢𝐺𝑆 (𝑦; 𝑐1 ) being a general solution for the above ODE, we have the fol-
lowing first order ODE for 𝑦(𝑥):

𝑑𝑦
= 𝑢𝐺𝑆 (𝑦; 𝑐1 ).
𝑑𝑥
Chapter 5

Linear ODEs

The general form of linear ODEs of order 𝑘 is

𝑑𝑘 𝑦 𝑑𝑘−1 𝑦 𝑑𝑦
𝛼𝑘 (𝑥) 𝑘
+ 𝛼𝑘−1 (𝑥) 𝑘−1 + ... + 𝛼1 (𝑥) + 𝛼0 (𝑥)𝑦 = 𝑓(𝑥),
𝑑𝑥 𝑑𝑥 𝑑𝑥
where 𝛼𝑘 (𝑥),.., 𝛼0 (𝑥) and 𝑓(𝑥) are functions of only the independent variable 𝑥.
The ODE is called homogeneous if 𝑓(𝑥) = 0 and inhomogeneous otherwise.
Some examples of linear ODEs:
• First order ODE
𝑑𝑦
+ 𝑝(𝑥)𝑦 = 𝑞(𝑥).
𝑑𝑥
• Bessel’s equation
𝑑2 𝑦 𝑑𝑦
𝑥2 +𝑥 + (𝑥2 − 𝑛2 )𝑦 = 0.
𝑑𝑥2 𝑑𝑥
• Legendre’s equation
𝑑2 𝑦 𝑑𝑦
(1 − 𝑥2 ) 2
− 2𝑥 + 𝑛(𝑛 + 1)𝑦 = 0.
𝑑𝑥 𝑑𝑥
Linear Operators
𝑑
We define the differential operator as 𝒟[𝑓] ≡ 𝑑𝑥 [𝑓].

Differential operator is a linear operator since we have:

𝒟[𝜆1 𝑓1 + 𝜆2 𝑓2 ] = 𝜆1 𝒟[𝑓1 ] + 𝜆2 𝒟[𝑓2 ].

𝑑𝑘
Defining differential operator of order 𝑘 as 𝒟𝑘 [𝑓] ≡ 𝑑𝑥𝑘 [𝑓], which is also a linear
operator as we have:

𝒟𝑘 [𝜆1 𝑓1 + 𝜆2 𝑓2 ] = 𝜆1 𝒟𝑘 [𝑓1 ] + 𝜆2 𝒟𝑘 [𝑓2 ].

35
36 CHAPTER 5. LINEAR ODES

Linear ODEs are associated with a linear operator defined using the differential
𝑘
operators: ℒ[𝑦] ≡ ∑𝑖=0 𝛼𝑖 (𝑥)𝒟𝑖 [𝑦] since:

ℒ[𝜆1 𝑓1 + 𝜆2 𝑓2 ] = 𝜆1 ℒ[𝑓1 ] + 𝜆2 ℒ[𝑓2 ].

A linear ODE can thus be simply written as ℒ[𝑦] = 𝑓(𝑥) and a homogenous
ODE as ℒ[𝑦] = 0. Linearity of ℒ has an important consequence. If we have
two solutions 𝑦1 and 𝑦2 of a homogenous linear ODE ℒ[𝑦] = 0, then any linear
combinations of these solutions are also solutions for this ODE, since:

ℒ[𝜆1 𝑓1 + 𝜆2 𝑓2 ] = 𝜆1 ℒ[𝑓1 ] + 𝜆2 ℒ[𝑓2 ] = 0.

Linear independence
A set of functions {𝑓𝑖 (𝑥)}𝑘𝑖=1 is said to be linearly independent if 𝑓𝑖 ’s satisfy the
following condition:

𝑐1 𝑓1 (𝑥) + 𝑐2 𝑓2 (𝑥) + ⋯ + 𝑐𝑘 𝑓𝑘 (𝑥) = 0,

if and only if 𝑐1 = 𝑐2 = ⋯ = 𝑐𝑘 = 0.
Linear ODEs are easier to solve because of the following important property of
their solutions. This is a basic consequence of linearity of differential operators.

Proposition 5.1. The solutions of the homogeneous linear ODE ℒ[𝑦] = 0 form
a vector space (see MATH40003: Linear Algebra and Groups) of dimension 𝑘,
where 𝑘 is the order of the ODE. Therefore, the general solution of a linear
homogeneous ODE can be written as
𝐻
𝑦𝐺𝑆 (𝑥; 𝑐1 , ..., 𝑐𝑘 ) = 𝑐1 𝑦1 + 𝑐2 𝑦2 + ... + 𝑐𝑘 𝑦𝑘 ,

where 𝐵 = {𝑦𝑖 (𝑥)}𝑘𝑖=1 is a set of linearly independent solutions forming a basis

for the linear homogeneous ODE’s solution vector space.

Proposition 5.2. To test the linear independence of a set of functions

{𝑦𝑖 (𝑥)}𝑘𝑖=1 , we calculate the Wronskian, which is the determinant of the
Wronskian matrix (𝕎𝑘×𝑘 ):

𝑦1 (𝑥) 𝑦2 (𝑥) ⋯ 𝑦𝑘 (𝑥)

⎡ 𝑑𝑦1 (𝑥) 𝑑𝑦2 𝑑𝑦𝑘 ⎤
⎢ 𝑑𝑥 𝑑𝑥 (𝑥) ⋯ 𝑑𝑥 (𝑥) ⎥
𝑊 [{𝑦𝑖 (𝑥)}𝑘𝑖=1 ] = det𝕎 = det ⎢ ⋮ ⋮ ⋮ ⎥
⎢ ⎥
𝑑𝑘−1 𝑦 𝑑𝑘−1 𝑦2 𝑑𝑘−1 𝑦𝑘
⎣ 𝑑𝑥𝑘−11 (𝑥) 𝑑𝑥𝑘−1 (𝑥) ⋯ 𝑑𝑥 𝑘−1 (𝑥) ⎦

The set {𝑦𝑖 (𝑥)}𝑘𝑖=1 is linearly independent if

𝑊 [{𝑦𝑖 (𝑥)}𝑘𝑖=1 ] ≠ 0.
5.1. GENERAL SOLUTION OF THE NON-HOMOGENEOUS LINEAR ODE37

Proof We prove this by contradiction; If we assume 𝑊 (𝑥) ≠ 0 but {𝑦𝑖 (𝑥)}𝑘𝑖=1

are linearly dependent:

𝑘
∃𝑖 ∈ {1, ⋯ , 𝑘}, 𝑐𝑖 ≠ 0, ∑ 𝑐𝑖 𝑦𝑖 (𝑥) = 0.
𝑖=1

By taking derivatives repeatedly with respect to 𝑥 from the above equation we

obtain:

𝑘
𝑑𝑦𝑖
∑ 𝑐𝑖 =0
𝑖=1
𝑑𝑥
⋮
𝑘 𝑘−1
𝑑 𝑦𝑖
∑ 𝑐𝑖 = 0,
𝑖=1
𝑑𝑥𝑘−1

which can be written as 𝕎.𝑐 ⃗ = 0, with 𝑐 ⃗ defined as

𝑐1
𝑐⃗= ⎡ ⎤
⎢ ⋮ ⎥,
⎣𝑐𝑘 ⎦
since 𝑐 ⃗ ≠ 0 then 𝕎.𝑐 ⃗ = 0 implies 𝑊 [{𝑦𝑖 (𝑥)}𝑘𝑖=1 ] = det𝕎 = 0, which is a
contradiction. So, if 𝑊 (𝑥) ≠ 0 then {𝑦𝑖 (𝑥)}𝑘𝑖=1 are linearly independent, as
required.

Example 5.1. Show that sin(𝑥) and cos(𝑥) are linearly independent.

sin(𝑥) cos(𝑥)
𝕎2×2 = [ ],
cos(𝑥) sin(𝑥)
then
𝑊 (𝑥) = det𝕎 = − sin2 (𝑥) − cos2 (𝑥) = −1 ≠ 0,
therefore sin(𝑥) and cos(𝑥) are linearly independent.
Note There exists examples in which the Wronskian vanishes without the func-
tions being linearly dependent. An example is given in a quiz in the lectures.

5.1 General solution of the non-homogeneous

linear ODE
To obtain the general solution of the non-homogeneous linear ODE
ℒ[𝑦] = 𝑓(𝑥),
38 CHAPTER 5. LINEAR ODES

we split the problem into two simpler steps:

1. We consider the corresponding homogeneous linear ODE ℒ[𝑦] = 0. We ob-
tain the general solution, which is also known as complementary function
(𝑦𝐶𝐹 ):
𝑘
𝐻
𝑦𝐶𝐹 = 𝑦𝐺𝑆 (𝑥; 𝑐1 , ⋯ , 𝑐𝑘 ) = ∑ 𝑐𝑖 𝑦𝑖 (𝑥),
𝑖=1

where, {𝑦𝑖 }𝑘𝑖=1 are the basis of the solution vector space (a set of linearly
independent solutions of the homogeneous linear ODE).
2. We obtain any/one solution of the full non-homogeneous ODE, which is
also known as particular integral (𝑦𝑃 𝐼 ):

ℒ[𝑦𝑃 𝐼 ] = 𝑓(𝑥).

Then for the solution to the full problem by combining the results above
and due to linearity, we have:

𝐻
ℒ[𝑦𝐺𝑆 (𝑥; 𝑐1 , ⋯ , 𝑐𝑘 )] = ℒ[𝑦𝐶𝐹 + 𝑦𝑃 𝐼 ] = ℒ[𝑦𝐺𝑆 ] + ℒ[𝑦𝑃 𝐼 ] = 𝑓(𝑥).
So the general solution of the non-homogeneous linear ODE is the sum of the
complementary function and a particular integral. As seen in a quiz in the
lectures different choices of particular integrals results in the same family of
general solutions.
One useful consequence of the linearity is that if the RHS of the ODE is sum of
two functions:
ℒ[𝑦] = 𝑓1 (𝑥) + 𝑓2 (𝑥).
We can break the second step of finding particular integral into additional steps.
𝐻
1. Find 𝑦𝐶𝐹 = 𝑦𝐺𝑆 (𝑥; 𝑐1 , ⋯ , 𝑐𝑘 ) such that ℒ[𝑦𝐶𝐹 ] = 0.
2. Find any solution to ℒ1𝒫ℐ [𝑦] = 𝑓1 (𝑥).
3. Find any solution to ℒ2𝒫ℐ [𝑦] = 𝑓2 (𝑥).
Then, we have 𝑦𝐺𝑆 = 𝑦𝐶𝐹 + 𝑦𝑃1 𝐼 + 𝑦𝑃2 𝐼 .
Linear ODEs with constant coefficients
The general linear ODE is not always analytically solvable. Next year, you
will see approximative and numerical methods to solve this kind of ODEs. In
the rest of this course, we will focus on the case of linear ODEs with constant
coefficients (𝛼𝑖 s not depending on independent variable 𝑥):

𝑘
ℒ[𝑦] = ∑ 𝛼𝑖 𝒟𝑖 [𝑦] = 𝑓(𝑥)
𝑖=0
5.2. FIRST ORDER LINEAR ODES WITH CONSTANT COEFFICIENTS39

5.2 First order linear ODEs with constant coef-

ficients
The general form of the first order linear ODEs with constant coefficients is:
𝑑𝑦
ℒ[𝑦] = 𝛼1 + 𝛼0 𝑦 = 𝑓(𝑥).
𝑑𝑥
As seen in chapter 4, rewriting this ODE as
𝑑𝑦 𝛼 𝑓(𝑥)
+ 0𝑦 = ,
𝑑𝑥 𝛼1 𝛼1

𝛼0
𝑥
we can obtain the general solution using the integrating factor 𝐼(𝑥) = 𝑒 𝛼1 . We
obtain

𝑥 𝑓(𝑥)
𝛼 𝛼 𝛼0
− 𝛼0 𝑥 − 𝛼0 𝑥
𝑦𝐺𝑆 = 𝑐1 𝑒 1 +𝑒 1 ∫ 𝑒 𝛼1 𝑑𝑥.
𝛼1

Example 5.2. Solve 𝑓(𝑥) = 𝑥.

Using the general solution above, and by integration by parts, we obtain:

𝛼
− 𝛼0 𝑥 𝑥 𝛼
𝑦𝐺𝑆 = 𝑐1 𝑒 1 +[ − 21 ] .
𝛼0 𝛼0

Alternative method
1. Solve the corresponding homogeneous ODE:
𝑑𝑦
ℒ[𝑦𝐶𝐹 ] = 𝛼1 + 𝛼0 𝑦 = 0.
𝑑𝑥
This is a separable ODE and by integration we obtain:
𝛼
𝐻 − 𝛼0 𝑥
𝑦𝐶𝐹 = 𝑦𝐺𝑆 (𝑥; 𝑐1 ) = 𝑐1 𝑒 1 .

2. Find a particular integral for the full ODE: ℒ[𝑦𝑃 𝐼 ] = 𝑓(𝑥) = 𝑥.

This is done by using ansatz, which is an educated guess using the method of
undetermined coefficients. In this case as 𝑓(𝑥) is polynomial, we could try a
polynomial ansatz:
𝑦𝑃 𝐼 = 𝐴𝑥2 + 𝐵𝑥 + 𝐶,
where, 𝐴, 𝐵 and 𝐶 are constants to be determined. By plugging this ansatz in
to the ODE, we check if here are suitable values for these constants that makes
our ansatz a particlular solution for the ODE:

ℒ[𝑦𝑃 𝐼 ] = 𝛼1 (2𝐴𝑥 + 𝐵) + 𝛼0 (𝐴𝑥2 + 𝐵𝑥 + 𝐶) = 𝑥.

40 CHAPTER 5. LINEAR ODES

This equation should be satisfied for all 𝑥 ∈ ℝ, so we equate the coefficients of

the powers of 𝑥:

𝑥2 ∶ 𝛼 0 𝐴 = 0 ⇒ 𝐴 = 0;
1
𝑥1 ∶ 2𝛼1 𝐴 + 𝛼0 𝐵 = 1 ⇒ 𝐵= ;
𝛼0
𝛼
𝑥0 ∶ 𝛼 1 𝐵 + 𝛼 0 𝐶 = 0 ⇒ 𝐶 = − 12 ,
𝛼0

which gives us the same general solution obtained using the first method:
𝛼
− 𝛼0 𝑥 𝑥 𝛼
𝑦𝐺𝑆 = 𝑦𝐶𝐹 + 𝑦𝑃 𝐼 = 𝑐1 𝑒 1 +[ − 12 ] .
𝛼0 𝛼0

Example 5.3. Solve 𝑓(𝑥) = 𝑒𝑏𝑥 ; 𝑏 ≠ − 𝛼

𝛼 .
0
1

The ODE is the following:

𝑑𝑦
+ 𝛼0 𝑦 = 𝑒𝑏𝑥 .
ℒ[𝑦] = 𝛼1
𝑑𝑥
Using the two step method, we have as before:
𝛼
𝐻 − 𝛼0 𝑥
1. 𝑦𝐶𝐹 = 𝑦𝐺𝑆 (𝑥; 𝑐1 ) = 𝑐1 𝑒 1 .
2. We try ansatz 𝑦𝑃 𝐼 = 𝐴𝑒𝑏𝑥 , plugging this into the ODE, we obtain:

𝛼1 𝐴𝑏𝑒𝑏𝑥 + 𝛼0 𝐴𝑒𝑏𝑥 = 𝑒𝑏𝑥 .

1
Solving this we obtain 𝐴 = 𝛼1 𝑏+𝛼0 . So we obtain:

− 𝛼0 𝑥
𝛼 1
𝑦𝐺𝑆 = 𝑦𝐶𝐹 + 𝑦𝑃 𝐼 = 𝑐1 𝑒 1 + 𝑒𝑏𝑥 .
𝛼1 𝑏 + 𝛼0

What about the case 𝑏 = − 𝛼 𝛼1 ? Naive ansatz 𝑦𝑃 𝐼 = 𝐴𝑒

0 𝑏𝑥
does not work,
since ℒ[𝑦𝑃 𝐼 ] = 0. A more general ansatz is:

𝑦𝑃 𝐼 = 𝐴(𝑥)𝑒𝑏𝑥 .

Here we are looking for an unknown function 𝐴(𝑥), so we will obtain an

ODE. This is called the method of variation of parameters, developed by
Euler and Lagrange. Plugging this ansatz into the ODE we obtain the
following simple ODE:
𝑑𝐴
𝛼1 = 1,
𝑑𝑥
which has the following general solution:
𝑥
𝐴(𝑥) = + 𝑐2 .
𝛼1
5.3. SECOND ORDER LINEAR ODES WITH CONSTANT COEFFICIENTS41

So, we obtain for the general solution:

𝛼
− 𝛼0 𝑥 𝑥 𝛼
− 0𝑥 𝑥 𝑏𝑥
𝑦𝐺𝑆 = 𝑦𝐶𝐹 + 𝑦𝑃 𝐼 = 𝑐1 𝑒 1 +( + 𝑐2 )𝑒𝑏𝑥 = 𝑐′ 𝑒 𝛼1 + 𝑒 ,
𝛼1 𝛼1
where in the last step we have renamed 𝑐1 + 𝑐2 as 𝑐′ a new constant of
integration.

5.3 Second order linear ODEs with constant co-

efficients
𝑑2 𝑦 𝑑𝑦
ℒ[𝑦] = 𝛼2 + 𝛼1 + 𝛼0 𝑦 = 𝑓(𝑥),
𝑑𝑥2 𝑑𝑥
𝐻
𝑦𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 ) = 𝑦𝐶𝐹 (𝑥; 𝑐1 , 𝑐2 ) + 𝑦𝑃 𝐼 = 𝑦𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 ) + 𝑦𝑃 𝐼 .
If 𝐵 = {𝑦1 (𝑥), 𝑦2 (𝑥)} is a basis for the solution vector space of the homogeneous
ODE: ℒ[𝑦𝐻 ] = 0. Then, we have:
𝐻
𝑦𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 ) = 𝑐1 𝑦1 (𝑥) + 𝑐2 𝑦2 (𝑥).

Solving the homogeneous second order linear ODE

We need to obtain two linearly independent solutions to the following ODE:

𝑑2 𝑦 𝑑𝑦
ℒ[𝑦] = 𝛼2 + 𝛼1 + 𝛼0 𝑦 = 0.
𝑑𝑥2 𝑑𝑥

We can try the ansatz: 𝑦𝐻 = 𝑒𝜆𝑥 ,

ℒ[𝑦𝐻 ] = 𝛼2 𝜆2 𝑒𝜆𝑥 + 𝛼1 𝜆𝑒𝜆𝑥 + 𝛼0 𝑒𝜆𝑥 = 0 ⇒ 𝛼2 𝜆2 + 𝛼1 𝜆 + 𝛼0 = 0.

This quadratic equation is called the characteristic equation of the linear ODE,
which has the following solutions:

−𝛼1 ± √𝛼21 − 4𝛼0 𝛼2

𝜆1 , 𝜆2 = .
2𝛼2

So, we have the following two candidate solutions 𝑦1𝐻 = 𝑒𝜆1 𝑥 and 𝑦2𝐻 = 𝑒𝜆2 𝑥 .
For these solutions to form a basis for the solution space of the homogeneous
linear ODE, they should be linear independence. We evaluate the Wronskian:

𝑒𝜆1 𝑥 𝑒𝜆2 𝑥
𝑊 (𝑥) = det [ 𝜆1 𝑥 ] = 𝑒(𝜆1 +𝜆2 )𝑥 (𝜆2 − 𝜆1 ).
𝜆1 𝑒 𝜆2 𝑒𝜆2 𝑥

So, if the roots of the characteristics equation are distinct (𝜆1 ≠ 𝜆2 ), then
𝑊 (𝑥) ≠ 0 and the solutions form a linearly independent set. So we have:
𝐻
𝑦𝐶𝐹 = 𝑦𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 ) = 𝑐1 𝑒𝜆1 𝑥 + 𝑐2 𝑒𝜆2 𝑥 .
42 CHAPTER 5. LINEAR ODES

𝛼1
For the case of 𝜆1 = 𝜆2 = − 2𝛼 , we have 𝑦1 = 𝑒𝜆1 𝑥 , what about the second
2
solution 𝑦2 ? We can try the ansatz 𝑦2 = 𝐴(𝑥)𝑦1 (𝑥) = 𝐴(𝑥)𝑒𝜆1 𝑥 . This is similar
to the method of variation of parameters. In the context of 2nd order linear
ODEs, when we have one of the solutions and looking for the second solution,
this method is called the method of reduction of order. Plugging this ansatz
into the ODE we obtain:

𝑑𝐴 𝑑𝑦 𝑑2 𝐴 𝑑𝐴 𝑑𝑦1 𝑑2 𝑦
𝛼0 [𝐴𝑦1 ] + 𝛼1 [ 𝑦1 + 𝐴 1 ] + 𝛼2 [ 2 𝑦1 + 2 + 𝐴 21 ] = 0.
𝑑𝑥 𝑑𝑥 𝑑𝑥 𝑑𝑥 𝑑𝑥 𝑑𝑥

This result in the following simple ODE and solution for 𝐴(𝑥):

𝑑2 𝐴
=0 ⇒ 𝐴(𝑥) = 𝐵1 𝑥 + 𝐵2 ⇒ 𝑦2 = (𝐵1 𝑥 + 𝐵2 )𝑒𝜆1 𝑥 .
𝑑𝑥2
We note that 𝑦2 we have obtained here contains 𝑦1 , so we can choose 𝑦2 =
𝑥𝑒𝜆1 𝑥 . Testing the linear independence of these solutions, we should evaluate
the Wronskian:

𝑒𝜆1 𝑥 𝑥𝑒𝜆1 𝑥
𝑊 (𝑥) = det [ 𝜆1 𝑥 𝜆1 𝑥 ] = 𝑒2𝜆1 𝑥 ≠ 0.
𝜆1 𝑒 𝑒 + 𝜆1 𝑥𝑒𝜆1 𝑥

So 𝑦1 and 𝑦2 are linearly independent and can span the solution space. So we
have the following general solution for the case characteristic equation has the
repeated root 𝜆1 :
𝐻
𝑦𝐶𝐹 = 𝑦𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 ) = 𝑐1 𝑒𝜆1 𝑥 + 𝑐2 𝑥𝑒𝜆1 𝑥 .

Possible behaviours of the 2nd order linear homogeneous ODE

If 𝜆1 ≠ 𝜆2 then 𝑦𝐶𝐹 = 𝑐1 𝑒𝜆1 𝑥 + 𝑐2 𝑒𝜆2 𝑥 .

𝛼1 𝛼2 − 4𝛼0 𝛼2
𝜆1,2 = − ±√ 1 .
2𝛼2 4𝛼22

𝛼21 − 4𝛼0 𝛼2 > 0 ⟹ 𝜆1,2 ∈ ℝ

1. 𝜆1,2 can be both positive, both negative or one positive/one negative.
• If 𝜆1 > 0 and 𝜆1 > 𝜆2

as 𝑥 → ∞, 𝑦𝐶𝐹 → 𝑒𝜆1 𝑥 → ∞.

• If 𝜆2 < 𝜆1 < 0
as 𝑥 → ∞, 𝑦𝐶𝐹 → 𝑒𝜆1 𝑥 → 0.
2. 𝛼21 − 4𝛼0 𝛼2 < 0 ⟹ 𝜆1,2 ∈ ℂ
5.3. SECOND ORDER LINEAR ODES WITH CONSTANT COEFFICIENTS43

𝛼21 − 4𝛼0 𝛼2 𝛼1
∣ ∣ = 𝜔2 ⟹ 𝜆1,2 = − ± 𝑖𝜔
4𝛼22 2𝛼2
So, we have for the general solution:
𝛼 𝛼
− 2𝛼1 𝑥 − 2𝛼1 𝑥
𝑦𝐶𝐹 = 𝑒 2 [𝑐1 𝑒𝑖𝜔𝑥 + 𝑐2 𝑒−𝑖𝜔𝑥 ] = 𝑒 2 [(𝑐1 + 𝑐2 ) cos 𝜔𝑥 + 𝑖(𝑐1 − 𝑐2 ) sin 𝜔𝑥] .

If the ODE has real coefficients the solution 𝑦𝐶𝐹 ∈ ℝ. Therefore, choosing 𝑐1 and
𝑐2 to be complex conjugate, we obtain 𝑐1′ = 𝑐1 + 𝑐2 ∈ ℝ and 𝑐2′ = 𝑖(𝑐1 − 𝑐2 ) ∈ ℝ.
So the we can write the general solution of the homogeneous ODE with complex
roots in following forms:
𝛼 𝛼
− 2𝛼1 𝑥 − 2𝛼1 𝑥
𝑦𝐶𝐹 = 𝑒 2 [𝑐1′ cos 𝜔𝑥 + 𝑐2′ sin 𝜔𝑥] = 𝑒 2 𝐴 cos(𝜔𝑥 − 𝜙),

where in the later, we have used the following change of constants of integrations
𝑐1′ = 𝐴 cos 𝜙 and 𝑐2′ = 𝐴 sin 𝜙. Figure 5.1 shows possible behaviors of 𝑦𝐶𝐹
𝛼1
depending on the value of the parameter 𝑑 = 2𝛼 .
2
4
2
y

0
−2

0 1 2 3 4 5 6

Figure 5.1: 𝑦𝐶𝐹 for 𝑑 > 0 (red), 𝑑 < 0 (green) and 𝑑 = 0 (blue); all three
solutions have the same phase but for clarity of visualisation different amplitudes
(𝐴) are used in each case.

Example 5.4. Find the general solution of

𝑑2 𝑦 𝑑𝑦
ℒ[𝑦] = 2
−3 + 2𝑦 = 𝑒8𝑥 .
𝑑𝑥 𝑑𝑥
• First step: The characteristic equation is 𝜆2 − 3𝜆 + 2 = 0, so we have
𝜆1 = 2 and 𝜆2 = 1, so
𝑦𝐶𝐹 = 𝑐1 𝑒2𝑥 + 𝑐2 𝑒𝑥 .
44 CHAPTER 5. LINEAR ODES

• Second step: we try ansatz 𝑦𝑃 𝐼 = 𝐴𝑒8𝑥 .

1
ℒ[𝑦𝑃 𝐼 ] = 𝐴𝑒8𝑥 [64 − 24 + 2] = 𝑒8𝑥 ⇒ 𝐴= ,
42
So, we have:
1 8𝑥
𝑦𝐺𝑆 = 𝑦𝐶𝐹 + 𝑦𝑃 𝐼 = 𝑐1 𝑒2𝑥 + 𝑐2 𝑒𝑥 + 𝑒 .
42

Example 5.5. Find the general solution of

𝑑2 𝑦 𝑑𝑦
ℒ[𝑦] = 2
+4 + 4𝑦 = 𝑒−2𝑥 .
𝑑𝑥 𝑑𝑥
• First step: This case has a charecteristic equation with repeated root
𝜆 = −2, so we have
𝑦𝐶𝐹 = 𝑐1 𝑒−2𝑥 + 𝑐2 𝑥𝑒−2𝑥 .

• Second step: Finding a particular integral ℒ[𝑦𝑃 𝐼 ] = 𝑓(𝑥). 1st try ansatz
𝑦𝑃 𝐼 = 𝐴𝑒−2𝑥 , which does not work. 2nd try ansatz 𝑦𝑃 𝐼 = 𝐴𝑥𝑒−2𝑥 , which
also does not work. Let’s try ansatz: 𝑦𝑃 𝐼 = 𝐴(𝑥)𝑒−2𝑥 using the method
of variation of parameters. By plugging into the ODE we obtain:
𝑑2 𝐴 𝑥2
=1 ⇒ 𝐴= + 𝐵1 𝑥 + 𝐵 2 .
𝑑𝑥2 2
So, we obtain the general solution:
𝑥2 −2𝑥
𝑦𝐺𝑆 = 𝑦𝐶𝐹 + 𝑦𝑃 𝐼 = 𝑐1 𝑒−2𝑥 + 𝑐2 𝑥𝑒−2𝑥 + 𝑒 .
2
Instead of using the method of variation of parameters, we could have
guessed the ansatz 𝑦𝑃 𝐼 = 𝐵𝑥2 𝑒−2𝑥 directly and obtaining value of 𝐵 using
the method of undetermined coefficients.

Example 5.6. Find the general solution of

𝑑2 𝑦 𝑑𝑦
ℒ[𝑦] = 2
−2 + 2𝑦 = 𝑒𝑥 sin(𝑥)
𝑑𝑥 𝑑𝑥

- First step: Solving the Homogeneous problem ℒ[𝑦𝐻 ] = 0. In this case roots
are complex, so we have:

𝑦𝐶𝐹 = 𝑐1 𝑒(1+𝑖)𝑥 + 𝑐2 𝑒(1−𝑖)𝑥 .

- Second step: Finding a particular integral ℒ[𝑦𝑃 𝐼 ] = 𝑓(𝑥).

Here 𝑓(𝑥) can be written as the sum of two functions

𝑒(1+𝑖)𝑥 𝑒(1−𝑖)𝑥
𝑓(𝑥) = 𝑒𝑥 sin 𝑥 = − .
2𝑖 2𝑖
5.4. 𝐾TH ORDER LINEAR ODES WITH CONSTANT COEFFICIENTS 45

So, we can use the following two ansatz to find particular integrals using the
method of undetermined coefficients: 𝑦𝑃1 𝐼 = 𝐴𝑥𝑒(1+𝑖)𝑥 and 𝑦𝑃2 𝐼 = 𝐴𝑥𝑒(1−𝑖)𝑥 . By
plugging into the ODE using the first and second part of 𝑓(𝑥), one finds values
for 𝐴 and 𝐵 respectively (left as an exercise for you) Then the general solution
is:
𝑦𝐺𝑆 = 𝑦𝐶𝐹 + 𝑦𝑃1 𝐼 + 𝑦𝑃2 𝐼 .

5.4 𝑘th order Linear ODEs with constant coeffi-

cients
𝑘
ℒ[𝑦] = ∑ 𝛼𝑖 𝒟𝑖 [𝑦] = 𝑓(𝑥); 𝛼𝑖 ∈ ℝ
𝑖=0

𝐻
𝑦𝐺𝑆 (𝑥; 𝑐1 , ⋯ , 𝑐𝑘 ) = 𝑦𝐶𝐹 + 𝑦𝑃 𝐼 = 𝑦𝐺𝑆 (𝑥; 𝑐1 , ⋯ , 𝑐𝑘 ) + 𝑦𝑃 𝐼 (𝑥)
First step: Solving the Homogeneous problem ℒ[𝑦𝐻 ] = 0
We can try the ansatz 𝑦𝐻 = 𝑒𝜆𝑥 :

𝑘 𝑘
ℒ[𝑒𝜆𝑥 ] = 𝑒𝜆𝑥 ∑ 𝛼𝑖 𝜆𝑖 = 0 ⇒ ∑ 𝛼𝑖 𝜆𝑖 = 0.
𝑖=0 𝑖=0

This is the characterstic equation of the 𝑘th order linear ODE. It has 𝑘 roots that
can be always obtained numerically (in the absence of an analytical solution).
• Case 1: 𝑘 roots of the characteristic polynomial are distinct:
The solutions 𝐵 = {𝑒𝜆𝑖 𝑥 }𝑘𝑖=1 can be shown to be linearly independent using the
Wronskian:

𝑒𝜆1 𝑥 𝑒𝜆2 𝑥 ⋯ 𝑒𝜆𝑘 𝑥

⎡ 𝜆 𝑒𝜆1 𝑥 𝜆2 𝑒𝜆2 𝑥 ⋯ 𝜆𝑘 𝑒𝜆𝑘 𝑥 ⎤
⎢ 1 ⎥
𝕎(𝑥) = ⎢ ⋮ ⋮ ⋮ ⎥
⎢ ⎥
⎣𝜆𝑘−1
1 𝑒
𝜆1 𝑥
𝜆𝑘−1
2 𝑒
𝜆2 𝑥
⋯ 𝜆𝑘−1
𝑘 𝑒 𝜆𝑘 𝑥
⎦

1 1 ⋯ 1
∣ ∣
𝜆 𝜆2 ⋯ 𝜆𝑘
𝑘 ∣ 1 ∣
𝑊 (𝑥) = det 𝕎(𝑥) = 𝑒∑𝑖=1 𝜆𝑖 𝑥 ∣ ⋮ ⋮ ⋮ ∣=
∣ ∣
∣𝜆𝑘−1
1 𝜆𝑘−1
2 ⋯ 𝜆𝑘−1
𝑘 ∣

𝑘
𝑒∑𝑖=1 𝜆𝑖 𝑥 ∏ (𝜆𝑖 − 𝜆𝑗 ) ≠ 0; (Vandermonde determinant)
1≤𝑖<𝑗≤𝑘
46 CHAPTER 5. LINEAR ODES

The determinant of the Vandermonde matrix (the matrix obtained above) is a

well known result in linear algebra and can be proven using induction.
𝑘
𝐵 = {𝑒𝜆1 𝑥 , 𝑒𝜆2 𝑥 , ⋯ , 𝑒𝜆𝑘 𝑥 } ⇒ 𝑦𝐶𝐹 = ∑ 𝑐𝑖 𝑒𝜆𝑖 𝑥 .
𝑖=1

• Case 2: Not all of the 𝑘 roots are distinct. Below, we consider the
particular case of having 𝑑 repeated roots and 𝑘 − 𝑑 distinct roots.

𝐵 = {𝑒𝜆1 𝑥 , 𝑒𝜆2 𝑥 , ⋯ , 𝑒𝜆𝑟 𝑥 , 𝑥𝑒𝜆𝑟 𝑥 , ⋯ , 𝑥𝑑−1 𝑒𝜆𝑟 𝑥 , 𝑒𝜆𝑟+1 𝑥 , ⋯ , 𝑒𝜆𝑘−𝑑+1 𝑥 } ⇒

𝜆1 𝑥 𝜆2 𝑥 𝜆𝑟 𝑥 𝜆𝑟 𝑥 𝑑−1 𝜆𝑟 𝑥 𝜆𝑟+1 𝑥
𝑦𝐶𝐹 = 𝑐1 𝑒 , 𝑐2 𝑒 , ⋯ , 𝑐𝑟 𝑒 , 𝑐𝑟+1 𝑥𝑒 , ⋯ , 𝑐𝑟+𝑑−1 𝑥 𝑒 , 𝑐𝑟+𝑑 𝑒 , ⋯ , 𝑐𝑘 𝑒𝜆𝑘−𝑑+1 𝑥 .
Second step: Finding a particular integral for example for: ℒ[𝑦𝑃 𝐼 ] = 𝑒𝑏𝑥 , for the
case 2 above, we use the following ansatz, using the method of undetermined
coefficients:
• if 𝑏 ≠ 𝜆𝑖 for ∀𝑖 then 𝑦𝑃 𝐼 = 𝐴𝑒𝑏𝑥 .
• if 𝑏 = 𝜆𝑖 for 𝑖 ≠ 𝑟 then 𝑦𝑃 𝐼 = 𝐴𝑥𝑒𝑏𝑥 .
• if 𝑏 = 𝜆𝑟 then 𝑦𝑃 𝐼 = 𝐴𝑥𝑑 𝑒𝑏𝑥 .

5.5 Euler-Cauchy equation

A (rare) example of a linear ODE with non-constant coefficients that we can
solve analytically is the Euler-Cauchy ODE:

𝑑𝑘 𝑦 𝑘−1 𝑑
𝑘−1
𝑦 𝑑𝑦
ℒ[𝑦] = 𝛽𝑘 𝑥𝑘 𝑘
+ 𝛽 𝑘−1 𝑥 𝑘−1
+ ⋯ + 𝛽1 𝑥 + 𝛽0 𝑦 = 𝑓(𝑥).
𝑑𝑥 𝑑𝑥 𝑑𝑥
Using the change of variable 𝑥 = 𝑒𝑧 , the Euler-Cauchy equation can be trans-
formed into a linear ODE with constant coefficients.
2
𝑑 𝑦 𝑑𝑦
Example 5.7. Solve the following ODE: 𝑥2 𝑑𝑥 3
2 + 3𝑥 𝑑𝑥 + 𝑦 = 𝑥 .

Using the change of variable 𝑥 = 𝑒𝑧 we have 𝑧 = ln 𝑥 and so:

𝑑𝑦 𝑑𝑦 𝑑𝑧 1 𝑑𝑦 𝑑𝑦 𝑑𝑦
= = ⇒ 𝑥 = ,
𝑑𝑥 𝑑𝑧 𝑑𝑥 𝑥 𝑑𝑧 𝑑𝑥 𝑑𝑧

𝑑2 𝑦 𝑑 𝑑𝑦 𝑑 𝑑𝑦 𝑑𝑧 1 𝑑2 𝑦 𝑑𝑦
2
= ( )= ( ) = 2[ 2 − ].
𝑑𝑥 𝑑𝑥 𝑑𝑥 𝑑𝑧 𝑑𝑥 𝑑𝑥 𝑥 𝑑𝑧 𝑑𝑧
So, in terms of the new independent variable we have the following linear ODE
with constant coefficients.

𝑑2 𝑦 𝑑𝑦
2
+2 + 𝑦 = 𝑒3𝑧 .
𝑑𝑧 𝑑𝑧
5.6. USING FOURIER TRANSFORMS TO SOLVE LINEAR ODES 47

By obtaining the complementary function and particular integral, we have the

following general solution:
1 3𝑧
𝑦𝐺𝑆 (𝑧; 𝑐1 , 𝑐2 ) = 𝑦𝐶𝐹 + 𝑦𝑃 𝐼 = 𝑐1 𝑒−𝑧 + 𝑐2 𝑧𝑒−𝑧 + 𝑒 .
16
So, in the general solution in terms of 𝑥 is:
𝑐1 ln 𝑥 1
𝑦𝐺𝑆 (𝑥; 𝑐1 , 𝑐2 ) = + 𝑐2 + 𝑥3 .
𝑥 𝑥 16

5.6 Using Fourier Transforms to solve linear

ODEs
As Fourier transform is a linear operation, and given the properties we had for
Fourier transforms of derivatives of a function seen in Section 2.1, one can use
Fourier transforms to solve linear ODEs or find particular integrals. This is
particularly relevant for solving partial differential equations as discussed in the
second year. Here we discuss an example.

Example 5.8. Find a solution for the following ODE, known as the Airy equa-
tion or the Stokes equation, which arises in different areas of physics. Assume
lim𝑥→±∞ 𝑦(𝑥) = 0.
𝑑2 𝑦
− 𝑥𝑦 = 0.
𝑑𝑥2

This is a linear 2nd order ODE with non-constant coefficients and so far we have
not seen a method of solving it. Note that this ODE is not also one of the types
that are discussed in Section 4.2. Our strategy is to take Fourier transform from
this ODE and see if we can solve for the Fourier transform. Using the properties
in Section 2.1, we obtain:
𝑑𝑦(𝜔)
̂
−𝜔2 𝑦(𝜔)
̂ −𝑖 =0
𝑑𝜔

This is a first order ODE for 𝑦(𝜔),

̂ by solving it we obtain:
𝑖𝜔3
𝑦(𝜔)
̂ = 𝑐𝑒 3 ,

where 𝑐 is an arbitrary constant of integration. Using the inverse transform we

obtain: ∞ ∞
𝑐 3 𝑐 𝜔3
𝑦(𝑥) = ∫ 𝑒𝑖(𝜔𝑥+𝜔 /3) 𝑑𝜔 = ∫ cos(𝜔𝑥 + ) 𝑑𝜔,
2𝜋 −∞ 𝜋 0 3
where, in the last step we have used the evenness of cosine and oddness of the
sine function. For 𝑐 = 1 the function 𝑦(𝑥) is known as the Airy function (of
the first kind) and is denoted by 𝐴𝑖(𝑥). It is defined as the above integral and
cannot be reduced further and is a solution of the Airy ODE.
48 CHAPTER 5. LINEAR ODES
Chapter 6

Introduction to Systems of
ODEs

So far we have discussed ordinary differential equations where the function we

have been looking for was a scaler function (𝑦(𝑥) ∶ ℝ → ℝ). Where the unknown
function is a vector (𝑦(𝑥)
⃗ ∶ ℝ → ℝ𝑛 ), we are dealing with systems of ODEs.
Definition Systems of Ordinary Differential Equations have the following gen-
eral form:

𝑑𝑦1 𝑑𝑦 𝑑 𝑘 1 𝑦1 𝑑𝑘𝑛 𝑦𝑛
𝐺1 (𝑥, 𝑦1 , 𝑦2 , ⋯ , 𝑦𝑛 , ,⋯, 𝑛,⋯, , ⋯ , ) = 0,
𝑑𝑥 𝑑𝑥 𝑑𝑥𝑘1 𝑑𝑥𝑘𝑛

𝑑𝑦1 𝑑𝑦 𝑑𝑘1 𝑦1 𝑑𝑘𝑛 𝑦𝑛

𝐺𝑛 (𝑥, 𝑦1 , 𝑦2 , ⋯ , 𝑦𝑛 , ,⋯, 𝑛,⋯, , ⋯ , ) = 0.
𝑑𝑥 𝑑𝑥 𝑑𝑥𝑘1 𝑑𝑥𝑘𝑛
The system is ordinary as we still have one independent variable 𝑥, but now
in contrast to single ODEs, we have 𝑛 functions of independent variables
𝑦1 (𝑥), 𝑦2 (𝑥), ⋯ , 𝑦𝑛 (𝑥) to solve for. This is the implicit form but the systems of
ODEs can be written in explicit from as well. Many problems in physics and
biology give rise to systems of ODEs. Here are few examples:

Example 6.1 (Predator-prey systems).

These models can be used to predict the dynamics of predator and prey systems
such as rabbits (𝑥(𝑡)) and foxes (𝑦(𝑡)). A classic model is Lotka-Volterra model
(1925/26) that can exhibit a periodic solution of the population of predator and
preys as one goes up and the other goes down.

49
50 CHAPTER 6. INTRODUCTION TO SYSTEMS OF ODES

𝑑𝑥
= 𝑎𝑥 − 𝑏𝑥𝑦,
𝑑𝑡
𝑑𝑦
= 𝑑𝑥𝑦 − 𝑐𝑦.
𝑑𝑡

Example 6.2 (Chemistry and biochemistry).

The chemical rate equations for a set of chemical reactions. For example, con-
sider the reversible binary reaction 𝐴 + 𝐵 ⇌ 𝐶 with forward rate of 𝑘1 and
backward rate of 𝑘2 . We have the following rate equations for the concentra-
tions [𝐴], [𝐵] and [𝐶].

𝑑[𝐴] 𝑑[𝐵]
= = 𝑘2 [𝐶] − 𝑘1 [𝐴][𝐵],
𝑑𝑡 𝑑𝑡
𝑑[𝐶] 𝑑[𝐴]
=− = −𝑘2 [𝐶] + 𝑘1 [𝐴][𝐵].
𝑑𝑡 𝑑𝑡
These kind of equations are used in mathematical modelling of biochemical
reaction networks in systems and synthetic biology.

Example 6.3 (Coupled spring-mass systems).

This is an example from mechanics. Consider the system of 3 masses and 4

springs that are fixed between two walls. We can write equations of motions
that describe the position of the 3 masses as a function of time (𝑥1 (𝑡), 𝑥2 (𝑡) and
𝑥3 (𝑡)) as seen in Figure 6.1.

Figure 6.1: Diagram of the mass and spring system

Using the second Newton law for each mass (𝐹 = 𝑚𝑎) and the Hook’s law
(𝐹 = −𝑘Δ𝑥) for the springs and given the relaxed lengths of each spring (𝑙1 to
6.1. SYSTEMS OF FIRST ORDER ODES 51

𝑙4 ), and assuming the distance between the walls is the sum of the the relaxed
lengths of the springs, we have the following system of ODEs:

𝑑 2 𝑥1
= −𝑘1 (𝑥1 − 𝑙1 ) + 𝑘2 (𝑥2 − 𝑥1 − 𝑙2 ),
𝑑𝑡2
2
𝑑 𝑥2
= −𝑘2 (𝑥2 − 𝑥1 − 𝑙2 ) + 𝑘3 (𝑥2 − 𝑥2 − 𝑙3 ),
𝑑𝑡2
𝑑 2 𝑥3
= −𝑘3 (𝑥3 − 𝑥2 − 𝑙3 ) + 𝑘4 (𝑙1 + 𝑙2 + 𝑙3 − 𝑥3 ).
𝑑𝑡2

Example 6.4 (SIR model of an epidemic).

One of the simplest but influential models of an epidemic is the compartmental

SIR model. In this model, disease propagates between the population compart-
ments through susceptible individuals (𝑆) becoming infected (𝐼) after encoun-
tering other infected individuals with rate 𝛽 and finally moving to a recovered
population (𝑅) with rate 𝛾. The following simple system of ODEs describes the
dynamics of this compartmental model.

𝑑𝑆
= −𝛽𝑆𝐼,
𝑑𝑡
𝑑𝐼
= 𝛽𝑆𝐼 − 𝛾𝐼,
𝑑𝑡
𝑑𝑅
= 𝛾𝐼.
𝑑𝑡

This model and its variations are the basis of a lot of mathematical modelling
that has been performed for predicting the effect of different interventions in
the Covid-19 world-wide pandemic. If you like to learn a bit more about the
SIR ODE system above check out this video. Also, the SIR model can be
modeled using a so-called agent-based stochastic approach, where the individuals
and their random interactions are specifically followed. You can check out this
interesting video from the 3blue1brown series to see some cool exploration of
this approach to SIR models.

6.1 Systems of first order ODEs

Systems of ODEs of general order can be rewritten in terms of systems of first
order ODEs, so the following system written in explicit form is more general
than it seems.

𝑑𝑦1
= 𝐹1 (𝑥, 𝑦1 , 𝑦2 , ⋯ , 𝑦𝑛 ),
𝑑𝑥
52 CHAPTER 6. INTRODUCTION TO SYSTEMS OF ODES

⋮
𝑑𝑦𝑛
= 𝐹𝑛 (𝑥, 𝑦1 , 𝑦2 , ⋯ , 𝑦𝑛 ).
𝑑𝑥

Example 6.5 (Turning a higher order ODE into systems of 1st order ODEs).

The following second order ODE is the equation of motion for damped harmonic
oscillator for a mass 𝑚 that is attached to an ideal spring which follows the
Hook’s law (𝐹𝑆 = −𝑘𝑥, where 𝑥 is position of the mass measured from the
spring relaxed position) and has a damping friction force that is propotional
and opposite in direction to its velocity 𝐹𝐷 = −𝜂 𝑑𝑥𝑑𝑡 and is acted on by a
deriving force 𝐹 (𝑡).

𝑑2 𝑥 𝑑𝑥
𝑚 +𝜂 + 𝑘𝑥 = 𝐹 (𝑡).
𝑑𝑡2 𝑑𝑡
By defining new variable 𝑢 = 𝑑𝑥/𝑑𝑡 as the velocity of the mass 𝑚, we can turn
this second order ODE to a system of two first order ODEs:
𝑑𝑥
= 𝑢,
𝑑𝑡
𝑑𝑢 1
= [𝐹 (𝑡) − 𝜂𝑢 − 𝑘𝑥].
𝑑𝑡 𝑚

We can use a general vector notation to write systems of 1st order ODEs as
𝑑𝑦𝑛×1
⃗ ⃗ (𝑡, 𝑦𝑛×1
= 𝐹𝑛×1 ⃗ ).
𝑑𝑡

Here 𝑛 is the number of equations, 𝑡 is the independent variable and 𝑦 ⃗ is the

function we are looking for. In the next section, we discuss an important subclass
of these systems.

6.2 Systems of linear 1st order ODEs with con-

stant coefficients
Systems of linear 1st order ODEs with constant coefficients is an important class
that we will discuss their solutions in detail. They have the following general
form.
𝑛
𝑑𝑦1
= ∑ 𝛼1𝑖 𝑦𝑖 + 𝑔1 (𝑡),
𝑑𝑡 𝑖=1
⋮
𝑛
𝑑𝑦𝑛
= ∑ 𝛼𝑛𝑖 𝑦𝑖 + 𝑔𝑛 (𝑡),
𝑑𝑡 𝑖=1
6.2. SYSTEMS OF LINEAR 1ST ORDER ODES WITH CONSTANT COEFFICIENTS53

where 𝛼𝑖𝑗 ∈ ℝ are the constant coefficients forming the matrix 𝐴𝑛×𝑛 . We can
write the system of linear ODEs in matrix form:
𝑑𝑦
𝑑𝑡
1
𝛼11 𝛼12 … 𝛼1𝑛 𝑦1 𝑔1 (𝑡)
⎡ 𝑑𝑦 2 ⎤ ⎡𝛼 𝛼22 … 𝛼2𝑛 ⎤ ⎡ 𝑦2 ⎤ ⎡ 𝑔2 (𝑡) ⎤
⎢ 𝑑𝑡 ⎥ = ⎢ 21 ⎥⎢ ⎥ + ⎢ ⎥.
⎢ ⋮ ⎥ ⎢ ⋮ ⋮ ⋱ ⋮ ⎥⎢ ⋮ ⎥ ⎢ ⋮ ⎥
𝑑𝑦
⎣ 𝑑𝑡𝑛 ⎦ ⎣𝛼𝑛1 𝛼𝑛2 … 𝛼𝑛𝑛 ⎦ ⎣𝑦𝑛 ⎦ ⎣𝑔𝑛 (𝑡)⎦

We can write this briefly as

𝑑𝑦 ⃗
= 𝐴𝑦 ⃗ + 𝑔(𝑡).
⃗
𝑑𝑡

𝑑
Now if we define the linear operator ℒ[𝑦]⃗ = [ 𝑑𝑡 − 𝐴]𝑦 ⃗ then we can write the
system of linear first order ODEs as

ℒ[𝑦]⃗ = 𝑔(𝑡).
⃗

𝑑
Since both matrix 𝐴 and derivative 𝑑𝑡 are linear operators, the operator asso-
ciated to systems of linear ODEs ℒ is also a linear operator. By linearity we
have:
ℒ[𝜆1 𝑦1⃗ + 𝜆2 𝑦2⃗ ] = 𝜆1 ℒ[𝑦1⃗ ] + 𝜆2 ℒ[𝑦2⃗ ].
Therefore, the solutions of the homogenous systems of 𝑛 linear ODEs ℒ[𝑦𝐻 ⃗ ]=0
forms a vector space of dimension 𝑛. So a set of linearly independent solutions
𝐵 = {𝑦𝑖⃗ }𝑛𝑖=1 form a basis for this space. Therefore, similar to linear ODEs, the
general solution can be written as
𝑛
𝐻
𝑦𝐺𝑆
⃗ = ∑ 𝑐𝑖 𝑦𝑖⃗ ,
𝑖=1

where 𝑐𝑖 s are 𝑛 arbitrary constants of integration.

The general solution of non-homogenous systems of 1st order linear
ODEs
Similar to the case of linear ODEs, here also we find the general solution in two
steps
1. Obtain complimentary function 𝑦𝐶𝐹
⃗ by solving the corresponding homoge-
nous systems of ODEs (ℒ[𝑦𝐶𝐹
⃗ ] = 0)
2. Find a particular integral 𝑦𝑃⃗ 𝐼 that satisfies the full non-homogenous sys-
tems of ODEs (ℒ[𝑦𝑃⃗ 𝐼 ] = 𝑔(𝑡)).
⃗
Then, for the general solution 𝑦𝐺𝑆
⃗ we have:

𝑦𝐺𝑆
⃗ (𝑡; 𝑐1 , 𝑐2 , ⋯ , 𝑐𝑛 ) = 𝑦𝐶𝐹
⃗ + 𝑦𝑃⃗ 𝐼 .

Solving the homogenous problem

54 CHAPTER 6. INTRODUCTION TO SYSTEMS OF ODES

𝑑𝑦𝐻
⃗
ℒ[𝑦𝐻
⃗ ]=0 ⟹ = 𝐴𝑦𝐻
⃗
𝑑𝑡
First, we consider the case where matrix 𝐴 has 𝑛 distinct roots and
therefore is diagonalizable.
That means that there exists a matrix 𝑉 where, we have:

𝜆1 0 0
𝑉 −1 𝐴𝑉 = Λ = ⎡
⎢0 ⋱ 0⎤ ⎥.
⎣0 0 𝜆𝑛 ⎦

where, 𝑖th column of the matrix 𝑉 is the eigenvector of matrix 𝐴 corresponding

to the eigenvalue 𝜆𝑖 :
𝐴𝑣𝑖⃗ = 𝜆𝑖 𝑣𝑖⃗ .

We use the eigenvectors matrix 𝑉 to obtain the solution of the homogenous

system.
𝑑𝑦 ⃗ 𝑑𝑦 ⃗
= 𝐴𝑦 ⃗ ⟹ 𝑉 −1 = 𝑉 −1 𝐴𝑉 𝑉 −1 𝑦.⃗
𝑑𝑡 𝑑𝑡
Letting 𝑧 ⃗ = 𝑉 −1 𝑦,⃗ we can write the system of ODEs as

𝑑𝑧 ⃗
= Λ𝑧.⃗
𝑑𝑡
𝑑𝑧𝑖
The 𝑖th row of this equation gives us 𝑑𝑡 = 𝜆𝑖 𝑧𝑖 , which can be solved, so we
obtain:

𝑐1 𝑒𝜆1 𝑡 𝑐 𝑒𝜆1 𝑡
⎡ 𝑐 𝑒𝜆2 𝑡 ⎤ ⋮ ⋮ ⋮ ⎡ 1 𝜆2 𝑡 ⎤
𝑐2 𝑒
𝑍⃗ = ⎢ 2 ⎥ ⟹ ⃗ = 𝑉 𝑍⃗ = ⎡
𝑦𝐻 ⎢𝑣1⃗ 𝑣2⃗ ⋯ 𝑣𝑛⃗ ⎤
⎥⎢ ⎥.
⎢ ⋮ ⎥ ⎢ ⋮ ⎥
𝜆𝑛 𝑡 ⎣⋮ ⋮ ⋮ ⎦ 𝜆𝑛 𝑡
⎣𝑐𝑛 𝑒 ⎦ ⎣𝑐𝑛 𝑒 ⎦

Therefore, we obtain the general solution of the homogenous system of first

order linear ODEs to be:
𝐻
𝑦𝐶𝐹 ⃗ = 𝑐1 𝑒𝜆1 𝑡 𝑣1⃗ + 𝑐2 𝑒𝜆2 𝑡 𝑣2⃗ + ⋯ + 𝑐𝑛 𝑒𝜆𝑛 𝑡 𝑣𝑛⃗ .
⃗ = 𝑦𝐺𝑆

Finding particular integrals

ℒ[𝑦𝑃⃗ 𝐼 ] = 𝑔(𝑡).
⃗
We will use Ansatz and use the methods of undetermined coefficients and vari-
ation of parameters as done for the linear ODEs.

Example 6.6. Solve the system of ODEs for {𝑥(𝑡), 𝑦(𝑡)}.

6.2. SYSTEMS OF LINEAR 1ST ORDER ODES WITH CONSTANT COEFFICIENTS55

𝑑𝑥
= −4𝑥 − 3𝑦 − 5,
𝑑𝑡
𝑑𝑦
= 2𝑥 + 3𝑦 − 2.
𝑑𝑡

We can write this ODE in vector form as:

𝑑𝑦 ⃗ −4 −3 𝑥 −5
= 𝐴𝑦 ⃗ + 𝑔(𝑡)
⃗ =[ ][ ] + [ ].
𝑑𝑡 2 3 𝑦 −2

First step is to obtain 𝑦𝐶𝐹

⃗ . We obtain the eigenvalues 𝜆1 and 𝜆2 and eigenvectors
𝑣1⃗ and 𝑣2⃗ .

𝜆2 + 𝜆 − 6 = 0 ⟹ 𝜆1 = 2, 𝜆2 = −3.
We have the corresponding eigenvectors:

1 3
𝑣1⃗ = [ ], 𝑣2⃗ = [ ].
−2 −1

𝐻 1 3
𝑦𝐶𝐹 ⃗ (𝑡; 𝑐1 , 𝑐2 ) = 𝑐1 𝑒2𝑡 [
⃗ = 𝑦𝐺𝑆 ] + 𝑐2 𝑒−3𝑡 [ ] .
−2 −1

2nd step is to find any particular integral 𝑦𝑃⃗ 𝐼 (𝑡)} that satisfies:

−5
ℒ[𝑦𝑃⃗ 𝐼 ] = 𝑔(𝑡)
⃗ =[ ].
−2
We use the ansatz:
𝑎
𝑦𝑃⃗ 𝐼 = [ ] .
𝑏
By plugging the ansatz into the ODE we obtain the undetermined coefficients
𝑎 and 𝑏:
𝑎 5 1 3 3 5 −7/2
𝑦𝑃⃗ 𝐼 = [ ] = 𝐴−1 [ ] = − [ ][ ] = [ ].
𝑏 2 6 −2 −4 2 3

So we have:
1 3 −7/2
𝑦𝐺𝑆 ⃗ + 𝑦𝑃⃗ 𝐼 = 𝑐1 𝑒2𝑡 [ ] + 𝑐2 𝑒−3𝑡 [ ] + [
⃗ (𝑡; 𝑐1 , 𝑐2 ) = 𝑦𝐶𝐹 ].
−2 −1 3

Example 6.7. Solve the problem of damped Harmonic spring with zero forcing

𝑑2 𝑥 𝑑𝑥
𝑚 2
+𝜂 + 𝑘𝑥 = 0
𝑑𝑡 𝑑𝑡
56 CHAPTER 6. INTRODUCTION TO SYSTEMS OF ODES

This is a second order linear ODE and we can solve it using the methods dis-
cussed in the last chapter. Using the ansatz 𝑒𝜆𝑡 , we obtain the following char-
acteristic equation:

−𝜂 ± √𝜂2 − 4𝑘𝑚
𝑚𝜆2 + 𝜂𝜆 + 𝑘 = 0 ⇒ 𝜆1,2 = .
2𝑚
If the roots are distinct we obtain:

𝑥𝐺𝑆 = 𝑐1 𝑒𝜆1 𝑡 + 𝑐2 𝑒𝜆2 𝑡 .

Alternatively, we can transform the second order ODE to systems of first order
ODEs as seen before:
𝑑𝑥
= 𝑢,
𝑑𝑡
𝑑𝑢 𝜂 𝑘
= − 𝑢 − 𝑥.
𝑑𝑡 𝑚 𝑚
We obtain the same 𝜆1,2 as above for the eigenvalues of corresponding matrix
𝐴 for this system of ODEs.
For the eigenvectors (𝑣1⃗ and 𝑣2⃗ ) of 𝐴, we have:

0 1 𝑣1𝑥 𝑣
𝑣1⃗ ∶ 𝐴𝑣1⃗ = 𝜆1 𝑣1⃗ ⟹ [ 𝑘 𝜂 ][ ] = 𝜆1 [ 1𝑥 ] ,
−𝑚 −𝑚 𝑣1𝑢 𝑣1𝑢

1
𝑣1𝑢 = 𝜆1 𝑣1𝑥 ⟹ 𝑣1⃗ = [ ].
𝜆1
Similarly, we obtain for the second eigenvector:

1
𝑣2⃗ ∶ 𝐴𝑣2⃗ = 𝜆2 𝑣2⃗ ⟹ 𝑣2⃗ = [ ].
𝜆2

This gives us the following general solution:

𝑥𝐺𝑆 1 1
𝑦𝐺𝑆
⃗ =[ ] = 𝑐1 𝑒𝜆1 𝑡 [ ] + 𝑐2 𝑒𝜆2 𝑡 [ ] ,
𝑢𝐺𝑆 𝜆1 𝜆2

which gives us the same general solution for 𝑥𝐺𝑆 as obtained using the previous
method.
When 𝐴 has repeated eigenvalues
Case 1: 𝐴 is still diagonalizable (it has 𝑛 linearly independent eigenvectors).
Then we can still use the method described. For example for 𝑛 = 2 we have:

𝜆 0 1 0
𝐴=[ ] ⇒ 𝑣1⃗ = [ ] , 𝑣2⃗ = [ ] .
0 𝜆 0 1
6.2. SYSTEMS OF LINEAR 1ST ORDER ODES WITH CONSTANT COEFFICIENTS57

So, we have
1 0
⃗ = 𝑐1 𝑒𝜆𝑡 [ ] + 𝑐2 𝑒𝜆𝑡 [ ] .
𝑦𝐶𝐹
0 1

Case 2: 𝐴 is not diagonalizable (it has less than 𝑛 linearly independent

eigenvectors). Then we will use the Jordan normal form. We first discuss an
example and then see the general case.

Example 6.8.
𝑑𝑦 ⃗ 1 −1
= 𝐴𝑦;⃗ 𝐴=[ ].
𝑑𝑡 1 3

Matrix 𝐴 in this case has repeated roots:

(1 − 𝜆)(3 − 𝜆) + 1 = 0 ⇒ 𝜆1 = 𝜆2 = 2.

We next find the Eigenvector(s):

1 −1 𝑣1𝑥 𝑣
𝐴𝑣1⃗ = 𝜆1 𝑣1⃗ ⟹ [ ] [ ] = 2 [ 1𝑥 ] .
1 3 𝑣1𝑦 𝑣1𝑦

Which gives us:

1
𝑣1𝑥 = −𝑣1𝑦 ⇒ 𝑣1⃗ = [ ].
−1
So, this matrix only has one eigenvector and the matrix is not diagonalizable.
We look for similarity transformation to a Jordan normal form (𝐽 ; an almost
diagonal form as defined below). We look for a matrix of the form:

1 𝛼
𝑊 =[ ].
−1 𝛽

So that:
2 1
𝑊 −1 𝐴𝑊 = [ ] = 𝐽,
0 2
where, 𝐽 is the Jordan normal form for a 2 by 2 matrix. We have:

2 1 1 −1 1 𝛼 1 𝛼 2 1
𝐴𝑤 = 𝑤 [ ] ⇒ [ ][ ]=[ ][ ].
0 2 1 3 −1 𝛽 −1 𝛽 0 2

So, we obtain the following equations for 𝛼 and 𝛽.

𝛼 − 𝛽 = 1 + 2𝛼 1
⟹ 𝛼 + 𝛽 = −1 ⟹ 𝑤⃗ 2 = [ ],
𝛼 + 3𝛽 = −1 + 2𝛽 −2
58 CHAPTER 6. INTRODUCTION TO SYSTEMS OF ODES

giving us
1 1
𝑊 =[ ],
−1 −2
and we can check that 𝑊 −1 𝐴𝑊 = 𝐽 . The Jordan normal form allows us to
solve the non-diagonalizable systems of ODEs:
𝑑𝑦 ⃗ 𝑑𝑦 ⃗
= 𝐴𝑦 ⃗ ⟹ 𝑊 −1 = [𝑊 −1 𝐴𝑊 ] 𝑊 −1 𝑦.⃗
𝑑𝑡 𝑑𝑡
𝑑𝑧 ⃗
Letting 𝑧 ⃗ = 𝑊 −1 𝑦,⃗ we obtain 𝑑𝑡 = 𝐽 𝑧,⃗ so we have:
𝑑𝑧1
= 2𝑧1 + 𝑧2 ,
𝑑𝑡
𝑑𝑧2
= 2𝑧2 .
𝑑𝑡
We can solve the second ODE to obtain 𝑧2 = 𝑐2 𝑒2𝑡 and we can then use this
solution to solve for 𝑧1 from the first equation above:
𝑧1 = 𝑐1 𝑒2𝑡 + 𝑐2 𝑡𝑒2𝑡 .
So in vector form:
𝑧 𝑐 𝑒2𝑡 + 𝑐2 𝑡𝑒2𝑡
⃗ = [ 1] = [ 1
𝑧𝐺𝑆 ].
𝑧2 𝑐2 𝑒2𝑡
Now we can obtain the general solution for 𝑦𝐺𝑆
⃗ .
𝑐1 𝑒2𝑡 + 𝑐2 𝑡𝑒2𝑡
𝑦𝐺𝑆 ⃗ = [𝑣1⃗
⃗ = 𝑊 𝑧𝐺𝑆 𝑤⃗ 2 ] [ ] = (𝑐1 𝑒2𝑡 + 𝑐2 𝑡𝑒2𝑡 )𝑣1⃗ + 𝑐2 𝑒2𝑡 𝑤⃗ 2 .
𝑐2 𝑒2𝑡

The case of non-diagonalizable 𝐴𝑛×𝑛 with one repeated eigenvalue 𝜆

Assume 𝜆 is associated with only a single eigenvector. We can use the Jordan
normal form (𝐽 ) to obtain a solution to the associated systems of linear ODEs.
We look for a similarity transformation 𝑊 to transform 𝐴 to 𝐽 :
𝜆 1 0 0 0
⎡0 𝜆 1 0 0⎤
⎢ ⎥
𝑊 −1 𝐴𝑊 = 𝐽 = ⎢ 0 0 ⋱ 1 0⎥ .
⎢0 0 0 𝜆 1⎥
⎣0 0 0 0 𝜆⎦
𝑑𝑧 ⃗
Letting 𝑧 ⃗ = 𝑊 −1 𝑦,⃗ we obtain 𝑑𝑡 = 𝐽 𝑧,⃗ so we have:
𝑑𝑧𝑛
= 𝜆𝑧𝑛 ⇒ 𝑧𝑛 = 𝑐𝑛 𝑒𝜆𝑡
𝑑𝑡
𝑑𝑧𝑛−1
= 𝜆𝑧𝑛−1 + 𝑧𝑛 ⇒ 𝑧𝑛−1 = 𝑐𝑛−1 𝑒𝜆𝑡 + 𝑐𝑛 𝑡𝑒𝜆𝑡
𝑑𝑡
𝑑𝑧𝑛−2 2
= 𝜆𝑧𝑛−2 + 𝑧𝑛−1 ⇒ 𝑧𝑛−2 = 𝑐𝑛−2 𝑒𝜆𝑡 + 𝑐𝑛−1 𝑡𝑒𝜆𝑡 + 𝑐𝑛 𝑡2 𝑒𝜆𝑡
𝑑𝑡
⋮
𝑑𝑧1 𝑡 𝑛−1
= 𝜆𝑧1 + 𝑧2 ⇒ 𝑧1 = 𝑐1 𝑒𝜆𝑡 + 𝑐2 𝑡𝑒𝜆𝑡 + ⋯ + 𝑐𝑛 (𝑛−1)! 𝑒𝜆𝑡
𝑑𝑡
6.2. SYSTEMS OF LINEAR 1ST ORDER ODES WITH CONSTANT COEFFICIENTS59

So, we can obtain 𝑦𝐺𝑆

⃗ = 𝑊 𝑧𝐺𝑆
⃗ .
There could be situations where the matrix has some distinct eigenvalues and
some repeated eigenvalues, which will result in different Jordan normal forms.
For example, consider a matrix 𝐴3×3 with two distinct eigenvalues one repeated.
The suitable Jordan normal form would have the following form:

𝜆1 1 0
𝐽 =⎡
⎢0 𝜆1 0⎤ ⎥.
⎣0 0 𝜆2 ⎦
60 CHAPTER 6. INTRODUCTION TO SYSTEMS OF ODES
Chapter 7

Qualitative analysis of
ODEs

So far we have focused to obtain analytical solution to ODEs, but this is not
always possible. Even, when it is possible, it is not always very insightful. In
this section, we will focus on qualitative analysis of ODEs and as before we
mostly focus on linear ODEs. We’ll discuss asymptotics behavior, fixed points
(and their stability) and phase plane analysis.

7.1 asymptotic behaviour

In qualitative analysis of an ODE, asymptotic behaviour of solution 𝑦(𝑡) as
𝑡 → ∞ is one aspect of solutions we look at.

Example 7.1 (Population growth).

𝑑𝑃 (𝑡)
= 𝐾𝑃 (𝑡)
𝑑𝑡

Here, the solution is an exponential 𝑃 (𝑡) = 𝑃 (0)𝑒𝐾𝑡 . So for the asymptotic

behaviour we have:

𝐾>0 ⇒ lim 𝑃 (𝑡) → ∞, (7.1)

𝑡→∞
𝐾<0 ⇒ lim 𝑃 (𝑡) → 0. (7.2)
𝑡→∞

Fixed points of systems of first order ODEs

61
62 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

𝑦∗⃗ is a fixed point or an equilibrium point of a system of first order ODEs, if

once 𝑦(𝑡⃗ 0 ) = 𝑦∗⃗ at some time 𝑡0 then for all future times 𝑡 > 𝑡0 , state vector 𝑦 ⃗
remains equal to 𝑦∗⃗ . Thus, at fixed point we have:

𝑑𝑦 ⃗
[ ] =0
𝑑𝑡 𝑦= ⃗ 𝑦∗⃗

Example 7.2 (Logistic growth). A modified model for population growth that
does not lead to exponential growth for 𝐾 > 0 is the logistic growth model,
with a carrying capacity 𝐶:

𝑑𝑃 𝑃
= 𝐾𝑃 (1 − ).
𝑑𝑡 𝐶

This ODE has two fixed points:

𝑑𝑃
=0 ⇒ 𝑃1∗ = 0, 𝑃2∗ = 𝐶,
𝑑𝑡
with the former being an unstable fixed point and the latter being an stable one
as defined below.

Example 7.3 (Systems of Linear homogeneous ODEs).

𝑑𝑦 ⃗
= 𝐴𝑦.⃗
𝑑𝑡

A system of linear ODEs can have one or infinitely many fixed points:

⎧ 0
𝑑𝑦 ⃗ { 𝑦∗⃗ = [ ] , if Det(𝐴) ≠ 0,
=0 ⇒ 𝐴𝑦∗⃗ = 0 ⇒ ⎨
0
𝑑𝑡 { ∗
⎩ 𝑦 ⃗ could be a line or plane, if Det(𝐴) = 0.

Stability of fixed points

Informally, a fixed point is stable if whenever the initial state is near that point,
the state remains near it, perhaps even tending toward the equilibrium point as
time increases. Formally, we have two types of stability as described below.
Lyapunov stability
A fixed point 𝑦∗⃗ is said to be Lyapunov stable, if for every 𝜖 > 0, there exists a
⃗ − 𝑦∗⃗ ‖ < 𝛿, then for ∀𝑡 ≥ 0, we have ‖𝑦(𝑡)
𝛿 > 0 such that, if ‖𝑦(0) ⃗ − 𝑦∗⃗ ‖ < 𝜖.
Intuitively, it means the solution does not blow up but also does not necessarily
approach to the fixed point.
Asymptotic stability
7.2. PHASE PLANE ANALYSIS 63

A fixed point 𝑦∗⃗ is said to be Asymptotically stable, if it is Lyapunov stable and

⃗ − 𝑦∗⃗ ‖ < 𝛿, then we have
there exists a 𝛿 > 0 such that, if ‖𝑦(0)

⃗ − 𝑦∗⃗ ‖ = 0.
lim ‖𝑦(𝑡)
𝑡→∞

Intuitively, in this case the solution does approach to the fixed point over long
times.

7.2 Phase plane analysis

The general solution of systems of ODEs is given by the family of parametric
curves, specified by the initial condition:

⃗ 𝑐 1 , ⋯ , 𝑐𝑛 ) ∈ ℝ 𝑛 .
𝑦(𝑡;

These solutions represent trajectories in ℝ𝑛 for a system of 𝑛 dimensional ODEs.

For a 2 dimensional systme the family of solutions starting from different initial
conditions can be represented in a so called phase plane as illustrated in Figure
7.1.

Figure 7.1: Illustration of phase plane for a 2 dimensional system. Two solutions
starting at different initial conditions are shown (blue). Also, some representa-
tive velocity vectors from the vector field are drawn (red).

One can interpret the phase plane in terms of dynamics. The solution 𝑦(𝑡) ⃗
corresponds to a trajectory of a point moving on the phase plane with velocity
𝑑𝑦 ⃗
(𝑡). For a system of first order ODEs of the form:
𝑑𝑡
𝑑𝑦 ⃗
= 𝐹 (𝑦),
⃗
𝑑𝑡
64 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

where, there is no explicit dependence on the independent variable (time) on

the right hand side, the velocity is a vector defined at every point of the phase
plane and is tangent to the trajectory. This is called the vector field. See Figure
7.1.
Uniqueness of solutions of ODEs
Solutions of ODEs are uniquely defined by initial conditions except at some
special points in the phase plane (no proof now, you will see in the second year
rigorous proof). Trajectories in the phase plane cannot cross (except at some
special points) as this would be equivalent of non-uniqueness of solutions. The
special points are fixed points or singular points where trajectories start or end.
Phase plane analysis for the linear systems of first order ODEs
For linear systems, the vector field has some very nice properties.

𝑑𝑦 ⃗
= 𝐴𝑦 ⃗
𝑑𝑡
Eigenvectors define very special directions in the phase plane (𝐴𝑣1⃗ = 𝜆1 𝑣1⃗ ) in a
linear system. The line defined by 𝑣1⃗ in the phase plane is an invariant, meaning
that if we start on 𝑣1⃗ , we will remain on it. Let 𝑦(0)
⃗ = 𝛼𝑣1⃗ with 𝛼 ∈ ℝ. We
have:
𝑑𝑦(0)
⃗
𝐴𝑦(0)
⃗ = 𝛼𝐴𝑣1⃗ = 𝛼𝜆1 𝑣1⃗ = ,
𝑑𝑡
0
so, if 𝜆1 > 0, 𝑦(𝑡) grows along 𝑣1⃗ and if 𝜆1 < 0, 𝑦(𝑡) decays along 𝑣1⃗ to [ ].
0
To check this explicitly, we go back to the general solution of systems of ODE
(2 dimensional case):
⃗ = 𝑐1 𝑒𝜆1 𝑡 𝑣1⃗ + 𝑐2 𝑒𝜆2 𝑡 𝑣2⃗
𝑦(𝑡)

Let 𝑦(0)
⃗ = 𝛼𝑣1⃗ , this gives 𝑐1 = 𝛼 and 𝑐2 = 0. So we have:

⃗ = 𝛼𝑒𝜆1 𝑡 𝑣1⃗ = 𝑒𝜆1 𝑡 𝑦(0),

𝑦(𝑡) ⃗

so, we see explicitly, from this solution that, if 𝜆1 > 0, 𝑦(𝑡) grows along 𝑣1⃗ and
0 0
if 𝜆1 < 0, 𝑦(𝑡) decays along 𝑣1⃗ to [ ]. [ ] is the fixed point of the system as
0 0
we have
𝑑𝑦 ⃗ 0 0 0
([ ]) = 𝐴 [ ] = [ ] .
𝑑𝑡 0 0 0

Example 7.4 (First example of qualitative and phase plane analysis of linear
systems of ODE).
𝑑𝑦 ⃗ −4 −3
= 𝐴𝑦;⃗ 𝐴 = [ ].
𝑑𝑡 2 3
7.2. PHASE PLANE ANALYSIS 65

We have the solution for this system of linear ODEs using the eigenvalues and
eigenvectors for the matrix 𝐴:

1 3
⃗ (𝑡) = 𝑐1 𝑒2𝑡 [
𝑦𝐺𝑆 ] + 𝑐2 𝑒−3𝑡 [ ] .
−2 −1

Next we can look at the asymptotic behavior. Asymptotically, solutions blow

0
up parallel to 𝑣1⃗ , unless we start on 𝑣2⃗ , which then we approach 𝑦∗⃗ = [ ]. This
0
is evident from the solution above as if 𝑐1 ≠ 0 as 𝑡 → ∞ then 𝑦𝐺𝑆 ⃗ → ∞.

Aim of phase plane analysis is to obtain the phase portrait of the system, which
is a summary of all distinct solutions, with qualitatively different trajectories in
the phase plane. To do this for our linear system we draw the lines corresponding
to the directions of the eigenvectors and trajectories that start on these lines.
We consider the asymptotic behavior and we also compute (some examples of
the) vector field at some points to draw some representative trajectories. For
1
example, we have at 𝑦 ⃗ = [ ]:
0

𝑑𝑦 ⃗ 1 1 −4
([ ]) = 𝐴 [ ] = [ ] .
𝑑𝑡 0 0 2

Figure 7.2 shows the phase portrait for this system. We can explicitly obtain
the equations for the trajectories using either the general solution (as seen in the
quiz in the lectures) or by obtaining an ODE for 𝑦(𝑥) by dividing the equation
for 𝑑𝑦 𝑑𝑥
𝑑𝑡 by 𝑑𝑡 :

𝑑𝑥
= −4𝑥 − 3𝑦 𝑑𝑦 2𝑥 + 3𝑦
𝑑𝑡 ⟹ =
𝑑𝑦 𝑑𝑥 −4𝑥 − 3𝑦
= 2𝑥 + 3𝑦
𝑑𝑡

This is a homogeneous first order ODE and by using the change of variable
𝑢 = 𝑦/𝑥, we can obtain the following solution.

(𝑥 + 3𝑦)3 (2𝑥 + 𝑦)2 = 𝑐,

where 𝑐 is a constant of integration and its different values gives us the different
trajectories in the phase plane as illustrated in the phase portrait in Figure 7.2.
This figure and all the figures in the next section are plotted using the following
online applet.
66 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

Figure 7.2: The phase portrait for the linear ODE in Example 7.4.

7.3 General system of linear ODEs in 2 dimen-

sion
In this section we present a catalogue of qualitative analysis of the general 2
dimensional system of linear ODEs:

𝑑𝑦 ⃗ 𝑎 𝑏
= 𝐴𝑦;⃗ 𝐴=[ ].
𝑑𝑡 𝑐 𝑑

The general solution of this system can be written in terms of the eigenvalues
and eigenvectors of matrix 𝐴 as seen in the last chapter. The eigenvalues can
be obtained by solving the following characteristic equation:

𝜆2 − 𝜏 𝜆 + Δ = 0,

where 𝜏 = 𝑎 + 𝑑 is the trace and Δ = 𝑎𝑑 − 𝑏𝑐 is the determinant of the matrix

𝐴. √
𝜏 ± 𝜏 2 − 4Δ
𝜏 = trace(𝐴); Δ = Det(𝐴); 𝜆1 , 𝜆2 =
2
The general solution is

𝑦𝐺𝑆 = 𝑐1 𝑒𝜆1 𝑡 𝑣1⃗ + 𝑐2 𝑒𝜆2 𝑡 𝑣2⃗ ,

where 𝑣𝑖⃗ is the eigenvector corresponding to eigenvalue 𝜆𝑖 . In the following

we consider the qualitative behaviours of this system for different values in the
(𝜏 , Δ) plane.
7.3. GENERAL SYSTEM OF LINEAR ODES IN 2 DIMENSION 67

1. Saddle-point or Hyperbolic profile. Δ < 0; Lower half of (𝜏 , Δ).

In this case we have 𝜆1 ∈ ℝ+ and 𝜆2 ∈ ℝ− since:

Δ<0 ⟹ 𝜏 2 − 4Δ > 𝜏 2 > 0

This is the case similar to Example 7.4 and we have asymptotically as 𝑡 → ∞,

𝑦(𝑡)
⃗ → 𝑐1 𝑒𝜆1 𝑣1⃗ , which grows exponentially as 𝜆1 is a real positive number.
However, if we start on the line characterised by 𝑣2⃗ direction (i.e. 𝑐1 = 0), the
solution goes to zero.
0
𝑡 → ∞ ⟹ 𝑦⃗ → [ ] .
0
An example of saddle point phase portrait can be seen in Figure 7.3.

Figure 7.3: The saddle point phase portrait. This figure is plotted
using this [online applet](http://mathlets.org/mathlets/linear-phase-portraits-
matrix-entry/)

𝜏2
2.1.1: Repelling or unstable node. 0 < Δ < 4 ; 𝜏 > 0.
In this case we have 𝜆1 , 𝜆2 ∈ ℝ+ and 𝜆1 > 𝜆2 > 0. So starting on 𝑣2⃗ blow-up
along the direction of 𝑣2⃗ . Otherwise, blow up in the direction of 𝑣1⃗ .

𝑡→∞ ⟹ ⃗ → 𝑐1 𝑒𝜆1 𝑣1⃗ → ∞.

𝑦(𝑡)

An example of repelling or unstable node phase portrait can be seen in Figure

7.4.
68 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

Figure 7.4: The repelling or unstable node phase portrait. This figure is plotted
using this [online applet](http://mathlets.org/mathlets/linear-phase-portraits-
matrix-entry/)

𝜏2
2.1.2: Attracting or stable node. 0 < Δ < 4 ;𝜏 < 0.
−
In this case we have 𝜆1 , 𝜆2 ∈ ℝ and 𝜆2 < 𝜆1 < 0. So starting on 𝑣2⃗ decays
0
to [ ] along the direction of 𝑣2⃗ . Otherwise, decays along the direction of 𝑣1⃗ to
0
0
[ ]. So, for 𝑐1 ≠ 0, we have:
0

0
𝑡→∞ ⟹ ⃗ → 𝑐1 𝑒−|𝜆1 | 𝑣1⃗ → [ ] .
𝑦(𝑡)
0

An example of attracting or stable node phase portrait can be seen in Figure

7.5.
𝜏2
2.2.1: Centre or elliptic profile. Δ > 4 ; 𝜏 =0
In this case 𝜆1,2 = ±𝑖𝜔 and the solution is periodic. Periodic behaviour corre-
sponds to closed curves in the phase plane. For linear systems the closed curves
are ellipses.

Example 7.5 (Harmonic oscillator: revisited).

𝑥2
+ 𝜔2 𝑥 = 0.
𝑑𝑡2
7.3. GENERAL SYSTEM OF LINEAR ODES IN 2 DIMENSION 69

Figure 7.5: The attracting or stable node phase portrait. This figure is plotted
using this [online applet](http://mathlets.org/mathlets/linear-phase-portraits-
matrix-entry/)

We can write this second order linear ODE as a system of two first order linear
ODEs, but defining 𝑦 = 𝑑𝑥
𝑑𝑡 . The general solution for this system can be written
as:
𝑥 𝐴 sin(𝜔𝑡 + 𝜙)
[ 𝐺𝑆 ] = [ 0 ].
𝑦𝐺𝑆 𝐴0 𝜔 cos(𝜔𝑡 + 𝜙)

From this result we see that the trajectory of solutions are ellipses.

𝑦2 𝑦2
𝑥2 + 2
= 𝐴20 = 𝑥20 + 02 ,
𝜔 𝜔
where 𝐴0 is a constant of integration and it depends on the initial conditions
𝑥0 and 𝑦0 .
The phase portrait illustrating the elliptic profile can be seen in Figure 7.6. To
figure out the direction of motion we evaluate the vector field at some points:

0 𝑑𝑦 ⃗ 𝑦
𝑦⃗ = [ ] ⇒ = 𝐴𝑦 ⃗ = [ ] ,
𝑦 𝑑𝑡 0

𝑥 𝑑𝑦 ⃗ 0
𝑦⃗ = [ ] ⇒ = 𝐴𝑦 ⃗ = [ 2 ] .
0 𝑑𝑡 −𝜔 𝑥
 70 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

0
Note that in this case the point 𝑦 ⃗ = [ ] is a stable fixed point, with Lyapunov
0
stability as the trajectories around the origin do not blow up but also do not
Linear Phase Portraits: Matrix Entry 14/02/2021, 17)10

asymptotically approach the origin.

LINEAR PHASE PORTRAITS: MATRIX ENTRY mode + help
3.40 y

-2

-4
det x
-4 -2 0 2 4
tr 0.00

Companion Matrix

0.00 1.00
A=
-3.40 0.00

Eigenvalues

Im
4 x= center
y= u' = A u Clear
2
λ 1 = −1.84 i
0 Re
λ 2 = 1.84 i
-2
-4 -2 0 2 4 -4 -2 0 2 4
-4
-4 -2 0 2 4 c -3.40 d 0.00

Figure 7.6: The centre or elliptic phase portrait. This figure is plotted
using this [online applet](http://mathlets.org/mathlets/linear-phase-portraits-
matrix-entry/)

𝜏2
2.2.2: Repelling or unstable spiral. Δ > 4 ; 𝜏 > 0.
The eigenvalues are complex with the real part being postive. So, for the general
solution we have:
𝜏
𝑦 ⃗ = 𝑒 2 𝑡 [𝑐1 𝑒𝑖𝜔𝑡 𝑣1⃗ + 𝑐2 𝑒−𝑖𝜔𝑡 𝑣2⃗ ] ,
which, asymptotically blow up in an oscilatory fashion. The phase portrait
illustrating the repelling or unstable spiral can be seen in Figure 7.7.
𝜏2
2.2.3: Attracting or stable spiral. Δ > 4 ; 𝜏 <0
The eigenvalues are complex with the real part being negative. So, for the
general solution we have:
https://mathlets.org/javascript/build/linPhasePorMatrix.html# Page 1 of 1

𝜏
𝑦 ⃗ = 𝑒 2 𝑡 [𝑐1 𝑒𝑖𝜔𝑡 𝑣1⃗ + 𝑐2 𝑒−𝑖𝜔𝑡 𝑣2⃗ ] ,
0
which, asymptotically as 𝑡 → ∞, 𝑦 ⃗ → [ ]. The phase portrait illustrating the
0
attracting or stable spiral can be seen in Figure 7.8.
3.1: Line of repelling or unstable fixed points. Δ = 0; 𝜏 > 0.
Linear Phase Portraits: Matrix Entry 14/02/2021, 17)11
7.3. GENERAL SYSTEM OF LINEAR ODES IN 2 DIMENSION 71
LINEAR PHASE PORTRAITS: MATRIX ENTRY mode + help
3.00 y

-2

-4
det x
-4 -2 0 2 4
tr 1.85

Companion Matrix

0.00 1.00
A=
-3.00 1.85

Eigenvalues

Im
4 x= spiral source
y= u' = A u Clear
2
λ 1 = 0.93 − 1.46 i
0 Re
λ 2 = 0.93 + 1.46 i
-2
-4 -2 0 2 4 -4 -2 0 2 4
-4
-4 -2 0 2 4 c -3.00 d 1.85

Figure 7.7: The repelling or unstable spiral phase portrait. This figure is plotted
using this [online applet](http://mathlets.org/mathlets/linear-phase-portraits-
matrix-entry/)

https://mathlets.org/javascript/build/linPhasePorMatrix.html# Page 1 of 1

Figure 7.8: The attracting or stable spiral phase portrait. This figure is plotted
using this [online applet](http://mathlets.org/mathlets/linear-phase-portraits-
matrix-entry/)
72 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

In this case we have 𝜆1 = 𝜏 and 𝜆2 = 0, so the general solution is:

𝑦 ⃗ = 𝑐1 𝑒𝜏𝑡 𝑣1⃗ + 𝑐2 𝑣2⃗ .
For the vector field we have:
𝑑𝑦 ⃗
= 𝑐1 𝜏 𝑒𝜏𝑡 𝑣1⃗ ,
𝑑𝑡
So, 𝑦∗ = 𝑐2 𝑣2⃗ is a line of unstable fixed points. Asymptotically as 𝑡 → ∞ then
⃗ → 𝑐1 𝜏 𝑒𝜏𝑡 𝑣1⃗ , which blows up exponentially in the direction of 𝑣1⃗ . The phase
𝑦(𝑡)
portrait for this case can be seen in Figure 7.9.

Figure 7.9: Line of repelling or unstable fixed points phase portrait. This figure
is plotted using this [online applet](http://mathlets.org/mathlets/linear-phase-
portraits-matrix-entry/)

3.2: Line of attracting or stable fixed points. Δ = 0; 𝜏 < 0.

In this case we have 𝜆1 = 𝜏 and 𝜆2 = 0 again similar to last case and the general
solution is
𝑦 ⃗ = 𝑐1 𝑒𝜏𝑡 𝑣1⃗ + 𝑐2 𝑣2⃗ .
For the vector field we have:
𝑑𝑦 ⃗
= 𝑐1 𝜏 𝑒𝜏𝑡 𝑣1⃗ .
𝑑𝑡
So, 𝑦∗ = 𝑐2 𝑣2⃗ is a line of stable fixed points. Asymptotically as 𝑡 → ∞ then
𝑦(𝑡)
⃗ → 𝑐1 𝜏 𝑒𝜏𝑡 𝑣1⃗ , which decays exponentially to the line of 𝑐2 𝑣2⃗ . The phase
portrait for this case can be seen in Figure 7.10.
7.3. GENERAL SYSTEM OF LINEAR ODES IN 2 DIMENSION 73

Figure 7.10: line of attracting or stable fixed points phase portrait. This figure
is plotted using this [online applet](http://mathlets.org/mathlets/linear-phase-
portraits-matrix-entry/)

4.1: Repelling (4.1.1) and attracting (4.1.2) star node. 𝜏 2 − 4Δ = 0

In this case the eigenvalues are repeated 𝜆1 = 𝜆2 = 𝜏2 and 𝐴 is diagonalizable.
We have:
𝜆 0 1 0
𝐴=[ ] ⇒ 𝑣1⃗ = [ ] , 𝑣2⃗ = [ ] .
0 𝜆 0 1
So, we have for the general solution

1 0 𝑐
𝑦 ⃗ = 𝑐1 𝑒𝜆𝑡 [ ] + 𝑐2 𝑒𝜆𝑡 [ ] = 𝑒𝜆𝑡 [ 1 ] .
0 1 𝑐2

We observe that in this case, the trajectory is always defined by 𝑦(0).

⃗ The
phase portrait for the repelling star node (4.1.1) can be seen in Figure 7.11.
The attracting star node phase portrait is similar with an asymptotically stable
0
fixed point at the origin [ ].
0
4.2: Unstable (4.2.1) and stable (4.2.2) improper or degenerate node.
𝜏 2 − 4Δ = 0
In this case the eigenvalues are repeated 𝜆1 = 𝜆2 = 𝜏2 but 𝐴 is non-
diagonalizable and 𝜏 > 0 (unstable 4.2.1) or 𝜏 < 0 (stable 4.2.2).
74 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

Figure 7.11: Repelling star node phase portrait. This figure is plotted
using this [online applet](http://mathlets.org/mathlets/linear-phase-portraits-
matrix-entry/)

Example 7.6.
𝑑𝑦 ⃗ 1 −1
= 𝐴𝑦;⃗ 𝐴=[ ].
𝑑𝑡 1 3

In this last chapter, using the Jordan normal form, we showed the general
solution of this system of ODEs to be:

⃗ = (𝑐1 𝑒2𝑡 + 𝑐2 𝑡𝑒2𝑡 )𝑣1⃗ + 𝑐2 𝑒2𝑡 𝑤⃗ 2 ,

𝑦𝐺𝑆

1 1
where 𝑣1⃗ = [ ] is the only eigenvector, and 𝑤⃗ 2 = [ ]. We see that as
−1 −2
𝑡 → ∞, 𝑦(𝑡)
⃗ blows up in the direction of 𝑣1⃗ . We can estimate the vector field at
specific points to helps draw the phase portrait. The phase portrait for this case
can be seen in Figure 7.12, which is of the type unstable improper or degenerate
node (4.2.1). Stable improper or degenerate node (4.2.2) have a similar phase
0
portrait to this with an asymptotically stable fixed point at the origin [ ].
0
The catalogue of phase portraits for the 2 dimensional linear systems of ODEs
can be seen in Figure 7.13 on the (𝜏 , Δ) plane. We note that the solutions can
be unstable, asymptotically stable or Lyapunov stable in the different regions
7.4. PHASE PLANE ANALYSIS FOR 2D NONLINEAR SYSTEMS 75

Figure 7.12: Unstable improper or degenerate node phase portrait. This figure
is plotted using this [online applet](http://mathlets.org/mathlets/linear-phase-
portraits-matrix-entry/)

of (𝜏 , Δ) plane. The system can have one fixed point or infinitely many fixed
points. Also, solutions could be oscillatory or non-oscillatory in different regions
of the parameter space.

7.4 Phase plane analysis for 2D nonlinear sys-

tems
Phase plane analysis is a very useful tool for 2 dimensional nonlinear systems and
the insights obtained from the 2D linear case is highly relevant. One obtains the
special points (fixed points) and could linearise the equations in the vicinity of
these points to get insight from the analysis of the corresponding linear system.
Vector field and asymptotic behaviour is useful to identify the trajectories in
the phase plane.

Example 7.7 (Synthetic Biology: Genetic Toggle Switch).

This model of a simple synthetic genetic network was proposed in a pioneering

paper by Gardener, Cantor and Colins, Nature 403:339-342 (2000). Consider
two genes 𝑢 and 𝑣, which inhibit expression of one another. The following
system of nonlinear ODEs characterises the dynamics between the two genes.
76 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

Figure 7.13: The catalogue of phase portraits for the 2 dimensional linear sys-
tems of ODEs

𝑑𝑢 𝛼1
= − 𝑢,
𝑑𝑡 1 + 𝑣𝛽
𝑑𝑣 𝛼2
= − 𝑣.
𝑑𝑡 1 + 𝑢𝛾

Characterising the fixed points of this model, allowed the authors to successfully
design and construct one of the first synthetic genetic networks. This system
has up to 3 fixed points (two stable and one unstable ones).

Example 7.8 (Lotka-Voltera Model).

Consider this classic model of predator-prey, where 𝑥 > 0 denotes number of

rabbits and 𝑦 > 0 number of foxes in a population (with 𝑎, 𝑏, 𝑐, 𝑑 > 0).

𝑑𝑥
= 𝑎𝑥 − 𝑏𝑥𝑦,
𝑑𝑡
𝑑𝑦
= 𝑑𝑥𝑦 − 𝑐𝑦.
𝑑𝑡
By setting the derivatives to zero, we obtain the following two fixed points for
the system.
𝑥∗ 0 𝑥∗ 𝑐/𝑑
[ ∗] = [ ] , [ ∗] = [ ].
𝑦 1 0 𝑦 2 𝑎/𝑏

We consider dynamics near each fixed points by considering

𝑥 = 𝑥∗ + Δ𝑥,

𝑦 = 𝑦∗ + Δ𝑦,
7.5. EXTENSION OF PHASE PLANE ANALYSIS TO HIGHER DIMENSIONAL SYSTEMS77

where, Δ𝑥, Δ𝑦 << 1. Linearising around the first fixed point by omitting terms
of the order of Δ𝑥Δ𝑦, we obtain:

𝑑Δ𝑥
= 𝑎Δ𝑥,
𝑑𝑡
𝑑Δ𝑦
= −𝑐Δ𝑦.
𝑑𝑡
This is a 2D linear system of ODEs that exhibits a saddle-point phase portrait
suggesting that the first fixed point is unstable. Similarly, linearising around
the second fixed point, we obtain:

𝑑Δ𝑥 𝑏𝑐
= − Δ𝑦,
𝑑𝑡 𝑑
𝑑Δ𝑦 𝑑𝑎
= Δ𝑥.
𝑑𝑡 𝑏
This is a 2D linear system of ODEs that exhibits a centre phase portrait suggest-
ing that the second fixed point has Lyapunove stability. Putting these together
we get the phase portrait in Figure 7.14 for the Lotka-Voltera Model that sug-
gests the system has periodic trajectories around the second fixed point in the
phase plane.

Figure 7.14: The phase portraits for the Lotka-Voltera model

7.5 Extension of phase plane analysis to higher

dimensional systems
⃗ can be considered a trajectory in ℝ𝑛 , where 𝑛 is the dimensionality
1. 𝑦(𝑡)
of the system.
2. From each initial condition there is a unique trajectory and trajectories
do not cross except at some special points.
78 CHAPTER 7. QUALITATIVE ANALYSIS OF ODES

3. We can consider asymptotic behavior to draw the trajectories.

4. We can compute vector field:

𝑑𝑦 ⃗
= 𝐹 ⃗ (𝑦).
⃗
𝑑𝑡
5. Fixed points (𝑦∗⃗ ) are obtained by:

𝑑𝑦 ⃗ ∗
(𝑦 ⃗ ) = 0.⃗
𝑑𝑡
The approach directly generalises for the linear systems as the solutions are
given by the eigenvectors and eigenvalues of matrix 𝐴. As in the 2 dimen-
sional case, solutions starting in the directions set by the eigenvectors, stay on
these directions and grow or decay depending on the sign of the corresponding
eigenvalue.
Stability of linear 𝑛 dimensional systems For the 2 dimensional case we
had stability where 𝜏 ≤ 0 and Δ ≥ 0. In terms of eigenvalue characterization
this meant the real part of the eigen values are negative. Similarly, for the
general 𝑛 dimensional linear systems of ODEs we have stability if the real part
of all the eigenvalues are negative.
Lorenz system (1933, Edward Lorenz)
More complex dynamics in phase planes are possible. Lorenz proposed a system
of 3 nonlinear equations that is a model of atmospheric convection. This rather
simple model for certain values of parameters (e.g. 𝜎 = 10, 𝛽 = 38 and 𝜌 = 28),
exhibits a complex non-periodic dynamics that is an example of chaos. This
dynamical behavior is characterised by the divergence of trajectories in the
phase plane starting from near identical initial conditions as illustrated in this
video.
𝑑𝑥
= 𝜎(𝑦 − 𝑥),
𝑑𝑡
𝑑𝑦
= 𝑥(𝜌 − 𝑧) − 𝑦,
𝑑𝑡
𝑑𝑧
= 𝑥𝑦 − 𝛽𝑧.
𝑑𝑡
Chapter 8

Introduction to Bifurcations

Bifurcations in a dynamical system (system of ODEs) describe the qualitative

change in behavior under a variation or change of some parameters of the system.
Parameters are constants that are tunable.

8.1 Bifurcations in linear systems

We start by looking at linear systems, first through an example:

Example 8.1 (Taking 𝑘 to be the tuning parameter in the damped harmonic

oscilator system).
𝑑2 𝑥 𝑑𝑥
+ 2𝑘 + 𝜔2 𝑥 = 0
𝑑𝑡2 𝑑𝑡

How does the qualitative behavior of 𝑥(𝑡; 𝑘) change when 𝑘 ∈ ℝ is varied?

We know this system is equivalent to a system of linear first order linear ODEs,
letting 𝑢 = 𝑑𝑥
𝑑𝑡 we have:

𝑑 𝑥 0 1 𝑥
[ ]=[ 2 ][ ].
𝑑𝑡 𝑢 −𝜔 −2𝑘 𝑢

Given the catalogue of phase portraits in the (𝜏 , Δ) plane we saw in the last
𝑥
⃗ 𝑘) = [ ]
chapter, we can use these to see how the qualitative behavior of 𝑦(𝑡;
𝑢
changes when 𝑘 is varied. For this system we have 𝜏 = −2𝑘 and Δ = 𝜔2 , so only
𝜏 depends on the tunable parameter 𝑘 and Δ is always positive. As Figure 8.1
shows there are 7 different phase portraits that can be observed as 𝑘 is varied.
Defining a bifurcation as a change in stability of the system, there is only one
bifurcation point at 𝑘 = 0 for this system.

79
80 CHAPTER 8. INTRODUCTION TO BIFURCATIONS

Figure 8.1: Qualitative behavior of the damped harmonic oscillator system as

𝑘 is varied

In linear systems the bifurcations are related to changes in the stability of the
system, as that is the main type of change that can happen in the dynamics of
the system.
In non-linear systems there is a whole zoo of bifurcations and we will not cover
these here but we will consider one dimensional nonlinear systems in the next
section.

8.2 Qualitative behavior of non-linear 1D sys-

tems
Consider the general one dimensional nonlinear first order ODE.

𝑑𝑦
= 𝑓(𝑦); 𝑦 ∈ ℝ1
𝑑𝑡

The phase plane for 1D systems can be considered. We have:

• 𝑦(𝑡) are trajectories on the real line.
• Vector field describing how we move is the velocity and is scalar in 1D
case (𝑓(𝑦) ∈ ℝ).
• special points include
1. Fixed points: 𝑓(𝑦∗ ) = 0.
2. Singularities where 𝑓(𝑦𝑠𝑖𝑛𝑔 ) is non-defined.
8.2. QUALITATIVE BEHAVIOR OF NON-LINEAR 1D SYSTEMS 81

Example 8.2 (First (trivial) example of the phase plane and bifurcation anal-
ysis in one dimension).
𝑑𝑦
= 𝑘𝑦 = 𝑓(𝑦; 𝑘).
𝑑𝑡

The fixed point for this system is 𝑦∗ = 0 as 𝑓(𝑦∗ = 0) = 0. The general solutio
for this ODE is
𝑦(𝑡; 𝑘) = 𝑦(0)𝑒𝑘𝑡 .
The general solution suggests that the fixed point is stable for 𝑘 < 0 and is
𝑑𝑦
unstable for 𝑘 > 0. Also, we can also use a plot of = 𝑓(𝑦; 𝑘) vs 𝑦 for different
𝑑𝑡
values of 𝑘 to draw the vector field as illustrated in Figure 8.2 to obtain the
stability of the fixed points. A stable fixed point has a flow towards it, while an
unstable fixed points has an outward flow.

Figure 8.2: The plot of 𝑓(𝑦) vs 𝑦 for different values of 𝑘, illustrates the vector
field and the stability of the fixed point for Example 8.2.

A bifurcation diagram summarises all possible behaviours of the system as a

parameter is varied. It represents all fixed points of the system and their stability
as a function of the varying parameter. The bifurcation diagram for this example
is drawn in Figure 8.3.
There are only 3 kinds of nonlinear 1D systems in terms of their bifurcation.

8.2.1 Saddle-node bifurcation

This is a basic mechanism for creation and destroying fixed points. The proto-
typical example of saddle-node bifurcation is given by:
𝑑𝑦
= 𝑟 + 𝑦2 ,
𝑑𝑡
√
where 𝑟 ∈ ℝ. We have 𝑦∗ = ± −𝑟. Figure 8.4 using the vector field, illustrates
number and stability of these fixed points for different values of 𝑟.
82 CHAPTER 8. INTRODUCTION TO BIFURCATIONS

Figure 8.3: The bifurcation diagram (plot of fixed points and their stability vs
the tuning parameter 𝑘) for Example 8.2.

Figure 8.4: The plot of 𝑓(𝑦) vs 𝑦 for different values of 𝑟, illustrates the vector
field and the stability of the fixed point for saddle-node bifurcation.
8.2. QUALITATIVE BEHAVIOR OF NON-LINEAR 1D SYSTEMS 83

For 𝑟 < 0 we have 2 fixed points (one stable, one unstable), at 𝑟 = 0 we have
one half-stable fixed point and for 𝑟 > 0 we have no fixed point. There is a
saddle-node bifurcation at 𝑟 = 0. These changes in the number and stability of
the fixed points can be summarised using the bifurcation diagram, which is a
plot of fixed points vs parameter 𝑟 (see Figure 8.5).

Figure 8.5: The bifurcation diagram (plot of fixed points and their stability vs
the tuning parameter 𝑟) for saddle-node bifurcation.

8.2.2 Transcritical bifurcation

In certain systems a fixed point must exist for all values of a parameter. The
prototypical example of this form of bifurcation is given by:
𝑑𝑦
= 𝑟𝑦 − 𝑦2 ,
𝑑𝑡
where 𝑟 ∈ ℝ. This ODE has up to two fixed points 𝑦∗ = 0 and 𝑦∗ = 𝑟. Figure
8.6 using the vector field, illustrates number and stability of these fixed points
for different values of 𝑟.

Figure 8.6: The plot of 𝑓(𝑦) vs 𝑦 for different values of 𝑟, illustrates the vector
field and the stability of the fixed point for transcritical bifurcation.

For 𝑟 < 0 we have 2 fixed points (one stable, one unstable), at 𝑟 = 0 we have
one half-stable fixed point and for 𝑟 > 0 go back to two fixed points. At the
84 CHAPTER 8. INTRODUCTION TO BIFURCATIONS

bifurcation point 𝑟 = 0 an exchange of stabilities takes place between the two

fixed points. The changes in the number and stability of the fixed points is
summarised in the bifurcation diagram (see Figure 8.7).

Figure 8.7: The bifurcation diagram (plot of fixed points and their stability vs
the tuning parameter 𝑟) for transcritical bifurcation.

8.2.3 Pitchfork bifurcation

This kind of bifurcation is common in physical systems that have a symmetry.
There are two subtypes of pitchfork bifurcation.
Supercritical pitchfork bifurcation
The prototypical example of supercritical pitchfork bifurcation is given by:

𝑑𝑦
= 𝑟𝑦 − 𝑦3 = 𝑓(𝑦; 𝑟)
𝑑𝑡

Note that the equation is invariant under the change of variable 𝑦 → −𝑦, which
signifies√the symmetry. The system has up to three fixed points 𝑦∗ = 0 and
𝑦∗ = ± 𝑟. Figure 8.8 using the vector field, illustrates number and stability of
these fixed points for different values of 𝑟.
For 𝑟 < 0 we have 1 stable fixed point, at 𝑟 = 0 we have still one stable fixed
point and for 𝑟 > 0 we have three fixed points (two stable and a middle one
that is unstable). At the bifurcation point 𝑟 = 0 an exchange of stabilities takes
place between the fixed points. The changes in the number and stability of the
fixed points is summarised in the bifurcation diagram (see Figure 8.9).
Subcritical pitchfork bifurcation
8.2. QUALITATIVE BEHAVIOR OF NON-LINEAR 1D SYSTEMS 85

Figure 8.8: The plot of 𝑓(𝑦) vs 𝑦 for different values of 𝑟, illustrates the vector
field and the stability of the fixed point for supercritical pitchforkbifurcation.

Figure 8.9: The bifurcation diagram (plot of fixed points and their stability vs
the tuning parameter 𝑟) for supercritical pitchfork bifurcation.
86 CHAPTER 8. INTRODUCTION TO BIFURCATIONS

In the subcritical pitchfork bifurcation in contrast to the supercritical case the

cubic term is destabilising:

𝑑𝑦
= 𝑟𝑦 + 𝑦3 = 𝑓(𝑦; 𝑟)
𝑑𝑡
√
The system again has up to three fixed points 𝑦∗ = 0 and 𝑦∗ = ± −𝑟. Figure
8.10 using the vector field, illustrates number and stability of these fixed points
for different values of 𝑟.

Figure 8.10: The plot of 𝑓(𝑦) vs 𝑦 for different values of 𝑟, illustrates the vector
field and the stability of the fixed point for subcritical pitchfork bifurcation.

For 𝑟 < 0 we have we have three fixed points (two unstable and a middle one
that is stable), at 𝑟 ≥ 0 we have one unstable fixed point. At the bifurcation
point 𝑟 = 0 an exchange of stabilities takes place between the fixed points. The
changes in the number and stability of the fixed points is summarised in the
bifurcation diagram (see Figure 8.11).

8.2.4 Singular points

As at the beginning of this section the special points can be the fixed points 𝑦∗
or the singularities 𝑦𝑠𝑖𝑛𝑔 , where the 𝑓(𝑦𝑠𝑖𝑛𝑔 ; 𝑟) is not defined.

Example 8.3 (An example of an ODE with singularity).

𝑑𝑦 𝑘
= = 𝑓(𝑦, 𝑘)
𝑑𝑡 𝑦

Here, 𝑓(𝑦; 𝑘) is not defined at 𝑦𝑠𝑖𝑛𝑔 = 0. Figure 8.12 using the vector field,
shows the stability of the singularity for different values of 𝑘.
For this example, we have an explicit solution that provides further insight in
the behavior of the system for different values of 𝑘 and initial condition.
8.2. QUALITATIVE BEHAVIOR OF NON-LINEAR 1D SYSTEMS 87

Figure 8.11: The bifurcation diagram (plot of fixed points and their stability vs
the tuning parameter 𝑟) for subcritical pitchfork bifurcation.

Figure 8.12: The plot of 𝑓(𝑦) vs 𝑦 for different values of 𝑘, illustrates the vector
field and the stability of the singularity in Example 8.3.
88 CHAPTER 8. INTRODUCTION TO BIFURCATIONS

𝑑𝑦 𝑘
= ⟹ ∫ 𝑦𝑑𝑦 = ∫ 𝑘𝑑𝑡 ⟹ 𝑦 = ±√2𝑘𝑡 + 𝑦02 ,
𝑑𝑡 𝑦
where 𝑦0 = 𝑦(𝑡 = 0) is the inital condition. For 𝑦0 > 0, we have the positive
solution and for 𝑦0 < 0, we have the negative solution. Also, for 𝑘 > 0 the solu-
tions blow up to infinity and for 𝑘 < 0 solutions approach 0. This is summarised
in the bifurcation diagram (see Figure 8.13).

Figure 8.13: The bifurcation diagram (plot of singular points and their stability
vs the tuning parameter 𝑘) for Example 8.3.

8.2.5 Impossibility of oscillations for one dimensional sys-

tems
Fixed points dominate the dynamics of first-order systems. The trajectory in
one dimension phase plane never reverses direction and the approach to equi-
librium is always monotonic, hence there is no over-shoot, damped oscillations
or periodic solutions. This is a topological constraint, if you flow monotonically
on a line, you’ll never come back to your starting point. Of course, if we were
moving on a circle rather than a line, we could eventually return to starting
point. For example the following ODE 𝑑𝜃 𝑑𝑡 = 𝜔 for the angle 𝜃 has the following
periodic solution 𝜃 = 𝜔𝑡 + 𝜃0 .

8.2.6 Linear stability analysis

So far we have relied on graphical methods to determine stability of fixed points
(using the flows of the vector field). A quantitative measure can be obtained by
linearizing about the fixed point for the one dimensional systems. Let 𝜂 = 𝑦 −𝑦∗
be a small perturbation away from the fixed point 𝑦∗ . We have using the Taylor
expansion of 𝑓(𝑦).
𝑑𝑦 𝑑𝜂 𝑑𝑓 𝑑𝜂 𝑑𝑓
= = 𝑓(𝑦∗ + 𝜂) = 𝑓(𝑦∗ ) + 𝜂 (𝑦 = 𝑦∗ ) + ⋯ ⇒ ≈ 𝜂 (𝑦∗ ).
𝑑𝑡 𝑑𝑡 𝑑𝑦 𝑑𝑡 𝑑𝑦
8.2. QUALITATIVE BEHAVIOR OF NON-LINEAR 1D SYSTEMS 89

𝑑𝑓 ∗
Now, if 𝑑𝑦 (𝑦 ) > 0 then the fixed point 𝑦∗ is unstable as the perturbations
around the fixed point grow. However, if 𝑑𝑓 ∗ ∗
𝑑𝑦 (𝑦 ) < 0 then the fixed point 𝑦 is
stable as the perturbations around the fixed point decay to zero.
90 CHAPTER 8. INTRODUCTION TO BIFURCATIONS
Part III: Introduction to
Multivariate Calculus

91
Chapter 9

Partial Differentiation

So far, in the course, we have considered functions of single independent vari-

ables (ordinary functions):
𝑓 ∶ ℝ → ℝ,
or in the case of systems of ODEs:

𝑓∶ ℝ → ℝ𝑛 .

In the third part of the module, we now consider functions of several variables
or Multivariable or Multivariate functions.

𝑓∶ ℝ𝑛 → ℝ.

For every 𝑛-tuple of {𝑥𝑖 }𝑛𝑖=1 , where 𝑥𝑖 ∈ ℝ, there exists an image in ℝ.

𝑓(𝑥1 , ⋯ , 𝑥𝑛 ) ∈ ℝ

An important example is functions of two variables:

𝑥
𝑓(𝑥, 𝑦) = 𝑓(𝑥);
⃗ 𝑥 ⃗ = [ ] ∈ ℝ2
𝑦

9.1 Representation
As seen in Figure 9.1), there are the following two representations for the func-
tions of two variables.
1. 3D representation where 𝑓(𝑥, 𝑦) is the height.
2. Level curves 𝑥𝐶⃗ = (𝑥, 𝑦)𝐶 , where 𝑓(𝑥𝐶 ⃗ ) = 𝐶. For each 𝐶 there will be a
set of points that fulfill this condition. This kind of representation is also
known as a contour plot.

93
94 CHAPTER 9. PARTIAL DIFFERENTIATION

Figure 9.1: The 3D and contour plot of function 𝑓(𝑥, 𝑦) = 𝑥2 + 2𝑦2 drawn using
[Wolfram Alpha](wolframalpha.com)
9.2. LIMIT AND CONTINUITY 95

9.2 Limit and continuity

The general notions from calculus can be naturally extended to multivariate
functions.
Limit of the function 𝑓(𝑥)⃗ as 𝑥⃗ → 𝑥∗⃗ exists and is equal 𝐶:

lim 𝑓(𝑥)⃗ = 𝐶,
⃗ 𝑥⃗∗
𝑥→

if we have ∀𝜖 > 0, ∃𝛿 > 0 so that

0 < |𝑥⃗ − 𝑥∗⃗ | < 𝛿 ⇒ |𝑓(𝑥)⃗ − 𝐶| < 𝜖.

Function 𝑓 is continuous at 𝑥∗⃗ if:

lim 𝑓(𝑥)⃗ = 𝑓(𝑥∗⃗ )

⃗ 𝑥⃗∗
𝑥→

For example 𝑓(𝑥, 𝑦) = 𝑥𝑦 is continuous at all points in ℝ2 . But, the following

0
function is not continuous at 𝑥⃗ = [ ].
0

𝑥𝑦, if 𝑥, 𝑦 ≠ 0,
𝑔(𝑥, 𝑦) = {
1, if 𝑥 = 𝑦 = 0.

9.3 Partial and Total Differentiation

Different derivatives are defined for functions of several variables. First we
introduce partial differentiation, which is differentiation with respect to one of
the variables while the other ones are held constant:
𝜕𝑓 𝑓(𝑥1 , ⋯ , 𝑥𝑖 + ℎ, ⋯ , 𝑥𝑛 ) − 𝑓(𝑥1 , ⋯ , 𝑥𝑛 )
= lim
𝜕𝑥𝑖 ℎ→0 ℎ

This is read as partial derivative of 𝑓 with respect to 𝑥𝑖 , it is also sometimes

denoted alternatively as 𝑓𝑥𝑖 , 𝑓𝑥′ 𝑖 , 𝜕𝑥𝑖 𝑓 and 𝐷𝑥𝑖 .
Higher order partial derivatives can also be defined. For example, consider a
function of two variable 𝑓(𝑥, 𝑦), we denote the first partial derivatives as:

𝜕𝑓 𝜕𝑓
𝑔1 (𝑥, 𝑦) = ( ) ; 𝑔2 (𝑥, 𝑦) = ( )
𝜕𝑥 𝑦 𝜕𝑦 𝑥

The subscript of 𝑦 and 𝑥 in each of these partial deriveatives, highlights which

variable is held constant. We have the following second order partial derivatives
for 𝑓(𝑥, 𝑦):
𝜕𝑔 𝜕2𝑓 𝜕𝑔 𝜕 𝜕𝑓
( 1) = 2
; ( 1) = ( )
𝜕𝑥 𝑦 𝜕𝑥 𝜕𝑦 𝑥 𝜕𝑦 𝜕𝑥
96 CHAPTER 9. PARTIAL DIFFERENTIATION

𝜕𝑔2 𝜕2𝑓 𝜕𝑔2 𝜕 𝜕𝑓

( ) = ; ( ) = ( )
𝜕𝑦 𝑥 𝜕𝑦2 𝜕𝑥 𝑦 𝜕𝑥 𝜕𝑦

Symmetry of mixed derivatives or equality of mixed derivatives: If

the second partial derivatives are continuous, the order of differentiation is not
important and we therefore have:

𝜕 𝜕𝑓 𝜕 𝜕𝑓 𝜕2𝑓 𝜕2𝑓
( )= ( ) ⇒ = .
𝜕𝑦 𝜕𝑥 𝜕𝑥 𝜕𝑦 𝜕𝑦𝜕𝑥 𝜕𝑥𝜕𝑦

This result is known as Schwarz’s theorem, Clairaut’s theorem, or Young’s the-

orem.

Operationally, calculations are simple. Partial derivatives are obtained by keep-

ing the other variables constant, using the laws of differentiation for functions
of single variables.

Example 9.1 (Obtain all the first and second partial derivatives of the following
function.).
𝑢(𝑥, 𝑦) = 𝑥2 sin 𝑦 + 𝑦3 .

We have for the first partial derivatives:

𝜕𝑢
( ) = 2𝑥 sin 𝑦,
𝜕𝑥 𝑦

𝜕𝑢
( ) = 𝑥2 cos 𝑦 + 3𝑦2 .
𝜕𝑦 𝑥

And for the second partial derivatives, we have:

𝜕2𝑢
= 2 sin 𝑦,
𝜕𝑥2

𝜕2𝑢
= 2𝑥 cos 𝑦,
𝜕𝑥𝜕𝑦

𝜕2𝑢
= −𝑥2 sin 𝑦 + 6𝑦,
𝜕𝑦2

𝜕2𝑢
= 2𝑥 cos 𝑦.
𝜕𝑦𝜕𝑥
We observe that the symmetry of the mixed derivatives in this example holds.
9.4. TOTAL DIFFERENTIATION OF A FUNCTION OF SEVERAL VARIABLES97

9.4 Total differentiation of a function of several

variables
Total derivative evaluates the infinitesimal change of 𝑓(𝑥)⃗ when all the variables
are allowed to change infinitesimally in contrast with partial derivatives that are
about change in only one of the variables. We first consider the case of a function
of two variables. We have

Δ𝑓 = 𝑓(𝑥 + Δ𝑥, 𝑦 + Δ𝑦) − 𝑓(𝑥, 𝑦)

= 𝑓(𝑥 + Δ𝑥, 𝑦 + Δ𝑦) − 𝑓(𝑥, 𝑦) + 𝑓(𝑥 + Δ𝑥, 𝑦) − 𝑓(𝑥 + Δ𝑥, 𝑦)
𝑓(𝑥 + Δ𝑥, 𝑦) − 𝑓(𝑥, 𝑦) 𝑓(𝑥 + Δ𝑥, 𝑦 + Δ𝑦) − 𝑓(𝑥 + 𝛿𝑥, 𝑦)
=[ ] Δ𝑥 + [ ] Δ𝑦.
Δ𝑥 Δ𝑦

The total derivative is obtained at the limit of Δ𝑥, Δ𝑦 → 0:

𝜕𝑓 𝜕𝑓
𝑑𝑓 = lim Δ𝑓 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦.
∆𝑥,∆𝑦→0 𝜕𝑥 𝑦 𝜕𝑦 𝑦

For function of several variables 𝑓(𝑥1 , ⋯ , 𝑥𝑛 ), total differentiation is generalized.

We have:
𝑛
𝜕𝑓
𝑑𝑓 = ∑ ( ) 𝑑𝑥𝑖 .
𝑖=1
𝜕𝑥𝑖

9.5 Chain rule for functions of several variables

Given ordinary functions 𝑢(𝑥) and 𝑥(𝑡), chain rule for ordinary functions is
recalled to be:
𝑑𝑢 𝑑𝑢 𝑑𝑥
= .
𝑑𝑡 𝑑𝑥 𝑑𝑡
What is the equivalent for multivariable functions? Consider a function of two
variables:

𝑢 = 𝑢(𝑥, 𝑦) = 𝑢(𝑥);
⃗ 𝑥 ⃗ ∈ ℝ2 .
𝑑𝑢
Now, if we have 𝑥 = 𝑥(𝑡) and 𝑦 = 𝑦(𝑡) with 𝑡 ∈ ℝ. What is ?
𝑑𝑡
Combining total differentiation and the chain rule for ordinary functions, one
can obtain:

𝜕𝑢 𝜕𝑢
𝑑𝑢 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦,
𝜕𝑥 𝑦 𝜕𝑦 𝑦
𝜕𝑢 𝑑𝑥 𝜕𝑢 𝑑𝑦
=( ) ( ) 𝑑𝑡 + ( ) ( ) 𝑑𝑡.
𝜕𝑥 𝑦 𝑑𝑡 𝜕𝑦 𝑦 𝑑𝑡
98 CHAPTER 9. PARTIAL DIFFERENTIATION

So, we have:
𝑑𝑢 𝜕𝑢 𝑑𝑥 𝜕𝑢 𝑑𝑦
= ( ) ( ) + ( ) ( ).
𝑑𝑡 𝜕𝑥 𝑦 𝑑𝑡 𝜕𝑦 𝑦 𝑑𝑡
This generalises to a function of 𝑛 variables 𝑢(𝑥1 , ⋯ , 𝑥𝑛 ) with 𝑥𝑖 = 𝑥𝑖 (𝑡) and
𝑡 ∈ ℝ:

𝑛
𝑑𝑢 𝜕𝑢 𝑑𝑥𝑖
= ∑( ) .
𝑑𝑡 𝑖=1
𝜕𝑥 𝑖 𝑑𝑡

Example 9.2 (Consider a cylinder that its radius and height are expanding
with time).
𝑟(𝑡) = 2𝑡; ℎ(𝑡) = 1 + 𝑡2 .

𝑑𝑉
Evaluate the rate of change in volume .
𝑑𝑡

We have 𝑉 = 𝜋𝑟2 ℎ, therefore

𝑑𝑉 𝜕𝑉 𝑑𝑟 𝜕𝑉 𝑑ℎ
=( ) +( ) = 2𝜋𝑟(2ℎ + 𝑟𝑡) = 8𝜋𝑡 + 16𝜋𝑡3 .
𝑑𝑡 𝜕𝑟 ℎ 𝑑𝑡 𝜕ℎ 𝑟 𝑑𝑡

Another example of chain rule when we have multiple dependencies. Consider

𝑢 = 𝑢(𝑥, 𝑦); with 𝑦 = 𝑦(𝑡, 𝑥).

To obtain ( 𝜕𝑢
𝜕𝑥 )𝑡 , we combine total differentiation for 𝑢(𝑥, 𝑦) and 𝑦(𝑡, 𝑥). We
get:
𝜕𝑢 𝜕𝑢
𝑑𝑢 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦,
𝜕𝑥 𝑦 𝜕𝑦 𝑥
𝜕𝑦 𝜕𝑦
𝑑𝑦 = ( ) 𝑑𝑥 + ( ) 𝑑𝑡.
𝜕𝑥 𝑡 𝜕𝑡 𝑥
Now, by plugging 𝑑𝑦 in the expression for 𝑑𝑢 and rearranging we get:

𝜕𝑢 𝜕𝑢 𝜕𝑦 𝜕𝑢 𝜕𝑦
𝑑𝑢 = [( ) + ( ) ( ) ] 𝑑𝑥 + ( ) ( ) 𝑑𝑡.
𝜕𝑥 𝑦 𝜕𝑦 𝑥 𝜕𝑥 𝑡 𝜕𝑦 𝑥 𝜕𝑡 𝑥
Now, thinking of the above expression as the total derivative of 𝑢(𝑥, 𝑡), we
obtain:
𝜕𝑢 𝜕𝑢 𝜕𝑢 𝜕𝑦
( ) =( ) +( ) ( ) ,
𝜕𝑥 𝑡 𝜕𝑥 𝑦 𝜕𝑦 𝑥 𝜕𝑥 𝑡
𝜕𝑢 𝜕𝑢 𝜕𝑦
( ) =( ) ( ) .
𝜕𝑡 𝑥 𝜕𝑦 𝑥 𝜕𝑡 𝑥

Dependencies on another set of coordinates

9.6. IMPLICIT FUNCTIONS 99

Let
ℎ = ℎ(𝑥, 𝑦); with 𝑥 = 𝑥(𝑢, 𝑣) and 𝑦 = 𝑦(𝑢, 𝑣).
Considering total derivative of ℎ(𝑥, 𝑦) and substituting total derivatives of
𝑥(𝑢, 𝑣) and 𝑦(𝑢, 𝑣), and rearranging, we obtain:

𝜕ℎ 𝜕𝑥 𝜕ℎ 𝜕𝑦 𝜕ℎ 𝜕𝑥 𝜕ℎ 𝜕𝑦
𝑑ℎ = [( ) ( ) + ( ) ( ) ] 𝑑𝑢+[( ) ( ) + ( ) ( ) ] 𝑑𝑣.
𝜕𝑥 𝑦 𝜕𝑢 𝑣 𝜕𝑦 𝑥 𝜕𝑢 𝑣 𝜕𝑥 𝑦 𝜕𝑣 𝑢 𝜕𝑦 𝑥 𝜕𝑣 𝑢

Now thinking of this expression as total derivative of ℎ(𝑢, 𝑣), we have:

𝜕ℎ 𝜕ℎ 𝜕𝑥 𝜕ℎ 𝜕𝑦
( ) =( ) ( ) +( ) ( ) ,
𝜕𝑢 𝑣 𝜕𝑥 𝑦 𝜕𝑢 𝑣 𝜕𝑦 𝑥 𝜕𝑢 𝑣

𝜕ℎ 𝜕ℎ 𝜕𝑥 𝜕ℎ 𝜕𝑦
( ) =( ) ( ) +( ) ( ) .
𝜕𝑣 𝑢 𝜕𝑥 𝑦 𝜕𝑣 𝑢 𝜕𝑦 𝑥 𝜕𝑣 𝑢

Note, that strictly speaking the transformed function ℎ(𝑥, 𝑦) should be denoted
as ℎ′ (𝑢, 𝑣) as the transformed function is a ‘different’ function of its variables.
But, very commonly, the prime on ℎ′ (𝑢, 𝑣) is not used. For example ℎ(𝑥, 𝑦) =
𝑥2 + 𝑦2 in polar coordinates is ℎ′ (𝑟,′ 𝜃) = 𝑟2 , while common notation of ℎ(𝑟, 𝜃)
could imply 𝑟2 + 𝜃2 , if one thinks of plugging 𝑟 and 𝜃 in the original function
ℎ(𝑥, 𝑦). One should be aware of this notational ambiguity.

9.6 Implicit functions

First, a reminder about the explicit form for an ordinary function:

𝑦 = 𝑓(𝑥); 𝑥∈ℝ

The implicit form for an ordinary function is

𝐹 (𝑥, 𝑦) = 0.

Trivially, if we have the explicit form we also have an implicit form:

𝐹 (𝑥, 𝑦) = 𝑦 − 𝑓(𝑥) = 0.

For functions of two variables, we also have explicit form:

𝑧 = 𝑧(𝑥, 𝑦)

And implicit form:

𝐹 (𝑥, 𝑦, 𝑧) = 0

Differentiation using the Implicit form

100 CHAPTER 9. PARTIAL DIFFERENTIATION

Taking total differential from the implicit form 𝐹 (𝑥, 𝑦, 𝑧) = 0, we obtain:

𝜕𝐹 𝜕𝐹 𝜕𝐹
𝑑𝐹 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦 + ( ) 𝑑𝑧 = 0.
𝜕𝑥 𝑦,𝑧 𝜕𝑦 𝑥,𝑧 𝜕𝑧 𝑥,𝑦

Taking total differential from the explicit form 𝑧 = 𝑧(𝑥, 𝑦), we obtain:

𝜕𝑧 𝜕𝑧
𝑑𝑧 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦.
𝜕𝑥 𝑦 𝜕𝑦 𝑥

Now, solving for 𝑑𝑧 in the 𝑑𝐹 equation above, we thus have the following rela-
tionship between derivatives of the implicit and explicit form:

𝜕𝐹
)(
𝜕𝑧 𝜕𝑥 𝑦,𝑧
( ) =− ,
𝜕𝑥 𝑦 𝜕𝐹
( )
𝜕𝑧 𝑥,𝑦
𝜕𝐹
)(
𝜕𝑧 𝜕𝑦 𝑥,𝑧
( ) =− .
𝜕𝑦 𝑥 𝜕𝐹
( )
𝜕𝑧 𝑥,𝑦

Example 9.3 (Obtain the partial derivatives of 𝑧 using the explicit and implicit
forms).
Let 𝑧(𝑥, 𝑦) = 𝑥2 + 𝑦2 − 5.

We have from the explicit form:

𝑑𝑧 = 2𝑥𝑑𝑥 + 2𝑦𝑑𝑦.

Using the implicit form 𝐹 (𝑥, 𝑦, 𝑧) = 𝑥2 + 𝑦2 − 5 − 𝑧, we have:

𝑑𝐹 = 2𝑥𝑑𝑥 + 2𝑦𝑑𝑦 − 𝑑𝑧 = 0.

Which results to the same total derivative for 𝑑𝑧, that we obtained above from
the explicit form.

9.7 Taylor expansion of multivariate functions

Taylor expansion for functions of one variable (reminder): Let 𝑓(𝑥) ∶ ℝ → ℝ
and consider 𝑥0 ∈ ℝ. We saw in the first term of the module that:

𝑑𝑓 1 𝑑2 𝑓 1 𝑑3 𝑓
𝑓(𝑥0 + Δ𝑥) = 𝑓(𝑥0 ) + ( ) Δ𝑥 + ( 2 ) (Δ𝑥)2 + ( 3 ) (Δ𝑥)3 + ⋯ .
𝑑𝑥 𝑥0 2 𝑑𝑥 𝑥 3! 𝑑𝑥 𝑥
0 0
9.7. TAYLOR EXPANSION OF MULTIVARIATE FUNCTIONS 101

⃗ 𝑥⃗ ∈ ℝ2 , we assume suitable conditions of differen-

Now, let us consider 𝑓(𝑥),
tiability. We can use Taylor expansion for ordinary functions first on the 𝑥
direction and then 𝑦 to obtain the Taylor expansion for 𝑓(𝑥, 𝑦). Up to to 3rd
order we have:

𝑓(𝑥0⃗ + Δ𝑥)⃗ = 𝑓(𝑥0 + Δ𝑥, 𝑦0 + Δ𝑦)

𝜕𝑓 1 𝜕2𝑓 1 𝜕3𝑓
= 𝑓(𝑥0 , 𝑦0 + Δ𝑦) + ( ) Δ𝑥 + ( 2 ) (Δ𝑥)2 + ( ) (Δ𝑥)3 + ⋯
𝜕𝑥 𝑥0 ,𝑦0 +∆𝑦 2 𝜕𝑥 𝑥 3! 𝜕𝑥3 𝑥
0 ,𝑦0 +∆𝑦 0 ,𝑦0 +∆𝑦

𝜕𝑓 1 𝜕2𝑓 1 𝜕 𝑓 3
= 𝑓(𝑥0 , 𝑦0 ) + ( ) Δ𝑦 + ( 2 ) (Δ𝑦)2 + ( 3 ) (Δ𝑦)3 + ⋯
𝜕𝑦 𝑥⃗ 2 𝜕𝑦 𝑥⃗ 3! 𝜕𝑦 𝑥⃗
0 0 0

𝜕𝑓 𝜕2𝑓 1 𝜕3𝑓
+ Δ𝑥 [( ) + ( ) Δ𝑦 + ( 2 ) (Δ𝑦)2 + ⋯]
𝜕𝑥 𝑥⃗0 𝜕𝑦𝜕𝑥 𝑥⃗ 2 𝜕𝑦 𝜕𝑥 𝑥⃗
0 0

2 3
1 𝜕 𝑓 𝜕 𝑓 1 𝜕3𝑓
+ (Δ𝑥)2 [( 2 ) + ( ) Δ𝑦 + ⋯] + (Δ𝑥) 3
[( ) + ⋯]
2 𝜕𝑥 𝑥⃗ 𝜕𝑦𝜕𝑥2 𝑥⃗ 3! 𝜕𝑥3 𝑥⃗
0 0 0

𝜕𝑓 𝜕𝑓
= 𝑓(𝑥0⃗ ) + [( ) Δ𝑥 + ( ) Δ𝑦] +
𝜕𝑥 𝑥⃗0 𝜕𝑦 𝑥⃗
0

2 2
1 𝜕 𝑓 𝜕 𝑓 𝜕2𝑓
[( 2 ) (Δ𝑥)2 + 2 ( ) Δ𝑥Δ𝑦 + ( 2 ) (Δ𝑦)2 ]
2! 𝜕𝑥 𝑥⃗ 𝜕𝑥𝜕𝑦 𝑥⃗ 𝜕𝑦 𝑥⃗
0 0 0

1 𝜕3𝑓 𝜕3𝑓 𝜕3𝑓 𝜕3𝑓

+ [( 3 ) (Δ𝑥)3 + 3 ( 2 ) (Δ𝑥)2 Δ𝑦 + 3 ( ) Δ𝑥(Δ𝑦) 2
+ ( ) (Δ𝑦)3 ]
3! 𝜕𝑥 𝑥⃗ 𝜕𝑥 𝜕𝑦 𝑥⃗ 𝜕𝑥𝜕𝑦2 𝑥⃗ 𝜕𝑦3 𝑥⃗
0 0 0 0

+ ⋯.

We can write the Taylor expansion up to the second order in a vector-matrix

form. We define Gradient of the function 𝑓 evaluated at point 𝑥0⃗ as:
𝜕𝑓

⃗ 𝑥⃗ ⎡ 𝜕𝑥 ⎤
∇𝑓 0
=⎢ ⎥ .
𝜕𝑓
⎣ 𝜕𝑦 ⎦𝑥⃗
0

Hessian Matrix associated with the function 𝑓 evaluated at the point 𝑥0⃗ is
defined as:
𝜕2𝑓
𝐻𝑖𝑗 (𝑥0⃗ ) = ( )
𝜕𝑥𝑖 𝜕𝑥𝑗
𝑥⃗0

We can write the Taylor expansion up to the second order in terms of the
Gradient and Hessian:

⃗ 𝑥0⃗ )𝑇 . Δ𝑥⃗ + 1 Δ𝑥𝑇⃗ 𝐻(𝑥0⃗ )Δ𝑥⃗ + ⋯

𝑓(𝑥0⃗ + Δ𝑥)⃗ = 𝑓(𝑥0⃗ ) + ∇𝑓(
2

This generalizes to functions of 𝑛 dimensions.

102 CHAPTER 9. PARTIAL DIFFERENTIATION

Example 9.4 (Approximation).

Let 𝐴(𝑥, 𝑦) = 𝑥𝑦.

Expand 𝐴 around 𝑥0⃗ = (𝑥0 , 𝑦0 ).

⃗ = 𝐴(𝑥⃗ ) + [𝑦 Δ𝑥 1 0 1 Δ𝑥
𝐴(𝑥0⃗ + Δ𝑥) 0 𝑜 𝑥0 ] [ ] + [Δ𝑥 Δ𝑦] [ ][ ] + ⋯
Δ𝑦 2 1 0 Δ𝑦
= 𝑥0 𝑦0 + (𝑦0 Δ𝑥 + 𝑥0 Δ𝑦) + Δ𝑥Δ𝑦.

Example 9.5 (Using Taylor Expansion for error analysis). What is the maxi-
mum error in ℎ given errors in 𝑥 and 𝜃 of Δ𝑥 and Δ𝜃, respectively:

ℎ(𝑥, 𝜃) = 𝑥 tan 𝜃.

We have 𝑥 = 𝑥0 ± Δ𝑥 and 𝜃 = 𝜃0 ± Δ𝜃, we are looking for Δℎ, using the Taylor
expansion of ℎ up to the first order we have:

𝜕ℎ 𝜕ℎ
ℎ(𝑥0 ± Δ𝑥, 𝜃0 ± Δ𝜃) = ℎ(𝑥0 , 𝑦0 ) ± ( ) Δ𝑥 ± ( ) Δ𝜃 + ⋯ .
𝜕𝑥 𝑥⃗0 𝜕𝜃 𝑥⃗0

So for maximum error we have:

2
|Δℎ| = | tan 𝜃0 ||Δ𝑥| + |𝑥0 sec 𝜃0 ||Δ𝜃|.

For relative error we have:

Δℎ Δ𝑥 2Δ𝜃
∣ ∣=∣ ∣+∣ ∣.
ℎ(𝑥0⃗ ) 𝑥0 sin 2𝜃0
Chapter 10

Applications of
Multivariate Calculus

In Chapter 9, we introduced multivariable functions and notions of differentia-

tion. In this chapter, we present several applications of multivariate calculus.

10.1 Change of Coordinates

In lots of situations, one may need a change of coordinates, which in general
could be a nonlinear transformation. In this section, we focus on the familiar
example of the change of coordinates from polar coordinates (𝑥⃗ = (𝑟, 𝜃)) to
cartesian (𝑥⃗ = (𝑥, 𝑦)) and vice versa to illustrate the concepts. We generalise
these results at the end of the section.
Let 𝑥 = 𝑥(𝑟, 𝜃) and 𝑦 = 𝑦(𝑟, 𝜃), we have:

𝑥 = 𝑟 cos 𝜃, 𝑦 = 𝑟 sin 𝜃.

Conversely, we have for 𝑟 = 𝑟(𝑥, 𝑦) and 𝜃 = 𝜃(𝑥, 𝑦):

𝑦
𝑟 = √𝑥2 + 𝑦 2 , 𝜃 = arctan ( ).
𝑥
Using total differentiation, we can obtain a relationship between the vectors of
infinitesimal change 𝑑𝑥⃗ and 𝑑𝑟.
⃗

𝜕𝑥 𝜕𝑥
𝑑𝑥 = ( ) 𝑑𝑟 + ( ) 𝑑𝜃,
𝜕𝑟 𝜃 𝜕𝜃 𝑟
𝜕𝑦 𝜕𝑦
𝑑𝑦 = ( ) 𝑑𝑟 + ( ) 𝑑𝜃.
𝜕𝑟 𝜃 𝜕𝜃 𝑟

103
104 CHAPTER 10. APPLICATIONS OF MULTIVARIATE CALCULUS

We can write this in vector-matrix form as

𝜕𝑥 𝜕𝑥
𝑑𝑥 ⎡( 𝜕𝑟 ) ( )
𝜕𝜃 𝑟 ⎤ 𝑑𝑟
[ ] = ⎢ 𝜕𝑦 𝜃 ⎥
𝜕𝑦 ⎥ [𝑑𝜃] .
𝑑𝑦 ⎢
( ) ( )
⎣ 𝜕𝑟 𝜃 𝜕𝜃 𝑟 ⎦

By defining matrix 𝐽 known as Jacobian of the transformation as below we can

write this relationship as 𝑑𝑥⃗ = 𝐽 𝑑𝑟⃗ in short.

𝜕𝑥 𝜕𝑥
⎡( 𝜕𝑟 ) ( )
𝜕𝜃 𝑟 ⎤ cos 𝜃 −𝑟 sin 𝜃
𝐽 = ⎢ 𝜕𝑦 𝜃 ⎥
𝜕𝑦 ⎥ = [ sin 𝜃 ].
⎢ 𝑟 cos 𝜃
( ) ( )
⎣ 𝜕𝑟 𝜃 𝜕𝜃 𝑟 ⎦

Similarly for the polar to cartesian transformation we have:

𝜕𝑟 𝜕𝑟
𝑑𝑟 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦
𝜕𝑥 𝑦 𝜕𝑦 𝑥
𝜕𝜃 𝜕𝜃
𝑑𝜃 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦
𝜕𝑥 𝑦 𝜕𝑦 𝑥

In vector-matrix form we can writh this using matrix 𝐾 defined below as 𝑑𝑟⃗ =
⃗
𝐾 𝑑𝑥:
𝜕𝑟 𝜕𝑟
⎡( 𝜕𝑥 ) ( 𝜕𝑦 ) ⎤ cos 𝜃 sin 𝜃
𝐾 = ⎢ 𝜕𝜃 𝑦 𝑥⎥
𝜕𝜃 ⎥ = [− sin𝑟 𝜃 cos𝑟 𝜃 ] .
⎢
( ) ( )
⎣ 𝜕𝑥 𝑦 𝜕𝑦 𝑥 ⎦
It is evident that 𝐾, the Jacobian of the transformation from polar to cartersian
is equal to the inverse of 𝐽 , the Jacobian of the transformation from cartesian
to polar.
𝐾 = 𝐽 −1 .

Application 1: Infinitesimal element of length

Consider the curve 𝑦(𝑥), the infinitesimal element of length 𝑑𝑠, along this curve
is
𝑑𝑠2 = 𝑑𝑥2 + 𝑑𝑦2
What is the infinitesimal element of length in polar coordinates? Given we have
𝑑𝑥⃗ = 𝐽 𝑑𝑟,
⃗ we have:

𝑑𝑥 𝑑𝑟
𝑑𝑠2 = (𝑑𝑥)2 + (𝑑𝑦)2 = [𝑑𝑥 𝑑𝑦] [ ] = [𝑑𝑟 𝑑𝜃] 𝐽 𝑇 𝐽 [ ] = (𝑑𝑟)2 + 𝑟2 (𝑑𝜃)2 .
𝑑𝑦 𝑑𝜃

Application 2: Infinitesimal element of area in polar coordinate

10.1. CHANGE OF COORDINATES 105

Infinitesimal element of area is useful in taking integrals of functions of two

variables (𝑓(𝑥, 𝑦)) over a domain in the (𝑥, 𝑦) plane. In cartesian coordinates
we have:
𝑑𝐴 = 𝑑𝑥𝑑𝑦.
We note that letting 𝑑𝑥 = 𝑑𝑥𝑖 ̂ and 𝑑𝑦⃗ = 𝑑𝑦𝑗 ̂ and using the fact that the area
⃗
of a parallelogram is equal to the magnitude of the cross-product of vectors
spanning its edges, we can also write 𝑑𝐴 as:

𝑖̂ 𝑗̂ 𝑘̂
⃗ = ‖det ⎡
𝑑𝐴 = ‖𝑑𝑥⃗ × 𝑑𝑦‖ ⎢𝑑𝑥 0 0⎤ ⎥‖.
⎣ 0 𝑑𝑦 0⎦
What is area element for a general transformation? Consider a general transfor-
mation in two-dimensions:
𝑥 = 𝑥(𝑢, 𝑣) and 𝑦 = 𝑦(𝑢, 𝑣).
We have:
𝑑𝑥 𝑑𝑢
[ ] = 𝐽 [ ],
𝑑𝑦 𝑑𝑣
where 𝐽 is the Jacobian of the transformation. We therefore have for 𝑑𝑢⃗ and 𝑑𝑣⃗
in cartesian coordinates:
𝑑𝑢 0
𝑑𝑢⃗ = 𝐽 [ ] , 𝑑𝑣⃗ = 𝐽 [ ] .
0 𝑑𝑣
Using the cross-product rule for the area of the parallelogram, we have:
𝑑𝐴′ = ‖𝑑𝑢⃗ × 𝑑𝑣‖
⃗ = |det𝐽 |𝑑𝑢𝑑𝑣.

For the polar coordinate we obtain:

𝑑𝐴′ = |det𝐽 |𝑑𝑟𝑑𝜃 = 𝑟𝑑𝑟𝑑𝜃.

Note that some texts do not use prime on the transformed area element, al-
though 𝑑𝐴 and 𝑑𝐴′ are not mathematically equal (one can easily check that
𝑑𝑥𝑑𝑦 ≠ 𝑟𝑑𝑟𝑑𝜃). However, there is a conceptual equivalence between 𝑑𝐴 and
𝑑𝐴′ .
This result generalizes to higher dimensions for a volume element. Given a
general transformation 𝑢 in 𝑛 dimensions we have:
𝑑𝑥⃗ 𝑛×1 = 𝐽𝑛×𝑛 𝑑𝑢⃗ 𝑛×1 ,
where 𝐽 is the Jacobian of the transformation. The infinitesimal volume element
𝑛
in cartesian coordinates is 𝑑𝑉 = ∏𝑖=1 𝑑𝑥𝑖 . For the volume element in the
transformed coordinates we have
𝑛
𝑑𝑉 ′ = |det𝐽 | ∏ 𝑑𝑢𝑖 .
𝑖=1
106 CHAPTER 10. APPLICATIONS OF MULTIVARIATE CALCULUS

10.2 Partial Differential Equations (PDEs)

Analogous to an ordinary differential equation (ODE), one can define a partial
⃗ 𝑥⃗ ∈ ℝ𝑛 satisfying:
differential equation or PDE with 𝑓(𝑥),

𝜕𝑓 𝜕𝑓 𝜕 2 𝑓
𝑓(𝑥1 , ⋯ , 𝑥𝑛 , 𝑓, ,⋯, , , ⋯) = 0.
𝜕𝑥1 𝜕𝑥𝑛 𝜕𝑥𝑖 𝑥𝑗

Consider the following 2 dimensional examples. 1. Laplace Equation for 𝑢(𝑥, 𝑦)

(relevant to multiple areas of physics including fluid dynamics):

𝜕2𝑢 𝜕2𝑢
+ 2 = 0.
𝜕𝑥2 𝜕𝑦
2. Wave Equation for 𝑢(𝑥, 𝑡) (describing the dynamics of a wave with speed 𝑐
in one spatial dimension and time):

𝜕2𝑢 1 𝜕2𝑢
2
− 2 2 = 0.
𝜕𝑥 𝑐 𝜕𝑦
Our discussion of PDEs here will be very brief, but this topic is a major part
of your multivariable calculus course and one of the applied elective courses in
the second year. If you would like to have a sneak preview and for some cool
connections to the Fourier series you saw last term, you can check out this video.
Transforming a PDE under a change of coordinates
We again consider the example of transformation from cartesian to polar coor-
dinates:
𝑢(𝑥, 𝑦) ⟷ 𝑢(𝑟, 𝜃),
with 𝐽 being Jacobian of the transformation. Using total differentiation we
have:

𝜕𝑢 𝜕𝑢
𝑑𝑢 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦
𝜕𝑥 𝑦 𝜕𝑦 𝑦
𝜕𝑢 𝜕𝑥 𝜕𝑥 𝜕𝑢 𝜕𝑦 𝜕𝑦
=( ) [( ) 𝑑𝑟 + ( ) 𝑑𝜃] + ( ) [( ) 𝑑𝑟 + ( ) 𝑑𝜃]
𝜕𝑥 𝑦 𝜕𝑟 𝜃 𝜕𝜃 𝑟 𝜕𝑦 𝑦 𝜕𝑟 𝜃 𝜕𝜃 𝑟
𝜕𝑢 𝜕𝑥 𝜕𝑢 𝜕𝑦 𝜕𝑢 𝜕𝑥 𝜕𝑢 𝜕𝑦
= [( ) ( ) + ( ) ( ) ] 𝑑𝑟 + [( ) ( ) + ( ) ( ) ] 𝑑𝜃.
𝜕𝑥 𝑦 𝜕𝑟 𝜃 𝜕𝑦 𝑦 𝜕𝑟 𝜃 𝜕𝑥 𝑦 𝜕𝜃 𝑟 𝜕𝑦 𝑦 𝜕𝜃 𝑟

Now, by equating the terms in the above and the total derivative of 𝑢(𝑟, 𝜃), and
also using the definition of 𝐽 , we obtain the following:
𝜕
𝜕
( 𝜕𝑟 )𝜃 ( 𝜕𝑥 )𝑦
𝑇
[ 𝜕 ] 𝑢(𝑟, 𝜃) = 𝐽 [ 𝜕 ] 𝑢(𝑥, 𝑦).
( 𝜕𝜃 )𝑟 ( 𝜕𝑦 )
𝑥
10.3. EXACT ODES 107

Example 10.1 (Laplace Equation in polar coordinates).

Laplace equation in Cartesian coordinates is:

𝜕2𝑢 𝜕2𝑢
+ 2 = 0.
𝜕𝑥2 𝜕𝑦

In polar coordinates we have:

𝜕 𝜕𝑢 𝜕𝑟 𝜕𝑢 𝜕𝜃
[𝑢] = +
𝜕𝑥 𝜕𝑟 𝜕𝑥 𝜕𝜃 𝜕𝑥
𝜕 sin 𝜃 𝜕
= [cos 𝜃 − ] 𝑢.
𝜕𝑟 𝑟 𝜕𝜃

Now taking second derivative we have:

𝜕2 𝜕 sin 𝜃 𝜕 𝜕 sin 𝜃 𝜕
[𝑢] = [cos 𝜃 − ] [cos 𝜃 − ]𝑢
𝜕𝑥2 𝜕𝑟 𝑟 𝜕𝜃 𝜕𝑟 𝑟 𝜕𝜃
𝜕 2 𝑢 2 cos 𝜃 sin 𝜃 𝜕𝑢 2 cos 𝜃 sin 𝜃 𝜕 2 𝑢 sin2 𝜃 𝜕𝑢 sin2 𝜃 𝜕 2 𝑢
= cos2 𝜃 2 + − + + .
𝜕𝑟 𝑟2 𝜕𝜃 𝑟 𝜕𝑟𝜕𝜃 𝑟 𝜕𝑟 𝑟2 𝜕𝜃2

Similarly, we have:

𝜕2 𝜕 2 𝑢 2 cos 𝜃 sin 𝜃 𝜕𝑢 2 cos 𝜃 sin 𝜃 𝜕 2 𝑢 cos2 𝜃 𝜕𝑢 cos2 𝜃 𝜕 2 𝑢

2
[𝑢] = sin2 𝜃 2 − + + + 2 .
𝜕𝑦 𝜕𝑟 𝑟2 𝜕𝜃 𝑟 𝜕𝑟𝜕𝜃 𝑟 𝜕𝑟 𝑟 𝜕𝜃2

Finally, we have:

𝜕2𝑢 𝜕2𝑢 𝜕 2 𝑢 1 𝜕𝑢 1 𝜕2𝑢

2
+ 2 = 2
+ + 2 2 = 0.
𝜕𝑥 𝜕𝑦 𝜕𝑟 𝑟 𝜕𝑟 𝑟 𝜕𝜃

For example, if one looking for function 𝑢(𝑟) (with no dependence on 𝜃) fulfilling
the Laplace equation, we could solve the following ODE:

𝑑𝑢 1 𝑑𝑢
+ = 0.
𝑑𝑟2 𝑟 𝑑𝑟

10.3 Exact ODEs

The concept of total differentiation provides an alternative method for solving
first order nonlinear ODEs. Consider the given first order ODE:

𝑑𝑦 −𝐹 (𝑥, 𝑦)
= .
𝑑𝑥 𝐺(𝑥, 𝑦)
108 CHAPTER 10. APPLICATIONS OF MULTIVARIATE CALCULUS

If we have a solution of the ODE in implicit form 𝑢(𝑥, 𝑦) = 0, assuming 𝑢 is

continuous with continuous derivatives. For total derivative of 𝑢 we have:

𝜕𝑢 𝜕𝑢 𝑑𝑦 − ( 𝜕𝑢
𝜕𝑥 )
𝑑𝑢 = ( ) 𝑑𝑥 + ( ) 𝑑𝑦 = 0 ⇒ = 𝜕𝑢 𝑦 .
𝜕𝑥 𝑦 𝜕𝑦 𝑦 𝑑𝑥 ( 𝜕𝑦 )
𝑦

For the solution 𝑢(𝑥, 𝑦) = 0 to exist then we need the RHS of the above equation
to be equal to the RHS of the ODE, we are trying to solve. This will be the
case if
𝜕𝑢 𝜕𝑢
𝐹 =( ) and 𝐺 = ( ) .
𝜕𝑥 𝑦 𝜕𝑦 𝑦

But, if that is the case, due to the symmetry of the partial second mixed deriva-
tives, we should have:

𝜕𝑢2 𝜕𝑢2 𝜕𝐹 𝜕𝐺
= ⇒ = .
𝜕𝑦𝜕𝑥 𝜕𝑥𝜕𝑦 𝜕𝑦 𝜕𝑥

This is known as the condition of integrability of the ODE and if it is satisfied

then there exists 𝑢(𝑥, 𝑦), where

𝜕𝑢 𝜕𝑢
𝐹 =( ) and 𝐺 = ( ) ,
𝜕𝑥 𝑦 𝜕𝑦 𝑦

and then 𝑢(𝑥, 𝑦) = 0 is a solution of the first order ODE. We call this kind of
ODE exact.

Example 10.2 ( Is the following ODE exact?).

𝑑𝑦 −2𝑥𝑦 − cos 𝑥 cos 𝑦

=
𝑑𝑥 𝑥2 − sin 𝑥 sin 𝑦

Letting

𝐹 (𝑥, 𝑦) = 2𝑥𝑦 + cos 𝑥 cos 𝑦 and 𝐺(𝑥, 𝑦) = 𝑥2 − sin 𝑥 sin 𝑦.

We can check the condition of integrability and see the ODE is exact:

𝜕𝐹 𝜕𝐺
= = 2𝑥 − cos 𝑥 sin 𝑦,
𝜕𝑦 𝜕𝑥

since the ODE is exact, we can look for a solution in implicit form 𝑢(𝑥, 𝑦) = 0
such that:

𝜕𝑢 𝜕𝑢
𝐹 (𝑥, 𝑦) = and 𝐺(𝑥, 𝑦) = .
𝜕𝑥 𝜕𝑦
10.3. EXACT ODES 109

We can do that in two steps. Firstly, we have:

𝜕𝑢
( ) = 2𝑥𝑦 + cos 𝑥 cos 𝑦
𝜕𝑥 𝑦

Integrating with respect to 𝑥 and assuming the constant of intgration could

depend on 𝑦, we obtain:

𝑢 = 𝑦𝑥2 + cos 𝑦 sin 𝑥 + 𝑓(𝑦).

Now in the second step, we use

𝜕𝑢 𝑑𝑓
𝐺(𝑥, 𝑦) = = 𝑥2 − sin 𝑦 sin 𝑥 ⇒ = 0.
𝜕𝑦 𝑑𝑦
This implies 𝑓 is a constant, so we obtain the general solution 𝑦(𝑥) of the ODE
in implicit form:
𝑢(𝑥, 𝑦) = 𝑦𝑥2 + cos 𝑦 sin 𝑥 + 𝑐 = 0.

When the ODE is not exact, sometimes we can find a function (an integrating
factor) that will make the equation exact. Given:

𝐹 (𝑥, 𝑦)𝑑𝑥 + 𝐺(𝑥, 𝑦)𝑑𝑦 = 0,

is not exact, we look for a function 𝜆(𝑥) or 𝜆(𝑦) such that:

𝜆𝐹 (𝑥, 𝑦)𝑑𝑥 + 𝜆𝐺(𝑥, 𝑦)𝑑𝑦 = 0,

is exact. Note that an integrating factor can in general be a function of both
𝑥 and 𝑦, but in this case we cannot find an explicit solution for 𝜆, and it is for
this reason we can not solve very many ODEs.

Example 10.3 (Is this ODE exact? If not find an integrating factor to make
it exact).
𝑑𝑦 𝑥𝑦 − 1
= .
𝑑𝑥 𝑥(𝑦 − 𝑥)

Letting 𝐹 = 𝑥𝑦 − 1 and 𝐺 = 𝑥2 − 𝑥𝑦, we see that the ODE is not exact as:
𝜕𝐹 𝜕𝐺
≠ .
𝜕𝑦 𝜕𝑥

So, we will try to find a 𝜆(𝑥) (or 𝜆(𝑦) that will make the ODE exact). We need
to find 𝜆(𝑥) such that:

𝜕[𝜆(𝑥)(𝑥𝑦 − 1)] 𝜕[𝜆(𝑥)(𝑥2 − 𝑥𝑦)]

= .
𝜕𝑦 𝜕𝑥
110 CHAPTER 10. APPLICATIONS OF MULTIVARIATE CALCULUS

After simplification we obtain:

𝑑𝜆
(𝑥 − 𝑦) [ 𝑥 + 𝜆] = 0.
𝑑𝑥
As we obtain an ODE for 𝜆(𝑥) (that does not depend on 𝑦), an integrating
factor of the form 𝜆(𝑥) exists and can be obtained by solving this ODE to be:
𝑐
𝜆= .
𝑥

Now we can solve the exact ODE that is obtained (left as a quiz):

1
(𝑦 − )𝑑𝑥 + (𝑥 − 𝑦)𝑑𝑦 = 0.
𝑥

10.4 Sketching functions of two variables

Similar to the sketching of functions of one variable, we will use the following
steps in sketching a function of two variables 𝑓(𝑥, 𝑦):
• Check continuity and find singularities.
• Find asymptotic behaviour
lim 𝑓(𝑥, 𝑦) and lim 𝑓(𝑥).
⃗
𝑥,𝑦→±∞ 𝑥→
⃗ 𝑥⃗𝑠𝑖𝑛𝑔

• Obtain some level curves, for example 𝑓(𝑥)⃗ = 0.

• Find stationary points: minimum, maximum, saddle points
Stationary points for functions of two variables
Reminder that for a function of one variable 𝑓(𝑥), we find stationary points by
setting the first derivative to zero:
𝑓(𝑥∗ ) = 0.
Then, using Taylor expansion of 𝑓(𝑥) near 𝑥∗ , we see that one can use the sign of
the second derivative of 𝑓(𝑥) at 𝑥∗ to decide if the stationary point is minimum
𝑑2 𝑓 ∗ 𝑑2 𝑓 ∗
(if 𝑑𝑥 2 (𝑥 ) > 0) or maximum (if 𝑑𝑥2 (𝑥 ) < 0).

Using a similar approach for the functions of two variables 𝑓(𝑥, 𝑦), we have
stationary points are the points where tangent plane at 𝑥∗ is parallel to (𝑥, 𝑦)
plane:
𝜕𝑓 ∗ 𝜕𝑓 ∗
(𝑥⃗ ) = (𝑥⃗ ) = 0.
𝜕𝑥 𝜕𝑦
Then, the type of stationary point can be determined using the Taylor expansion
around the stationary point 𝑥∗⃗ and by the Hessian matrix.
𝜕2𝑓 ∗ 𝜕2𝑓 ∗
𝜕𝑓 ∗ 𝜕𝑓 ∗ 1 ⎡ 2
(𝑥⃗ ) ( 𝑥 ⃗ )⎤
𝑓(𝑥∗⃗ +Δ𝑥)⃗ = 𝑓(𝑥∗⃗ )+[ (𝑥 ⃗ ) ⃗ Δ𝑥𝑇⃗ ⎢ 𝜕 2 𝑥
(𝑥⃗ )] Δ𝑥+ 𝜕𝑥𝜕𝑦
2 ⎥ Δ𝑥.⃗
𝜕𝑥 𝜕𝑦 2 ⎢ 𝜕 𝑓 ∗ 𝜕 𝑓 ∗ ⎥
(𝑥 ⃗ ) ( 𝑥 ⃗ )
⎣ 𝜕𝑦𝜕𝑥 𝜕2𝑦 ⎦
10.4. SKETCHING FUNCTIONS OF TWO VARIABLES 111

Given the fact that the first partial derivatives of 𝑓(𝑥)⃗ are zero at the stationary
points, we have:

1
𝑓(𝑥∗⃗ + Δ𝑥)⃗ − 𝑓(𝑥∗⃗ ) = Δ𝑥𝑇⃗ 𝐻(𝑥∗⃗ )Δ𝑥.⃗
2
Under the assumption of continuity, we have the symmetry of the second mixed
partial derivatives:
𝜕2𝑓 𝜕2𝑓
= ,
𝜕𝑦𝜕𝑥 𝜕𝑥𝜕𝑦
therfore the Hessain is symmetric (𝐻 = 𝐻 𝑇 ), which implies that 𝐻 is diago-
nalizable and it has real eigenvalues 𝜆1 and 𝜆2 . So, there exists a similarity
transformation 𝑉 that diagonalise the Hessian:

𝜆1 0
𝑉 −1 𝐻𝑉 = Λ = [ ].
0 𝜆2

Also as 𝐻 is symmetric, we have 𝑉 −1 = 𝑉 𝑇 . So, we have:

1
𝑓(𝑥∗⃗ + Δ𝑥)⃗ − 𝑓(𝑥∗⃗ ) = Δ𝑥𝑇⃗ [𝑉 ((𝑥∗⃗ )Λ(𝑥∗⃗ )𝑉 𝑇 (𝑥∗⃗ )] Δ𝑥.⃗
2

Now, if we let Δ𝑧 ⃗ = 𝑉 𝑇 (𝑥∗⃗ )Δ𝑥,⃗ we have:

1 𝑇 1
Δ𝑓 = 𝑓(𝑥∗⃗ + Δ𝑥)⃗ − 𝑓(𝑥∗⃗ ) = Δ𝑧 ⃗ Λ(𝑥∗⃗ )Δ𝑧 ⃗ = [(Δ𝑧1 )2 𝜆1 + (Δ𝑧2 )2 𝜆2 ] .
2 2
Given this expression, we can use the sign of the eigenvalues to classify the
stationary points.
• If 𝜆1 , 𝜆2 ∈ ℝ+ , we have Δ𝑓 > 0 as we move away from the stationary
point, suggesting 𝑥∗⃗ is a minimum.
• If 𝜆1 , 𝜆2 ∈ ℝ− , we have Δ𝑓 < 0 as we move away from the stationary
point, suggesting 𝑥∗⃗ is a maximum.
• If 𝜆1 ∈ ℝ+ and 𝜆2 ∈ ℝ− , we classify 𝑥∗⃗ is a saddle point, since Δ𝑓 can be
positive or negative depending the direction of Δ𝑥.⃗
We note that, we could use the trace (𝜏 ) and determinant Δ of the matrix 𝐻
to know the sign of the eigenvalues with explicityly calculating the eigenvalues
as done in section 7.3 in the analysis of the 2D linear ODEs. In particular,
Δ > 0, 𝜏 > 0 (Δ > 0, 𝜏 < 0) suggests eigenvalues are positive (negative) and we
have a minima (maxima). Δ < 0 indicates a saddle point stationary point.

Example 10.4 (Sketch the following function of the two variables).

𝑢(𝑥, 𝑦) = (𝑥 − 𝑦)(𝑥2 + 𝑦2 − 1)
112 CHAPTER 10. APPLICATIONS OF MULTIVARIATE CALCULUS

We note the function is continuous and there are no singularities. The asymp-
totic behavior is that 𝑢(𝑥, 𝑦) → ±∞ as 𝑥, 𝑦 → ±∞.
Next, we find the level curves at zero
𝑢(𝑥, 𝑦) = 0 ⇒ 𝑥=𝑦 and 𝑥2 + 𝑦2 − 1 = 0
In the next step, we obtain the stationary points.

𝜕𝑢 ∗ 𝜕𝑢 ∗
(𝑥⃗ ) = (𝑥⃗ ) = 0
𝜕𝑥 𝜕𝑦
𝜕𝑢
= (𝑥2 + 𝑦2 − 1) + 2𝑥(𝑥 − 𝑦) = 0,
𝜕𝑥
𝜕𝑢
= −(𝑥2 + 𝑦2 − 1) + 2𝑦(𝑥 − 𝑦) = 0.
𝜕𝑦
Adding these two equations we obtain 𝑥∗ = 𝑦∗ or 𝑥∗ = −𝑦∗ .
2
1. 𝑥∗ = 𝑦∗ ⇒ 2𝑥∗ − 1 = 0 ⇒ 𝑃1 = ( √12 , √12 ), 𝑃2 = (− √12 , − √12 ).
2
2. 𝑥∗ = −𝑦∗ ⇒ 6𝑥∗ − 1 = 0 ⇒ 𝑃3 = ( √16 , − √16 ), 𝑃4 = (− √16 , √16 ).
We classify the stationary points using the Hessain:

6𝑥 − 2𝑦 2𝑦 − 2𝑥
𝐻(𝑥)⃗ = [ ]
2𝑦 − 2𝑥 2𝑥 − 6𝑦
Now we use the determinant (Δ) and trace (𝜏 ) of matrix 𝐻 at each stationary
point to classify each stationary point:

1 1 4 √1 0
𝑃1 = ( √ , √ ) ⇒ 𝐻(𝑃1 ) = [ 2 ] ⇒ Δ < 0 ⇒ 𝑃1 is a saddle point.
2 2 0 −4 √12

1 1 −4 √12 0
𝑃2 = (− √ , − √ ) ⇒ 𝐻(𝑃2 ) = [ ] ⇒ Δ < 0 ⇒ 𝑃2 is a saddle point.
2 2 0 4 √12

1 1 √8 − √46
𝑃3 = ( √ , − √ ) ⇒ 𝐻(𝑃3 ) = [ √64 √8
] ⇒ Δ > 0, 𝜏 > 0 ⇒ 𝑃3 is a minimum.
6 6 − 6 6

1 1 − √8 √4
𝑃4 = (− √ , √ ) ⇒ 𝐻(𝑃4 ) = [ √4 6 6 ] ⇒ Δ > 0, 𝜏 < 0 ⇒ 𝑃4 is a maximum.
6 6 6
− √86

Given the location and stability of the stationary points, we complete our sketch
of

𝑢(𝑥, 𝑦) = (𝑥 − 𝑦)(𝑥2 + 𝑦2 − 1),

as seen in Figure 10.1, by sketching some level curves, specifying the sign of the
funcion and the location of the stationary points.
10.4. SKETCHING FUNCTIONS OF TWO VARIABLES 113

Figure 10.1: The contour plot and sketch of function 𝑢(𝑥, 𝑦) = (𝑥−𝑦)(𝑥2 +𝑦2 −1)