Introduction to finite difference methods

Simone Calogero
December 11, 2015

1 Introduction
The finite difference methods are techniques to find (numerically) approximate solutions
to ordinary differential equations (ODEs) and partial differential equations (PDEs). They
are based on the idea to replace the ordinary/partial derivatives with a finite difference
quotient, e.g., y 0 (x) ≈ (y(x + h) − y(x))/h. The various methods differ by the choice of
the finite difference used in the approximation. We shall present a number of methods by
examples. As this suffices for our applications to the Black-Scholes PDE, we consider only
linear equations.

2 ODEs
Consider the first order ODE
dy
= ay + bt, y(0) = 0, t ∈ [0, T ], (1)
dt
for some constants a, b ∈ R and T > 0. The solution is given by
b at
y(t) = y0 eat + (e − at − 1). (2)
a2
We shall apply three different finite difference methods to approximate the solution of (1).
In all cases we divide the time interval [0, T ] in a uniform partition,

T T
0 = t0 < t1 < · · · < tn = T, tj = j , ∆t = tj+1 − tj =
n n
and define
y(tj ) = yj , j = 0, . . . , n.

1
2.1 Forward Euler method
In this method we do the following approximation of dy/dt at time t:

dy y(t + ∆t) − y(t)

(t) = + O(∆t),
dt ∆t
i.e.,
dy
y(t + ∆t) = y(t) + (t)∆t + O(∆t2 ). (3)
dt
For Equation (1) this becomes

y(t + ∆t) = y(t) + (ay(t) + bt)∆t + O(∆t2 ).

Setting t = tj , ∆T = T /n, t + ∆t = tj + T /n = tj+1 and neglecting second order terms we

obtain
T
yj+1 = yj + (ayj + btj ) , j = 0, . . . , n − 1. (4)
n
As y0 is known, the previous iterative equation can be solved at any step j. This method is
called explicit, because the solution at the step j +1 is given explicitly in terms of the solution
at the step j. It is a simple matter to implement this method numerically, for instance using
the following Matlab function:1
function [time,sol]=exampleODEexp(T,y0,n)
dt=T/n;
sol=zeros(1,n+1);
time=zeros(1,n+1);
a=1; b=1;
sol(1)=y0;
for j=2:n+1
sol(j)=sol(j-1)+(a*sol(j-1)+b*time(j-1))*dt;
time(j)=time(j-1)+dt;
end

Exercise 1. Compare the approximate solution with the exact solution for increasing values
of n. Compile a table showing the difference between the approximate solution and the exact
solution at time T for increasing value of n.

Remark 1. Note carefully that when the method is implemented with Matlab, both the com-
puted approximate solution (i.e., the solution to (4)) and the computed exact solution (2)
will contain numerical (truncation) errors!
1
The Matlab codes presented in this text are not optimized. Moreover the powerful vectorization tools
of Matlab are not employed, so as to make the codes easily adaptable to other computer softwares and
languages.

2
2.2 Backward Euler method
This method consists in approximating dy/dt at time t as

dy y(t) − y(t − ∆)
(t) = + O(∆t),
dt ∆t
hence
dy
y(t + ∆t) = y(t) + (t + ∆t)∆t + O(∆t2 ). (5)
dt
The iterative equation for (1) now is

T
yj+1 = yj + (ayj+1 + btj+1 ) , j = 0, . . . , n − 1. (6)
n
This method is called implicit, because the solution at the step j + 1 depends on the solution
at both the step j and the step j + 1 itself. Therefore implicit methods involve an extra
computation, which is to find yj+1 in terms of yj only. For the present example this is a
trivial step, as we have
aT
−1 T

yj+1 = 1 − yj + btj+1 , (7)
n n
provided n 6= aT . Here is a Matlab function implementing the backward Euler method for
the ODE (1):
function [time,sol]=exampleODEimp(T,y0,n)
dt=T/n;
sol=zeros(1,n+1);
time=zeros(1,n+1);
a=1; b=1;
sol(1)=y0;
for j=2:n+1
time(j)=time(j-1)+dt;
sol(j)=1/(1-a*dt)*(sol(j-1)+b*time(j)*dt);
end

Exercise 2. Compare the approximate solution obtained with the backward Euler method
with the exact solution and the approximate one obtained via the forward Euler method.
Compile a table for increasing values of n as in Exercise 1.

2.3 Central difference method

By a Taylor expansion,

dy 1 d2 y
y(t + ∆) = y(t) + (t)∆t + (t)∆t2 + O(∆t3 ), (8)
dt 2 dt2

3
and replacing ∆t with −∆t,
dy 1 d2 y
y(t − ∆) = y(t) − (t)∆t + 2
(t)∆t2 + O(∆t3 ). (9)
dt 2 dt
Subtracting the two equations we obtain the following approximation for dy/dt at time t:

dy y(t + ∆t) − y(t − ∆)

(t) = + O(∆t2 ),
dt 2∆t
which is called central difference approximation. Hence
dy
y(t + ∆t) = y(t − ∆t) + 2 (t)∆t + O(∆t3 ). (10)
dt
Note that, compared to (3) and (5), we have gained one order in accuracy. The iterative
equation for (1) becomes
T
yj+1 = yj−1 − 2(ayj + btj ) , j = 0, . . . , n − 1. (11)
n
Note that the first step j = 0 requires y−1 . This is fixed by the backward method
T
y−1 = y0 − ay0 , (12)
n
which is (6) for j = −1.
Exercise 3. Write a Matlab function that implements the central difference method for (1).
Compile a table comparing the exact solution with the approximate solutions at time T ob-
tained by the three methods presented above for increasing value of n.

2.4 A second order ODE

Consider the second order ODE for the harmonic oscillator:
d2 y
= −ω 2 y, y(0) = y0 , ẏ(0) = ye0 . (13)
dt2
The solution to this problem is given by
ye0
y(t) = y0 cos(ωt) + sin(ωt). (14)
ω
One can define forward/backward/central difference approximations for second derivatives
in a way similar as for first derivatives. For instance, adding (8) and (9) we obtain the
following central difference approximation for d2 y/dt2 at time t:

d2 y y(t + ∆t) − 2y(t) + y(t − ∆t)

(t) = + O(∆t),
dt2 ∆t2
4
which leads to the following iterative equation for (13):
 2
T
yj+1 = 2yj − yj−1 − ω 2 yj , j = 1, . . . , n − 1, (15)
n
T
y1 = y0 + ye0 . (16)
n
Note the approximate solution at the first node is computed using the forward method and
the initial datum ẏ(0) = ye0 . The Matlab function solving this iteration is the following.
function [time,sol]=harmonic(w,T,y0,N)
dt=T/N;
sol=zeros(1,N+1);
time=zeros(1,N+1);
sol(1)=y0(1);
sol(2)=sol(1)+y0(2)*dt;
for j=3:N+1
sol(j)=2*sol(j-1)-sol(j-2)-dt^2*w^2*sol(j-1);
time(j)=time(j-1)+dt;
end

Exercise 4. Compare the exact and approximate solutions at time T for increasing values
of n.

3 PDEs
In this section we present three finite difference methods to find approximate solutions to
the one-dimensional heat equation

∂t u = ∂x2 u, u(x, 0) = u0 (x), (17)

where u0 is continuous. We refer to t as the time variable and to x as the spatial variable,
since this is what they typically represent in the applications of the heat equation. As before
we let t ∈ [0, T ] and introduce the partition

T T
0 = t0 < t1 < · · · < tn = T, tj = j , ∆t = tj+1 − tj = .
n n
As to the domain of the spatial variable x, we distinguish two cases

(i) x runs over the whole real line, i.e., x ∈ (−∞, ∞), and we are interested in finding an
approximation to the solution u ∈ C 1 (R × (0, T )) such that ∂x u ∈ C 1 (R × (0, T )).

5
 (ii) x runs over a finite interval, say x ∈ (xmin , xmax ), and we want to find an approximation
of the solution u which satisfies the boundary conditions2

u(xmin , t) = uL (t), u(t, xmax ) = uR (t), t ∈ [0, T ],

for some given continuous functions uL , uR . We also require uL (0) = u0 (xmin ), uR (0) =
u0 (xmax ), so that the solution is continuous on the boundary.

In fact, for numerical purposes, problem (i) is a special case of problem (ii), for the
domain (−∞, ∞) needs to be approximated by (−A, A) for A >> 1 when we solve problem
(i) in a computer. Note however that in the finite domain approximation of problem (i), the
boundary conditions at x = ±A cannot be prescribed freely! Rather they have to be given by
suitable approximations of the limit values at x = ±∞ of the solution to the heat equation
on the real line.
By what we have just said we can focus on problem (ii). To simplify the discussion we
assume that the domain of the x variable is given by x ∈ (0, X) and we assign zero boundary
conditions, i.e., uL = uR = 0. Hence we want to study the problem

∂t u = ∂x2 u, (x, t) ∈ (0, X) × (0, T ), (18a)

u(x, 0) = u0 (x), u(0, t) = u(X, t) = 0, x ∈ [0, X], t ∈ [0, T ]; u0 (0) = u0 (X) = 0.
(18b)

We introduce the partition of the interval (0, X) given by

X X
0 = x0 < x1 < · · · < xm = X, xi = i , ∆x = xi+1 − xi = .
m m
We let
ui,j = u(xi , tj ), i = 0, . . . , m, j = 0, . . . , n.
Hence ui,j is a m + 1 × n + 1 matrix. The j th column contains the value of the approximate
solution at each point of the spatial mesh at the fixed time tj . For instance, the zeroth
column is the initial datum: ui,0 = u0 (xi ), i = 0, . . . n. The rows of the matrix ui,j contain
the values of the approximate solution at one spatial point for different times. For instance,
the row u0,j are the values of the approximate solution at x0 = 0 for different times tj , while
um,j contains the values at xm = X. By the given boundary conditions we then have

u0,j = um,j = 0, j = 0, . . . , n.

We define
∆t T m2
d= = , (19)
∆x2 X2 n
2
These are called Dirichlet type boundary conditions. Other types of boundary conditions can be imposed,
but the Dirichlet type is sufficient for our applications to the Black-Scholes PDE.

6
3.1 Method 1: Forward in time, centered in space
In this method we use a forward difference approximation for the time derivative and a
centered difference approximation for the second spatial derivative:

u(x, t + ∆t) − u(x, t)

∂t u(x, t) = + O(∆t),
∆t
u(x + ∆x, t) − 2u(x, t) + u(x − ∆x, t)
∂x2 u(x, t) = + O(∆x).
∆x2
We find
u(x, t + ∆t) = u(x, t) + d(u(x + ∆x, t) − 2u(x, t) + u(x − ∆x, t)).
Hence we obtain the following iterative equation

ui,j+1 = ui,j + d(ui+1,j − 2ui,j + ui−1,j ), i = 1, . . . , m − 1, j = 0, . . . , n − 1, (20)

where we recall that ui,0 = u0 (xi ), u0,j = um+1,j = 0, i = 0, . . . , m, j = 0, . . . n. This method

is completely explicit. A Matlab function solving the iteration (20) with the initial datum
u0 (x) = exp(X 2 /4) − exp((x − X/2)2 ) is the following.
function [time,space,sol]=heatexp(X,T,m,n)
dt=T/n; dx=X/m;
d=dt/dx^2;
sol=zeros(m+1,n+1);
time=zeros(1,n+1);
space=zeros(1,m+1);
for j=2:n+1
time(j)=time(j-1)+dt;
end
for i=2:m+1
space(i)=space(i-1)+dx;
end
for i=1:m+1
sol(i,1)=exp(X^2/4)-exp((space(i)-X/2)^2);
end
sol(1,:)=0; sol(m+1,:)=0;
for j=2:n+1
for i=3:m+1
sol(i-1,j)=sol(i-1,j-1)+d*(sol(i,j-1)-2*sol(i-1,j-1)+sol(i-2,j-1));
end
end

To visualize the result it is convenient to employ an animation which plots the approx-
imate solution at each point on the spatial mesh for some increasing sequence of times in

7
the partition {t0 , t1 , . . . , tn }. This visualization can be achieved with the following simple
Matlab function:
function anim(r,F,v)
N=length(F(1,:));
step=round(1+N*v/10);
figure
for i=1:step:N
plot(r,F(:,i));
axis([0 1 0 1/2]);
drawnow;
pause(0.3);
end

Upon running the command anim(space,sol,v), the previous function will plot the
approximate solutions at different increasing times with speed v (the speed v must be between
0 and 1).
Let us try the following: [time,space,sol]=heatexp(1,1,50,2500). Hence we solve
the problem on the unit square (x, t) ∈ (0, 1)2 on a mesh of 50 × 2500 points. The value of
the parameter (19) is
d = 1.
If we now try to visualize the solution by running anim(space,sol,0.1), we find that the
approximate solution behaves very strangely (it produces just random oscillations). However
by increasing the number of time steps with [time,space,sol]=heatexp(1,1,50,5000),
so that
d = 0.5,
and visualize the solution, we shall find that the approximate solution converges quickly
and smoothly to u ≡ 0, which is the equilibrium of our problem (i.e., the time independent
solution of (18)). In fact, this is not a coincidence, for we have the following

Theorem 1. The forward-centered method for the heat equation is unstable if d ≥ 1 and
stable for d < 1.

The term unstable here refers to the fact that numerical errors, due for instance to the
truncation and round-off of the initial datum on the spatial grid, will increase in time. On the
other hand, stability of a finite difference method means that the error will remain small at
all times. The stability condition d < 1 for the forward-centered method applied to the heat
equation is very restrictive: it forces us to choose a very high number of points on the time
partition. To avoid such a restriction, which could be very costly in terms of computation
time, implicit methods are preferred, such as the one we present next.

8
3.2 Method 2: Backward in time, centered in space
In this method we employ the backward finite difference approximation for the time derivative
and the central difference for the second spatial derivative (same as before). This results in
the following iterative equation:

ui,j+1 = ui,j + d(ui+1,j+1 − 2ui,j+1 + ui−1,j+1 ), i = 1, . . . , m − 1, j = 0, . . . , n − 1, (21)

where we recall that ui,0 = u0 (xi ), u0,j = um+1,j = 0, i = 0, . . . , m, j = 0, . . . n. This method

is implicit and we need therefore to solve for the solution at time j +1 in terms of the solution
at time j. To this purpose we rewrite (21) in matrix form as follows:

Auj+1 = uj , (22)

where uj is the m + 1-dimensional vector with components (u0,j , u1,j , . . . , um,j ) and A is the
m + 1 × m + 1 matrix with non-zero entries given by

A0,0 = Am,m = 1, Ak,k = 1 + 2d, Ak,k−1 = Ak,k+1 = −d, k = 1, . . . m − 1.

The matrix A is invertible, hence we can invert (22) to express uj+1 in terms of uj as

uj+1 = A−1 uj . (23)

This method is unconditionally stable, i.e., it is stable for all values of the parameter d.
We can test this property by using the following Matlab function, which solves the iterative
equation (23):
function [time,space,sol]=heatimp(X,T,m,n)
dt=T/n; dx=X/m;
d=dt/dx^2;
sol=zeros(m+1,n+1);
time=zeros(1,n+1);
space=zeros(1,m+1);
A=zeros(m+1,m+1);
A(1,1)=1; A(m+1,m+1)=1;
for j=2:n+1
time(j)=time(j-1)+dt;
end
for i=2:m+1
space(i)=space(i-1)+dx;
end
for i=1:m+1
sol(i,1)=exp(X^2/4)-exp((space(i)-X/2)^2);
end
sol(1,:)=0; sol(m+1,:)=0;
for i=2:m

9
A(i,i-1)=-d;
A(i,i)=1+2*d;
A(i,i+1)=-d;
end
for i=2:n+1
sol(:,i)=inv(A)*sol(:,i-1);
end

If we now run [time,space,sol]=heatexp(1,1,50,500), for which d = 5, and visualize

the solution we shall obtain that the approximate solution behaves smoothly as expected,
indicating that the instability problem of the forward-centered method has been solved.

3.3 Method 3: Crank-Nicholson

This is an implicit method with higher order of accuracy than the backward-centered method.
It is obtained by simply averaging between methods 1 and 2 above, i.e.,
1 1
ui,j+1 = ui,j+1 + ui,j+1 ,
2 2
where the first term in the right hand side is computed with method 1 and the second term
with method 2. Thus we obtain the following iterative equation
d
ui,j+1 = ui,j + [(ui+1,j − 2ui,j + ui−1,j ) + (ui+1,j+1 − 2ui,j+1 + ui−1,j+1 )]. (24)
2
As the backward-centered method, the Crank-Nicholson method is also unconditionally sta-
ble.
Exercise 5. Write (24) in matrix form and solve for the solution at the time step j + 1 in
terms of the solution at the time step j.
Exercise 6. Write a Matlab function that implements the Crank-Nicholson method.
Exercise 7. Compare methods 2 and 3.

4 Black-Scholes model (assignment)

The Black-Scholes PDE is
1
∂t f + rx∂x f + σ 2 x2 ∂x2 f = rf, (x, t) ∈ (0, ∞) × (0, T )
2
with the terminal value f (x, T ) = g(x). Upon defining u(x, t) = f (x, T − t), we obtain the
following initial value problem:
1
− ∂t u + rx∂x u + σ 2 x2 ∂x2 u = ru, (25a)
2
u(x, 0) = g(x). (25b)

10
Note that in this new formulation the time t = T corresponds to t = 0 in the old formulation,
i.e., to the “present time”. The following project consists of two parts.
Part 1 (0.5 points)

• Write a short introduction to the Black-Scholes PDE, max 3 pages; in this part you
should discuss in particular the properties of the Black-Scholes PDE which are needed
for the numerical analysis in the second part of the project.

• Write a finite difference approximation for the PDE (25a) on the domain (x, t) ∈
(0, X) × (0, T ) with initial data g(x) at time t = 0 and boundary conditions u(0, t) =
uL (t), u(X, t) = uR (t). Use an implicit method.

Part 2 (0.5 points)

• Write a Matlab function that implements the finite difference approximation derived
in part 1. The parameters r, σ, X, T , must appear as input variables of your function,
while the boundary/initial data may appear in the code itself.

• Plot the solutions at time t = T in the case of call and put options. Focus on nearly-
at-the-money options, say x/K ∈ (4/5, 6/5). Show the exact solution on the same
plot. Run several tests for different values of the parameters. Discuss possible sources
of error and how to remove them.

• Compute solutions for call and put options for different values of volatility and show in a
plot how the the solution at time t = T depends on volatility (fix the other parameters,
including the initial price of the stock close to the strike price). Do a similar analysis
to depict the depence on the time of maturity and discuss your findings.

• Verify numerically the validity of the put-call parity at time t = T .

Include your matlab codes in an appendix. The deadline to submit the project is January
7th. You can work on the project with max one other student.

11