Von Neumann stability analysis

In numerical algorithms for differential equations the concern is the growth of round-off errors and/or
initially small fluctuations in initial data which might cause a large deviation of final answers from the
exact solution. The method of stability analysis shown next was developed by the mid-twentieth
century Hungarian mathematician — and father of the electronic computer — John von Neumann. The
von Neuman stability analysis is based on the decomposition of numerical errors of numerical
approximations into Fourier series. Fourier series decomposes any periodic function or periodic signal
into the sum of a (possibly infinite) set of simple oscillating functions, namely sines and cosines (or,
equivalently, complex exponentials). We have chosen complex exponentials to represent errors as they
are much easier to work with than real trigonometric functions.

Consider the one-dimensional heat equation

𝜕𝑈 𝜕2𝑈
=𝛼 2 [1]
𝜕𝑡 𝜕𝑥

Stability Analysis of FTCS scheme.

Approximate the numerical error. Using the FTCS method the discretized form of equation [1] is:

𝑈𝑗𝑛+1 = 𝑈𝑗𝑛 + 𝜆(𝑈𝑗+1

𝑛
− 2𝑈𝑗𝑛 + 𝑈𝑗−1
𝑛
) [2]

where 𝜆 = 𝛼∆𝑡/∆𝑥 2 (called the mesh ratio), 𝑗 is the space index, and 𝑛 is the time index.

Define the error in the numerical approximation as

𝜖𝑗𝑛 = 𝑈𝑗𝑛 − 𝑢𝑗𝑛 [3]

where 𝑢𝑗𝑛 is the exact solution at grid point (𝑗, 𝑛). Both the numerical solution 𝑈𝑗𝑛 and the exact solution
𝑢𝑗𝑛 satisfy equation [2], therefore, the error 𝜖𝑗𝑛 also follows the discretized ODE equation [2]

𝜖𝑗𝑛+1 = 𝜖𝑗𝑛 + 𝜆(𝜖𝑗+1

𝑛
− 2𝜖𝑗𝑛 + 𝜖𝑗−1
𝑛
) [4]

The equations [2] and [4] show that both the error and the numerical solution have the same growth or
decay behavior with respect to time. For linear differential equations with periodic boundary condition,
the spatial variation of error may be expanded in a finite Fourier series, in the interval L, as

𝜖(𝑥) = ∑ 𝐴𝑚 𝑒 𝑖𝑘𝑚 𝑥 [5]

𝑚=1

𝜋𝑚
where the wavenumber 𝑘𝑚 = 𝐿
where 𝑚 = 1,2, … , 𝑀 and 𝑀 = 𝐿/∆𝑥 , 𝑖 = √−1, and 𝑥 is the
𝑖𝑘𝑚 𝑥
independent space variable. 𝑒 is the complex exponential (recall the Euler’s formula: 𝑒 𝑖𝜑 =
cos 𝜑 + 𝑖 sin 𝜑, where 𝑒 is the base of natural logarithm and 𝜑 is a real number).

[Von Neumann stability analysis-Presentation-V5] Page 1 of 6

The time dependence of the error is included by assuming the amplitude of error 𝐴𝑚 is a function of
time. Since the error tends to grow or decay exponentially with time, it is reasonable to assume that the
amplitude varies exponentially with time; hence

𝜖(𝑥, 𝑡) = 𝜖𝑗𝑛 = ∑ 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚𝑥 [6]

𝑚=1

where 𝑎 is a constant.

Since the difference equation for error is linear (the behavior of each term of the series is the same
as the series itself), it is enough to consider the growth of error of a typical term:

𝜖𝑚 (𝑥, 𝑡) = 𝜖𝑗𝑛 = 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 𝑥 [7]

The stability characteristics can be studied using just this form for the error with no loss in generality.

Define a Stability Criterion. The goal is to show that the error incurred by a particular numerical
scheme doesn’t grow in the evolving steps of time. Define the amplification factor

𝜖𝑗𝑛+1
𝐺≡ [8]
𝜖𝑗𝑛

Then, the necessary and sufficient condition for the error to remain bounded, maintaining numerical
stability is

|𝐺| ≤ 1 𝑜𝑟 [−1 ≤ 𝐺 ≤ 1 ] [9]

How the FTCS scheme complies with the Stability Criterion. The goal now is to show that equation
[9] holds for the particular FTCS numerical scheme. To find out how the error varies in steps of time,
substitute equation [7] into each term of equation [4], as shown below

𝜖𝑗𝑛 = 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 𝑥 [10. 𝑎]

𝜖𝑗𝑛+1 = 𝑒 𝑎(𝑡+∆𝑡) 𝑒 𝑖𝑘𝑚 𝑥 [10. 𝑏]

𝑛
𝜖𝑗+1 = 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 (𝑥+∆𝑥) [10. 𝑐]
𝑛
𝜖𝑗−1 = 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 (𝑥−∆𝑥) [10. 𝑑]

By eqs. [10] in equation [4], the last yields

𝑒 𝑎(𝑡+∆𝑡) 𝑒 𝑖𝑘𝑚 𝑥 = 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 𝑥 + 𝜆(𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 (𝑥+∆𝑥) − 2𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 𝑥 + 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 (𝑥−∆𝑥) ) [11]

Divide by 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 𝑥 to yield after simplification

𝑒 𝑎∆𝑡 = 1 + 𝜆(𝑒 𝑖𝑘𝑚 ∆𝑥 + 𝑒 −𝑖𝑘𝑚 ∆𝑥 − 2) [12]

Using the identities (see Appendix)

[Von Neumann stability analysis-Presentation-V5] Page 2 of 6

 𝑖𝑘𝑚∆𝑥 𝑖𝑘𝑚 ∆𝑥
𝑘𝑚 ∆𝑥 𝑒 2 −𝑒 − 2
𝑠𝑖𝑛 ( 2
)= 2𝑖
[13], and

𝑘𝑚 ∆𝑥 [𝑒 𝑖𝑘𝑚 ∆𝑥 + 𝑒 −𝑖𝑘𝑚 ∆𝑥 − 2]
sin2 ( )=− [14]
2 4

By eq. [14] in eq. [12], the last may be written as

𝑘𝑚 ∆𝑥
𝑒 𝑎∆𝑡 = 1 − 4λ sin2 ( ) [15]
2

However, by eqs [10] in eq [8], the last yields

𝜖𝑗𝑛+1 𝑒 𝑎(𝑡+∆𝑡) 𝑒 𝑖𝑘𝑚 𝑥

𝐺= = = 𝑒 𝑎∆𝑡 [16]
𝜖𝑗𝑛 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚 𝑥

Thus, by equations [15], [16] plugged into [9], the condition for stability is given by

𝑘𝑚 ∆𝑥
(1 − 4𝜆 sin2 ( )) ≥ −1 [17. 𝑎]
[17] |𝐺| = |1 − 4𝜆 sin2(𝑘𝑚 ∆𝑥/2)| ≤ 1 { 2
𝑘𝑚 ∆𝑥
(1 − 4𝜆 sin2 ( )) ≤ 1 [17. 𝑏]
2

For the above conditions, equations [17.a] and [17.b], to hold for all m (and therefore all
sin2(𝑘𝑚∆𝑥 /2) ), we need to consider [17.a] and [17.b], separately.

Equation [17.a] yields

[18] 4𝜆 sin2 (𝑘𝑚 ∆𝑥/2) ≤ 2

Note that the term 4𝜆 sin2 (𝑘𝑚 ∆𝑥/2) is always positive, i.e., always in the range [0,1]. The worst
case is when sin2 (𝑘𝑚 ∆𝑥/2) = 1, therefore, equation [18] yields

1 𝛼∆𝑡 1
[19] 𝜆 ≤ or 𝜆= ≤
2 ∆𝑥 2 2

Equation [17.b] yields

[20] 4𝜆 sin2 (𝑘𝑚 ∆𝑥/2) ≥ 0

Since sin2 (𝑘𝑚 ∆𝑥/2) range is [0,1] equation [20] yields

[21] 𝜆 ≥ 0
1
which always holds. Hence, combining [19] and [21]: 0 ≤ 𝜆 ≤ 2

Equation [19] gives the stability requirement for the FTCS scheme as applied to the one-dimensional
heat equation. We say that the method is conditionally stable; for a given ∆𝑥, the allowed value of ∆𝑡
must be small enough to satisfy equation [19] or

[Von Neumann stability analysis-Presentation-V5] Page 3 of 6

 ∆𝑥 2
[22] ∆𝑡 ≤
2𝛼

Usually we establish the space discretization by assuming ∆𝑥, then we compute the allowed ∆𝑡 with
equation [22] so to be sure our method won’t exceed the stability requirement. For instance, if we have
∆𝑥 = 0.01, and 𝛼 = 1, then we can only use a time step size ∆𝑡 ≤ 0.00005, which is minuscule. It
would take an inordinately large number of time steps to compute the value of the solution at even
moderate final times, e.g., t = 1. Moreover, owing to the limited accuracy of computers, the propagation
of round-off errors might then cause a significant reduction in the overall accuracy of the final solution
values. Since not all choices of space and time steps lead to a convergent scheme, that’s the reason the
explicit scheme FTCS is called conditionally stable, capisce my friend?

Stability Criterion for the Crank-Nicolson Scheme

By the CN scheme, the discretized form of equation [1]
𝜆 𝑘+1 𝜆 𝑘
𝑈𝑖𝑘+1 − 𝑈𝑖𝑘 = [𝑈𝑖−1 − 2𝑈𝑖𝑘+1 + 𝑈𝑖+1
𝑘+1
] + [𝑈𝑖−1 − 2𝑈𝑖𝑘 + 𝑈𝑖+1
𝑘
] [23]
2 2

Using similar arguments as in the FTCS method, the error 𝜖𝑗𝑚 also follows the discretized ODE equation
of the CN scheme
𝜆 𝑘+1 𝜆 𝑘
𝜖𝑖𝑘+1 − 𝜖𝑖𝑘 = [𝜖𝑖−1 − 2𝜖𝑖𝑘+1 + 𝜖𝑖+1
𝑘+1
] + [𝜖𝑖−1 − 2𝜖𝑖𝑘 + 𝜖𝑖+1
𝑘
] [24]
2 2

Divide [24] by 𝜖𝑖𝑘

𝜖𝑖𝑘+1 𝜖𝑖𝑘 𝑘+1

𝜆 𝜖𝑖−1 2𝜖𝑖𝑘+1 𝜖𝑖+1
𝑘+1 𝑘
𝜆 𝜖𝑖−1 2𝜖𝑖𝑘 𝜖𝑖+1
𝑘
− = [ − + ] + [ − + ] [25]
𝜖𝑖𝑘 𝜖𝑖𝑘 2 𝜖𝑖𝑘 𝜖𝑖𝑘 𝜖𝑖𝑘 2 𝜖𝑖𝑘 𝜖𝑖𝑘 𝜖𝑖𝑘

Simplify each term in equation [25]:

𝜖𝑖𝑘+1 𝑒 𝑎(𝑡+∆𝑡)
= 𝐺 = [26. 𝑎]
𝜖𝑖𝑘 𝑒 𝑎𝑡

𝑘+1
𝜖𝑖−1 𝑒 𝑎(𝑡+∆𝑡) 𝑒 𝑖𝑘𝑚(𝑥−∆𝑥) 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚(−∆𝑥)
= =𝐺 = 𝐺𝑒 −𝑖𝑘𝑚∆𝑥 [26. 𝑏]
𝜖𝑖𝑘 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚𝑥

𝑘+1
𝜖𝑖+1 𝑒 𝑎(𝑡+∆𝑡) 𝑒 𝑖𝑘𝑚(𝑥+∆𝑥) 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚∆𝑥
= = 𝐺 = 𝐺𝑒 𝑖𝑘𝑚∆𝑥 [26. 𝑐]
𝜖𝑖𝑘 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚𝑥

𝑘
𝜖𝑖−1 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚(𝑥−∆𝑥) 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚(−∆𝑥)
= = = 𝑒 −𝑖𝑘𝑚∆𝑥 [26. 𝑑]
𝜖𝑖𝑘 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚𝑥

[Von Neumann stability analysis-Presentation-V5] Page 4 of 6

 𝑘
𝜖𝑖+1 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚(𝑥+∆𝑥) 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚∆𝑥
= = = 𝑒 𝑖𝑘𝑚∆𝑥 [26. 𝑒]
𝜖𝑖𝑘 𝑒 𝑎𝑡 𝑒 𝑖𝑘𝑚𝑥 𝑒 𝑖𝑘𝑚𝑥

Plugging [26.a,b,c,d,e] into [25] yields

𝜆 𝜆
𝐺 − 1 = ( ) 𝐺(𝑒 −𝑖𝑘𝑚Δ𝑥 − 2 + 𝑒 𝑖𝑘𝑚Δ𝑥 ) + ( ) (𝑒 −𝑖𝑘𝑚Δ𝑥 − 2 + 𝑒 𝑖𝑘𝑚Δ𝑥 ) [27]
2 2

Using a trigonometric identity involving complex exponentials, such as

2 cos(𝜃) = 𝑒 𝑖𝜃 + 𝑒 −𝑖𝜃 [28]

Using [28] into [24]
𝜆 𝜆
𝐺 − 1 = ( ) 𝐺(cos(𝑘𝑚 Δ𝑥) − 2) + ( ) (cos(𝑘𝑚 Δ𝑥) − 2) [29]
2 2

Few kilometer later

1 − 𝜆(1 − cos(𝑘𝑚 Δ𝑥))

𝐺= [30]
1 + 𝜆(1 − cos(𝑘𝑚 Δ𝑥))

Remembering that the cosine range is [-1,+1], the value we should explore for cosine values are
cos(𝑘𝑚 Δ𝑥) = −1, 0, +1, hence

Case 1: cos(𝑘𝑚 Δ𝑥) = −1

1−𝜆(1−(−1)) 1−2𝜆
𝐺 = 1+𝜆(1−(−1)) = 1+2𝜆 ⟹ 𝑠𝑖𝑛𝑐𝑒 𝜆 > 0, 𝑡ℎ𝑒𝑛 𝐺 ≤ 1, always [31.a]

Case 2: cos(𝑘𝑚 Δ𝑥) = 0

1−𝜆(1−0) 1−𝜆
𝐺 = 1+𝜆(1−0) = 1+𝜆 ⟹ 𝑠𝑖𝑛𝑐𝑒 𝜆 > 0, 𝑡ℎ𝑒𝑛 𝐺 ≤ 1, always [31.b]
Case 3: cos(𝑘𝑚 Δ𝑥) = +1:

1−𝜆(1−1) 1
𝐺 = 1+𝜆(1−1) = 1 ⟹ 𝑡ℎ𝑒𝑛 𝐺 ≤ 1, always [31.c]

Therefore, we can simple say,

|𝐺| ≤ 1 [32]

Hence, unconditionally stable

Stability criterion for the BTCS scheme

For this method the recurrence formula is

𝑘+1
−𝜆𝑢𝑖−1 + (1 + 2𝜆)𝑢𝑖𝑘+1 − 𝜆𝑢𝑖+1
𝑘+1
= 𝑢𝑖𝑘 [33]
For the error is

𝑘+1
−𝜆𝜖𝑖−1 + (1 + 2𝜆)𝜖𝑖𝑘+1 − 𝜆𝜖𝑖+1
𝑘+1
= 𝜖𝑖𝑘 [34]

[Von Neumann stability analysis-Presentation-V5] Page 5 of 6

Ten and half miles later, equation [30] yields a magnification factor of

1 1
𝐺= = [35]
1 + 2𝜆(1 − cos(𝑘𝑚 Δ𝑥) 1 + 4𝜆 sin2 1 𝑘 Δ𝑥
2 𝑚
1
Since 𝜆 > 0, and sin2 2 𝑘𝑚 Δ𝑥 worst case is zero for the current condition, the magnification factor
always is less than or equal to 1 in absolute value (i.e., |𝐺| ≤ 1), and so the stability criterion of
equation [35] is satisfied for any choice of step sizes. We conclude that the implicit BTCS scheme is
unconditionally stable.

REFERENCES
(1) MIT18_336S109_lec14.pdf
(2) https://en.wikipedia.org/wiki/Von_Neumann_stability_analysis
(3) Richard Fitzpatrick 2006-03-29
http://farside.ph.utexas.edu/teaching/329/lectures/node79.html
(4) Exponential trigonometric identities:
https://wstein.org/edu/winter06/20b/notes/html/node30.html

APPENDIX

𝑖𝑘𝑚∆𝑥 𝑖𝑘𝑚 ∆𝑥
𝑘𝑚 ∆𝑥 𝑒 2 − 𝑒− 2
[10. 𝑎] 𝑠𝑖𝑛 ( )=
2 2𝑖
𝑖𝑘𝑚∆𝑥 𝑖𝑘𝑚 ∆𝑥 𝑖𝑘𝑚∆𝑥 𝑖𝑘𝑚 ∆𝑥
𝑘𝑚 ∆𝑥 𝑒 2 − 𝑒− 2 𝑒 2 − 𝑒− 2
sin2 ( )=[ ][ ]
2 2𝑖 2𝑖
𝑖𝑘𝑚 ∆𝑥 2 𝑖𝑘𝑚∆𝑥 𝑖𝑘𝑚 ∆𝑥 𝑖𝑘 ∆𝑥 2
− 𝑚
{(𝑒 2 ) − 2𝑒 2 𝑒 − 2 + (𝑒 2 ) }
= =
(2𝑖)(2𝑖)
𝑖𝑘𝑚 ∆𝑥
{𝑒 − 2 + 𝑒 −𝑖𝑘𝑚∆𝑥 }
= =
(4)(−1)
{𝑒 𝑖𝑘𝑚∆𝑥 + 𝑒 −𝑖𝑘𝑚∆𝑥 − 2}
=−
(4)

[Von Neumann stability analysis-Presentation-V5] Page 6 of 6