Taylor Series and Numerical Approximations

Hilary Weller [email protected]

August 7, 2015

An introduction to the concept of a Taylor series and how these are used in
numerical analysis to find numerical approximations and estimate their accuracy.
This is a series of four short videos to accompany the printed notes. You can
download the printed notes and fill parts in as we go along. Alternatively, you
can work through the notes without the videos.

1 Taylor Series
• Many functions can be expressed as Taylor series
• Taylor series are infinite polynomials

For example an exponential:

6
ex
5 1
1 +x
4 1 + x + x2!2
1 + x + x2!2 + x3!3
3
1 + x + x2!2 + x3!3 + x4!4
2

2
2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0

2
 For example a sine wave:

1.5

1.0 sinx sinx sinsin

xx
x x xx
x− x3!3x− x3!3x− x3!3
0.5
x− x3!3x−+ x3!x5!35 + x5!5
x− x3!3 + x5!5 − x7!7
0.0

0.5

1.0

1.5
6 4 2 0 2 4 6

The more terms you include, the more accurate it should get.

Functions with discontinuities cannot be expressed as Taylor series:

1.5
square wave
1.0

0.5

0.0

0.5

1.0

1.5
6 4 2 0 2 4 6

A polynomial cannot jump between 1 and -1

4
Functions with discontinuities in some of their derivatives cannot be expressed as
Taylor series:
(
1
/2 (1 + cos x) |x| < π
Eg. cosine bell, f (x) =
0 otherwise

1.5

cosine bell
1−12 x2
1.0 2!
1−12 x2 + 12 x4
2! 4!
1−12 x2 + 12 x4 −1 x6
2! 4! 2 6!

0.5

0.0

0.5
6 4 2 0 2 4 6

A polynomial cannot be uniform in one place and non-uniform in another

The Taylor Series:

A function, f , can be represented as a Taylor series about position a if:
• is continuous near a and
• all of its derivatives are continuous near a

Using the notation ∆x = x − a:

∆x2 00 ∆x3 000 ∆x j ( j)

f (x) = f (a) + ∆x f 0 (a) + f (a) + f (a) + · · · + f (a) + · · ·
2! 3! j!
df d2 f
where f 0 (x) = (x), f 00 (x) = 2 (x), ...
dx dx
• If infinitely many terms are used then this approximation is exact near a.
• If all terms of order n and above are discarded then the error is approximately proportional
to ∆xn (assuming that ∆x is small). Then the approximation is said to be nth order accurate
• A third order accurate approximation for f (x) has error proportional to ∆x3 :
∆x2 00

f (x) = f (a) + ∆x f 0 (a) + f (a) + O ∆x3 .
2!


We say that the error is of order ∆x3 or O ∆x3 .
• If ∆x is small, then higher order accuracy generally means higher accuracy

6
1.1 Exercises (answers at end)
1. Write down the infinite Taylor series for function f at position x + ∆x about x

2. Write the third order approximation for f (x − ∆x) in terms of f (x), f 0 (x) and f 00 (x). Write
the error term using the O (∆xn ) notation.

3. Write the third order approximation for f (x + ∆x) in terms of f (x − ∆x), f 0 (x − ∆x) and
f 00 (x − ∆x).

2 Numerical Differentiation
Consider a set of points, x0 , x1 , · · · x j , · · · xn where x j = j∆x (the points are distance ∆x apart).
Assume that we know the value of the function f (x) at these points, as shown in figure 1.

f f j−1 fj
f j+1

x0 x1 x2 ∆x x j−1 xj x j+1 x
Figure 1: Values of a function f at points x0 , x1 , · · · , x j , · · · .

Some possible estimate of f j0 = f 0 (x j ) are:

forward difference backward difference centred difference
f j+1 − f j f j − f j−1 f j+1 − f j−1
f j0 ≈ f j0 ≈ f j0 ≈
∆x ∆x 2∆x
Taylor series can be used to derive estimates of derivatives and to find their order of accuracy.
8
3 Taylor Series to find Finite Difference Gradients
In order to use a Taylor series (below) to find an approximation for f 0
∆x2 00 ∆x3 000 ∆x j ( j)
f (x + ∆x) = f (x) + ∆x f 0 (x) + f (x) + f (x) + · · · + f (x) + · · ·
2! 3! j!
1. write down the knowns
2. consider where we want to find f 0
3. consider what order of accuracy we want
4. write down Taylor series for some of the knowns
5. eliminate the additional unknowns to find f 0

3.1 Example
1. Assume that we know f j = f (x j )
f j−1 = f (x j−1 ) = f (x − ∆x)
f j+1 = f (x j+1 ) = f (x + ∆x)
2. and we want to find f j0 .
3. For 3 knowns we wonder if we can get second order accuracy
0 0
4. We do not want to generate too many unknowns. We don’t know f j−1 or f j+1 so no Taylor
series about x j−1 or x j+1 . So let’s try Taylor series for f j+1 and f j−1 about x j
∆x2 00 ∆x3 000
f j+1 = f j + ∆x f j0 + f + f + O(∆x4 )
2! j 3!
∆x2 00 ∆x3 000
f j−1 = f j − ∆x f j0 + f − f + O(∆x4 )
2! j 3!

5. Eliminate f j00 by taking the difference of the two equations

∆x3 000
f j+1 − f j−1 = 2∆x f j0 + f + O(∆x4 )
3 j
Rearrange to get f j0
f j+1 − f j−1 ∆x2 000
f j0 = − f j + O(∆x3 )
2∆x 3!
We cannot eliminate f j000 so this is part of the error:
f j+1 − f j−1
f j0 = + O(∆x2 )
2∆x
The error, ε , is proportional to ∆x2 (ε ∝ ∆x2 ) so this approximation is second order accu-
rate.

This is a worked example on my YouTube page:

https://www.youtube.com/channel/UCO0YwmerBCvW-BrR8kcjLzw
called TaylorSeries2

10
3.2 Exercises (answers at the end)
1. Use the Taylor series to find an approximation for f j0 in terms of f j and f j−1 . What order
accuracy is it?

2. Derive an uncentred, second order difference formula for f j0 that uses f j , f j+1 and f j+2 .
(And show that it is second order accurate)

3. Find an uncentred approximation for f j00 using f j , f j+1 and f j+2 . What order accurate is it?
4. Derive a second order approximation for fb0 from fa , fb and fc at x locations a < b < c
when the grid spacing is not regular. (And show that it is second order accurate)
11

4 Order of Accuracy of Numerical Solutions

In order to demonstrate the order of accuracy (or order of convergence) of a numerical
method, we can calculate the error of solving a problem that has an analytic solution. For
example our numerical method calculates the gradient of sin x and gives these results:
∆x numerical gradient of sin x at x = 0 Error, ε (Difference from cos(0))
0.4 0.97355 -0.02645
0.2 0.99335 -0.00666
0.1 0.99833 -0.00167
n
Assume that ε = A∆x where n is the order of accuracy and A is unknown. From the data,
eliminate A and find two possible values for n.
10-1
|error|

10-2

10-310-2 10-1 100

∆x
Plot |ε | as a function of ∆x on log paper. Then the order of accuracy is the gradient. Does
this value agree with the values found from calculating n directly from the data above? (both
should give about n = 2. See YouTube video for a worked example.
12
5 Interpolation
f2

An Example:
fi
• Function f is known at points x1 and x2
(values f1 and f2 ) f1
• We want to estimate the value of f at point
xi in between x1 and x2 x1 xi x2
x
β∆x (1−β)∆ x
∆x
• Exercise: Use linear interpolation (ie assume that fi lies on a straight line between f1 and
f2 ): to find f at xi
Hint: First write down expressions for ∆x, β and the gradient, f 0 between x1 and x2 . Then
find an expression for f at xi along the straight line between x1 and x2 .

xi − x1 f2 − f1
∆x = x2 − x1 β= f0 = =⇒ fi = (1 − β ) f1 + β f2
x2 − x1 x2 − x1
• If f is known at n points then a polynomial of degree n − 1 can be fit to estimate f

Eg. Cubic Lagrange interpolation for constant grid spacing, ∆x:

1 1
fˆ(x) = − β (1 − β )(2 − β ) fk−1 + (1 + β )(1 − β )(2 − β ) fk
6 2
1 1
+ (1 + β )β (2 − β ) fk+1 − (1 + β )β (1 − β ) fk+2
2 6
13

5.1 Finding Finite Difference Formulae for Interpolation using Taylor Series
An Example:
Assume that we know f j = f (x j ) and f j+1 = f (x j+1 ) and we want to find the interpolated
value, f j+1/2 , mid-way between x j and x j+1 .
• Start by writing down Taylor series for f j and f j+1 about f j+1/2
   
∆x 0 1 ∆x 2 00 1 ∆x 3 000
f j+1 = f j+1/2 + f j+1/ + f j+1/ + f j+1/ + O(∆x)4
2 2 2! 2 2 3! 2 2
 2  3
∆x 0 1 ∆x 00 1 ∆x 000 4
f j = f j+1/2 − f j+1/ + f j+ 1/ − f j+ 1/ + O(∆x)
2 2 2! 2 2 3! 2 2

0
• Eliminate the largest unknown, f j+ 1/ by adding the two equations
2
∆x2 00
f j + f j+1 = 2 f j+1/2 + f 1 + O(∆x)4
4 j+ /2
• Rearrange to find f j+1/2 and express the error based on the largest unknown
f j+1/2 = ( f j + f j+1 )/2 + O(∆x)2
• So this is a second-order accurate approximation

14
5.2 Exercises (answers at the end)
1. Derive a centred, second order difference interpolation formula for f j that uses f j−1 and
f j+1 . (And show that it is second order accurate)
0
2. Derive a centred fourth order difference formula for f j+1/ that uses f j−1 , f j , f j+1 and f j+2 .
2
(And show that it is fourth order accurate)
3. Show that the first order forward difference formula for f j0 is exact for linear functions,
f (x) = ax + b.
4. Show that the centred second order difference formula for f j0 is exact for quadratic func-
tions f (x) = ax2 + bx + c.

15

Solutions
Solutions to Exercises 1.1
1. Write down the infinite Taylor series for function f at position x + ∆x about x
∆x2 00 ∆x3 000 ∆x j ( j)
f (x + ∆x) = f (x) + ∆x f 0 (x) + f (x) + f (x) + · · · + f (x) + · · ·
2! 3! j!
2. Write the third order approximation for f (x − ∆x) in terms of f (x), f 0 (x) and f 00 (x). Write
the error term using the O (∆xn ) notation.
∆x2 00

f (x) + O ∆x3
f (x − ∆x) = f (x) − ∆x f 0 (x) +
2!
3. Write the third order approximation for f (x + ∆x) in terms of f (x − ∆x), f 0 (x − ∆x) and
f 00 (x − ∆x).
4∆x3 000

f (x + ∆x) = f (x − ∆x) + 2∆x f 0 (x − ∆x) + 2∆x2 f 00 (x − ∆x) + f (x − ∆x) + O ∆x3
3
Solutions to Exercises 3.2
1. Use the Taylor series to find an approximation for f j0 in terms of f j and f j−1 . What order
accuracy is it?
Write the Taylor series for f j−1 in terms of f j :
f j−1 = f j − ∆x f j0 + O(∆x2 )
Rearrange to find f j0 :
f j0 = ( f j − f j−1 )/∆x + O(∆x)
Note dividing O(∆x2 ) by ∆x gives O(∆x) so the approximation is first order accurate
2. Derive an uncentred, second order difference formula for f j0 that uses f j , f j+1 and f j+2 .
(And show that it is second order accurate)
16
 Taylor approximations for f j+1 and f j+2 about f j :
∆x2 00 ∆x3 000 ∆x4 0000
f j+1 = f j + ∆x f j0 + f + f + f + O(∆x5 )
2! j 3! j 4! j
4∆x3 000 2∆x4 0000
f j+2 = f j + 2∆x f j0 + 2∆x2 f j00 + f + f + O(∆x5 )
3 j 3 j
Eliminate the largest unknown, f j00 by calculating f j+2 − 4 f j+1 :
f j+2 − 4 f j+1 = −3 f j − 2∆x f j0 + O(∆x3 )
Rearrange to find f j0 :


f j0 = − f j+2 + 4 f j+1 − 3 f j /(2∆x) + O(∆x2 )
3. Find an uncentred approximation for f j00 using f j , f j+1 and f j+2 . What order accurate is it?
Taylor approximations for f j+1 and f j+2 about f j :
∆x2 00 ∆x3 000 ∆x4 0000
f j+1 = f j + ∆x f j +
0
fj + fj + f j + O(∆x5 )
2! 3! 4!
2 00 4∆x3 000 2∆x4 0000
0
f j+2 = f j + 2∆x f j + 2∆x f j + fj + f j + O(∆x5 )
3 3
Eliminate the largest unknown, f j0 by calculating f j+2 − 2 f j+1 :
f j+2 − 2 f j+1 = − f j − ∆x2 f j00 + O(∆x3 )
Rearrange to find f j00 :


f j00 = − f j+2 + 2 f j+1 − f j /∆x2 + O(∆x)
4. Derive a second order approximation for fb0 from fa , fb and fc at x locations a < b < c
when the grid spacing is not regular. (And show that it is second order accurate)
Define ∆x1 = b − a and ∆x2 = c − b and ∆x = max (∆x1 , ∆x2 )
Taylor approximations for fa and fc about fb :
∆x2 ∆x3 ∆x4
fa = fb − ∆x1 fb0 + 1 fb00 − 1 fb000 + 1 fb0000 + O(∆x15 )
2! 3! 4!
17

∆x22 00 ∆x23 000 ∆x24 0000

fc = fb + ∆x2 fb0 + f + f + f + O(∆x25 )
2! b 3! b 4! b
Eliminate the largest unknown,
 fb00 : 
∆x1 fc − ∆x2 fa = ∆x1 fb + ∆x2 fb0 + O(∆x23 ) − ∆x22 fb − ∆x1 fb0 + O(∆x13 )
2 2 2

Rearrange to find fb0 :

∆x12 fc − ∆x22 fa = ∆x12 fb + ∆x12 ∆x2 f
b0 + O(∆x23 ∆x12 ) − ∆x22 fb + ∆x1 ∆x22 fb0 + O(∆x13 ∆x22 )
=⇒ ∆x12 fc − ∆x22 fa = ∆x12 − ∆x22 fb +
∆x1 ∆x2 (∆x1 + ∆x2 ) fb0 + O(∆x5 )
∆x2 fc − ∆x22 fa − ∆x12 − ∆x22 fb
=⇒ fb0 = 1 + O(∆x2 )
∆x1 ∆x2 (∆x1 + ∆x2 )
Solutions to Exercises 5.2
1. Use 2 Taylor series about f j so that they both contain f j0 so that we can eliminate it:
1
f j−1 = f j − ∆x f j0 + ∆x2 f j00 + O(∆x3 )
2!
1
f j+1 = f j + ∆x f j + ∆x2 f j00 + O(∆x3 )
0
2!
Eliminate f j0 by adding the two equations:
f j−1 + f j+1 = 2 f j + ∆x2 f j00 + O(∆x3 )
Rearrange for f j and express the unknowns as the order of accuracy:
f j = ( f j−1 + f j+1 )/2 + O(∆x2 )
2. Taylor series about f j+1/2 :
3 1 9 2 00 1 27 3 000 1 81 4 0000
f j−1 = f j+1/2 − ∆x f j+
0
1/ + ∆x f j+1/ − ∆x f j+1/ + ∆x f j+1/ + O(∆x5 )
2 2 2! 4 2 3! 8 2 4! 16 2
1 1 1 1 1 1 1
f j = f j+1/2 − ∆x f j+
0
1/ + ∆x2 f j+
00
1/ − ∆x3 f j+
000
1/ + ∆x4 f j+
0000 5
1/ + O(∆x )
2 2 2! 4 2 3! 8 2 4! 16 2
1 1 1 2 00 1 1 3 000 1 1 4 0000
f j+1 = f j+1/2 + ∆x f j+
0
1/ + ∆x f j+1/ + ∆x f j+1/ + ∆x f j+1/ + O(∆x5 )
2 2 2! 4 2 3! 8 2 4! 16 2
18
 3 1 9 2 00 1 27 3 000 1 81 4 0000
f j+2 = f j+1/2 + ∆x f j+ 0
1/ + ∆x f j+1/ + ∆x f j+1/ + ∆x f j+1/ + O(∆x5 )
2 2 2! 4 2 3! 8 2 4! 16 2
00
First eliminate f j , the biggest unknown (we only need to leave two equations):
9 3 000 5
1/ + ∆x f j + O(∆x )
0
f j+2 − f j−1 = 3∆x f j+
2 8
1 1 3 000
f j+1 − f j = ∆x f j+1/ +
0
∆x f j + O(∆x5 )
2 38
Next eliminate f j000 leaving one equation for f j0 :
0 5
f j+2 − f j−1 − 27 f j+1 + 27 f j = −24∆x f j+ 1/ + O(∆x )
2
f j−1 − 27 f j + 27 f j+1 − f j+2
=⇒ f j+ 0
1/ = + O(∆x4 )
2 24∆x
f j+1 − f j
3. If f (x) = ax + b and we use the approximation f j0 = between any two points x j+1
∆x
and x j then we get:
(ax j+1 + b) − (ax j + b)
f j0 = = a which is the exact solution.
x j+1 − x j
f j+1 − f j−1
4. If f (x) = ax2 + bx + c and we use the approximation f j0 = for f j at x j between
2∆x
two points x j+1 and x j−1 then we get:
(ax2j+1 + bx j+1 + c) − (ax2j−1 + bx j−1 + c) a(x j + ∆x)2 − a(x j − ∆x)2 + 2b∆x
f j0 = =
x j+1 − x j−1 2∆x
= 2ax j + b which is the exact solution.

19