Lecture 2:

Interpolation
J.K. [email protected]
WI3097TU
Delft Institute of Applied Mathematics
Delft University of Technology

10 September 2012

()

Interpolation // J.K. Ryan

10 September 2012

1 / 43

Outline

Review

Lagrange Interpolation
Definition
Forming the approximation
Approximation properties
Approximation error

Spline Interpolation

()

Interpolation // J.K. Ryan

10 September 2012

2 / 43

Last lecture
Important Concepts

Important tools: Taylors theorem, Triangle inequality

How numbers are stored
Errors
I
I
I

Roundoff error: = fl(x ) x

Absolute error: |e| = |uexact uapproximate |
Relative error:
|uexact uapproximate |
absolute error
=
|uexact |
|uexact |

Landaus O symbol

()

Interpolation // J.K. Ryan

10 September 2012

3 / 43

Beginning of Numerics
Overall Goal: to solve
d
u(x ) = f (u(x ))
dx
.
Approximate a function (today).
Approximate the derivative (next time).
Approximate an integral (later).
Todays Goal: Given a finite set of data points that represent a
function,y = f (x ),
(x0 , y0 ), (x1 , y1 ), (x2 , y2 ), . . . , (xn , yn ),
we want to form an approximation to y = f (x ).
()

Interpolation // J.K. Ryan

10 September 2012

4 / 43

Beginning of Numerics

To approximate a function, one method is through

interpolation.
Lagrange interpolation.
Cubic spline interpolation.

()

Interpolation // J.K. Ryan

10 September 2012

5 / 43

Interpolation/Extrapolation
Given a set of data points,
(x0 , y0 ),

(x1 , y1 ),

(x2 , y2 ),

. . . , (xn , yn )

where yn = f (xn ).

Definition (Interpolation)
For interpolation, we try to approximate a value in between the
given data points. That is, an approximation, yh f (x ), where
x0 x xn .

Definition (Extrapolation)
For extrapolation, we try to approximate a value outside the
range of the given data points. That is, an approximation,
yh f (x ), where either x < x0 or x > xn .
()

Interpolation // J.K. Ryan

10 September 2012

6 / 43

Interpolation

Example
Given a set of data points,


2,

1
,
2


5 2
,
,
2 5


1
,
4


4,

we want to find the value at

x = 3, interpolate
x = 5, extrapolate.
Simplest method: use piecewise constants.

()

Interpolation // J.K. Ryan

10 September 2012

7 / 43

Linear interpolation
Linear interpolation is a
straight line between 2 points:

(x0 , y0 ), (x1 , y1 ) where

y0 = f (x0 ) and y1 = f (x1 ).
Left: We use the endpoints
for the approximation.

()

Interpolation // J.K. Ryan

10 September 2012

8 / 43

Linear interpolation

The approximation to f (x ) is given by the linear interpolation

formula

p(x ) =

x x0
x x1
y0 +
y1 .
x0 x1
x1 x0

Note: Quite similar to the point-slope formula,

y = mx + b, where m =slope and b = y intercept.

()

Interpolation // J.K. Ryan

10 September 2012

9 / 43

Linear interpolation
We can right this in another way:

x x0
x x1
y0 +
y1
x0 x1
x1 x0
= L0,1 (x )y0 + L1,1 (x )y1

p(x ) =

where
L0,1 (x ) =

()

x x1
x x0
and L1,1 (x ) =
x0 x1
x1 x0

Interpolation // J.K. Ryan

10 September 2012

10 / 43

Linear interpolation
Notice:
Polynomial of degree 1 (with order 2)
Constructed using 2 points
Can relate to Linearization:
L(x ) = f (x0 ) + f 0 (x0 )(x x0 )

At x0 :

p(x0 ) = y0 ,

L0,1 (x0 ) = 1,

L1,1 (x1 ) = 0

At x1 :

p(x1 ) = y1 ,

L0,1 (x0 ) = 0,

L1,1 (x1 ) = 1

()

Interpolation // J.K. Ryan

10 September 2012

11 / 43

Interpolation
Generalization

Generalization to higher order:

Given n + 1 points:
(x0 , f (x0 )),

(x1 , f (x1 )),

. . . , (xn , f (xn ))

Is it possible to construct a polynomial of degree n that

interpolates f (x ) at those points?

()

Interpolation // J.K. Ryan

10 September 2012

12 / 43

Interpolation
Generalization

Definition (Lagrange interpolating polynomial)

Given n + 1 data points,
(x0 , f (x0 )),

(x1 , f (x1 )),

. . . , (xn , f (xn )),

we can construct a polynomial of degree n at those points,

p(x ) =

n
X

f (xk )Lk,n (x ),

k=0

where p(x ) is the Lagrange interpolating polynomial and

Lk,n (x ) are the Lagrange coefficients which are given by
Lk,n (x ) =

()

(x x0 )(x x1 ) (x xk1 )(x xk+1 ) (x xn )

.
(xk x0 )(xk x1 ) (xk xk1 )(xk xk+1 ) (xk xn )

Interpolation // J.K. Ryan

10 September 2012

13 / 43

Interpolation
Theorem (Lagrange interpolation)
If points x0 , x1 , . . . , xn are n + 1 distinct points and f (x ) is
a function whose values are given at these points, then there
exists a unique polynomial, p(x ), of degree at most n such that
f (xk ) = p(xk ),

k = 0, 1, . . . , n.

The polynomial is given by

p(x ) =

n
X

f (xk )Lk,n (x ),

k=0

where
Lk,n (x ) =

n
Y
x xj
j =0

xk xj

k = 0, 1, . . . , n.

j6=k
()

Interpolation // J.K. Ryan

10 September 2012

14 / 43

Interpolation

Notice:
p(x ) is a polynomial of degree n.
Constructed using n + 1 points.
(

Lk,n (xj ) =

0,
1,

j=
6 k
j =k

Polynomial is unique.
p(xj ) = f (xj ) for j = 0, 1, . . . , n.

()

Interpolation // J.K. Ryan

10 September 2012

15 / 43

Interpolation

Example
Given x0 = 2, x1 = 2.5, x2 = 4, find an interpolating
polynomial of degree 2 for f (x ) = x1 .
This means our given data is
1
2,
,
2

5 2
,
,
2 5


1
4,
.
4


We have 3 = n + 1 points. We should be able to construct a

polynomial of degree 2 = n.

()

Interpolation // J.K. Ryan

10 September 2012

16 / 43

Interpolation
Example (continued)

In this case,
p(x ) =

n
X

f (xk )Lk,n (x ) =

k=0

where
Lk,2 (x ) =

2
X

f (xk )Lk,2 (x ),

k=0
2
Y
x xj
j =0

xk xj

k = 0, 1, 2.

j6=k

More specifically,
L0,2 (x ) =

(x x1 )(x x2 )
,
(x0 x1 )(x0 x2 )
L2,2 (x ) =

()

L1,2 (x ) =

(x x0 )(x x2 )
,
(x1 x0 )(x1 x2 )

(x x0 )(x x1 )
(x2 x0 )(x2 x1 )

Interpolation // J.K. Ryan

10 September 2012

17 / 43

Interpolation
Example (continued)

Plugging in our data:

p(x ) =

2
X

f (xk )Lk,2 (x )

k=0

= f (x0 )L0,2 (x ) + f (x1 )L1,2 (x ) + f (x2 )L2,2 (x )

1
2
1
= L0,2 (x ) + L1,2 (x ) + L2,2 (x )
2
5
4
where


L0,2 (x ) = 

5
2

5
2

(x 4)
(2 4)

(x 2)(x 4)

,
L1,2 (x ) = 
5
5

4
2
2


5
2

5
2

(x 2) x
L2,2 (x ) =
()

(4 2) 4

Interpolation // J.K. Ryan

10 September 2012

18 / 43

Interpolation
Example (continued)

Simplifying, we have that

p(x ) = (0.05x 0.425)x + 1.15

interpolates f (x ) =
2 x 4.

()

1
x

for

Interpolation // J.K. Ryan

10 September 2012

19 / 43

Interpolation
Error

Once we have constructed the approximation, we should always

ask:
How good is this approximation?

Absolute Error = |Exact Approximate|

= |f (x ) p(x )|


1


=  ((0.05x 0.425)x + 1.15)
x

()

Interpolation // J.K. Ryan

10 September 2012

20 / 43

Interpolation
Error

At x = 3 (interpolation):



1
Absolute Error =  ((0.05(3) 0.425)3 + 1.15) = 0.00833
3

At x = 5 (extrapolation):


1


Absolute Error =  ((0.05(5) 0.425)5 + 1.15) = 0.075
5

()

Interpolation // J.K. Ryan

10 September 2012

21 / 43

Interpolation
Error

Interpolation, x = 3 :
|f (x ) p(x )| = 0.0083
3

Extrapolation, x = 5 :

|f (x ) p(x )| = 0.075

()

Interpolation // J.K. Ryan

10 September 2012

22 / 43

Interpolation

Theorem (Interpolation error)

Given n + 1 distinct points in [a, b], and a function with n + 1
continuous derivatives, f C n+1 (a, b), then for each x [a, b]
there exists a (a, b) with




1
(n+1)

|f (x ) p(x )| = (x x0 )(x x1 ) (x xn )
f
((x )) ,
(n + 1)!

where p(x ) is the Lagrange interpolating polynomial of degree

n.

()

Interpolation // J.K. Ryan

10 September 2012

23 / 43

Interpolation
Error

From the previous example: f (x ) =

1
x

f (3) = x64

The error formula would then be:




1 (3) 
f ()
3!






5
1
6 
(x 4)
4 
= (x 2) x
2
3!

 







1
6
5

max  4  (x 2) x
(x 4)
3! [2,4]
2

|f (x ) p(x )| = (x x0 )(x x1 )(x x2 )

()

Interpolation // J.K. Ryan

10 September 2012

24 / 43

Interpolation

Sometimes, it is better to use a lower order polynomial

approximation, but use smaller intervals.

Example
Let f (x ) = e x and x [0, 1]. Suppose we want to use a
piecewise linear interpolation using points x0 , x1 , . . . , xn .
Find a piecewise linear interpolant for f (x ).
What should the spacing be between data to keep the error
less than 106 ?

()

Interpolation // J.K. Ryan

10 September 2012

25 / 43

Interpolation
Example

Define xj = 0 + jh,
between the data,

j = 0, . . . , n. h then represents the spacing

h = xj+1 xj .

For linear interpolation, the Lagrange interpolant for

x [xj , xj+1 ] is given by
p(x ) = f (xj )Lj,1 (x ) + f (xj+1 )Lj+1,1 (x ),
x xj+1
x xj
+ f (xj+1 )
= f (xj )
xj xj+1
xj+1 xj

()

Interpolation // J.K. Ryan

x (xj , xj+1 )

10 September 2012

26 / 43

Interpolation
Example

This gives
p(x ) = e xj

x xj+1
x xj
+ e xj+1
xj xj+1
xj+1 xj

But h = xj+1 xj

()

p(x ) =

1
((xj+1 x )e xj + (x xj )e xj+1 )
h

Interpolation // J.K. Ryan

10 September 2012

27 / 43

Interpolation
Example

We want
|f (x ) p(x )| 106
For which h does this hold?
Using our interpolation error theorem



|f (x ) p(x )| (x xj )(x xj+1 )

1 (2) 
f ()
2!

1 (2)
| f () ||(x xj )(x xj+1 )|
2! | {z }
d2 x
e =e x
dx 2

1
e h2
max e | (x jh) (x (j + 1)h) |
| {z } |
{z
}
2 (0,1)
2 4
h
2

()

Interpolation // J.K. Ryan

h
2

10 September 2012

28 / 43

Interpolation
Example

We want
|f (x ) p(x )| 106
where
e
|f (x ) p(x )| h2 106
8

h2 8e 1 106

()

h 8e 1 106

Interpolation // J.K. Ryan

1/2

10 September 2012

29 / 43

Lagrange Interpolation
Summary

One method of constructing an approximation to a function, f (x ) is

1
2

Take n + 1 data points: x0 , x1 , . . . , xn .

Define the Lagrange coefficients:
Lk,n (x ) =

n
Y
x xj
j =0

xk xj

k = 0, 1, . . . , n.

j6=k

Then the approximating polynomial is given by

p(x ) =

n
X

f (xk )Lk,n (x )

k=0
4

The error is given by

n
Y



1




|f (x ) p(x )|
(x xk ) max f (n+1) ()

 (x0 ,xn )
(n + 1)! k=0

()

Interpolation // J.K. Ryan

10 September 2012

30 / 43

Spline Interpolation

Another way to form a piecewise polynomial approximation is

through Spline interpolation.
Break the intervals into subintervals
{(x0 , y0 ), (x1 , y1 )}, {(x1 , y1 ), (x2 , y2 )}, . . . {(xn1 , yn1 ), (xn , yn )}

Use piecewise polynomial approximation

Use differentiability at endpoints of submitervals

()

Interpolation // J.K. Ryan

10 September 2012

31 / 43

Spline Interpolation
Linear interpolation

We could use piecewise linear polynomials:

Definition
Given a function f on [a, b] and a set of nodes
a = x0 < x1 < < xn = b, a linear spline interpolant is a
function s(x ) C[a, b] such that
si (x ) is a linear polynomial on the interval
[xi , xi+1 ], i = 0, . . . , n 1
si (xi ) = f (xi ),

i = 0, . . . , n

But this gives something like piecewise Lagrange interpolation.

()

Interpolation // J.K. Ryan

10 September 2012

32 / 43

Spline Interpolation
Cubic interpolation

What about a higher order piecewise interpolant?

Definition
Given a function f on [a, b] and a set of nodes
a = x0 < x1 < < xn = b, a cubic spline interpolant is a
function s(x ) C 2 [a, b] such that
1

si (x ) is a polynomial of degree 3 on each subinterval

[xi , xi+1 ], i = 0, . . . , n 1

si (xi ) = f (xi ),
dk

i = 0, . . . , n 1 and si (xi+1 ) = f (xi+1 )

dk

s (x )
dx k i i+1

= dx k si+1 (xi+1 ),
i = 0, . . . , n 2, k = 0, 1, 2

(x ) = 0
(One possible condition) s0 (x0 ) = sn1
n

()

Interpolation // J.K. Ryan

10 September 2012

33 / 43

Cubic Spline Interpolation

This means that given equally spaced data,

(x0 , y0 ), (x1 , y1 ), . . . , (xn , yn )
we can define
si (x ) = ai (x xi )3 + bi (x xi )2 + ci (x xi ) + di
on [xi , xi+1 ], where h = xi+1 xi .
We must solve for the coefficients, ai , bi , ci , di .

()

Interpolation // J.K. Ryan

10 September 2012

34 / 43

Cubic Spline Interpolation

Step 1

Plug in x = xi into
si (x ) = ai (x xi )3 +bi (x xi )2 +ci (x xi )+di ,

i = 0, . . . , n1

si (xi ) = f (xi )
= ai (xi xi )3 + bi (xi xi )2 + ci (xi xi ) + di ,
= di
di = f (xi ),

()

i = 0, . . . , n 1

Interpolation // J.K. Ryan

10 September 2012

35 / 43

Cubic Spline Interpolation

Step 2
(x

Use si (xi+1 ) = si+1

i+1 ) and s0 (x0 ) = sn1 (xn ) = 0 where

si (x ) = 6ai (x xi ) + 2bi
This gives
si (xi+1 ) = 6ai (xi+1 xi ) + 2bi = 6ai h + 2bi

si+1
(xi+1 ) = 2bi+1

ai =

(1)

1
(bi+1 bi ),
3h

i = 0, . . . , n 2

and
b0 = 0,

()

an1 =

1
bn1
3h

Interpolation // J.K. Ryan

i = 0, . . . , n 2

10 September 2012

36 / 43

Cubic Spline Interpolation

Step 3

Use si (xi+1 ) = si+1 (xi+1 ) and sn1 (xn ) = yn Which gives,

si (xi+1 ) = ai (xi+1 xi )3 + bi (xi+1 xi )2 + ci (xi+1 xi ) + yi
= ai h3 + bi h2 + xi h + yi
si+1 (xi+1 ) = yi+1
So that we have
ci =

1
h
(yi+1 yi ) (bi+1 + 2bi ),
h
3

i = 0, . . . , n 2

and
cn1 =

()

1
2h
(yn yn1 ) bn ,
h
3

Interpolation // J.K. Ryan

10 September 2012

37 / 43

Cubic Spline Interpolation

Step 4

0 (x
From si0 (xi+1 ) = si+1
i+1 ) we obtain a linear system:

2
1
1
si0 (xi+1 ) = hbi+1 + hbi + (yi+1 yi )
3
3
h
1
0
si (xi+1 ) = (bi+2 + 2bi+1 ) + 1h(bi+1 + 2bi+1 )
3
Which gives
bi+2 + 4bi+1 + bi =

()

3
(yi+2 2yi+1 + yi ),
h2

Interpolation // J.K. Ryan

i = 0, . . . , n 3

10 September 2012

38 / 43

Cubic Spline Interpolation

Algorithm

Given data,
(x0 , y0 ), (x1 , y1 ), . . . , (xn , yn )
we define
si (x ) = ai (x xi )3 + bi (x xi )2 + ci (x xi ) + di
on [xi , xi+1 ], where hi = xi+1 xi . To find the coefficients:
1

Solve linear system Mb = g for b = [b0 , b1 , . . . , bn1 ]T

Calculate ci .

Calculate ai .

Form si (x ) = ai (x xi )3 + bi (x xi )2 + ci (x xi ) + di for
x [xi , xi+1 ].

()

Interpolation // J.K. Ryan

10 September 2012

39 / 43

Cubic Spline Interpolation

Example
Given data
1
2,
,
2

(1, 1),

1
3,
,
3

1
4,
,
4


The cubic spline interpolant that approximation f (x ) =

this data is given by

S(x ) =

()

1
3
2

12 (x 3x 4x + 18),
1
3
2
12 (x + 9x

1 (x + 7),
12

28x + 34),

Interpolation // J.K. Ryan

1
x

using

1 x 2,
2 < x 3,
3 < x 4.

10 September 2012

40 / 43

Cubic Spline Interpolation

How did we find this Spline interpolant?
Piecewise polynomial intervals: [x0 , x1 ], [x1 , x2 ], [x2 , x3 ].
This means we have 3 spline interpolants s0 (x ), s1 (x ),
and s2 (x ).
1
i = 0, . . . , n 1.
xi ,

s2 (x3 ) = 0 s0 (1) = s2 (4)

We know di = f (xi ) =
Boundaries: s0 (x0 ) =

1
a2 = b2
3

b0 = 0
Solve the system for bi ,
"

()

4 1
1 4

#"

b1
b2

=0

i = 1, 2 :

"

1
1
4

"

Interpolation // J.K. Ryan

b1
b2

"

1
4

10 September 2012

41 / 43

Cubic Spline Interpolation

Using ci =

Using ai =

1
hi (yi+1

yi )

c0 =

7
,
12

1
2hi (bi+1

a0 =

hi
3 (bi+1

+ 2bi ) :

1
c1 = ,
3

c2 =

1
12

bi ) :

1
,
12

a1 =

1
,
12

a2 = 0

Gives

1
7
3

s0 (x ) = 12 (x 1) 12 (x 1) + 1,
1
3
s(x ) = s1 (x ) = 12 (x 2) + 14 (x 2)2 13 (x 2) + 21 ,

1
s2 (x ) = 12
(x 3) + 31 ,
()

Interpolation // J.K. Ryan

10 September 2012

x
2<x
3<x

42 / 43

Material addressed

Review

Lagrange Interpolation
Definition
Forming the
approximation
Approximation properties
Approximation error

()

Spline Interpolation
Material in book:
Chapter 2, Sections 1-3, 5-6

Interpolation // J.K. Ryan

HIGHLY Recommended
exercises:
All from Ch. 2.

10 September 2012

43 / 43