Interpolation and

Approximation with
Python
 Interpolation
Interpolation is used to estimate data points between two known points
Known points ?

?
• The most common
interpolation technique is
? Linear Interpolation.
• Others are Quadratic, Cubic, … (Splines)
Examples of Polynomial Interpolation
Linear Interpolation Quadratic Interpolation

 Given any two points, Given any three points there is

there is one polynomial of one polynomial of
order ≤ 1 that passes order ≤ 2 that passes
through the two points. through the three points.

3
Existence and Uniqueness
Given a set of n+1 points:
x0 , f ( x0 ), x1, f ( x1), ...., xn , f ( xn )
Assumption: x0 , x1 ,..., xn are distinct

Theorem:
There is a unique polynomial fn(x) of order ≤ n
that passes through all points
f n ( xi )  f ( xi ) for i  0,1,...,n
There is one and only one nth-order polynomial that fits n+1 points 4
 Interpolation
We can use the following packages:
• numpy.interp
https://docs.scipy.org/doc/numpy/reference/generated/numpy.interp.html

• scipy.interpolate
https://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html

y_new = np.interp(x_new, x, y)
f = interpolate.interp1d(x, y)
 Interpolation - Example
We can plot the data points like this:
Assume the following Data:

𝑥 𝑦
1 3
2 2
𝑦
3 0

Assume we want to find the value for

𝑦 when 𝑥 = 2.5

From the plot we see that 𝑦 = 1 is a good guess 𝑥

 Interpolation - Example
Python Code:

import numpy as np
import matplotlib.pyplot as plt
x = [1, 2, 3] One-dimensional linear interpolation for
y = [3, 2, 0]
monotonically increasing sample points.
x_new = 2.5
y_new = np.interp(x_new,x, y)

print("New Interpolated Value:")

print(y_new)

plt.plot(x,y,'o-’) New Interpolated Value:

plt.show()
1.0
Linear vs Cube Interpolation
import numpy as np We start with a cos function
import matplotlib.pyplot as plt and compare Linear and Cubic
from scipy.interpolate import interp1d interpolation
x = np.linspace(0, 10, num=11, endpoint=True)
y = np.cos(-x**2/9.0)
f = interp1d(x, y) #linear is default
f2 = interp1d(x, y, kind='cubic')
xnew = np.linspace(0, 10, num=41, endpoint=True)
plt.plot(x, y, 'o', xnew, f(xnew), '-*', xnew, f2(xnew), '--*')

plt.legend(['data', 'linear', 'cubic'], loc='best') plt.show()

 Linear vs Cube cont.

From Figure we see that the cubic

interpolation gives a good results based
on the cosine function we used in this
example.
Newton’s Divided
Differences
NEWTON’S POLYNOMIAL
Newton’s polynomial interpolation is another popular way to exactly fit a set of data points.
The general form of the nth order Newton’s polynomial that goes through n+1 points is
NEWTON’S
POLYNOMIAL

-1

-1

Given data points

11
NEWTON’S POLYNOMIAL

12 12
NEWTON’S POLYNOMIAL

f [ xk ]  f ( xk ) Zeroth order DD
f [ x1 ]  f [ x0 ]
f [ x0 , x1 ]  First order DD
x1  x0
f [ x1 , x2 ]  f [ x0 , x1 ]
f [ x0 , x1 , x2 ]  Second order DD
x2  x0
............
f [ x1 , x2 ,..., xk ]  f [ x0 , x1 ,..., xk 1 ]
f [ x0 , x1 ,..., xk ] 
xk  x0 13
 Divided Difference Table
 Divided difference are calculated as follows:
f xi   f x j 
f xi , x j 
xi  x j
f xi , x j  f x j , xk 
f xi , x j , xk 
xi  x k
f xn , xn1 , , x2   f xn1 , xn2 , , x1 
f xn , xn1, , x2 , x1  
xn  x1

 Divided differences are calculated using divided difference of a smaller number of

terms:


14
Divided Difference Table
x f[ ] f[ , ] f[ , , ] f[ , , ,]
x0 f[x0]= 𝑎0 f[x0,x1]= 𝑎1 f[x0,x1,x2] = 𝑎2 f[x0,x1,x2,x3]= 𝑎3
x1 f[x1] f[x1,x2] f[x1,x2,x3]
x2 f[x2] f[x2,x3]
x3 f[x3]

𝑛 𝑖−1

𝑓𝑛 (𝑥) = ෍ 𝑓[𝑥0 , 𝑥1 , . . . , 𝑥𝑖 ] ෑ 𝑥 − 𝑥𝑗
𝑖=0 𝑗=0

𝑓𝑛 (𝑥) = 𝑎0 + 𝑎1 𝑥 − 𝑥0 + 𝑎2 𝑥 − 𝑥0 𝑥 − 𝑥1 +. . . +𝑎𝑛 𝑥 − 𝑥0 . . . 𝑥 − 𝑥𝑛−1

𝑎0 = 𝑓(𝑥0 )
𝑎1 = 𝑓[𝑥0 , 𝑥1 ]
....
𝑎𝑛 = 𝑓[𝑥0 , 𝑥1 , . . . , 𝑥𝑛 ] 15
Divided Difference Table
x f[ ] f[ , ] f[ , , ]
xi f(xi)
0 -5 2 -4
1 -3 6
0 -5
-1 -15 1 -3

Entries of the divided difference -1 -15

table are obtained from the data
table using simple operations.

16
Divided Difference Table
x f[ ] f[ , ] f[ , , ] xi f(xi)
0 -5 2 -4 0 -5
1 -3 6 1 -3
-1 -15 -1 -15

The first two column of the table are the data columns.
Third column: First order differences.
Fourth column: Second order differences.

17
Divided Difference Table
x f[ ] f[ , ] f[ , , ] xi yi
0 -5 2 -4 0 -5
1 -3 6
-1 -15 1 -3
-1 -15
 3  (5)
2
1 0
f [ x1 ]  f [ x0 ]
f [ x0 , x1 ] 
x1  x0

18
Divided Difference Table
x f[ ] f[ , ] f[ , , ] xi yi
0 -5 2 -4 0 -5
1 -3 6
-1 -15 1 -3
-1 -15
 15  (3)
6
1 1

f [ x2 ]  f [ x1 ]
f [ x1 , x2 ] 
x2  x1

19
Divided Difference Table
x f[ ] f[ , ] f[ , , ] xi yi
0 -5 2 -4 0 -5
1 -3 6
-1 -15 1 -3
-1 -15

6  (2)
 4
 1  (0)
f [ x1 , x2 ]  f [ x0 , x1 ]
f [ x0 , x1 , x2 ] 
x2  x0
20
Divided Difference Table
xi yi
x f[ ] f[ , ] f[ , , ]
0 -5 2 -4 0 -5
1 -3 6
-1 -15 1 -3
-1 -15

f 2 ( x)  5  2( x  0)  4( x  0)( x 1)

f2(x)= f[x0]+f[x0,x1] (x-x0)+f[x0,x1,x2] (x-x0)(x-x1)

21
Two Examples
Obtain the interpolating polynomials for the two examples:

x y x y

1 0 2 3

2 3 1 0

3 8 3 8

What do you observe?

22
Two Examples

x Y
x Y
1 0 3 1 2 3 3 1
2 3 5 1 0 4
3 8 3 8
P2 ( x)  3  3( x  2)  1( x  2)( x  1)
P2 ( x)  0  3( x  1)  1( x  1)( x  2)
 x2 1
 x2 1
Ordering the points should not affect the divided difference:
f [ x0 , x1 , x2 ]  f [ x1 , x2 , x0 ]  f [ x2 , x1 , x0 ]
23
Example
 Find a polynomial to x f(x)
interpolate the data.
2 3

4 5

5 1

6 6

7 9

24
Example
x f(x) f[ , ] f[ , , ] f[ , , , ] f[ , , , , ]
2 3 1 -1.6667 1.5417 -0.6750
4 5 -4 4.5 -1.8333
5 1 5 -1
6 6 3
7 9
f 4  3  1( x  2)  1.6667( x  2)( x  4)  1.5417( x  2)( x  4)( x  5)
 0.6750( x  2)( x  4)( x  5)( x  6)

25
# Newton Interpolation
import numpy as np
import matplotlib.pyplot as plt
f [ x1 , x2 ]  f [ x0 , x1 ]
f [ x0 , x1 , x2 ] 
x = [0, 1, -1]; y = [-5,-3,-15] x2  x0
n = len(y)
coef = np.zeros([n, n])
coef[:,0] = y # the first column is y

for j in range(1,n): # j denotes column

for i in range(n-j):
coef[i][j] = \
 3  (5)
(coef[i+1][j-1] - coef[i][j-1]) / (x[i+j]-x[i]) 2
1 0
yp = coef[0,0] #first term of the polynomial 𝑐𝑜𝑒𝑓(1,0)−𝑐𝑜𝑒𝑓(0,0)
xp=np.arange(-4,2,0.1) #xp values for the calculation coef(0,1) = 𝑥1 −𝑥0

for i in range(1,n): #from a1 to an-1 cal.

xprod = 1
for j in range(i): # i is not included
xprod *= xp - x[j]

yp += coef[0, i] * xprod

plt.plot(x,y,"*", markersize=20)
plt.plot(xp,yp,"+-"); plt.grid()
Lagrange Interpolation

27
Lagrange Interpolation
Lagrange polynomial interpolation finds a single polynomial that goes through all the data
points. This polynomial is referred to as a Lagrange polynomial, L(x).
As an interpolation function, it should have the property L(xi ) = yi for every point in the dataset.
When computing Lagrange polynomials, it is useful to write them as a linear combination of
Lagrange basis polynomials, Pi(x), where

𝑛
𝑥 − 𝑥𝑗
ℓ𝑖 (𝑥) = ෑ
𝑥𝑖 − 𝑥𝑗
𝑗=0,𝑗≠𝑖
𝑛

𝑓𝑛 (𝑥) = ෍ 𝑓 𝑥𝑖 ℓ𝑖 (𝑥)
𝑖=0
28
Lagrange Interpolation
Problem: xi x0 x1 …. xn
Given ….
yi y0 y1 yn
Find the polynomial of least order f n (x) such that:

f n ( xi )  f ( xi ) for i  0,1,..., n
n
Lagrange Interpolation Formula: f n ( x)   f xi   i ( x)
i 0

x  x 
n
 i ( x)   x  x 
j 0, j i
j

i j
29
Lagrange Interpolation Example
P2 ( x)  f ( x0 ) 0 ( x)  f ( x1 ) 1 ( x)  f ( x2 ) 2 ( x)
x  x1  x  x2   x  1 / 4 x  1 x 1/3 1/4 1
 0 ( x)  y 2 -1 7
x0  x1  x0  x2  1 / 3  1 / 4 1 / 3  1
 1 ( x) 
x  x0  x  x2   x  1 / 3 x  1
x1  x0  x1  x2  1 / 4  1 / 3 1 / 4  1
 2 ( x) 
x  x0  x  x1   x  1 / 3 x  1 / 4 𝑛
x2  x0  x2  x1  1  1 / 3 1  1 / 4 ℓ𝑖 (𝑥) = ෑ
𝑥 − 𝑥𝑗
𝑥𝑖 − 𝑥𝑗
P2 ( x)  2 18( x  1 / 4)( x  1) 116( x  1 / 3)( x  1) 𝑗=0,𝑗≠𝑖

 72( x  1 / 3)( x  1 / 4)

𝑛

𝑓𝑛 (𝑥) = ෍ 𝑓 𝑥𝑖 ℓ𝑖 (𝑥)
𝑖=0
30
Example
Find a polynomial to interpolate: x y

0 1
Both Newton’s interpolation
method and Lagrange 1 3
interpolation method must
give the same answer. 2 2

3 5

4 4

31
Interpolating Polynomial Using Lagrange
Interpolation Method


4

f4 ( x)  f ( xi )  i   0  31  2 2  5 3  4 4
i 0
( x  1) ( x  2) ( x  3) ( x  4) ( x  1)( x  2)( x  3)( x  4)
0  
(0  1) ( 0  2) (0  3) ( 0  4) 24
( x  0) ( x  2) ( x  3) ( x  4) x ( x  2)( x  3)( x  4)
1  
(1  0) (1  2) (1  3) (1  4) 6
( x  0) ( x  1) ( x  3) ( x  4) x ( x  1)( x  3)( x  4) 𝑛
2   𝑥 − 𝑥𝑗
( 2  0) ( 2  1) ( 2  3) ( 2  4) 4 ℓ𝑖 (𝑥) = ෑ
𝑥𝑖 − 𝑥𝑗
( x  0) ( x  1) ( x  2) ( x  4) x ( x  1)( x  2)( x  4) 𝑗=0,𝑗≠𝑖
3   𝑛
( 3  0) (3  1) ( 3  2) ( 3  4) 6
𝑓𝑛 (𝑥) = ෍ 𝑓 𝑥𝑖 ℓ𝑖 (𝑥)
( x  0) ( x  1) ( x  2) ( x  3) x ( x  1)( x  2)( x  3)
4   𝑖=0
( 4  0) ( 4  1) ( 4  2) ( 4  3) 24
32
Code

n
f n ( x)   f xi   i ( x)
i 0

x  x 
n
 i ( x)   x  x 
j 0, j i
j

i j

33
Interpolation using
spline functions
Why Splines ?
1
f ( x) 
1  25 x 2
Table : Six equidistantly spaced points in [-1, 1]
1
x y
1  25 x 2

-1.0 0.038461

-0.6 0.1

-0.2 0.5

0.2 0.5

0.6 0.1

1.0 0.038461 Figure : 5th order polynomial vs. exact function

35
Why Splines ?
1.2

0.8

0.4
y

0
-1 -0.5 0 0.5 1
-0.4

-0.8
x
19th Order Polynomial f (x) 5th Order Polynomial

Figure : Higher order polynomial interpolation is a bad idea 36

Spline Interpolation
 There are cases where polynomials can lead to
erroneous results because of round off error and
overshoot.
 Alternative approach is to apply lower-order
polynomials to subsets of data points. Such
connecting polynomials are called spline functions.

37
 Alternative approach is to apply
lower-order polynomials to subsets of
Spline Interpolation data points.

“n” is the number of equations

There are 3*n unknowns

38
Quadratic Splines (contd) “n” is the number of equations

“n-1” equations can be defined for the solution 39

Quadratic Interpolation (contd)

40
Quadratic Splines (contd)
This gives us ‘3n’ equations and ‘3n’ unknowns. Once we find the ‘3n’ constants,
we can find the function at any value of ‘x’ using the splines,

f ( x)  a1 x 2  b1 x  c1 , x 0  x  x1

 a 2 x 2  b2 x  c 2 , x1  x  x 2
.
.
.
 a n x 2  bn x  c n , x n 1  x  x n

41
Quadratic Spline Example
The upward velocity of a rocket is given as a function of time. Using
quadratic splines, find the velocity at t=16 seconds

Table: Velocity as a
function of time

t (s) v (t ) (m/s)
0 0
10 227.04
15 362.78
20 517.35
22.5 602.97
30 901.67
Figure. Velocity vs. time data
for the rocket example

42
Solution
v(t )  a1t 2  b1t  c1 , 0  t  10
 a2 t 2  b2 t  c2 , 10  t  15
t (s) v (t ) (m/s)
0
10
0
227.04  a3t  b3t  c3 , 15  t  20
2

15 362.78
20
22.5
517.35
602.97
 a4 t 2  b4 t  c4 , 20  t  22.5
30 901.67
 a5 t 2  b5 t  c5 , 22.5  t  30
Let us set up the equations
43
Each Spline Equation Goes Through Two
Consecutive Data Points
v(t )  a1t  b1t  c1 , 0  t  10
2

a1 (0)  b1 (0)  c1  0
2

a1 (10) 2  b1 (10)  c1  227.04

t (s) v (t ) (m/s)
0 0
10 227.04
15 362.78
20 517.35
22.5 602.97
44
30 901.67
Each Spline Goes Through Two
Consecutive Data Points
a2 (10) 2  b2 (10)  c2  227.04
t v(t)
s m/s
a2 (15) 2  b2 (15)  c2  362.78
0 0 a3 (15) 2  b3 (15)  c3  362.78
3th equ.
10 227.04 a3 (20)  b3 (20)  c3  517.35
2

15 362.78
a4 (20) 2  b4 (20)  c4  517.35
20 517.35 4th equ.
a4 (22.5) 2  b4 (22.5)  c4  602.97
22.5 602.97
30 901.67 a5 (22.5) 2  b5 (22.5)  c5  602.97
a5 (30) 2  b5 (30)  c5  901.67
45
Derivatives are Continuous at Interior Data
Points
v(t )  a1t 2  b1t  c1 , 0  t  10
 a2 t 2  b2 t  c2 ,10  t  15
d
dt

a1t 2  b1t  c1  
d
dt
a2t 2  b2t  c2 
t 10 t 10

2a1t  b1  t 10  2a2t  b2  t 10

2a1 10  b1  2a2 10  b2
20a1  b1  20a2  b2  0 46
Derivatives are continuous at
Interior Data Points
At t=10
2a1 (10)  b1  2a2 (10)  b2  0
At t=15
2a2 (15)  b2  2a3 (15)  b3  0
At t=20
2a3 (20)  b3  2a4 (20)  b4  0
At t=22.5
2a4 (22.5)  b4  2a5 (22.5)  b5  0
47
Last Equation

a1  0

48
Final Set of Equations
0 0 1 0 0 0 0 0 0 0 0 0 0 0 0  a1   0 
100 10 1 0 0 0 0 0 0 0 0 0 0 0 0  b1  227 .04 
 0 0 0 100 10 1 0 0 0 0 0 0 0 0
   227 .04 
0 c1
    
 0 0 0 225 15 1 0 0 0 0 0 0 0 0 0 a 2  362 .78 
0 0 0 0 0 0 225 15 1 0 0 0 0 0 0  b2  362 .78 
0 0 0 0 0 0 400 20 1 0 0 0 0 0
  517 .35 
0 c2
    
0 0 0 0 0 0 0 0 0 400 20 1 0 0 0  a3  517 .35 
0 0 0 0 0 0 0 0 0 506 .25 22 .5 1 0 0 0  b3   602 .97 
0 0 0 0 0 0 0 0 0 0 0 0 506 .25 22 .5 1   c3  602 .97 
    
0 0 0 0 0 0 0 0 0 0 0 0 900 30 1  a 4  901 .67 
 20 1 0  20  1 0 0 0 0 0 0 0 0 0 0  b4   0 
 0 0 0 30 1 0  30  1 0 0 0 0 0 0 0  c4   0 
    
0  40 1 0  a5 
0 0 0 0 0 0 40 1 0 0 0
  0 
0 0 0 0 0 0 0 0 0 45 1 0  45 1 0  b5   0 
 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0  c5   0 

a1  0
49
Coefficients of Spline
i ai bi ci
1 0 22.704 0
2 0.8888 4.928 88.88
3 −0.1356 35.66 −141.61
4 1.6048 −33.956 554.55
5 0.20889 28.86 −152.13

50
Final Solution
v(t )  22.704t , 0  t  10
 0.8888t 2  4.928t  88.88, 10  t  15
 0.1356t 2  35.66t  141.61, 15  t  20
 1.6048t 2  33.956t  554.55, 20  t  22.5
 0.20889t 2  28.86t  152.13, 22.5  t  30

51
Velocity at a Particular Point
a) Velocity at t=16
v(t )  22.704t , 0  t  10
 0.8888t 2  4.928t  88.88, 10  t  15
 0.1356t 2  35.66t  141.61, 15  t  20
 1.6048t 2  33.956t  554.55, 20  t  22.5
 0.20889t 2  28.86t  152.13, 22.5  t  30

v16  0.135616  35.6616  141.61

2

 394.24 m/s
52