CHAPTER IV INTERPOLATION

4.1 Method of Undetermined Coefficient

 Example: Consider the following discrete data
n 0 1 2 3
xn 1 2 3 5.5
yn 0 6 20 129.4

* Find y(x=4) using a 3rd order polynomial that fits the above data.
* Start with a 3rd order polynomial:
2 3
P3(x) = a0 + a1 x + a2 x + a3 x
* Set P3(xn) = yn for n=0, 1, 2, and 3 
a0 + a1 + a2 + a3 = 0
a0 + 2a1 + 4a2 + 8a3 = 6
a0 + 3a1 + 9a2 + 27a3 = 20
a0 + 5.5a1 +30.25a2 + 166.375a3 = 129.375
1 x0 x02 x03  a0   y0 
    
1 x1 x12 x13   a1   y1 
=
Or in general, 1 x2 x22 x23  a2   y2 
    
1 x3 x32 x33   a3   y3 

or [V] a = y
where the matrix V is called Vandermonde matrix.
* Gaussian elimination  (a0, a1, a2, a3) = (-4, 5, -2, 1)
* Interpolation value: y(x = 4) ~ P3(x = 4) = 48.0

Notes organized by Prof. Renwei Mei, University of Florida 1

 Comments:
* The solution for (a0, a1, a2, a3) exists and is unique

because det(V) =  ( x j − xi ) ≠0.

1i  j  n

* There are two serious problems with this approach.

i. If the polynomial is of higher order, the system may be
ill-conditioned (Cond(V)=1956.45»1 in the present
example) so that the solution for (a0, a1, a2, a3) may be
unreliable.
ii. When a new data point is encountered, say (x4, y4), we need
to start all over.
 Need more efficient and robust methods.

4.2 Lagrangian Interpolation

Consider the data given below:
xi x0 x1 x2 x3
yi y0 y1 y2 y3
• We can use a 3rd order polynomial to interpolate
• Express the third order polynomial as
3
P3(x) =  li(x) yi
i =0
with li(x) = 3rd order polynomial.

Notes organized by Prof. Renwei Mei, University of Florida 2

4.2.1 Lagrangian interpolating polynomials
* Motivational example: First we consider fitting the following
data using a 3rd order polynomial:
xi x0 x1 x2 x3
yi 1 0 0 0

From algebra, we can express the 3rd order polynomial as

q3(x) = c(x- x1)(x- x2)(x- x3) (because of the 3 roots!)
Set q3(x0) = y0 = 1
 1= c (x0 - x1) (x0 - x2) (x0 - x3)
 c = 1 / [(x0 - x1) (x0 - x2) (x0 - x3)]
( x − x1)( x − x2 )( x − x3 )
 q0(x) = ( l0(x))
( x0 − x1)( x0 − x2 )( x0 − x3 )

Graphical illustration for q0(x) when (x0, x1, x2, x3) = (0, 1, 2, 3):

1.2
L0
1
0.8
0.6
0.4
0.2
0
x
-1 -0.2 0 1 2 3

Notes organized by Prof. Renwei Mei, University of Florida 3

* Similarly, by considering
xi x0 x1 x2 x3
yi 0 1 0 0

we obtain
( x − x0 )( x − x2 )( x − x3 )
q1(x) = (=l1(x))
( x1 − x0 )( x1 − x2 )( x1 − x3 )
Graphical illustration for q1(x):

1.2
L1
1
0.8
0.6
0.4
0.2
0
-1 -0.2 0 1 2 3 x
-0.4

* And similarly,

q2(x) = ( x − x0 )( x − x1)( x − x3 ) (=l2(x)),

( x2 − x0 )( x2 − x1)( x2 − x3 )

q3(x) = ( x − x0 )( x − x1)( x − x2 ) (=l3(x)).

( x3 − x0 )( x3 − x1)( x3 − x2 )

Notes organized by Prof. Renwei Mei, University of Florida 4

 1.2
L2
1
0.8
0.6
0.4
0.2
0
-1 -0.2 0 1 2 3 x
-0.4

1.2
L3
1
0.8
0.6
0.4
0.2
0
x
-1 -0.2 0 1 2 3

* We further note that:

0 i j
li(xj) = ij =  = Kronecker delta.
1 i = j
* To summarize, for the data shown below,
xk x0 x1 … xi … xn
yk 0 0 0 1 0 0
we can write such an nth order polynomial as
x - xj
li(x) =  x - x , i=0, 1, 2, ..., n.
j≠i i j
which is called Lagrangian interpolating polynomial.

Notes organized by Prof. Renwei Mei, University of Florida 5

4.2.2 Lagrangian interpolation and its properties
 To obtain the 3rd order polynomial, we simply superimpose:
3
P3(x) = l0(x) y0 +l1(x) y1 +l2(x) y2+ l3(x) y3 =  li(x) yi.
i =0
* By direct substitution, we can easily verify that
P3(x0) = y0, P3(x1) = y1, P3(x2) = y2, P3(x3) = y3.
 In general, for (y0(x0), y1(x1), …, yn(xn)), we can construct
n
Pn(x) =  li(x) yi = nth order polynomial fit
i =0
 Note:
i. There is no need to solve ill-conditioned system.
ii. (x0, x1, x2, x3,...) can be unevenly spaced and out of order.
iii. The polynomial is unique. Why?
Let qn(x) be another polynomial of degree n with
qn(xi) = yi = Pn(xi)
Then r(x) = Pn(x) - qn(x) is also a polynomial of degree n.
A polynomial of degree n has at most n roots!
However, r(xi) = 0 for i=0, 1, 2, ..., n  it has (n+1) roots.
 the only possibility to have (n+1) roots is r(x)  0.
 qn(x)  Pn(x) & you will never find 2 different polynomials.
iv) Error is exactly zero at x = xi, i.e., E(xi) = 0 (i=1, 2,...n).
v) If f(x) is a polynomial of order lower than (n+1),
then Pn(x) is exactly equal to f(x) since f(n+1)()=0
(see below for more detailed error analysis).
Notes organized by Prof. Renwei Mei, University of Florida 6
4.2.3 Error in polynomial fitting
 Error of Pn(x) in fitting f(x) using the data at (x0, x1, x2,..., xn) is
E(x) = f(x) - Pn(x)
It can be shown that
1 (n+1)
E(x) = (x-x0)(x-x1)... (x-xn) (n+1)! f () (*)

where  lies among (x0, x1, x2, ..., xn).

 Derivation of error formula:
* Let (x) = (x-x0)(x-x1)...(x-xn) = polynomial of (n+1)th degree,
( Note: (xi) = 0 for i=0, 1, ..., n;
(n+1)
  (x) = (n+1)! )
(x)
Let G( x) = E(x) - E(t). (E(x) is an UNKNOWN function!)
(t)
(xi)
* Note: G(xi) = E(xi) - E(t) = 0 (i=0, 1, ..., n)
(t)

because E(xi) = 0 & (xi) = 0 for i=0, 1, ..., n.

(t)
* Furthermore, G(x=t)= E(t) - E(t) = 0.
(t)

Thus, G( x) has (n+2) roots 

G'(x) has (n+1) roots, ...
(j)
& G (x) has (n+2-j) roots for j=0,1, … n+1
(n+1)
G (x) has one root, say x=,
(n+1)
so that G () = 0.

Notes organized by Prof. Renwei Mei, University of Florida 7

 (n+1)
* Note: P (x) = 0 (since P(x) is only nth order)
(n+1) (n+1)
 E (x) = f (x).
(n+1)
 (x) = (n+1)! 200
0
(n+1) (n+1) (n+1)! -200 0 1 2 3 4 5 6 7 8
G (x) = f (x) - E(t).
(t) -400  (x )
(n+1)
At x=, G () = 0 -600 n=8
-800
(t) (n+1)
 E(t) = (n+1)! f ()

(x) (n+1)
or E(x) = (n+1)! f () (=Eq. (*))

◊ Example: Given the following data, find y = f(x=3) using

i) P2(x); and ii) P1(x).
(data generated using y=f(x) = x3-5; f(3)=22)
n0 1 2
1
xn 2 4
-4
yn 3 59
(x-x1) (x-x2) (x-x0) (x-x2)
* Sol. i. P2(x) = (x -x ) (x -x ) y0 + (x -x ) (x -x ) y1
0 1 0 2 1 0 1 2

(x-x0) (x-x1)
+ ( x -x ) (x -x ) y2
2 0 2 1

(x-2)(x-4) (x-1)(x-4) (x-1)(x-2)

= (-1)(-3) (-4) + (1)(-2) 3 + 3*2 59

4 59
 P2(x=3) = 3 + 3 + 3 = 24.

Notes organized by Prof. Renwei Mei, University of Florida 8

 ii. If we use a linear interpolation between x1 and x2,

P1(x) = x − x2 y1 + x − x1 y2
( x1 − x2 ) ( x2 − x1)
x-4 x-2
= -2 3 + 2 59

1 1
P1(x=3) = 2 * 3 + 2 * 59 = 31.

iii. Comparison:
f(3) = 22 (exact)
P1(x=3) = 31 (error =22-31= -9; it is large)
P2(x=3) = 24 (error =22-24= -2; closer to f(3) =22)
* Comparison of fits with y = f(x):
100 f(x)
P2 fit
80

60 P1(x)

40

20

0
-2 -1 0 1 2 x 3 4 5
-20

◊ Further comments:
• Both P1(x) and P2(x) work fine inside the interval (1, 4).
• Both are bad outside the interval
• Error(x) oscillates between node points.
Notes organized by Prof. Renwei Mei, University of Florida 9
 * Error in the fit: Error2(x) = f(x) – P2(x)

4
3 Error2

2
1
0 x
-1 0 1 2 3 4
-2
-3
-4

* Outside the interval, consider the third order base functions

l2(x) and l3(x) shown below:

0
The contribution from
-1 0 1 2 3 4 x x2=2 has too much
-2
L2 influence at x=4.
-4

-6

4 The contribution from

3.5
L3
3 x3=3 has too much
2.5
2
1.5
influence at x=4
1 (outside the interval).
0.5
0
-1 -0.5 0 1 2 3 4 x

* Thus extrapolation outside the interval can be dangerous!

Notes organized by Prof. Renwei Mei, University of Florida 10
◊ Application of error analysis in interpolation
* Truncation error in linear interpolation between (x0, x1)
Let h = x1-x0.
x-x1 x-x0
E1(x) = f(x) - P1(x) = f(x) - x -x f0 - x -x f1
0 1 1 0

1
= 2 (x-x0) (x-x1) f"().

for some  in [x0, x1].

Let g(x) = (x-x0)(x-x1) (Notice that g(x) = 0 at x = x0 & x1)
h h h2
 max |g(x)| = 2 2 = 4 for x in [x0, x1].

h2 Sup
Thus, |E1(x)| ≤ 8 [x0‚x1] |f"()|.

* This is useful in estimating the dependence of the error

in linear interpolation as a function of h when one

has a general idea about the magnitude of |f"(x)|.

* Effect of round-off error (R) in interpolation:
o Note: the values of f(x) at x0 and x1, f0 & f1, are only
approximate due to rounding,
f0T = f0A + 0, f1T = f1A + 1;  = max(|0|, |1|)
x-x1 x-x0 x-x1 x-x0
 R = (x - x ) 0 + (x - x ) 1 ≤ [(x - x ) + (x - x ) ]  = .
0 1 1 0 0 1 1 0

* Total interpolation error = E1 + R

h2 Sup
≤ 8 [x0‚x1] |f"()| +  .

Notes organized by Prof. Renwei Mei, University of Florida 11

 Example: If we interpolate f(x) = x in x = [10, 1000],
-1
it is seen that f"(x) = 4x3/2 .

1
Thus, max(|f''(x))| = 4*103/2 for x in [10, 1000].

1 1
For h=1, E1≤ 8 4*103/2 = 9.88 E-4 using linear interpolation.

Notes organized by Prof. Renwei Mei, University of Florida 12

4.3 Newton Divided Difference
4.3.1 Motivation and definitions
◊ Lagrangian interpolation is difficult to use when an additional point,
(xn+1, fn+1), is to be added.
◊ The discrete data are given as follows.
x x0 x1 x2 ... xn xn+1
f f0 f1 f2 ... fn fn+1
◊ Want to fit the data using Pn(x) in the form of
Pn(x) = a0 +a1(x-x0) +a2(x-x0)(x-x1) +... +an(x-x0) (x-x1)...(x-xn-1)
so that when we want Pn+1(x), we simply add another term to it,
Pn+1(x) = Pn(x) + an+1(x-x0)(x-x1)...(x- xn).
◊ Define
first divided difference between x0 and x1:
f1-f0 [1]
f[x0, x1] = x -x = f0
1 0

first divided difference between xr and xs:

fs-fr [1]
f[xr, xs] = x -x = fr
s r

second order divided difference:

f[x1‚ x2] - f[x0‚ x1] [2]
f[x0, x1, x2] = x2 - x0 = f 0

high order divided difference:

f[x1‚ x2‚…xn]- f[x0 x2‚…xn-1] [n]
f[x0, x1, x2,..., xn] = xn - x0 = f 0

Notes organized by Prof. Renwei Mei, University of Florida 13

4.3.2 Construction of interpolating polynomials
◊ Back to fitting by Pn(x)
At x= x0, Pn(x0) = a0 = f0  a0 = f0
f1-f0 [1]
At x= x1, Pn(x1) = a0 + a1(x1- x0) = f1  a1 = x -x = f0
1 0

At x= x2, Pn(x2) = a0 + a1(x2- x0) + a2(x2- x0)(x2- x1) = f2

f1-f0
f2-f0 - (x2- x0) x - x
1 0
 a2 = (x2- x0)(x2- x1)
f2-f1 f1-f0 x2- x0 f1-f0
x2- x1 x2- x1 - x2- x1 x1- x0
+
= x2- x0
f1-f0 x2- x0 f1-f0
Since x2- x1 - x2- x1 x1- x0 =
f1-f0 x1- x0 x2- x0 f1-f0
x1- x0 ( x2- x1 - x2- x1 ) = - x1- x0
f2-f1 f1-f0
-
x2- x1 x1- x0 [2]
 a2 = x2 - x0 = f 0

…
[n]
Similarly, an = f0 .
[1] [2]
Hence, Pn(x) = f0 + (x-x0) f0 + (x-x0)(x-x1) f0 +...
[n]
+ (x-x0) (x-x1)...(x-xn-1) f0 .
◊ Error: same as in Lagrangian interpolation (uniqueness)
◊ Note: (x0, x1, x2, ..., xn) can be in any order in the table.

Notes organized by Prof. Renwei Mei, University of Florida 14

4.3.3 Advantage over Lagrangian interpolation:
i. It is easy to improve from Pk(x) to Pk+1(x) by just adding
[k+1]
f0 (x-x0)… (x-xk) to Pk(x).
In Lagrangian interpolation, we have to start all over for
x - xj
li(x) =  , i=1, ...k, k+1
j≠i xi - xj
[k+1]
ii. f0 is easy to construct (see the table below for example).
iii. It is much easier to compute Pk(x) at different x's.
iv. Lagrangian interpolation is valuable theoretical tool.

* Lagrange basis polynomials are used in numerical integration

to derive the Newton-Cotes formulas.

* Lagrange interpolation is often used in digital signal

processing of audio for the implementation of fractional
delay FIR filters (e.g., to precisely tune digital waveguides
in physical modeling synthesis).

v. Divided difference is a more practical computational tool.

Notes organized by Prof. Renwei Mei, University of Florida 15

 ◊ Example: Given the data below,
Find: i) P2(3) & P1(3); ii) P2(1.5) & P1(1.5)

i xi fi [1] [2] [n]

fi fi The table for fi
0 1 -4 7 7
is easy to build
1 2 3 28
2 4 59 on spreadsheet

Solution: computing divided difference first:

[1] [1]
f0 = (3-(-4))/(2-1) = 7; f1 = (59-3)/(4-2) = 28
[2]
f0 = (28 - 7)/(4-1) = 7 (use of spreadsheet is ideal!)
[1] [2]
i) P2(x) = f0 + (x-x0) f0 + (x-x0)(x-x1) f0
P2(x) = -4 + 7(x-1) + 7(x-1)(x-2)
(simpler expression comparing with Lagrangian:
(x-2)(x-4) (x-1)(x-4) (x-1)(x-2)
P2(x) = (-1)(-3) (-4) + (1)(-2) 3 + 3*2 59 )

P2(x=3)= -4 +7(3-1) + 7(3-1)(3-2) = 24.

* For linear interpolation, only two points are needed.
In order to avoid extrapolation, we use x1 & x2,
[1]
P1(x=3) = f1 + f1 (x-x1) = 3 + 28(x-2) =31
ii) At x=1.5, we first compute P1(x=1.5)
P1(x=1.5) = -4+7 (1.5-1) = -0.5
P2(x=1.5)= P1(x=1.5)+ 7(1.5-1)(1.5-2)= - 0.5-1.75= -2.25
Improvement of P2 over P1 is easily accomplished!

Notes organized by Prof. Renwei Mei, University of Florida 16

4.4 Finite Difference
4.4.1 Forward difference and divided difference
◊ If x1-x0=x2-x1=x3-x2 =... = xn-xn-1 = h ---- equidistance !
we can use "forward difference", f, without division:
f0 = f1-f0 & fi = fi+1- fi

y = f(x)

h h h h

x
x0 x1 x2 xn-1 xn

[1] f1-f0 f1-f0 [1]

◊ Note: f0 = x1-x0 = h  f1-f0 = h f0

[2] f[x1‚x2] - f[x0‚ x1] f1/h-f0/h

f0 = x2-x0 = 2h
f1-f0 f2-2f1+f0
= 2 = 2
2h 2h
[2] [2] [2] [2]
[3]
f1 -f0 f1 -f0
f0 = x -x = 3h
3 0

1 1
= 3h 2 [ (f3-2f2+f1) - (f2-2f1+f0)]
2h
1
= 3!h3 (f3-3f2+3f1-f0)

Notes organized by Prof. Renwei Mei, University of Florida 17

 [1]
◊ Denote: f0 = f1-f0 = h f0 , ...
[1]
fi = fi+1 - fi = hfi ,

◊ Note, the 2nd order forward difference is the difference of the

1st order difference fi:
[2]
2f0  f1-f0 = f2-2f1-f0 = 2h2f0 ...

[2]
2fi  fi+1 -fi = fi+2 - 2fi+1 + fi = 2h2fi .

Similarly, the relationship between the divided difference and

forward difference for higher order can be established:
[3]
3f0  2f1 - 2f0 =...= f3 - 3f2 + 3f1 - f0 = 3!h3f0

...
k k-1 k-1
 fi =  fi+1 -  fi =...
k(k-1)
= fi+k - kfi+k-1 + 2 fi+k-2 +...

[k]
= k! hk fi

[k] kfi
OR fi = k!hk

Notes organized by Prof. Renwei Mei, University of Florida 18

4.4.2 Interpolation using forward difference
x-x0
◊ Define s = h  x – xk = (x - x0) + (x0 – xk) = sh -kh = (s-k)h
[1] [2]
Hence Pn(x) = f0 + (x-x0) f0 + (x-x0)(x-x1) f0 +...
[n]
+ (x-x0) (x-x1)...(x-xn-1) f0 .
s(s-1) 2 s(s-1)...(s-n+1) n
= f0 + sf0 + 2!  f0 +...+ n!  f0

s(s-1)...(s-k+1) s
Note: k! = (k) (combinatoric notation)

n
s k
so that Pn(x) =  (k)  f0
k =1
= Newton-Gregory forward polynomial.

Notes organized by Prof. Renwei Mei, University of Florida 19

* Example: Consider the evenly spaced data (xi, fi) given below.

(f(x) = x3)
i xi fi f0 2
 f0
3
 f0
0 1 1 7 12 6
1 2 8 19 18
2 3 27 37
3 4 64

Find: P2(1.8), P3(1.8).

Solution:
2 3
* Note: h=1; values of f0,  f0,  f0 are listed on the table.
* For x=1.8, we use starting point x0=1 for both P2 & P3;
for P2, if we use x1=2 as starting point => extrapolation;
for P3, there is no other choice since only 4 data are given.
 s = x - x 0= x -1= 0.8

 P2(x)=1 + 7s + 12/2*s(s-1) = 1+ 7*0.8+6*0.8*(-0.2)=5.64

3
(error =1.8 – 5.64= 0.192)
[3] s 3
 P3(x) = P2(x) + (x-x0)(x-x1)(x-x2) f0 = P2(x) +(3)  f0

P3(x)=5.64+6/3! s(s-1)(s-2) = 5.64+ 0.192=5.832 (exact)

(note: P2(1.8) is used to compute P3(1.8) => efficiency!)

Notes organized by Prof. Renwei Mei, University of Florida 20

* Example: For the data f(xi, fi) given below with h=0.5,
i xi fi f0 2
 f0
3
 f0
0 1.5 3.375 4.625 3 0.75
1 2 8 7.625 3.75
2 2.5 15.625 11.375
3 3 27
Find: P2(1.8) and P3(1.8)
Solution:
2 3
* Values of f0,  f0,  f0 are listed on the table.
* For x=1.8 we use starting point x0=1.5 for both P2 & P3.

 s = (x-1.5)/h= 0.3/0.5 = 0.6

 P2(x) = 3.375 + 4.625s + 1.5s(s-1)
=3.375 + 4.625*0.6+1.5*0.6*(-0.4)= 5.79
(Error = 5.832-5.79 = 0.042; much smaller for h=0.5 than
0.192 for h=1.0)
s 3
 P3(x) = P2(x) +(3)  f0

= 5.79+ 0.125 s(s-1)(s-2),

= 5.79 +0.125*0.6(0.6-1)(0.6-2) = 5.832

Notes organized by Prof. Renwei Mei, University of Florida 21

  if i = k
 Consider e(xi) = 
0 if i  k
2 3
xi ei ei  ei  ei
... ... 0 ... ...
xk-3 0 0
0 0
xk-2 0 0
0 
xk-1 0 
 -3
xk  -2
- 3
xk+1 0 
0 -
xk+2 0 0
0 0
xk+3 0 0
n
* The maximum error in the column  ei increases with n as
1
2 n(n-1) .
 We can use this behavior to extract isolated error e(xk) from
original data and make correction.

Notes organized by Prof. Renwei Mei, University of Florida 22

 ~
 Example: Consider the discrete data fi on an equal interval which
contain an isolated error.
Solution: The kth order forward difference error is
k k ~ k k k k ~
 fi =  fi +  ei   ei =  fi -  fi
r k k~
Expect  fi to be small for smooth f(x) so that  ei ~ - fi .
~ ~ 2~ 3~
xi fi  fi  fi  fi
Error
guess
xk-3 0.10396
0.01700
xk-2 0.12096 -0.00014
0.01686 -0.00003 0
xk-1 0.13782 -0.00017
0.01669 -0.00002 0
xk 0.15451 -0.00019
0.01650 0.00006 
xk+1 0.17101 -0.00013
0.01637 -0.00025 −
xk+2 0.18738 -0.00038
0.01599 0.00021 
xk+3 0.20337 -0.00017
0.01582 -0.00010 
xk+4 0.21919 -0.00027
0.01555
xk+5 0.23474

As a first estimate, we try -3 ~ -0.00025   ~ 0.00008.

If we try 3~0.00021   ~ 0.00007.
If - ~-0.00010   ~ 0.0001.
 Various estimates for the order of magnitude of  are close.
Notes organized by Prof. Renwei Mei, University of Florida 23
 3 3~ 3
* Note:  fi will also contribute to  fi besides  ei.

3.E-04
3f
2.E-04

1.E-04

0.E+00

-1.E-04

-2.E-04

-3.E-04
-4 -3 -2 -1 0 1 2 i

3
It is reasonable to assume that  fi ~ -0.00002 based on the 1st
3~
two numbers (i=-3 & -2 in above chart) in  fi column.
Hence,   -0.00006 - 0.00002 = -0.00008
or -  0.00010 - 0.00002 = 0.00008,
or -3 ~ 0.00025 - 0.00002   ~ -0.000077,
or 3 ~ -0.00021 - 0.00002   ~ -0.000077.
Using  = -0.00008, we trace back the error to xk+2 and correct
the data as fk+2 = ~fk +2 +  = 0.18738 -0.00008 = 0.18730.
* Comment: errors of this magnitude (0.043%) are impossible to
spot on graphs.

Notes organized by Prof. Renwei Mei, University of Florida 24

* Results after correction
~ ~ 2~ 3~
xi fi  fi  fi  fi
xk-3 0.10396
0.01700
xk-2 0.12096 -0.00014
0.01686 -0.00003
xk-1 0.13782 -0.00017
0.01669 -0.00002
xk 0.15451 -0.00019
0.01650 -0.00002
xk+1 0.17101 -0.00021
0.01629 -0.00001
xk+2 0.18730 -0.00022
0.01607 -0.00003
xk+3 0.20337 -0.00025
0.01582 -0.00002
xk+4 0.21919 -0.00027
0.01555
xk+5 0.23474

3~
 fi column is smoother now!

Notes organized by Prof. Renwei Mei, University of Florida 25

4.6 Interpolation Error in General
◊ Truncation error
1
En(x) = f(x) - Pn(x) = (x-x0)(x-x1)... (x-xn) (n+1)!f(n+1)()

1
 |En(x)| ≤ max{| (x-x0)(x-x1)...(x-xn)|} (n+1)!Cn+1

where Cn+1 = Sup {|f(n+1)()|} is a constant.

(x0‚x1‚...,xn)

Q: How does En(x) depend on x?

Note Cn+1 /(n+1)! is independent on x.
Let n(x) = (x-x0)(x-x1)... (x-xn).
We consider equal interval case, h = x1-x0 = ... = xn - xn-1.
 n=1: 1(x) = (x-x0)(x-x1)
 max[1(x)] = h2/4 is reached when x-x0 = h/2 (middle).
 n=2: 2(x) = (x-x0)(x-x1)(x-x2)
0.5
2
0.4

0.3

0.2

0.1

0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
-0.1

-0.2

-0.3

-0.4

-0.5

 max[2(x)] = 0.385 h3 at x-x1 = ±h/ 3 .

Notes organized by Prof. Renwei Mei, University of Florida 26
 3 4 4
0.4
x0 x1 x2 x3 2
0
x x1 x3 x4
0
-0.4 x0 x2 x
-0.8 -2

-1.2 -4

6 5
100 6
x2 x4 x5
0 50
x0 x1 x3 x
x1 x3 x4 x5 x6
-6 0
x0 x2 x
-12 -50

-18 -100

 n=3: 3(x) = (x-x0)(x-x1)(x-x2)(x-x3)

 max[3(x)] = 0.5625h4 for x1 < x < x2 .
max[3(x)] = h4 for x < x1 or x> x2.
 Largest error occurs when x is near the far end of the interval.
 Choose x in the middle to reduce interpolation error.
 n=6: max[6(x)] = 12.36 h7 for x2<x<x4 (middle).
But max[6(x)] = 95.8 h7 for x0<x<x6 (whole).
◊ QUESTION: Does increasing n help to reduce the error En(x)?
Answer: it does in many cases.
In some cases, it does not—Runge’s phenomenon.
Notes organized by Prof. Renwei Mei, University of Florida 27
 * Consider the following data generated by
-5
f(x) = (1+x2) for -2< x ≤2.
x -2 -1.5 -1 -0.5 0 ... 2
f 0.00032 0.002758 0.03125 0.32768 1 ... 0.00032

Polynomials of nth degree (n=2, 4, 8, & 16) fitting the given data in
-2<x≤2 are shown below.
1.2
y y
1.0
1.0
P 2 (x) P 4 (x)
0.8 0.5

0.6 f(x)
0.0
0.4
f(x) -0.5
0.2

0.0 -1.0
-2 -1 0 1 2 -2 -1 0 1 2
x x
1.5 2
y y
1.0 0 f(x)
0.5 P (x)
8
f(x) -2
0.0 P16 (x)
-4
-0.5

-1.0 -6

-1.5 -8
-2 -1 0 1 2 -2 -1 0 1 2
x x
=> Runge’s phenomenon – error increases as n increases.
-1
* Same problem appears if f(x) = (1+x8) , cos10x, ..., in the same
interval.

Notes organized by Prof. Renwei Mei, University of Florida 28

 * If this function is interpolated at equidistant points xi between
−2 and 2 with a polynomial Pn(x) which has a degree , the
resulting interpolation oscillates toward the end of the interval.

* It can even be proven that the interpolation error tends toward

infinity when the degree of the polynomial increases:

 
lim  max | f ( x) − pn ( x) |  = 
n→ − 2 x  2 
* Runge’s phenomenon demonstrates that high degree
polynomials are generally unsuitable for interpolation.

* How to solve the problem of Runge's phenomenon

• The oscillation can be minimized by using Chebyshev
nodes

Notes organized by Prof. Renwei Mei, University of Florida 29

 instead of equidistant nodes. In this case the maximum
error is guaranteed to diminish with increasing polynomial
order.
• The problem can be avoided by using spline curves which
are piecewise polynomials.
4.7 Hermite Interpolation
4.7.1 Simple illustration using two points (x1, x2)
Given: y1 & y1' at x=x1 and y2 & y2' at x=x2.
Goal: find the interpolating curve that will match the slopes
(y1', y2') at the end of interval in addition to (y1, y2).

y2, y2′

y1, y1′

• P1: the starting value, y1, of the curve

• T1: the tangent, y1', (e.g. direction and speed) to tell us

how the curve enters the starting point into the interval
• P2: the ending point, y2, of the curve
• T2: the tangent y2' to tell us how the curve meets the
ending point
=> Similar to the development of Lagrangian polynomial li(x)

Notes organized by Prof. Renwei Mei, University of Florida 30

◊ Consider the following 4 functions: h1(x), h2(x), h3(x), & h4(x)

1 h1 1
slope=0 slope=0
y=1 y=1
h2
0.5 0.5

0 0
x
01 x12 x x01 x12 x

0
h3
0.2 -0.05 h4
slope=1,y=0
0.15 -0.1
0.1 -0.15
slope=1
0.05 -0.2 y=0
0 -0.25
x01 x12 x x01 x12 x

h1(x1)=1, h1(x2)=0; h1'(x1)=0, h1'(x2)=0.

h2(x1)=0, h2(x2)=1; h2'(x1)=0, h2'(x2)=0.
h3(x1)=0, h3(x2)=0; h3'(x1)=1, h3'(x2)=0.
h4(x1)=0, h4(x2)=0; h4'(x1)=0, h4'(x2)=1.
Thus, to obtain the desired fit,
we can simply superimpose: y2, y2′
desired fit
“ y1 * h1 + y2 * h2
+ y1' * h3 + y2' * h4 ”
y1, y1′
◊ What about n points?

Notes organized by Prof. Renwei Mei, University of Florida 31

4.7.2 Hermite interpolation using n points
◊ Now given (y1, y2, ..., yn) & (y1', y2', ..., yn') at (x1, x2, ..., xn).
We want to find a polynomial P(x) so that
P(xi) = yi
and P'(xi) = yi' for i = 1, 2, ..., n.
◊ Given 2n conditions  P(x) is a polynomial of degree (2n-1).
◊ Consider Hn(x) be expressed in the form of
n n ~
Hn(x) =  yi hi(x) +  yi'hi(x).
i =1 i =1
~
◊ How to decide hi(x) & hi(x)? We must require:
~
• hi(x) and hi(x) be polynomials of degree (2n-1);
~
• hi(xk) = ik, hi(xk) = 0 so that Hn(xk) = yk;
~'
• hi'(xk) = 0, hi (xk) = ik so that Hn'(xk) = yk'.
Recall the Lagrangian polynomial of degree n
x-xj
li(x) =  xi-xj , i=1, 2, ...n.
j≠i

~ 2
= Choose hi(x) = c l i (x)(x-xi) (2n-1 degree)

~
so that hi(xk) = 0 (because li(xk) = ik),
~'
hi (xk) = cl i (xk) + c(xk - xi) 2 li(xk) li'(xk)
2
&

= c ik + 0.

Notes organized by Prof. Renwei Mei, University of Florida 32

 ~'
* In order for hi (xk) = ik we simply set c = 1.

* Now choose hi(x) = l i (x)(ax+b)

2

need: hi(xk) = ik = axk + b = 1

hi'(xk) = 0 = al i (xk)+2li(xk)li'(xk)(axk+b)=0
2
&

For i≠k, since li(xk) = ik=0, al i (xk)+2li(xk)li'(xk)(axk+b)=0

2

is satisfied automatically

For i=k, li(xk)=1, we need a + 2 li'(xi) * 1 = 0.

= a = - 2 li'(xi)

& b = 1 + 2 xi li'(xi)

= a x + b = 1 - 2(x- xi) li'(xi).

◊ To summarize,
n n
- 2(x- xi) li'(xi)] + 
2 2
Hn(x) =  yi l i (x) [1 yi' l i (x) (x- xi).
i=1 i=1
◊ Since we derived Hn(x), the existence for Hn(x) is proven.
◊ Uniqueness of Hn(x):
Suppose there exist another polynomial Gn(x) of degree (2n-1).
Then R(x) = Hn(x) - Gn(x) = polynomial of degree (2n-1).
Since R(xi) = R'(xi) = 0 for i = 1, 2, ..., n,
2
= R(x) = A(x- x1)2(x- x2)2...(x- xn)2 = An(x)

= R(x) has 2n roots!

Notes organized by Prof. Renwei Mei, University of Florida 33

 = R(x) = 0 = Gn(x) = Hn(x)

◊ Example:
Find the desired fit for n=2

desired fit

Solution:
i) Let x1 = a, x2 = b. Then
2 2
H2(x) = 1 + 2 x − a   b − x  f (a) + 1 + 2 b − x   x − a  f (b)
   
b − a  b − a    
b − a b − a 

2
+ ( x − a)(b − x) f ' (a) + ( x − a) (b − x) f ' (b)
2
2 2
(b − a) (b − a)

ii) For a=0, b=1, f(a)=2, f(b)=3, f'(a)=-2, f'(b)=-3,

H2(x) = 2(1+2x)(1-x)2+3[1+2(1-x)]x2-2x(1-x)2 – 3 x2(1- x)
H2(x) in the interval [0, 1] is shown below.

Notes organized by Prof. Renwei Mei, University of Florida 34

 3.4
H2(x )
3.2
3
2.8
2.6 fit
2.4
2.2
2
1.8
0 0.2 0.4 0.6 0.8 1 x

Notes organized by Prof. Renwei Mei, University of Florida 35

4.7.3 More computable form of Hermite interpolation
* Lagrangian interpolation is not easy to compute compare with
Newton divided difference.
* Hermite interpolation in the form of
n n
- 2(x- xi) li'(xi)] +  yi' l i (x) (x- xi).
2 2
Hn(x) =  yi l i (x) [1
i=1 i=1
is not easy to compute practically.
* Can we convert it to a form similar to divided difference?
 Consider 2n nodes: z1, z2, ..., z2n
Newton divided difference form gives :
P2n-1(x) = f(z1) + (x- z1) f [z1, z2] +(x- z1) (x- z2) f [z1, z2, z3]
+...+ (x- z1)... (x- z2n-1) f [z1, z2, ..., z2n]
with E(x) = f(x) - P2n-1(x)
= (x- z1)... (x- z2n) f [z1, z2, ..., z2n, x]
 Now set z1 = z2 = x1, z3 = z4 = x2, ...
z2n-1 = z2n = xn,
lim f(x1)-f()
Note: f [x1, x1] =
→x1 x1- =
f ' ( x1 ) is well defined/given.

 Thus P2n-1(x) = f(x1) + (x- x1) f ' ( x1 ) + (x- x1)2 f[x1, x1, x2]
+ (x- x1)2 (x- x2) f[x1, x1, x2, x2]
+ (x- x1)2 (x- x2)2 f[x1, x1, x2, x2, x3]+ …
+ (x- x1)2...(x- xn-1)2 (x- xn) f[x1, x1, x2, x2, ..., xn, xn]

Notes organized by Prof. Renwei Mei, University of Florida 36

 * In the above f[x1, x1, x2] can be computed as
1 f(x2) - f(x1) lim f(x1)-f()
f[x1, x1, x2] = x - x [ x2- x1 - →x1 x1- ]
2 1

1
= x - x { f[x1, x2] - f ' ( x1 ) }
2 1

1
f[x1, x1, x2, x2] = x - x { f[x1, x2, x2] - f[x1, x1, x2]}
2 1

1
where f[x1, x2, x2] = x - x { f ' ( x2 ) - f[x1, x2]}
2 1

…
 Example:
Given: a=0, b=1, f(a)=2, f(b)=3, f'(a)=-2, f'(b)=-3,
use finite difference to find H2(x).
Solution:
x f f' f[x1,x2] f[x1,x1,x2] f[x1,x2,x2] f[x1,x1,x2,x2]
0 2 -2 1 3 -4 -7
1 3 -3

In order to evaluate H2(x), various divided differences are needed.

f[x1, x2] = (3-2)/(1-0) = 1; f[x1, x1, x2] = (1-(-2))/(1-0)= 3
f[x1, x2, x2] = (-3-1)/(1-0)= -4; f[x1, x1, x2, x2] = (-4-3)/(1-0)=-7.
Their values are listed in the table above. Thus,
H2(x) = 2 -2x + 3 x2 -7 x2(x-1)
This is much simpler than the 4-term expression
H2(x) = 2(1+2x)(1-x)2+3[1+2(1-x)]x2-2x(1-x)2-3x2(1- x)

Notes organized by Prof. Renwei Mei, University of Florida 37

  Error in Hermite interpolation:
E(x) = (x-x1)2(x-x2)2...(x-xn-1)2(x-xn)2 f[x1, x1, x2, x2, ..., xn, xn, x]
1
= (x-x1)2(x-x2)2...(x-xn-1)2(x-xn)2 (2n)! f (2n) ( ) .

2 1
= n(x) (2n)! f (2n) ( ) .

◊ Comparison between cubic H- and cubic L-interpolations

 Hermite H2(x): (x1, x2), (y1, y2), and (y1', y2') are given
1 1
Error: E(x) = 4! f(4)() (x-x1)2(x-x2)2 = 4! f(4)() 2(x)
2

2 2 1 1
Max 2(x) = 2(x=(x1+ x2)/2) = 16 (x2-x1)4 = 16 h4.

1 1
=> E(x) ≤ max|f(4)()| 4! 16 h4.

0.8
0.6
0.4
0.2 ψ 22
0
-0.2 0 0.5 1 1.5 2 2.5 3 x
-0.4
-0.6
-0.8 4
-1
-1.2

Notes organized by Prof. Renwei Mei, University of Florida 38

  Lagrangian L3(x): (x0, x1, x2, x3) and (y0, y1, y2, y3) are given
1 1
Error: E(x)= 4! f(4)()(x-x0)(x-x1)(x-x2)(x-x3) = 4! f(4)()4(x)

Max |4(x)| = |4[x=( x1+ x2)/2) ±1.118h]|

= h2 for x0≤ x ≤ x3;
9
Max |4(x)| = 16 h2 for x1≤ x ≤ x2.

1
=> E(x) ≤ max|f(4)()| 4! h4 for x0≤ x ≤ x3;

1 9
& E(x) ≤ max| f(4)()| 4! 16 h4 for x1≤ x ≤ x2.

 This is at least 9 times larger than the error in H2(x)!

Notes organized by Prof. Renwei Mei, University of Florida 39

4.8 Cubic Spline
* Applications: airplane, automobile, ship building,…

* Traditional practice: use drafting spline to draw smooth curves.

Zero curvatures
at end points

Notes organized by Prof. Renwei Mei, University of Florida 40

4.8.1 Fitting with a cubic polynomial
y gi(x) yi+1
yi
gi+1(x)
gi-1(x) yi+2
yi-1
left mid right
interval interval interval

xi-1 xi xi+1 xi+2 x

* Fit y=f(x) in (xi, xi+1) using a cubic polynomial gi(x) as

gi(x) = di + ci (x- xi) + bi (x- xi)2 + ai (x- xi)3 (1)
Our task: determine FOUR coefficients (ai, bi, ci, di).
* Two obvious constraints are:
gi(xi) = yi & gi(xi+1) = yi+1 (2)
 di = yi (3)
2 3
& cihi + bi hi + ai hi = yi+1 - yi (4)
with hi = xi+1- xi. (5)
 Need two more conditions.
* These two extra conditions will be obtained by imposing
smoothness condition at the joint xi between the left interval
and the mid interval.

Notes organized by Prof. Renwei Mei, University of Florida 41

4.8.2 Smoothness conditions at joints
* We pose "smoothness conditions" between adjacent intervals
y gi-1"(xi)= gi "(xi) gi"(xi+1)= gi+1"(xi+1)
gi-1′(xi)= gi′(xi) gi′(xi+1)= gi+1′(xi+1)

gi-1(x) gi(x) gi+1(x) yi+2

yi-1
left mid right

xi-1 xi xi+1 xi+2 x

i.e. the 1st /2nd derivatives at xi evaluated from the LEFT interval
= that at xi from the MID interval.
 gi-1'(xi) = gi'(xi)
 at x = xi (6a)
 gi-1"(xi) = gi"(xi)
 gi'(xi+1) = gi+1'(xi+1)
 at x = xi+1 (6b)
 gi"(xi+1) = gi+1"(xi+1)
* Differentiate gi(x) to get the 1st order derivative
• gi'(x) = ci + 2bi (x-xi) + 3ai (x-xi)2
x=xi  gi'(xi) = ci, (7)
x=xi+1  gi'(xi+1) = ci + 2bi hi + 3ai hi2, (8)
* If we move back by one interval to (xi-1, xi), we have
gi-1'(x) = ci-1 + 2bi-1 (x-xi-1) + 3ai-1 (x-xi-1)2
2
x=xi  gi-1'(xi) = ci-1+ 2bi-1hi-1 + 3ai-1 hi-1 (9)

* For the 2nd order derivative,

• gi"(x) = 2 bi + 6 ai (x-xi)

Notes organized by Prof. Renwei Mei, University of Florida 42

 * Let Si  gi"(xi) = 2bi, (10a)
Si+1  gi"(xi+1) = 2bi + 6ai hi (10b)
Express (ai, bi) from Eq. (10) in terms of (Si, Si+1) 
bi = Si /2, ai = (Si+1 - Si)/(6hi) (11)
Solve for ci using (ai, bi, di) and Si
yi+1-yi 2hiSi + hiSi+1
 ci = hi - 6 (12)

* This gives yi'(xi) = ci (see (7)) based on right interval (xi, xi+1).
* What about yi'(xi) from the left interval (xi-1, xi)? We have
2
yi'(xi) = gi-1'(xi) = ci-1 + 2bi-1 hi-1 + 3ai-1 hi-1 (13)

* Repeat the same procedure from Eqns. (10-12),

obtain (ai-1, bi-1, ci-1) based on (Si-1, Si, yi-1, yi),
Si - Si-1
 bi-1 = Si-1/2, ai-1 = 6hi-1 (14)

yi - yi-1 2Si-1+Si
 ci-1 = hi-1 - 6 hi-1 (15)

(9)
yi - yi-1 2Si-1+Si Si-Si-1
 gi-1'(xi) = h - 6 h i-1 + hi-1Si-1+
2 hi-1
i-1

yi - yi-1 1
= hi-1 + 6 hi-1(Si-1 + 2Si) (16)

* Smoothness at xi requires gi-1'(xi) = gi'(xi) 

yi+1-yi yi - yi-1
hi-1Si-1+2(hi-1+hi)Si+hiSi+1 = 6( h - h )
i i-1

= 6(f[xi+1, xi] - f[xi, xi-1]) (17)

Notes organized by Prof. Renwei Mei, University of Florida 43

 * Simplification: hi = constant (as in many applications);
... ...   S1   ... 
1 4 1  S  y − 2y + y 
  2   i +1 i i −1 
 1 4 1   S3  6  ... 
(17)     S = 2  
 1 4 1  4  h  ... 
 1 4 1   S5   ... 
    
 1 4 1   ...   ... 
 ... ...  Sn −1   ... 

* Each equation for Si depends on Si-1 & Si+1.

* Hence a set of coupled equations for yi" = Si must be solved.
Total number of unknowns: (S0, S1, S2,..., Sn)  n+1
Total number of equations for (17) = n-1
since there are (n-1) internal nodes: (x1, x2,..., xn-1).
 Need TWO additional conditions to close the system.
4.8.3 End conditions
 There several possibilities for providing TWO end conditions.
i. If we know the values of S0 & Sn, use the given values.
ii. Take S0 = Sn = 0  ends of curve (at x0 & xn) are straight
→ natural spline;
then solve (n-1) Eqns. for (S1, S2,..., Sn-1).
iii. Take S0 = S1 & Sn = Sn-1  a1 = an = 0
(i.e. 1st & last intervals are fit by parabolas so that y"'=0)

Notes organized by Prof. Renwei Mei, University of Florida 44

 Set S0 = S1 & Sn = Sn-1 from (17), the 1st & last Eqns. 
(3h0+2h1) S1 + h1S2 = 6(f[x2, x1] - f[x1, x0]) (18)
hn-2Sn-2+(2hn-2+3hn-1)Sn-1 = 6 (f[xn, xn-1]- f[xn-1, xn-2]) (19)
The number of unknowns is (n-1) now.
iv. “Not-a-knot” (extrapolation) condition
* Assume that y"' is continuous at x1, it implies that
y"'(x1) = g0"'(x1) = 6a0 =g1"'(x1) = 6a1
 6 a0 = 6 a1  (S1 - S0)/h0 = (S2 - S1)/h1
Express S0 in terms of S1 & S2 
S0 = (h0/h1+1) S1 - h0/h1 S2 (20)
(This is a linear extrapolation for S0 given S1 & S2)
* Similarly, at the other end,
6an-2 = 6an-1  (Sn-1-Sn-2)/hn-2 = (Sn-Sn-1)/hn-1
 Sn = (1+hn-1/hn-2) Sn-1 - hn-1/hn-2 Sn-2 (21)
* Now the first equation for S1 becomes
2 2
(h0+ h1)(h0+2h1) h1 - h0 [1] [1]
h1 S 1+
h1 S 2 = 6(f1 -f0 ) (22)

Similarly, the last equation becomes

2 2
hn-2-hn-1 (hn-1+hn-2)(hn-1+2hn-2) [1] [1]
hn-2 S n-2 +
hn-2 S n-1 = 6(fn-1-fn-2) (23)

 The resulting equations for Si form an (n-1)x(n-1) tri-diagonal

system which can be solved using Thomas algorithm
(i.e. tri-diagonal solver).

Notes organized by Prof. Renwei Mei, University of Florida 45

◊ Example: Reconsider the data generated by
f(xi)=1/(1+xi2)5 for -2≤ xi ≤2 with  xi = hi = 0.5.
Sol.: We consider three types of the "end conditions" (type ii-iv).
x Si --(ii) Si --(iii) Si --(iv)
-2 0 0.044086 -0.180246
-1.5 0.055903 0.044086 0.104217
-1 0.401686 0.404868 0.388679
-0.5 4.767861 4.766953 4.771578
0 -10.45177 -10.45132 -10.45363
0.5 4.767861 4.766953 4.771578
1 0.401686 0.404868 0.388679
1.5 0.055903 0.044086 0.104217
2 0 0.044086 -0.180246
6
S
4

-2
End
-4 condition
-6 (ii)
-8 (iii)
(iv)
-10
-12
-3 -2 -1 0 1 2 x

* Although Si’s near the ends of the interval are a little different from
using 3 different conditions, the coefficients (ai, bi, ci, di) differ very
little. Hence the curves for y(x) should be nearly the same.

Notes organized by Prof. Renwei Mei, University of Florida 46

* The graph below shows how closely the cubic splines (using three
end condition) agree with the exact curve.

1.2
End
1 conditions
0.8 f-ii
f-iii
0.6 f-iv
exact
0.4

0.2
0

-0.2
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 x

More detailed comparison can be seen from the table given below.
x exact ii iii iv
-2 0.00032 0.00032 0.00032 0.00032
-1.9 0.00048 0.00036 -7.4E-05 0.002137
-1.8 0.00073 0.000513 -2.7E-05 0.002720
-1.7 0.001123 0.000888 0.00046 0.002639
-1.6 0.001749 0.001600 0.001389 0.002462
-1.5 0.002758 0.002758 0.002758 0.002758
-1.4 0.004401 0.004572 0.004688 0.004096
-1.3 0.007100 0.007637 0.007781 0.007046
-1.2 0.011562 0.012644 0.012758 0.012175
-1.1 0.018969 0.020284 0.020340 0.020054
-1 0.031250 0.031250 0.031250 0.031250
-0.9 0.051476 0.047573 0.047542 0.047699
-0.8 0.084291 0.076645 0.076607 0.076801
-0.7 0.136166 0.127199 0.127169 0.127321
-0.6 0.214934 0.207966 0.207952 0.208026
-0.5 0.327680 0.327680 0.327680 0.327680
-0.4 0.476113 0.488544 0.488551 0.488514
-0.3 0.649931 0.666647 0.666655 0.666614
-0.2 0.821927 0.831550 0.831556 0.831528
-0.1 0.951466 0.952814 0.952816 0.952807

Notes organized by Prof. Renwei Mei, University of Florida 47

4.9 Least Square Method
4.9.1 Motivation and objective
* Consider the following data from an experiment measurement.
The data clearly shows a trend but also has random noise in it.
* It is impractical or pointless to use a Lagrangian interpolation
because of the "high frequency" random noise in the data.
30
y

20

10

0
0 1 2 x 3 4 5

* We observe, on average, the data may be represented by:

P2(x) = a0 + a1x + a2x2 (1)
* Obviously, there are many more data available than needed to
just determine these 3 coefficients.
* How to "best" choose the coefficients (a0, a1, a2)?
* Define "best" coefficients (a0, a1, a2) in the sense that the mean
square error E between the data and the fit P2(x) is smallest
among all possible choices of (a0, a1, a2).

Notes organized by Prof. Renwei Mei, University of Florida 48

4.9.2 Minimization of mean square error
◊ Consider the total values of the error squared,
M
E = Error =  [ yi - (a0 + a1xi + a2xi2)]2 (2)
i =1

where M = number of data points.

* Taking partial derivatives with respect to ai:
E M
=0   (-2) [yi - (a0 + a1xi + a2xi2)] = 0
a0 i =1

E M
=0   (-2xi) [yi - (a0 + a1xi + a2xi2)] = 0
a1 i =1

E M
=0   (-2xi2)[yi - (a0 + a1xi + a2xi2)] = 0
a2 i =1
M M M
 a0M + a1  xi + a2  xi2 =  yi
i =1 i =1 i =1
M M M M
a0  xi + a1  xi2 + a2  xi3 =  xiyi (3)
i =1 i =1 i =1 i =1
M M M M
a0  xi2 + a1  xi3 + a2  xi4 =  xi2yi
i =1 i =1 i =1 i =1

* Solve the above to get "best" coefficients (a0, a1, a2).

◊ Fitting with an nth order polynomial using a =(a0, a1, a2,..., an)T.
Let Pn(x) = a0 + a1x + a2x2 + ... + anxn, (4)
a system of equations for a similar to (3) can be obtained:
[X] a = b.
* Major challenge: [X] can be very ill-conditioned.

Notes organized by Prof. Renwei Mei, University of Florida 49

◊ Example: Given the data below, find the least square fit
using P1(x) and P2(x).
xi 0.05 0.11 0.15 0.31 0.46 0.52 0.70 0.74 0.82 0.98 1.17
yi 0.956 0.89 0.832 0.717 0.571 0.539 0.378 0.370 0.306 0.242 0.104
Solution:
* M = 11,
M M M M
*  xi = 6.01,  xi2 = 4.6545,  xi3 = 4.115,  xi4 = 3.9161,
i =1 i =1 i =1 i =1
M M M
 yi = 5.905,  xi yi = 2.1839,  xi2 yi = 1.3357.
i =1 i =1 i =1

* For P1(x), we solve

M M
a0M + a1  xi =  yi
i =1 i =1
M M M
a0  xi + a1  xi2 =  xi yi
i =1 i =1 i =1

 11 6.01  a0   5.905 

 6.01 4.6545  a  = 2.1839
  1   

 a0 = 0.9523, a1 = -0.7604
1 M
 E=  [ yi - (0.9523-0.7604 xi)]2 = 0.00102
M − 1 i =1

* For P2(x), we solve

 11 6.01 4.6545 a0   5.905 
 6.01 4.6545 4.115   a  = 2.1839
  1   
4.6545 4.115 3.9161 a2  1.3357

 a0 = 0.9980, a1 = -1.0180, a2 = 0.2247

Notes organized by Prof. Renwei Mei, University of Florida 50
 1 M
 E=  [ yi - (a0 + a1 xi + a2 xi2)]2 = 0.00023
M − 1 i =1

* The comparison of the two fits are shown below

Linear least square fit Quadratic least square fit
1.0 1.0
y y = 0.9523 - 0.7604x y y = 0.9980 - 1.0180x
+ 0.2247x^2
0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0.0 0.0
0.0 0.2 0.4 0.6 0.8 1.0 1.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2
x x

◊ Note:
* The matrix of the system is a symmetric one.
* If n≤3, there is no practical difficulty in obtaining the
coefficients (a0, a1, a2, a3).
* If n>3, the system [X]a = b can be very ill-conditioned.
* For very ill-conditioned system, we need more sophisticated
method to reduce the round off error.
* This method minimize the norm of the error, ||yi -
pn(xi)||2. In many cases, one is only interested in
minimizing ||a-a*||2. This is difficult since we do not
know the exact a* in advance.
Notes organized by Prof. Renwei Mei, University of Florida 51
◊ Matlab implementation for this same problem for least
square fit:
>> x=[0.05 0.11 0.15 0.31 0.46 0.52 0.7 0.74 0.82 0.98
1.17]';
>> y=[0.956 0.89 0.832 0.717 0.571 0.539
0.378 0.37 0.306 0.242 0.104]';
>> p=polyfit(x,y,1)
p =
-0.76040691274189 0.95227686777989

>> p=polyfit(x,y,2)
p =
0.22468213278795 -1.01804246473857 0.99796838418339
* Note: given N=11 data point, the use of the lower order (n<10)
polynomials in the fitting implies that a least square
procedure has been adopted.
* For n=1, Matlab gives: a1 = -0.76040691274189
a0 = 0.95227686777989
and they agree with what were obtained earlier.
* For n=2, Matlab gives: a2 =0.22468213278795
a1 = -1.01804246473857
a0 = 0.99796838418339
and they agree with what were obtained earlier.

Notes organized by Prof. Renwei Mei, University of Florida 52