Newton Divided Differences

I Given m + 1 distinct points

(z0 , f (z0 )), (z1 , f (z1 )), · · · , (zm , f (zm )),

I Interpolating polynomial of degree ≤ m

P(x) = a0 + a1 (x − z0 ) + a2 (x − z0 )(x − z1 ) +
· · · + am (x − z0 )(x − z1 ) · · · (x − zm−1 ),
with P(z0 ) = f (z0 ), P(z1 ) = f (z1 ), · · · , P(zm ) = f (zm ).

I where

aj = f [z0 , z1 , · · · , zj ], for j = 0, 1, · · · , m.
 Newton Divided Difference Table
zi f [zi ] f [zi−1 , zi ] f [zi−2 , zi−1 , zi ] ··· f [z0 , z1 , · · · , zm ]
z0 f[z0 ] = a0
f[z0 , z1 ] = a1
z1 f [z1 ] f[z0 , z1 , z2 ] = a2
f [z1 , z2 ]
z2 f [z2 ] f [z1 , z2 , z3 ]
f [z2 , z3 ]
z3 f [z3 ] f [z2 , z3 , z4 ]
f [z3 , z4 ]
z4 f [z4 ] f [z3 , z4 , z5 ] ··· f[z0 , z1 , · · · , zm ] = am
f [z4 , z5 ]
z5 f [z5 ] f [z4 , z5 , z6 ]
f [z5 , z6 ]
z6 f [z6 ] f [z5 , z6 , z7 ]
f [z6 , z7 ]
z7 f [z7 ] f [z6 , z7 , z8 ]
f [z7 , z8 ]
z8 f [z8 ]
 Double nodes, m = 2n + 1, xj = zbj/2c , f [xj , xj ] = f 0 (xj )
xi f [xi ] f [zi−1 , zi ] f [zi−2 , zi−1 , zi ] ··· f [z0 , z1 , · · · , zm ]
x0 f[x0 ] = a0
f[x0 , x0 ] = a1
x0 f [x0 ] f[x0 , x0 , x1 ] = a2
f [x0 , x1 ]
x1 f [x1 ] f [x0 , x1 , x1 ]
f [x1 , x1 ]
x1 f [x1 ] f [x1 , x1 , x2 ]
f [x1 , x2 ]
x2 f [x2 ] f [x1 , x2 , x2 ] ··· f[x0 , x0 , · · · , xn , xn ]
f [x2 , x2 ]
x2 f [x2 ] f [x2 , x2 , x3 ]
f [x2 , x3 ]
x3 f [x3 ] f [x2 , x3 , x3 ]
f [x3 , x3 ]
x3 f [x3 ]
Hermite Interpolation

I Given n + 1 distinct points

(x0 , f (x0 ), f 0 (x0 )), (x1 , f (x1 ), f 0 (x1 )), · · · , (xn , f (xn ), f 0 (xn )),

I Interpolating polynomial H(x) of degree ≤ 2n + 1 with

H(x0 ) = f (x0 ), H 0 (x0 ) = f 0 (x0 ),

H(x1 ) = f (x1 ), H 0 (x1 ) = f 0 (x1 ),
.. ..
. .
H(xn ) = f (xn ), H 0 (xn ) = f 0 (xn ).

I 2n + 2 conditions, 2n + 2 coefficients in H(x).

hermit-1.jpg 496×412 pixels 2/17/15, 12:29 AM
Hermite Interpolation: Newton Divided Differences

I Given n + 1 distinct points

(x0 , f (x0 ), f 0 (x0 )), (x1 , f (x1 ), f 0 (x1 )), · · · , (xn , f (xn ), f 0 (xn )),

I Interpolating polynomial of degree ≤ 2n + 1

H(x) = a0 + a1 (x − x0 ) + a2 (x − x0 )2 +
· · · + a2n+1 (x − x0 )2 (x − x1 )2 · · · (x − xn−1 )2 (x − xn ).
aj = f [z0 , z1 , · · · , zj ], for j = 0, 1, · · · , 2n + 1,

with z2i = z2i+1 = xi for i = 0, 1, · · · , n.

 Hermite Interpolation: m = 2n + 1, f [xj , xj ] = f 0 (xj )
xi f [xi ] f [zi−1 , zi ] f [zi−2 , zi−1 , zi ] ··· f [z0 , z1 , · · · , zm ]
x0 f[x0 ] = a0
f[x0 , x0 ] = a1
x0 f [x0 ] f[x0 , x0 , x1 ] = a2
f [x0 , x1 ]
x1 f [x1 ] f [x0 , x1 , x1 ]
f [x1 , x1 ]
x1 f [x1 ] f [x1 , x1 , x2 ]
f [x1 , x2 ]
x2 f [x2 ] f [x1 , x2 , x2 ] ··· f[x0 , x0 , · · · , xn , xn ]
f [x2 , x2 ]
x2 f [x2 ] f [x2 , x2 , x3 ]
f [x2 , x3 ]
x3 f [x3 ] f [x2 , x3 , x3 ]
f [x3 , x3 ]
x3 f [x3 ]
Newton Divided Differences for Hermite Interpolation
Evaluate Hermite Polynomial given coefficients
Hermite Interpolation on e x on [−1, 1]
Hermite Interpolation (red) vs. exact function (blue) for ex on [−1, 1], 11 points
2.5

1.5

0.5
−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

Interpolation Error, semilog scale

−15.7
10

−15.8
10

−15.9
10

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2

Coefficients aj for Hermite Interpolation
Divided Difference Coefficients, semilog scale
0
10

−2
10

−4
10

−6
10

−8
10

−10
10

−12
10
0 5 10 15 20 25
 1
Hermite Interpolation on 0.2+x 2 on [−1, 1]
Hermite Interpolation (red) vs. exact function (blue) for 1/(0.2+x2) on [−1, 1], 21 points
5

0
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

Interpolation Error, semilog scale

0
10

−5
10

−10
10

−15
10
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1
Coefficients aj for Hermite Interpolation
Divided Difference Coefficients, semilog scale
8
10

7
10

6
10

5
10

4
10

3
10

2
10

1
10

0
10

−1
10
0 5 10 15 20 25 30 35 40 45
Hermite Interpolation

I Given n + 1 distinct points

(x0 , f (x0 ), f 0 (x0 )), (x1 , f (x1 ), f 0 (x1 )), · · · , (xn , f (xn ), f 0 (xn )),

I Interpolating polynomial H(x) of degree ≤ 2n + 1 with

H(x0 ) = f (x0 ), H 0 (x0 ) = f 0 (x0 ),

H(x1 ) = f (x1 ), H 0 (x1 ) = f 0 (x1 ),
.. ..
. .
H(xn ) = f (xn ), H 0 (xn ) = f 0 (xn ).

I 2n + 2 conditions, 2n + 2 coefficients in H(x).

(Equivalent) Hermite Interpolation Solution

I Given n + 1 distinct points

(x0 , f (x0 ), f 0 (x0 )), (x1 , f (x1 ), f 0 (x1 )), · · · , (xn , f (xn ), f 0 (xn )),

I Interpolating polynomial H(x) is

n
X n
X
H(x) = f (xj ) Hj (x) + f 0 (xj ) Hbj (x),
j=0 j=0

where

Hj (x) = 1 − 2(x − xj )L0j (xj ) L2j (x), Hbj (x) = (x − xj )L2j (x),

Q x − xi
with Lj (x) = i6=j .
xj − xi
(Equivalent) Hermite Interpolation Solution

I Given n + 1 distinct points

(x0 , f (x0 ), f 0 (x0 )), (x1 , f (x1 ), f 0 (x1 )), · · · , (xn , f (xn ), f 0 (xn )),

I Interpolating polynomial H(x) is

n
X n
X
H(x) = f (xj ) Hj (x) + f 0 (xj ) Hbj (x),
j=0 j=0

where

Hj (x) = 1 − 2(x − xj )L0j (xj ) L2j (x), Hbj (x) = (x − xj )L2j (x),

Q x − xi
with Lj (x) = i6=j .
xj − xi
(Equivalent) Hermite Interpolation Solution
I Interpolating polynomial H(x) is
n
X n
X
H(x) = f (xj ) Hj (x) + f 0 (xj ) Hbj (x),
j=0 j=0

where

Hj (x) = 1 − 2(x − xj )L0j (xj ) L2j (x), Hbj (x) = (x − xj )L2j (x).

I For i 6= j,

Hj (xi ) = 0, Hj0 (xi ) = 0, Hbj (xi ) = 0, Hbj0 (xi ) = 0.

I For i = j,

Hi (xi ) = 1, Hi0 (xi ) = 0, Hbi (xi ) = 0, Hbi0 (xi ) = 1.

Hermite Interpolation Error

Theorem: Suppose x0 , x1 , · · · , xn are distinct numbers in the

interval [a, b] and f ∈ C 2n+2 [a, b]. Then, for each x ∈ [a, b], a
number ξ(x) between , x0 , x1 , · · · , xn (hence ∈ (a, b)) exists with

f (2n+2) (ξ(x))
f (x) = H(x) + (x − x0 )2 (x − x1 )2 · · · (x − xn )2 ,
(2n + 2)!

where H(x) is the interpolating polynomial.

Hermite Interpolation Error: Proof
If x = x0 , x1 , · · · , xn , then error = 0 and theorem is true. Now
let x be not equal to any node. Define function g for t ∈ [a, b]

def (t − x0 )2 (t − x1 )2 · · · (t − xn )2
g (t) = (f (t) − H(t)) − (f (x) − H(x))
(x − x0 )2 (x − x1 )2 · · · (x − xn )2
n
Y (t − xj )2
= (f (t) − H(t)) − (f (x) − H(x)) ∈ C 2n+2 [a, b].
(x − xj )2
j=0

Then g (t) vanishes at n + 2 distinct points:

g (x) = 0, g (xk ) = 0, , for k = 0, 1, · · · , n.

and g 0 (t) vanishes at n + 1 distinct points:

g 0 (xk ) = 0, , for k = 0, 1, · · · , n.

There must be a ξ between x and nodal points such that

g (2n+2) (ξ) = 0.
Hermite Interpolation Error: Proof

Since
n
Y (t − xj )2 (2n+2)
g (2n+2) (ξ) = (f (t) − H(t))(2n+2) |t=ξ − (f (x) − H(x))( ) |ξ
(x − xj )2
j=0
(2n + 2)!
= f (2n+2) (ξ) − (f (x) − H(x)) Qn 2
j=0 (x − xj )
=0

Therefore

f (2n+2) (ξ(x))
f (x) = H(x) + (x − x0 )2 (x − x1 )2 · · · (x − xn )2 ,
(2n + 2)!
The Splines Idea

I Given n + 1 distinct points

(x0 , f (x0 )), (x1 , f (x1 )), · · · , (xn , f (xn )),

I Find cubic spline interpolant S(x),
I for x ∈ [xj , xj+1 ], j = 0, 1, · · · , n − 1,

def
S(x) = Sj (x) = aj + bj (x − xj ) + cj (x − xj )2 + dj (x − xj )3 .

(S(x) piece-wise low-degree polynomial)

I S(xj ) = f (xj ) j = 0, 1, · · · , n.
(S(x) matches f (x) at all nodal points)

I S(x) ∈ C 2 [x0 , xn ].
(S(x) remains smooth-enough)
Carl de Boor: The book about Splines
Carl de Boor
The Splines Idea
I Spine Interpolant S(x) for given n + 1 distinct points
(x0 , f (x0 )), (x1 , f (x1 )), · · · , (xn , f (xn )),
The Splines Equations (I)
For x ∈ [xj , xj+1 ], j = 0, 1, · · · , n − 1,

def
S(x) = Sj (x) = aj + bj (x − xj ) + cj (x − xj )2 + dj (x − xj )3 .

I for j = 0, 1, · · · , n − 1,

aj = Sj (xj ) = f (xj ).

I let hj = xj+1 − xj , for j = 0, 1, · · · , n − 1,

aj+1 = Sj+1 (xj+1 ) = Sj (xj+1 ) = aj + bj hj + cj hj2 + dj hj3

aj+1 − aj
bj + cj hj + dj hj2 = , j = 0, 1, · · · , n − 1.
hj
This ensures S(x) ∈ C [x0 , xn ].
The Splines Equations (II)

For x ∈ [xj , xj+1 ], j = 0, 1, · · · , n − 1,

Sj (x) = aj + bj (x − xj ) + cj (x − xj )2 + dj (x − xj )3
Sj0 (x) = bj + 2cj (x − xj ) + 3dj (x − xj )2

I 0
Define bn = Sn−1 (xn ); for j = 0, · · · , n − 1,
0
bj+1 = Sj+1 (xj+1 ) = Sj0 (xj+1 ) = bj + 2cj hj + 3dj hj2 .

This ensures S 0 (x) ∈ C [x0 , xn ].

The Splines Equations (III)

For x ∈ [xj , xj+1 ], j = 0, 1, · · · , n − 1,

Sj (x) = aj + bj (x − xj ) + cj (x − xj )2 + dj (x − xj )3
Sj00 (x) = 2cj + 6dj (x − xj )

I 00 (x )/2; for j = 0, · · · , n − 1,
Define cn = Sn−1 n

00
2cj+1 = Sj+1 (xj+1 ) = Sj00 (xj+1 ) = 2cj + 6dj hj .

This ensures S 00 (x) ∈ C [x0 , xn ].

The Splines Equations (IV)

For j = 0, · · · , n − 1,
cj+1 − cj
2cj+1 = 2cj + 6dj hj , so dj =
3hj

aj+1 − aj hj aj+1 − aj
bj +cj hj +dj hj2 = , so bj = − (2cj +cj+1 )+ .
hj 3 hj

bj+1 = bj + 2cj hj + 3dj hj2 , so bj+1 = bj + hj (cj + cj+1 ).

Last two equations do not involve dj . Next get rid of bj .

The Splines Equations (V)

For j = 0, · · · , n − 2,
hj+1 aj+2 − aj+1
− (2cj+1 + cj+2 ) +
3 hj+1
hj aj+1 − aj
= − (2cj + cj+1 ) + + hj (cj + cj+1 ),
3 hj

which simplifies to (for j = 1, · · · , n − 1)

 
aj+1 − aj aj − aj−1
hj−1 cj−1 + 2(hj−1 + hj ) cj + hj cj+1 = 3 − .
hj hj−1

n − 1 equations with n + 1 unknowns.

Natural Splines: S000 (x0 ) = Sn−1
00
(xn ) = 0

I c0 = S000 (x0 )/2 = 0, cn = Sn−1

00 (x )/2 = 0.
n
I Equations for {cj }n−1
j=1 ,
 
a2 − a1 a1 − a0
2(h0 + h1 ) c1 + h1 c2=3 − ,
h1 h0
 
aj+1 − aj aj − aj−1
hj−1 cj−1 + 2(hj−1 + hj ) cj + hj cj+1=3 − , 2 ≤ j ≤ n − 2,
hj hj−1
 
an − an−1 an−1 − an−2
hn−2 cn−2 + 2(hn−2 + hn−1 ) cn−1=3 −
hn−1 hn−2
 Natural Splines: equations in matrix form

I Equations for {cj }n−1

j=1 ,
  
2(h0 + h1 ) h1 c1  a2 −a1 a −a

h1 2(h1 + h2 )   c2 h1
− 1h 0

 

  0 
 . . .  .   . 
 = 3 . .

 . . .  .
. . . .
  .  
    
 an −a a −an−2
n−1
− n−1

 hn−3 2(hn−3 + hn−2 ) hn−2  c
n−2

hn−2 2(hn−2 + hn−1 ) cn−1 h n−1 h
n−2

I Equations for {dj }nj=0 , {bj }nj=0 ,

cj+1 − cj hj aj+1 − aj
dj = , and bj = − (2cj + cj+1 ) + .
3hj 3 hj
Clamped Splines: S00 (x0 ) = f 0 (x0 ), Sn−1
0
(xn ) = f 0 (xn )
I Equation for c0 , c1 :
h0 a1 − a0
f 0 (x0 ) = S00 (x0 ) = b0 = − (2c0 + c1 ) + .
3 h0
 
a1 − a0
2h0 c0 + h0 c1 = 3 − f 0 (x0 ) .
h0
I Equation for cn−1 , cn :

f 0 (xn ) = Sn−1
0
(xn ) = bn = bn−1 + hn−1 (cn−1 + cn )
hn−1 an − an−1
= − (2cn−1 + cn ) + + hn−1 (cn−1 + cn ),
3 hn−1

which leads to
 
0 an − an−1
hn−1 cn−1 + 2hn−1 cn = 3 f (xn ) − .
hn−1
Clamped Splines: equations in matrix form

I Equations for {cj }nj=0 ,

 a −a
1 0 − f 0 (x )

h0 0
  
2h0 h0 c0  a2 −a1 a −a 

h1
− 1h 0 
 h0 2(h0 + h1 )  c1 
 0


.
  
.
 
 . . .  
 = 3 . .

 . . .  . .
. . .

  .  
a −a an−1 −an−2
  
hn−2 2(hn−2 + hn−1 ) hn−1  cn−1 
 n n−1 
−
  
 hn−1 hn−2 
hn−1 2hn−1 cn a n −a
 
f 0 (xn ) − h n−1
n−1

I Equations for {dj }nj=0 , {bj }nj=0 ,

cj+1 − cj hj aj+1 − aj
dj = , and bj = − (2cj + cj+1 ) + .
3hj 3 hj
Clamped Splines
Natural Splines, f (x) = e x , x0 = 0, x1 = 1, x2 = 2, x3 = 3
Clamped Splines, f (x) = e x ,
x0 = 0, x1 = 1, x2 = 2, x3 = 3, f 0 (0) = 1, f 0 (3) = e 3
Splines
I Given n + 1 distinct points
(x0 , f (x0 )), (x1 , f (x1 )), · · · , (xn , f (xn )),
I Find cubic spline interpolant S(x) ∈ C 2 [x0 , xn ],
def
S(x) = Sj (x) = aj + bj (x − xj ) + cj (x − xj )2 + dj (x − xj )3 ,
for x ∈ [xj , xj+1 ], 0 ≤ j ≤ n − 1.
I S(xj ) = f (xj ), 0 ≤ j ≤ n.
Two flavors of Splines

I Natural Splines: S000 (x0 ) = Sn−1

00 (x ) = 0.
n

I Clamped Splines: S00 (x0 ) = f 0 (x0 ), Sn−1

0 (xn ) = f 0 (xn ).
A duck in Splines

I A duck in flight

I Goal: to approximate the top profile.

duck top profile
Duck top profile in Natural Splines, a coefficients
Duck top profile in 20-degree polynomial interpolation
Parametric Curves: x = x(t), y = y (t)
Parametric Curve Approximation
I Given n + 1 distinct points

(x0 , y0 ), (x1 , y1 ), · · · , (xn , yn ),

where

xj = x(tj ), yj = y (tj ), j = 0, 1, · · · , n.
I Example
Parametric Curve Approximation
I Polynomial interpolation on x = x(t), and y = y (t),
respectively.
   
352 14
x(t) = 64t − t + 60 t − t − 1,
3 3
   
64 116
y (t) = − t + 48 t − t + 11 t.
3 3
Parametric Curves in Computer Graphics
I Design: Piece-wise cubic Hermite polynomial.
I Feature: Each cubic Hermite polynomial is completely
determined by function/derivative at endpoints.
I Consequence:, Each portion of the curve can be changed
while leaving most of the curve the same.

As duck flies, parametric curve can effectively evolve.

Parametric Curves: Piece-wise cubic Hermite polynomials
I Given: n + 1 data points (x(t0 ), y (t0 )), · · · , (x(tn ), y (tn )).
dy
I Given: n + 1 derivatives |t=ti , 0 ≤ i ≤ n.
dx

I Find: Piece-wise cubic Hermite polynomial:

x = xi (t), y = yi (t), for i ∈ [ti , ti+1 ], 0 ≤ i ≤ n,

such that

xi (ti ) = x(ti ), yi (ti ) = y (ti ), xi (ti+1 ) = x(ti+1 ), yi (ti+1 ) = y (ti+1 ),

dy y 0 (ti ) dy  yi0 (ti+1 )
|t=ti = i0 , and t=ti+1 =
dx xi (ti ) dx xi0 (ti+1 )

8 parameters in xi (t), yi (t) with 6 given conditions for each i.

Guidepoints guide slopes (assume [ti , ti+1 ] = [0, 1])
I The cubic Hermite polynomial x(t) on [0, 1] satisfies
x(0) = x0 , x(1) = x1 , x 0 (0) = α0 , and x 0 (1) = α1 .
I The cubic Hermite polynomial y (t) on [0, 1] satisfies
y (0) = y0 , y (1) = y1 , y 0 (0) = β0 , and y 0 (1) = β1 .
I Guidepoints guide slopes
dy β0 dy β1
|t=0 = , and |t=1 = .
dx α0 dx α1
Guidepoints, example

I Given end points

(x0 , y0 ) = (0, 0), and (x1 , y1 ) = (1, 0).

Guidepoints, example

I Given end points

(x0 , y0 ) = (0, 0), and (x1 , y1 ) = (1, 0).

I Given end points

(x0 + α0 , y0 + β0 ) = (1, 1), and (x1 − α1 , y1 − β1 ) = (0, 1).

Guidepoints, example

I Given end points

(x0 , y0 ) = (0, 0), and (x1 , y1 ) = (1, 0).

I Given end points

(x0 + α0 , y0 + β0 ) = (1, 1), and (x1 − α1 , y1 − β1 ) = (0, 1).

I The cubic Hermite polynomial x(t) = t satisfies
x(0) = 0, x(1) = 1, x 0 (0) = 1, and x 0 (1) = 1.
I The cubic Hermite polynomial x(t) = t satisfies
x(0) = 0, x(1) = 1, x 0 (0) = 1, and x 0 (1) = 1.

I The cubic Hermite polynomial y (t) = −t 2 + t satisfies

y (0) = 0, y (1) = 0, y 0 (0) = 1, and y 0 (1) = −1.
I The cubic Hermite polynomial x(t) = t satisfies
x(0) = 0, x(1) = 1, x 0 (0) = 1, and x 0 (1) = 1.

I The cubic Hermite polynomial y (t) = −t 2 + t satisfies

y (0) = 0, y (1) = 0, y 0 (0) = 1, and y 0 (1) = −1.

I Slopes
dy 1 dy −1
|t=0 = = 1, and |t=1 = = −1.
dx 1 dx 1
I The cubic Hermite polynomial x(t) = t satisfies
x(0) = 0, x(1) = 1, x 0 (0) = 1, and x 0 (1) = 1.

I The cubic Hermite polynomial y (t) = −t 2 + t satisfies

y (0) = 0, y (1) = 0, y 0 (0) = 1, and y 0 (1) = −1.

I Slopes
dy 1 dy −1
|t=0 = = 1, and |t=1 = = −1.
dx 1 dx 1
Numerical Differentiation

Derivative of given function f (x) at x0

f (x0 + h) − f (x0 )
f 0 (x0 ) = lim
h→0 h
f (x0 + h) − f (x0 )
≈ for small values of h.
h

How good is this approximation?

By Taylor expansion, there is a ξ between x0 and x0 + h,

1
f (x0 + h) = f (x0 ) + h f 0 (x0 ) + h2 f 00 (ξ),
2
0 f (x0 + h) − f (x0 ) 1 00
therefore f (x0 ) = − h f (ξ).
h 2
Numerical Differentiation

f (x0 + h) − f (x0 ) 1 00
f 0 (x0 ) = − h f (ξ)
h 2
f (x0 + h) − f (x0 )
≈ .
h
I forward-difference formula for h > 0,
I backward-difference formula for h < 0.
Numerical Differentiation

f (x0 + h) − f (x0 ) 1 00
f 0 (x0 ) = − h f (ξ)
h 2
f (x0 + h) − f (x0 )
≈ .
h
I forward-difference formula for h > 0,
I backward-difference formula for h < 0.
Numerical Differentiation, example
Example: Use the forward-difference formula to approximate the
derivative of f (x) = ln(x) at x0 = 1.8, using = 0.1, 0.05, 0.01.

Because f 00 (ξ) = −1/ξ 2 for ξ ∈ [x0 , x0 + h] ⊂ [1.8, 1.9], it follows

that the approximation error
 
1  00  1  h 
h f (ξ) ≤  2  .
2 2 1.8
Numerical Differentiation, example
Example: Use the forward-difference formula to approximate the
derivative of f (x) = ln(x) at x0 = 1.8, using = 0.1, 0.05, 0.01.

Because f 00 (ξ) = −1/ξ 2 for ξ ∈ [x0 , x0 + h] ⊂ [1.8, 1.9], it follows

that the approximation error
 
1  00  1  h 
h f (ξ) ≤  2  .
2 2 1.8
Numerical Differentiation, via polynomial interpolation
Suppose that {x0 , x1 , · · · , xn } are n + 1 distinct numbers,
 
n n
X f (n+1) (ξ(x)) Y Y (x − xi )
f (x) =  f (xj )Lj (x) +
 (x−xi ), Lj (x) = ,
(n + 1)! (xj − xi )
j=0 i=0 i6=j

for some ξ(x).

Numerical Differentiation, via polynomial interpolation
Suppose that {x0 , x1 , · · · , xn } are n + 1 distinct numbers,
 
n n
X f (n+1) (ξ(x)) Y Y (x − xi )
f (x) =  f (xj )Lj (x) +
 (x−xi ), Lj (x) = ,
(n + 1)! (xj − xi )
j=0 i=0 i6=j

for some ξ(x). So

 
n n
! !
X f (n+1) (ξ(x)) d Y
f 0 (x) =  f (xj )L0j (x) + (x − xi )
(n + 1)! dx
j=0 i=0
! n !
d f (n+1) (ξ(x)) Y
+ (x − xi ) .
dx (n + 1)!
i=0

Messy. But last term = 0 for x = xk , k = 0, 1, · · · , n

Numerical Differentiation, via polynomial interpolation

  ! 
n
X f (n+1) (ξ(x k ))
Y
f 0 (xk ) =  f (xj )L0j (xk ) +  (xk − xi )
(n + 1)!
j=0 i6=k
n
X
≈ f (xj )L0j (xk ).
j=0

(n + 1)-point formula, works for any nodal choices.

3-point formulas (n = 2), j = 0, 1, 2
3-point formulas (n = 2), j = 0, 1, 2
3-point formulas (n = 2), j = 0, 1, 2
3-point formulas (n = 2), equi-spaced nodes
Choose x1 = x0 + h, x2 = x0 + 2h, with h 6= 0.
3-point formulas (n = 2), equi-spaced nodes
Choose x1 = x0 + h, x2 = x0 + 2h, with h 6= 0.
3-point formulas (n = 2), equi-spaced nodes
Choose x1 = x0 + h, x2 = x0 + 2h, with h 6= 0.
3-point formulas (n = 2), equi-spaced nodes
Choose x1 = x0 + h, x2 = x0 + 2h, with h 6= 0.
5-point formulas, equi-spaced nodes

Choose x1 = x0 − h, x2 = x0 − 2h, x3 = x0 + h, x4 = x0 + 2h,with

h 6= 0.
Second order derivatives, equi-spaced points

1 1 000
f (x0 + h) = f (x0 ) + f 0 (x0 ) h + f 00 (x0 ) h2 + f (x0 ) h3
2 6
1 (4) 4
+ f (ξ+ ) h ,
24
1 1 000
f (x0 − h) = f (x0 ) − f 0 (x0 ) h + f 00 (x0 ) h2 − f (x0 ) h3
2 6
1 (4) 4
+ f (ξ− ) h .
24
Second order derivatives, equi-spaced points

1 1
f (x0 + h) = f (x0 ) + f 0 (x0 ) h + f 00 (x0 ) h2 + f 000 (x0 ) h3
2 6
1 (4) 4
+ f (ξ+ ) h ,
24
1 1
f (x0 − h) = f (x0 ) − f 0 (x0 ) h + f 00 (x0 ) h2 − f 000 (x0 ) h3
2 6
1 (4) 4
+ f (ξ− ) h .
24
Adding up, terms with difference signs cancel,
h4  (4) 
f (x0 + h) + f (x0 − h) = 2f (x0 ) + f 00 (x0 ) h2 + f (ξ+ ) + f (4) (ξ− )
24
2h4 (4)
= 2f (x0 ) + f 00 (x0 ) h2 + f (ξ).
24
Second order derivatives, equi-spaced points

1 1
f (x0 + h) = f (x0 ) + f 0 (x0 ) h + f 00 (x0 ) h2 + f 000 (x0 ) h3
2 6
1 (4) 4
+ f (ξ+ ) h ,
24
1 1
f (x0 − h) = f (x0 ) − f 0 (x0 ) h + f 00 (x0 ) h2 − f 000 (x0 ) h3
2 6
1 (4) 4
+ f (ξ− ) h .
24
Adding up, terms with difference signs cancel,
h4  (4) 
f (x0 + h) + f (x0 − h) = 2f (x0 ) + f 00 (x0 ) h2 + f (ξ+ ) + f (4) (ξ− )
24
2h4 (4)
= 2f (x0 ) + f 00 (x0 ) h2 + f (ξ).
24
Therefore
f (x0 + h) + f (x0 − h) − 2f (x0 ) h2 (4)
f 00 (x0 ) = − f (ξ).
h2 12
Round-Off Error Instability

(For example) three-point midpoint formula

f (x0 + h) − f (x0 − h) h2 (3)

f 0 (x0 ) = − f (ξ).
2h 6

I every step in the computation could incur round-off error.

I division by 2h could magnify round-off error.
Assume round-off error model

f (x0 +h) = fb(x0 +h)+e(x0 +h) and f (x0 −h) = fb(x0 −h)+e(x0 −h),

where |e(x0 + h)| ≤ , |e(x0 − h)| ≤ . No other round-off errors.

 It follows

fb(x0 + h) − fb(x0 − h) e(x0 + h) − e(x0 − h) h2 (3)

f 0 (x0 ) − = − f (ξ)
2h 2h 6
 It follows

fb(x0 + h) − fb(x0 − h) e(x0 + h) − e(x0 − h) h2 (3)

f 0 (x0 ) − = − f (ξ)
 2h  2h 6
e(x0 + h) − e(x0 − h)  h2 (3)
 

 0 f (x0 + h) − f (x0 − h) 
b b   
f (x0 ) −  ≤   + 6 |f (ξ)|.
 2h  2h

Assume |e(x0 + h)| ≤ , |e(x0 − h)| ≤  and |f (3) (ξ)| ≤ M, an

upper bound on |f (3) (x)|,
 

 0 fb(x0 + h) − fb(x0 − h)   M h2 def
f (x0 ) − ≤ + = e(h).
 2h  h 6
Round-Off Error Instability: optimal h choice
 M h2
e(h) = +
h 6
is smallest at
 1
3 3  1
hmin = = O 3 ,
M
with
 13
92

1  2
e(hmin ) = = O 3 .
2 M