CSC475: Numerical Computing

Tanveer Ahmed Siddiqui

Department of Computer Science

COMSATS University, Islamabad
 Question Answer Session

Week 05

Department of Computer Science 2

 Today Covered
,

After completing this lecture you should be able to

know
 Differentiation using difference operators.
 Differentiation using Interpolation.

Department of Computer Science

 Recap
,

Department of Computer Science

 Recap
,

Department of Computer Science

 Recap
,

Department of Computer Science

 Recap
,

Department of Computer Science

 Numerical Differentiation

Department of Computer Science 8

 Differentiations

 Differentiation gives a measure of the rate at which a

quantity changes.
 Rates of change of quantities appear in many disciplines,
especially science and engineering.
 The function to be differentiated will typically be in one
of the following three forms:
 1. A simple continuous function such as a polynomial, an
exponential, or a trigonometric function.
 2. A complicated continuous function that is difficult or
impossible to differentiate directly.
 3. A tabulated function where values of x and f (x) are
given at a number of discrete points, as is often the case
with experimental or field data.

Department of Computer Science 9

Why Numerical methods for differentiations

 Consider a simple continuous function of single

variable y = f (x).
 If the function is known and simple, we can
easily obtain its derivative (s) analytically using
calculus.
 How we compute derivative if:
 We do not know the function as such
 The function is complicated
 Function is given in a tabular form at a set of points
x0,x1,…,xn,
 In all these situation, numerical methods helps
us for computing derivatives.
Department of Computer Science 10
 Approaches to numerical differentiation

 For a given set of points, two approaches can be

used to calculate a numerical approximation of
the derivative at one of the points.
 One approach is to use a finite difference
approximation for the derivative.
 The second approach is to approximate the
points with an analytical expression that
can be easily differentiated, and then to
calculate the derivative by differentiating the
analytical expression.

Department of Computer Science 11

 Finite difference approximation

 A finite difference approximation of a derivative

at a point x; is an approximate calculation based
on the value of points in the neighborhood of xi.
 The derivative at point xi is approximated by the
slope of the line that connects the point before xi
with the point after xi.
 This approach is illustrated in following fig.

Department of Computer Science 12

 Accuracy of finite difference approximation

 The accuracy of a finite difference approximation

depends on the accuracy of:
 The data points,
 The spacing between the points
 The specific formula used for the approximation.
 The simplest formula approximates the
derivative as the slope of the line that connects
two adjacent points.

Department of Computer Science 13

 Approximate analytical expression

 The approximate analytical expression can be

derived by using curve fitting.
 The points are curve fitted by f(x), and the
derivative at point xi is obtained by analytically
differentiating the approximating function and
evaluating the result at the point.
 This approach is illustrated in following fig.

Department of Computer Science 14

 DIFFERENTIATION USING
DIFFERENCE OPERATORS

Department of Computer Science 15

 Difference operators: Shift Operator

 Let us first revise the definitions of various

difference operators.
 Shift Operator, E:

E f ( x )  f ( x  h)
E f ( x)  f ( x  nh)
n

E y x  y x  nh
n

Department of Computer Science 16

 Difference operators: Inverse Shift

 Inverse Shift Operator, E-1:

 The inverse operator E-1 is defined as
1
E f ( x )  f ( x  h)
 Similarly
n
E f ( x)  f ( x  nh)

Department of Computer Science 17

Difference operators: Forward difference operator

y x  y x  h  y x  f ( x  h)  f ( x )
r 1 r 1
 yi  
r
yi 1   yi

  E 1
Department of Computer Science 18
Difference operators: Backward difference operator

y x  y x  y x  h  f ( x )  f ( x  h )
k 1 k 1
 yi  
k
yi   yi 1 ,
i  n, (n  1),..., k

E 1
1
  1 E 
Department of Computer Science
E 19
Difference operators: Differential Operator

d 
Df ( x)  f ( x)  f ( x) 
dx 
2 
d
D f ( x)  2 f ( x)  f ( x) 
2 n

dx 

hD  log E
Department of Computer Science 24
Differentiation using forward difference operator

 We assume that the function y = f (x) is given

for the values of the independent variable
x = x0 + ph, for p = 0, 1, 2, ….
 To find the derivatives of such a tabular function,
we proceed as follows
 Case I:
 Using forward difference operator ∆ and
combining equations
E f ( x )  f ( x  h)
 and
  E 1
hD  log E  log(1   )
Department of Computer Science 27
Differentiation using forward difference operator

 Remember the Differential operator, D is known

to represents the property
d
 
 Df ( x)  f ( x)  f ( x) 
hD  log E  log(1   ) dx 
2 
d
D f ( x)  2 f ( x)  f " ( x) 
2

dx 
 This would mean that
in terms of ∆ :
1     2 3
 4 5
D        
h 2 3 4 5 
Department of Computer Science 28
Differentiation using forward difference operator

Df ( x0 )  f ( x0 )
1  2 f ( x0 )  3 f ( x0 )
  f ( x0 )  
h 2 3
 4 f ( x0 )  5 f ( x0 )  d
     f ( x0 )
4 5  dx
 In other words
Dy0  y0

1  y0  y0  y0
2
 3 4
  y0     
h 2 3
Department of Computer Science
4 29 
Differentiation using forward difference operator

 Also, we can easily

2
1      2 3 4
D  2  
2
   
h  2 3 4 
1  2 11 4 5 5 
 2          
3

h  12 6 

Department of Computer Science 30

Differentiation using forward difference operator

1  2 11 4 5 5 
 2          
3

h  12 6 
2
d y0
D y0 
2
2
 y0
dx
  y0   y0
2 3

1  
 2 11 4 5 5
h    y0   y0   

 12 6 
Department of Computer Science 31
 Example

Compute f (0.2) and f (0) from the following

tabular data.

Department of Computer Science 32

 Example

Solution: Since x = 0 and 0.2 appear at and near

beginning of the table, it is appropriate to use
formulae based on forward differences to find
the derivatives. The difference table for the given
data is depicted below:

Department of Computer Science 33

 Example

using the formula for Df ( x) we have

1  2 f ( x)
Df ( x )   f ( x ) 
h 2
 f ( x)
3
 f ( x) 
4
  
3 4 
1  8.00
f (0.2)   2.40 
0.2  2
9.60 3.84 
    3.2
3 4 

Department of Computer Science 34

 Example

Using forward difference formula for D 2 f ( x ),

1   2 f ( x)   3 f ( x)
D f ( x)  2
2

h 
11 4 5 5 
  f ( x)   f ( x) 
12 6 

1  11
f (0)  2  2.24  5.76  12 (3.84)
(02)
5 
 (0)   0.0
6  35
Department of Computer Science
Differentiation using backward difference operator

 Remember the Differential operator, D is known

to represents the property
d
 
hD
   log(1   ). Df ( x)  dx f ( x)  f ( x) 
2 
d
D f ( x)  2 f ( x)  f " ( x) 
2

dx 
 On expansion, we have

1     2 3 4
D       
h 2 3 4 
Department of Computer Science 36
Differentiation using backward difference operator

 1     2 3 4
D       
h 2 3 4 

d
yn  Dyn  yn
dx
1  yn  yn  y n
2 3 4

  yn     
h 2 3 4 
Department of Computer Science 37
Differentiation using backward difference operator

1     2 3 4
D       
h 2 3 4 
2
1     
2 3 4
D 2
 2      
h  2 3 4 

1  2 11 4 5 5 
 2          
3

h  12 6 
Department of Computer Science 38
Differentiation using backward difference operator

1  2 11 4 5 5 
 2          
3

h  12 6 

yn  D yn 2

1 2 11 4 5 5 
 2   yn   y n   yn   yn   
3

h 12 6 

Department of Computer Science 39

 Example

Find y(2.2) and y(2.2) from the table

Department of Computer Science 40

 Example

Solution: Since x=2.2 occurs at the end of the

table, it is appropriate to use backward difference
formulae for derivatives. The backward difference
table for the given data is shown below:

Department of Computer Science 41

 Example

1  yn  yn  yn  2 3 4
yn   yn    
h 2 3 4 

1  0.2964 0.0535
y(2.2)  1.6359  
0.2  2 3
0.0094 
   5(1.8043)  9.0215
4 

Department of Computer Science 42

 Example

1  2 11 4 
yn  2   yn   y n   y n 
3

h  12 
1 
y(2.2)  2 
0.2964  0.0535
(0.2) 
11 
 (0.0094) 
12 
 25(0.3585)  8.9629
Department of Computer Science 43
 Derivation of Two and
three point formulae:
Retaining only the first term
in equation :

Department of Computer Science 49

 Differentiation using forward difference operator

 The derivative f'(x) of a function f(x) at the point

x =a is defined by:

 Graphically, the definition is illustrated in Fig.

Department of Computer Science 50

 Differentiation using forward difference operator

 The derivative is the value of the slope of the

tangent line to the function at x = a.
 The derivative is obtained by taking a point x
near x = a and calculating the slope of the line
that connects the two points.
 The accuracy of calculating the derivative in this
way increases as point x is closer to point a.
 In the limit as point x approaches point a, the
derivative is the slope of the line that is tangent
to f(x) at x =a .

Department of Computer Science 51

 Differentiation using forward difference operator

 In finite difference approximations of the

derivative, values of the function at different
points in the neighborhood of the point x=a are
used for estimating the slope.
 Various finite difference approximation formulas
exist.
 Three such formulas, where the derivative is
calculated from the values of two points, are
presented in this section

Department of Computer Science 52

 Differentiation using forward difference operator

 The forward, backward, and central finite

difference formulas are the simplest finite
difference approximations of the derivative.
 In these approximations, the derivative at point
(xi) is calculated from the values of two points.

Department of Computer Science 53

Differentiation using forward difference operator

 The derivative is estimated as the value of the

slope of the line that connects the two points.
 This is illustrated in following fig.

Department of Computer Science 54

Differentiation using forward difference operator

 The derivative is estimated as the value of the

slope of the line that connects the two points.

Department of Computer Science 55

 Example

Department of Computer Science 56

 Example

Department of Computer Science 57

Finite difference formulas using Taylor Series expansion

 The number of points used in the calculation

varies with the formula, and the points can be
ahead, behind, or on both sides of the point at
which the derivative is calculated.
 Here we do not derive these formulas, but
simply use them for approximating derivative of
function given in a tabular form at a set of points
x0,x1,…,xn, which are equally spaced
 A summary of finite difference formulas for
evaluating derivatives up to the fourth derivative
is presented in this lecture.
Department of Computer Science 59
Finite difference formulas using Taylor Series expansion

Department of Computer Science 60

 Example

Department of Computer Science 61

 Example

Department of Computer Science 62

Finite difference formulas using Taylor Series expansion

Department of Computer Science 63

 Example

Department of Computer Science 64

Finite difference formulas using Taylor Series expansion

Department of Computer Science 65

Finite difference formulas using Taylor Series expansion

Department of Computer Science 66

Differentiation formulas using Lagrange polynomials

 The differentiation formulas can also be derived

by using Lagrange polynomials.
 For the first derivative, the two-point central,
three-point forward, and three-point backward
difference formulas are obtained by considering
three points (xi ,yi), (xi+1, yi+1) , and (xi+2 , yi+2) .
 The polynomial, in Lagrange form, that passes
through the points is given by:

Department of Computer Science 67

Differentiation formulas using Lagrange polynomials

 The polynomial, in Lagrange form, that passes

through the points is given by:

Department of Computer Science 68

Differentiation formulas using Lagrange polynomials

 The first derivative at either one of the three

points is calculated by substituting the
corresponding value of x (xi, xi+1 or xi+2) in Eq.
(8.34).

 This gives the following three formulas for the

first derivative at the three points.

Department of Computer Science 69

Differentiation formulas using Lagrange polynomials

Department of Computer Science 70

Differentiation formulas using Lagrange polynomials

 When the points are equally spaced,

 Eq. (8.35) reduces to the three point forward
difference formula (Eq. (8.24)),
 Eq. (8.36) reduces to the two-point central difference
formula (Eq. (8.20)),
 Eq. (8.37) reduces to the three-point backward
difference formula (Eq. (8.25)).
 Equation (8.34) has two other important
features.
 It can be used when the points are not spaced
equally,
 It can be used for calculating the value of the first
derivative at any point between xi and xi+2 .
Department of Computer Science 71
 Formula
 Lagrange interpolation polynomial for
unequally spaced data

f ( x )  Li  1 ( x ) f ( x i  1 )  Li ( x ) f ( x i )  Li  1 ( x ) f ( x i  1 )
( x  x i )( x  x i  1 )
 f ( x i 1 )
( x i  1  x i )( x i  1  x i  1 )
( x  x i  1 )( x  x i  1 )
 f ( xi )
( x i  x i  1 )( x i  x i  1 )
( x  x i  1 )( x  x i )
 f ( xi1 )
( x i  1  x i  1 )( x i  1  x i )

Department of Computer Science

 Interpolation
 1st-order Lagrange polynomial
x  x1 x  x0
f 1 ( x )  L0 f ( x0 )  L1 ( x ) f ( x 1 )  f ( x0 )  f ( x1 )
x0  x 1 x 1  x0
 Second-order Lagrange polynomial

( x  x 1 )( x  x 2 )
f2 ( x )  f ( x0 )
( x0  x 1 )( x0  x 2 )
( x  x0 )( x  x 2 )
 f ( x1 )
( x 1  x0 )( x 1  x 2 )
( x  x0 )( x  x 1 )
 f ( x2 )
( x 2  x0 )( x 2  x 1 )
Department of Computer Science
 Interpolation
 Third-order Lagrange polynomial

( x  x 1 )( x  x 2 )( x  x 3 )
f3 ( x )  f ( x0 )
( x0  x 1 )( x0  x 2 )( x0  x 3 )
( x  x0 )( x  x 2 )( x  x 3 )
 f ( x1 )
( x 1  x0 )( x 1  x 2 )( x 1  x 3 )
( x  x0 )( x  x 1 )( x  x 3 )
 f ( x2 )
( x 2  x0 )( x 2  x 1 )( x 2  x 3 )
( x  x0 )( x  x 1 )( x  x 2 )
 f ( x3 )
( x 3  x0 )( x 3  x 1 )( x 3  x 2 )

Department of Computer Science

 Formula
 Lagrange interpolation polynomial for
unequally spaced data
f ( x )  Li  1 ( x ) f ( x i  1 )  Li ( x ) f ( x i )  Li  1 ( x ) f ( x i  1 )
( x  x i )( x  x i  1 ) ( x  x i  1 )( x  x i  1 )
 f ( xi 1 )  f ( xi )
( x i  1  x i )( x i  1  x i  1 ) ( x i  x i  1 )( x i  x i  1 )
( x  x i  1 )( x  x i )
 f ( xi1 )
( x i  1  x i  1 )( x i  1  x i )

 First derivative
2 x  xi  xi1 2 x  xi 1  xi 1
f ( x )  f ( x i  1 )  f ( xi )
( x i  1  x i )( x i  1  x i  1 ) ( x i  x i  1 )( x i  x i  1 )
2 x  xi 1  xi
 f ( xi1 )
( x i  1  x i  1 )( x i  1  x i )
Department of Computer Science
 Second Derivative
 First Derivative for unequally spaced
data
2 x  xi  xi1 2 x  xi 1  xi 1
f ( x )  f ( x i  1 )  f ( xi )
( x i  1  x i )( x i  1  x i  1 ) ( x i  x i  1 )( x i  x i  1 )
2 x  xi 1  xi
 f ( xi1 )
( x i  1  x i  1 )( x i  1  x i )

 Second Derivative for unequally spaced

data
f ( x )  f ( x )
2
 f (x )
2
i 1 i
( x i  1  x i )( x i  1  x i  1 ) ( x i  x i  1 )( x i  x i  1 )
2
 f ( xi1 )
( xi1  x i  1 )( x i  1  x i )
Department of Computer Science