Chapter 1

Chapter 1 : Error Analysis

1
Chapter 2

Chapter 2 : Numerical
resolution of an algebraic
equation

2
Chapter 3

Chapter 3 : Interpolation
and Approximation

3.1 Introduction
One of the most useful and well-known classes of functions mapping the set of
real numbers into itself is the algebraic polynomials, the set of functions of the
form
P
n
Pn (x) = ak xk ;
k=0

where n is a nonnegative integer and a0 ; an , are real constants. One reason

for their importance is that they uniformly approximate continuous functions.
By this we mean that given any function, de…ned and continuous on a closed and
bounded interval, there exists a polynomial that is as “close”to the given func-
tion as desired. This result is expressed precisely in the following Weierstrass
Approximation Theorem:

Theorem 1 (Weierstrass Approximation Theorem) Suppose that f is de…ned

and continuous on [a; b]. For each " > 0, there exists a polynomial P (x), with
the property that
jf (x) P (x)j < "; for all x 2 [a; b] :

Another important reason for considering the class of polynomials in the

approximationof functions is that the derivative and inde…nite integral of a
polynomial are easy to determine and are also polynomials. For these reasons,
polynomials are often used for approximating continuous functions.

3
 Part I

Interpolation

4
3.2 Interpolation Problem
The basic interpolation problem can be posed in one of two ways:
1: Given a set of nodes fxi ; 0 i ng and corresponding data values
fyi ; 0 i ng, …nd the polynomial pn (x) of degree less than or equal to n,
such that
pn (xi ) = yi ; 0 i n:
2: Given a set of nodes fxi ; 0 i ng and a continuous function f (x), …nd
the polynomial pn (x) of degree less than or equal to n, such that

pn (xi ) = f (xi ); 0 i n:

Note that in the …rst case we are trying to …t a polynomial to the data, and in
the second we are trying to approximate a given function with the interpolating
polynomial.

3.3 Lagrange Interpolation

Theorem 2 (Lagrange Interpolation) Suppose that n 1: Let the distinct nodes
xi 2 [a; b] ; 0 i n be given, then there exists a unique polynomial pn ; of degree
less than or equal to n, which satis…es either of

pn (xi ) = yi ; 0 i n:

for a given set of data values yi ; or

pn (xi ) = f (xi ); 0 i n:

for a given continous function in [a; b].

Proof. We begin by de…ning the family of the following Lagrange polynomials

n
Y x xk
Lni (x) = ; for all 0 i n;
xi xk
k=0;k6=i

which have the interesting property :

1; if i = j;
Lni (xj ) = ij (x) = ; for all 0 j n;
0; if i 6= j;

where ij denotes the Kronecker symbol.

Then, if we consider the following polynomial
n
X
pn (x) = Lni (x)yi ;
i=0

This polynomial, called the nth Lagrange interpolating polynomial.

5
 Then, for all j : 0 j n we obtain
n
X
pn (xj ) = ij (x) yi = yi :
k=0

Thus, the interpolation conditions are satis…ed, and it remains only to prove
the uniqueness of the polynomial.
To this end, assume that there exists a second interpolation polynomial qn
of degree less than or equal to n.
Then
rn (xi ) = pn (xi ) qn (xi ) = 0; 8i : 0 i n:
Thus, rn (x) is a polynomial of degree less than or equal to n that has n + 1
roots, which, according to the fundamental theorem of algebra, implies that
rn (x) = 0: Consequently,
pn (x) = qn (x) ;
and thus pn is unique.

Remark 3 Although the statement of previous theorem required that n 1,

the trivial case of n = 0 mentioned at the beginning of the section can also be
included by de…ning, for n = 0; L0 (x) 1, and observing that the function p0
de…ned by p0 (x) = L0 (x)y0 y0 is the unique polynomial of degree than or equal
to 0 that satis…es p0 (x0 ) = y0 .

Example 4 To illustrate this construction let f (x) = ex and consider the prob-
lem of constructing an approximation to this function on the interval [ 1; 1],
using the nodes :
1
x0 = 1; x1 = 2 ; x2 = 0; x3 = 21 ; x4 = 1:

Solution 5 Using

Q
4 x xk
L4i (x) = ;0 i 4
k=0;k6=i xi xk

to have
x x1 x x2 x x3 x x4
L40 (x) = =
x0 x1 x0 x2 x0 x3 x0 x4
2 1 1
= 3 x+ 2 (x) x 2 (x 1) ;

x x0 x x2 x x3 x x4
L41 (x) = =
x1 x0 x1 x2 x1 x3 x1 x4
8 1
= 3 (x + 1) (x) x 2 (x 1) ;

6
 x x0 x x1 x x3 x x4
L42 (x) = =
x2 x0 x2 x1 x2 x3 x2 x4
1 1
= 4 (x + 1) x + 2 x 2 (x 1) ;

x x0 x x1 x x2 x x4
L43 (x) = =
x3 x0 x3 x1 x3 x2 x3 x4
8 1
= 3 (x + 1) x + 2 (x) (x 1) ;

x x0 x x1 x x2 x x3
L44 (x) = =
x4 x0 x4 x1 x4 x2 x4 x3
2 1 1
= 3 (x + 1) x + 2 (x) x 2 :

From where, it follows

4
X
p4 (x) = L4i (x) exp(xi ) = 1+0:997853749x+0:499644939x2 +0:177347443x3 +0:043435696x4 :
i=0

The approximation is su¢ ciently accurate that it is impossible to discern that

there are two curves present, so it is probably better to look at Figures, which
plots the error E(x) = f (x) p4 (x).

3 3
y y
2 2

1 1

-1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0
x x
p4 (x) exp(x)

y 4

-1.0 -0.5 0.5 1.0

-2 x
-4

E(x) = exp x p4 (x)

7
Example 6 Find the polynomial of degree 2 that interpolates at the data points
xi 0 1 4
yi 1 2 2
Use this polynomial to approximate f (2):

Solution 7 We have
2
X 2
Y x xk
p2 (x) = L2i (x)yi ; where L2i (x) = ; i = 0; 1; 2:
i=0
xi xk
k=0;k6=i

Thus, we get
1 2
p2 (x) = 4x + 54 x + 1:
So,
f (2) ' p2 (2) = 52 :

Exercise 8 Let Pn be the vector space of polynomials of degree less than or

equal to n; i:e:
Pn = fqn : deg qn ng :
Show that fLni ; i = 0; :::; ng forms a basis of Pn :

Solution 9 See Worksheet 03

3.4 Newton Interpolation and divided di¤erences

The Lagrange form of the interpolating polynomial gives us a very tidy con-
struction, but it does not lend itself well to actual computation. With Lagrange
form, it is inconvenient to pass from one interpolation polynomials to another
of degree one greater. Such a comparison of di¤erent degree interpolation poly-
nomials is a useful technique in deciding what degree polynomial to use.
An alternate form of the polynomial, known as the Newton form, avoids this
problem, and allows us to easily write pn+1 in terms of pn .

Theorem 10 (Newton Interpolation Construction) Let pn be the polynomial

that interpolates f at the nodes xi ; i = 0; 1; :::; n. The polynomial pn+1 that
interpolates f at the nodes xi ; i = 0; 1; :::; n + 1 is given by

p0 (x) = a0 = f (x0 ) ;
pn+1 (x) = pn (x) + an+1 n (x);

where
Q
n
n (x) = (x xi ) ;
i=0
f (xn+1 ) pn (xn+1 )
an+1 = n (xn+1 )
;

8
Proof. Since we know that the interpolation polynomial is unique, all we have
to do is show that pn+1 , as given in this theorem, satis…es the interpolation
conditions.
For all k = 0; 1; :::; n we have

n (xk ) = 0;

so
pn+1 (xk ) = pn (xk ) + an+1 n (xk ) = pn (xk ):
| {z }
=0

Hence, pn+1 interpolates f at all nodes xi ; i = 0; 1; :::; n but the last point.
So, it remains to check the interpolation at point xn+1 ; thus we have

pn+1 (xn+1 ) = pn (xn+1 ) + an+1 n (xn+1 ) =

f (xn+1 ) pn (xn+1 )
= pn (xn+1 ) + n (xn+1 )
n (xn+1 ) =
= pn (xn+1 ) + f (xn+1 ) pn (xn+1 ) = f (xn+1 ):

Thus, pn+1 is a polynomial of degree less than or equal to n + 1 and satis…es

pn+1 (xi ) = f (xi ); for all nodes xi ; i = 0; 1; :::; n + 1:

For reason divided below, the coe¢ cients

(
a0 = f (x0 ) ;
an = f (xn )n p1n(xn1)(xn ) ; for all n 1:

is called the nth order Newton divided di¤erence of f and it is denoted by

f [x0 ; x1 ; : : : ; xn ] :

Also, we can construct an algorithm for computing them. It is also possible

to slightly modify this algorithm to allow the user to update an existing set of
divided di¤erence coe¢ cients by adding new points to the set of nodes.
Thus, we obtain the following Newton interpolation formula

pn (x) = f (x0 )+f [x0 ; x1 ] (x x0 )+f [x0 ; x1 ; x2 ] (x x0 ) (x x1 )+ +f [x0 ; : : : ; xn ] n 1 (x):

How to calculate the divided di¤erences ?

We start by 1st order : We have

f (x1 ) = f (x0 ) + f [x0 ; x1 ] (x1 x0 ) :

So,
f (x1 ) f (x0 )
f [x0 ; x1 ] = :
x1 x0

9
 For 2nd order, we get

f (x2 ) = f (x0 ) + f [x0 ; x1 ] (x2 x0 ) + f [x0 ; x1 ; x2 ] (x2 x0 ) (x2 x1 ) :

Thus,

(f (x2 ) f (x1 )) + (f (x1 ) f (x0 )) f [x0 ; x1 ] (x2 x0 )

f [x0 ; x1 ; x2 ] = =
(x2 x0 ) (x2 x1 )
f [x1 ; x2 ] (x2 x1 ) + f [x0 ; x1 ] (x1 x0 ) f [x0 ; x1 ] (x2 x0 )
=
(x2 x0 ) (x2 x1 )
f [x1 ; x2 ] f [x0 ; x1 ]
= :
(x2 x0 )

Then by, inductively, it results

f [x1 ; : : : ; xn ] f [x0 ; : : : ; xn 1]
f [x0 ; x1 ; : : : ; xn ] = ; for all n 2:
(xn x0 )

Example 11 1) Use the divided di¤ erences to …nd the polynomial of degree 2
that interpolates f at the following data points
xi 0 1 4
f (xi ) 1 2 2
2) Deduce the polynomial of degree 3 by adding a new point x3 = 5; y4 = 3:

Solution 12 1)
0 1
2 1
f [x0 ; x1 ] = 1 0 =1
0 1 1
1 2 f [x0 ; x1 ; x2 ] = 4 0 = 4
2 2
f [x1 ; x2 ] = 4 1 =0
4 2
Thus,

p2 (x) = f [x0 ] + f [x0 ; x1 ] (x x0 ) + f [x0 ; x1 ; x2 ] (x x0 )(x x1 )

1 1 2 5
= 1 + (x 0) (x 0)(x 1) = x + x + 1:
4 4 4
2) To deduce p3 , we consider the following divided di¤ erences

0 1
2 1
f [x0 ; x1 ] = 1 0 =1
0 1 1
1 2 f [x0 ; x1 ; x2 ] = 4 0 = 4
1 1
2 2 4+4 1
f [x1 ; x2 ] = 4 1 =0 f [x0 ; x1 ; x2 ; x3 ] = 5 0 = 10
1 0 1
4 2 f [x1 ; x2 ; x3 ] = 5 1=4
3 2
f [x2 ; x3 ] = 5 4= 1
5 3

10
 So,

p3 (x) = p2 (x) + f [x0 ; x1 ; x2 ; x3 ] (x 0)(x 1)(x 4) =

1 2
= 4x + 54 x + 1 + 1
10 (x 0)(x 1)(x 4) =
1 3 3 2 33
= 10 x 4x + 20 x + 1:

Example 13 Using divided di¤ erences to show that

n
X n (n + 1) (2n + 1)
k2 = ; all n 1:
6
k=1

Proposition 14 Suppose x0 ; x1 ; :::; xn are distinct numbers in the interval [a; b]

and f 2 C n ([a; b]). Then, there is a number " between x0 ; x1 ; :::; xn , and hence
in [a; b] such that
f (n) (")
f [x0 ; x1 ; :::; xn ] = :
n!
Proof. Let
g(x) = f (x) pn (x);
where pn (x) is a ploynomial that interpolates f at x0 ; x1 ; :::; xn :
Since f 2 C n ([a; b]); and pn (t) 2 C 1 ([a; b]) then g 2 C n ([a; b]) satis…es
g(xi ) = 0; i = 0; :::; n: Then according to the generalized Roll’s Theorem, there
is " 2 [a; b] such that
g (n) (") = 0:
Thus,
f (n) (") = p(n)
n ("):

Since

pn (x) = f [x0 ] + (x x0 ) f [x0 ; x1 ] + ::: + f [x0 ; :::; xn ] n 1 (x);

then
p(n)
n (") = n!f [x0 ; :::; xn ]; for all x 2 [a; b]:

Consequently,
f (n) (")
f [x0 ; :::; xn ] = :
n!

Proposition 15 The divided di¤ erences f [x0 ; x1 ; :::; xn ] are symmetric func-
tions of their arguments, i:e:, there are invariant to permutations of the nodes
x0 ; x1 ; :::; xn :

Comments Lagrange method is mostly a theoretical tool used for proving

theorems. Not only it is not very e¢ cient when a new point is added (which
requires computing the polynomial again, from scratch), it is also numerically
unstable. Therefore, Newton’s method is usually used.

11
3.5 Polynomial Interpolation Error
Theorem 16 Suppose x0 ; x1 ; :::; xn are distinct numbers in the interval [a; b]
and f 2 C n+1 ([a; b]). Then, for each x in [a; b], a number " (x) (generally
unknown) between x0 ; x1 ; :::; xn , and hence in [a; b] exists with

f (n+1) (" (x))

f (x) pn (x) = n (x):
(n + 1)!

We can estimate the error as follows

Mn+1
jf (x) pn (x)j max j n (x)j ; with Mn+1 = max f (n+1) (x) :
(n + 1)! x2[a;b] x2[a;b]

Proof. In order to prove this theorem, we observe that it is trivially true if

x = xi ; i = 0; 1; :::; n:
Thus, it su¢ ces to consider x 2 [a; b] such that x 6= xi ; i = 0; 1; :::; n: For
such x, let’s de…ne the function g for t 2 [a; b] by

n (t)
g(t) = f (t) pn (t) (f (x) pn (x)) :
n (x)

Since f 2 C n+1 ([a; b]); and pn (t) 2 C 1 ([a; b]) then g 2 C n+1 ([a; b]):
As for t = xk , we have

f (xk ) = pn (xk ); for all k = 0; :::; n

and since
n (xk ); for all k = 0; :::; n;
then, it follows

n (xk )
g(xk ) = f (xk ) pn (xk ) (f (x) pn (x)) = 0; for all k = 0; :::; n
n (x)
and
n (x)
g(x) = f (x) pn (x) (f (x) pn (x)) = 0:
n (x)

Hence g 2 C n+1 ([a; b]); and is zero at n + 2 distinct points x; xk: ; k = 0; ::::; n:
According to generalized Rolle’s theorem, there is a number " (x) between
xk: ; k = 0; ::::; n, such that

g (n+1) (" (x)) = 0:

So
1 (n+1)
g (n+1) (" (x)) = f (n+1) (" (x)) p(n+1)
n (" (x)) (f (x) pn (x)) ( n (t))t="(x) = 0:
n (x)

12
As deg pn n and deg ( n (t)) = n + 1, then

p(n+1)
n (t)t="(x) = 0;
(n+1)
( n (t))t="(x) = (n + 1)!:

From where, it becomes

(n + 1)!
f (n+1) (" (x)) (f (x) pn (x)) = 0;
n (x)

which completes the proof, by writing

f (n+1) ("(x))
f (x) pn (x) = (n+1)! n (x):

Example 17 (a) Use the nodes x0 = 2; x1 = 2:75; and x2 = 4 to …nd the

second Lagrange interpolating polynomial for f (x) = 1=x.
(b) Use this polynomial to approximate f (3) = 1=3:
(c) Determine the error form for this polynomial, and the maximum error
when the polynomial is used to approximate f (x) for x 2 [2; 4]:

Solution 18 (a) We determinate the coe¢ cient polynomials L2i ; i = 0; 1; 2: In

nested form they are

x x1 x x2
L20 (x) = = 1
( 0:75)( 2) (x 2:75) (x 4) ;
x0 x1 x0 x2
x x0 x x2
L21 (x) = = 1
(0:75)( 1:25) (x 0) (x 4) ;
x1 x0 x1 x2
x x0 x x1
L22 (x) = = 1
(2)(1:25) (x 0) (x 2:75) ;
x2 x0 x2 x1
so,
2
X
p2 (x) = L2i (x)f (xi ) = 1 2
22 x
35
88 x + 49
44 :
i=0

(b) An approximation to f (3) = 1=3 is

f (3) p2 (3) = 0:32955:

(c) The second Lagrange polynomial has the error form

f 000 (" (x))

f (x) p2 (x) = 2 (x);
3!
where
6
f 000 (x) = ;
x4

13
which has a maximum value 264 = 83 in [2; 4] :
Now, it remains to determinate the maximum of j 2 (x)j in [2; 4] :
So, we calculate the critical points by solving the equation
0
( 2 (x)) = 0:

We have
0 35 49
( 2 (x)) = 3x2 x+ = 0;
2 2
which gives x = 7=2 or x = 7=3:
Hence,

max j( 2 (x))j = max (jg (7=2)j ; jg (7=3)j ; jg (2)j ; jg (4)j)

[2;4]

= jg (7=2)j = j 9=16j :

Consequently,
6 1 9
jf (x) p2 (x)j 0:03516:
24 6 16

3.6 Hermite interpolation

So far, we have studied interpolation that involved only knowledge of function
values. What happens if we include information about the derivative
as well ?
To be speci…c, given the distinct interpolation points xi ; i = 0; : : : ; n; and
two sets of real numbers yi ; (or f (xi ), and zi ,(or f 0 (xi )) ; i = 0; 1; : : : ; n; we need
to …nd a polynomial p2n+1 of degree less than or equal to 2n + 1 such that

p2n+1 (xi ) = yi ; or (= f (xi )) ; p02n+1 (xi ) = zi ; or (= f 0 (xi )) ; 1 i n:

This is known as the Hermite interpolation problem.

The construction is similar to that of the Lagrange interpolation polynomial,
but now requires two sets of polynomials Hk and Kk with k = 0; :::; n; these
will be de…ned in the proof of the next theorem.

Theorem 19 (Hermite Interpolation Theorem) Let n 0, and suppose

that xi ; i = 0; : : : ; n; are distinct nodes. Then, given two sets of real values yi ;
zi ; i = 0; : : : ; n; there is a unique p2n+1 of degree less than or equal to 2n + 1
such that
p2n+1 (xi ) = yi ; p02n+1 (xi ) = zi ; 1 i n:

Proof. Let us begin by supposing that n 1. As in the case of Lagrange in-

terpolation, we start by constructing a set of auxiliary polynomials; we consider
the polynomials Hi and Ki with i = 0; :::; n; de…ned by
2
Hi (x) = (Lni (x)) (1 2L0i (xi ) (x xi )) ;
2
Ki (x) = (Lni (x)) (x xi ) ;

14
where
n
Y x xj
Lni (x) = :
xi xj
i=0;j6=k

Clearly Hi and Ki with i = 0; :::; n; are polynomials of degree less or equal to

2n + 1:
Since Lni (xj ) = 0; i 6= j; then we have

Hi (xj ) = Ki (xj ) = 0;
for all i; j = 0; 1; : : : ; n and i 6= j:
Hi0 (xj ) = Ki0 (xj ) = 0;

Moreover, a straightforward calculation veri…es their values when i = j; showing

that

Hi (xi ) = 1; and Hi0 (xi ) = 0; for i = 0; 1; : : : ; n;

Ki (xi ) = 0; and Ki0 (xi ) = 1; for i = 0; 1; : : : ; n:

Which means
Hi (xj ) = Ki0 (xj ) = ij ;
; for all i; j = 0; 1; : : : ; n:
Hi0 (xj ) = Ki (xj ) = 0;

We deduce that
n
X
p2n+1 (x) = [Hi (x) yi + Ki (x) zi ] ; or
i=0
Xn
p2n+1 (x) = [Hi (x) f (xi ) + Ki (x) f 0 (xi )] ;
i=0

satis…es the interpolation conditions

p2n+1 (xi ) = yi ; p02n+1 (xi ) = zi ; 1 i n:

To show the uniqueness, we suppose otherwise; then, there exists a polyno-

mial q2n+1 of degree less than equal to 2n + 1 distinct from p2n+1 , such that
0
q2n+1 (xi ) = yi ; q2n+1 (xi ) = zi ; 1 i n:

Consequently, p2n+1 (x) q2n+1 (x) has n + 1 distinct zeros; therefore, Rolle’s
Theorem implies that

p02n+1 (ci ) 0
q2n+1 (ci ) = 0; with ci 2 ]xi ; xi+1 [ ; i = 0; 1; :::; n 1;

which means that, in addition to the n + 1 zeros xi ; i = 0; 1; :::; n; p02n+1 (x)

0
q2n+1 (x) vanishes at another n points ci ; i = 0; :::; n 1:
Hence p02n+1 (x) q2n+1
0
(x) is a polynomial of degree less than equal to 2n,
and has 2n + 1 zeros, which means that

p02n+1 (x) 0
q2n+1 (x) = 0;

15
so that p2n+1 (x) q2n+1 (x) is a constant function.
However, p2n+1 (xi ) q2n+1 (xi ) = 0 for i = 0; : : : ; n; and hence
p2n+1 (x) q2n+1 (x) 0;
contradicting the hypothesis that p2n+1 and q2n+1 are distinct. Thus, p2n + 1
is unique.
When n = 0; we de…ne H0 (x) 1 and K0 (x) x x0 ; which correspond
to taking L0 (x) 1. Clearly,
p1 (x) = H0 (x) y0 + K0 (x) z0 = y0 + (x x0 ) z0
is the unique polynomial of degree less than or equal to 1 such that p1 (x0 ) = y0
and p01 (x0 ) = z0 .
De…nition 20 Let n 0, and suppose that xi ; i = 0; : : : ; n; are distinct nodes.
Then, given two sets of real numbers yi ; zi ; i = 0; : : : ; n; are real values. The
polynomial p2n+1 of degree less than or equal to 2n + 1 de…ned by
n
X
p2n+1 (x) = [Hi (x) yi + Ki (x) zi ] ;
i=0

where
2 2
Hi (x) = (Li (x)) (1 2L0i (xi ) (x xi )) ; and Ki (x) = (Li (x)) (x xi )
is called the Hermite interpolation polynomial for the set of values given
in f(xi ; yi ; zi ) ; i = 0; 1; :::; ng :
Example 21 We shall construct a cubic polynomial p3 such that
p3 (0) = y0 = 0; p3 (1) = y1 = 1; p03 (0) = z0 = 1 and p03 (0) = z1 = 0:
Since n 1 and y0 = z1 = 0; y1 = z0 = 1 then
1
X
p3 (x) = [Hi (x) yi + Ki (x) zi ]
i=0
= K0 (x) z0 + H1 (x) y1
= K0 (x) + H1 (x) :
We easily …nd that, with n = 1; x0 = 0 and x1 = 1;
L0 (x) = (x 1) ; L1 (x) = x;
and then
2
H1 (x) = (L1 (x)) (1 2L01 (x1 ) (x x1 )) = x2 (1 2 (x 1)) = x2 (3 2x) ;
2 2
K0 (x) = (L0 (x)) (x x0 ) = x (x 1) :
These yield the required Hermite interpolation polynomial,
p3 (x) = x3 + x2 + x:

16
Theorem 22 (Hermite Interpolation Error Theorem) Suppose that n 0
and let f 2 C 2n+2 ([a; b]) be a real-valued function. Further, let p2n+1 denote
the Hermite interpolation polynomial of f . Then, for each x 2 [a; b] there exists
"(x) in ]a; b[ such that

f (2n+2) (" (x)) 2

f (x) p2n+1 (x) = [ n (x)] ;
(2n + 2)!

where
n
Y
n (x) = (x xk ) :
k=0

Moreover,
M2n+2 2
jf (x) p2n+1 (x)j max [ n (x)] ;
(2n + 2)! [a;b]
where M2n+2 = max f (2n+2) (x) :
x2[a;b]

Proof. In order to prove this theorem, we observe that it is trivially true if

x = xi ; i = 0; 1; :::; n: thus, it su¢ ces to consider x 2 [a; b] such that x 6= xi ; i =
0; 1; :::; n: For such x, let’s de…ne the function for t 2 [a; b] by
2
[ n (t)]
(t) = (f (t) p2n+1 (t)) (f (x) p2n+1 (x)) 2:
[ n (x)]

Then
(x) = (xi ) = 0; i = 0; 1; :::; n:
Hence, by Rolle’s Theorem, 0 (t) vanishes at n + 1 points which lie strictly
between each pair of consecutive points from the set fx; x0 ; :::; xn g :
Also, 0 (t) de…ned by
0
0 2 n (t) n (t)
(t) = f 0 (t) p02n+1 (t) (f (x) p2n+1 (x)) 2
[ n (x)]

vanishes at xi ; i = 0; 1; :::; n:
Hence 0 (t) vanishes at a total of 2n + 2 distinct points in [a; b]. Applying
Rolle’s Theorem repeatedly, we …nd eventually that (2n+2) (t) = 0 at some
point " in ]a; b[, the location of " being dependent on the position of x. This
implies

(2n+2) (2n+2) f (x) p2n+1 (x) (2n+2)

(") = [f (t) p2n+1 (t)]t=" 2 [ n (t)]t=" = 0:
[ n (x)]

Since
(2n+2) (2n+2)
p2n+1 (t) = 0; [ n (t)]t=" = (2n + 2)!;
then it follows the required result.

17
3.7 Di¤erentiation
From the Lagrange interpolation polynomial pn , which is an approximation to
f , it is easy to obtain the polynomial p0n , which is an approximation to the
derivative f 0 . The polynomial p0n is given by
n
X 0
p0n (x) = (Lni ) (x)f (xi ); n 1:
i=0

The degree of the polynomial p0n is clearly at most n 1; p0n is a linear

combination of the derivatives of the polynomials Lni , the coe¢ cients being the
values of f at the interpolation points xk ; k = 0; 1; :::; n:
In order to …nd an expression for the error, we might simply the expression
of the previous polynomial error to give

d f n+1 (" (x))

f 0 (x) p0n (x) = n (x) :
dx (n + 1)!

However, the result is not helpful: on application of the chain rule, the right-
hand side involves the derivative d"(x)
dx ; the value of " depends on x, but not in
any simple manner. In fact, it is not a priori clear that the function x 7! "(x)
is continuous, let alone di¤erentiable. An alternative approach is given by the
following theorem.

Theorem 23 Let n 1, Suppose x0 ; x1 ; :::; xn are distinct numbers in the

interval [a; b] and f 2 C n+1 ([a; b]). Suppose further that pn is the Lagrange
interpolation polynomial for f at these nodes.Then, there exist distinct nodes
i ; i = 1; :::; n; in [a; b], and corresponding to each x in [a; b] there exists a point
" (x) in [a; b], such that

f (n+1) (" (x))

f 0 (x) p0n (x) = n (x); (3)
n!
where
n
Y
n (x) = (x k) :
k=1

Proof. Since
q(x) = f (xi ) pn (xi ) = 0; i = 0; 1; :::; n;
then by using Roll’s theorem there exist i 2 ]xi 1 ; xi [ such that q 0 ( i ) = 0; for
each i ; i = 1; :::; n: This de…nes the points i ; i = 1; :::; n:
When x = i ; i = 1; :::; n, both sides of (3) are zero. Suppose then that x is
distinct from all the i ; i = 1; :::; n; and de…ne the function (t) by

n (t)
(t) = q 0 (t) q 0 (x) :
n (x)

18
This function vanishes at every point i ; i = 1; :::; n; and also at the point t = x:
By successively applying Rolle’s Theorem we deduce that (n) (t) vanishes at
some point ".
As deg pn n and deg ( n (t)) = n, then
(n)
(n) n (t)
(t) = q 0 (t) q 0 (x)
n (x) t="(x)
(n)
0 (n)
( n (t))t="(x)
= (f (t) p0n (t))t="(x) 0
(f (x) p0n (x)) = 0:
n (x)

Hence,
n!
f (n+1) (" (x)) p(n+1) (" (x)) (f 0 (x) p0n (x)) =0
| n {z } n (x)
=0

From where, it becomes

n!
f (n+1) (" (x)) (f 0 (x) p0n (x)) = 0:
n (x)

The result then follows as in the proof of the Theorem.

19
 Part II

Approximation

20
3.8 Introduction
An important area in approximation is the problem of …tting a curve to experi-
mental data. Since the data is experimental, we must assume that it is polluted
with some degree of error, most commonly measurement error, so we do not
necessarily want to construct a curve that goes through every data point, it is
the case of Interpolation. Rather, we want to construct a function that repre-
sents the "sense of the data" and which is, in some sense, a close approximation
to the data.

3.8.1 Normed vector spaces

De…nition 24 (Normed vector space) Suppose that E is a vector space over the
…eld | (| = R; or C). The nonnegative real-valued function k:k is said to be a
norm on the space E provided that it satis…es the following axioms:
kxk 0; kxk () x = 0; for all x 2 E;
k xk = j j kxk for all 2 | and all x 2 E;
kx + yk kxk + kyk for all x; y 2 E (the triangle inequality).
A vector space E, equipped with a norm, is called a normed vector space.
Example 25 Let E = Rn ; n 1: Three vector norms are in common use in
numerical linear algebra :
n
X
kxk1 = jxi j ; for all x 2 Rn ;
i=1
" n
# 12
X 2
kxk2 = (xi ) ; for all x 2 Rn ;
i=1
n
kxk1 = max jxi j ; for all x 2 Rn ;
i=1

It is easy to show that kxk1 and kxk1 obey all axioms of a norm. For the
kxk2 the …rst two axioms are still trivial to verify; to show that the triangle
inequality is satis…ed by the kxk2 requires use of the following Cauchy–Schwarz
inequality :
Xn
xi yi kxk2 kyk2 8x; y 2 Rn :
i=1

Example 26 The set C [a; b] = ff : [a; b] ! R continuousg is a normed vector

space with norm
kf k1 = max jf (x)j ; for all f 2 C [a; b] :
a x b

The norm k:k1 is called the 1 norm or maximum norm.

De…nition 27 " Weight function " A real-valued function ! is said to be a
weight function it it is de…ned, continuous, positive and integrable on the interval
]a; b[ :

21
 The assumptions on ! allow for singular weight functions, such as w(x) =
p 1 , which is continuous, positive and integrable on the open interval ] 1; 1[,
1 x2
but is not continuous on the closed interval [ 1; 1].
Example 28 Let ! be a weight function. Then C[a; b] is a normed vector space
equipped with the norm
"Z # 21
b
2
kf k2 = !(x) jf (x)j dx :
a

The norm kf:k2 is called the 2 norm.

The next result provides a comparison of the 1 norm with the 2 norm,
de…ned on C[a; b].
Lemma 29 (i) For any function f 2 C[a; b] we have
kf k2 K kf k1 ;
hR i 21
b
where K = a
!(x)dx and ! is a weight function.

Lemma 30 (ii) Given any two positive numbers " (however small) and M
(however large), there exists a function f 2 C[a; b] such that
kf k2 < "; kf k1 > M:
Theorem 31 (Weierstrass Approximation Theorem) Suppose that f 2 C[a; b]:
Given any " > 0; there exists a polynomial p such that
kf pk1 < ":
Further, if ! is a weight function, then for any " > 0; there exists a polyno-
mial q such that
kf qk2 < "
Proof. This is an important theorem in classical analysis, and several proofs
are known. It is evidently su¢ cient to consider only the interval [0; 1]; a simple
change of variable will then extend the proof to any bounded closed interval
[a; b]. For a real-valued function f, de…ned and continuous on the interval [0; 1],
Bernstein’s proof uses the polynomial
n
X
k
pn (x) = pnk (x) f n ; x 2 [0; 1] ;
k=0

where the Bernstein polynomials pnk (x) are de…ned by

n k
pnk (x) = Cnk xk (1 x) ; x 2 [0; 1] :
It can then be shown that, for any " > 0, there exists n = n (") such that
kf pk1 < ".

22
3.8.2 Orthogonal Polynomials
Orthogonal polynomials are of fundamental importance in many branches of
mathematics in addition to approximation theory and their applications are
numerous but we shall be mainly concerned with two special cases, the Legendre
polynomials and the Chebyshev polynomials. More general applications are
however easily worked out once the general principles have been understood.
De…nition 32 (Vector Space of Polynomials ) For any n 0 de…ne Pn as
the vector space of polynomials of degree less than or equal to n: Note that
this space has a standard basis consisting of 1; x; x2 ; :::; xn , and thus is an
(n + l) dimensional space, i.e.
Pn = fp a polynomial : deg p ng :
De…nition 33 (Inner Product on Real Vector Spaces) Let f and g be elements
of a real vector space V . Let hf; gi denote any operation on f and g that satis…es
the following three properties :
1) hf; f i 0; for all f 2 V; and hf; f i = 0 () f = 0;
2) hf; gi = hg; f i ; for all f; g 2 V ;
3) hf; g + hi = hf; gi + hf; hi ; for all f; g; h 2 V and all scalairs ; :
Then hf:; :i is called inner product on V .
If we want to consider our overlying vector space to be C([a; b]), that is,
continuous functions on a closed interval, then we can easily establish that
the positively weighted integral of a product of two functions will be an inner
product.
Example 34 Let ! be weight function on the interval (a; b). Then the following
expression de…ned by
Zb
hf; gi! = f (x)g(x)!(x)dx; for all f; g 2 C ([a; b])
a

is an inner product on C ([a; b]) :

It is easy to see that
Zb
2
hf; f i! = f 2 (x)!(x)dx = kf k2 :
a

De…nition 35 A system of real functions '0 (x); '1 (x); .... de…ned in C ([a; b])
is said to be orthogonal with respect to the weight function !(x) over [a; b] if
0; if i 6= j;
'i (x); 'j (x) =
! i 6= 0; if i = j:

If i = 1; for all i; the system is said to be normal. An orthogonal system

which is also normal is sometimes referred to as an orthonormal system.
At this point it might be useful to look at some examples of orthogonal
polynomial families.

23
3.8.3 Examples
Legendre Polynomials
De…nition 36 The Rodrigues’ Formula for generating the Legendre polynomi-
als is given by
1 dn h 2 n
i
Pn (x) = n x 1 ; n 0:
2 n! dxn
From the de…nition given above, it will be observed that an nth derivative
must be carried out before a polynomial of degree n is obtained. Thus the …rst
few set of Legendre polynomials can be obtained as follows:

P0 (x) = 1;
P1 (x) = x;
P2 (x) = 1
2 3x2 1 ;
1 3
P3 (x) = 2 5x 3x ;
4
P4 (x) = 1
8 3x 30x2 + 3 :

The recurrence formula for the Legendre Polynomial Pn (x) is given by the
equation

P0 (x) = 1; P1 (x) = x;
2n+1 n
Pn+1 (x) = n+1 xPn (x) n+1 Pn 1 (x); for all n 1:

where pn (x) is known to have satis…ed the Legendre di¤erential equation

(1 x2 )Pn00 (x) 2xPn0 (x) + n(n + 1)Pn (x) = 0:

It is known that the Legendre Polynomial pn (x) satis…es the following prop-
erty:
Z
+1
0; if m =6 n;
hPn (x); Pm (x)i = Pn (x)Pm (x)dx = 2
2n+1 ; if m = n:
1

This is the orthogonality property which permits it to be a polynomial ap-

proximation to any continuous function within its range [ 1; 1].
n
X
Remark 37 All polynomial q(x) = am xm de…ned in [ 1; +1] ; can be ap-
m=0
proximated with a series of Legendre polynomials by writing all polynomial terms
xm as follows
Xm
xm = ak Pk (x):
k=0

Which means that the system fPk gk=0;:::;n forms an orthogonal basis of Pn =
fp a polynomial : deg p ng :

24
Example 38 Find an approximation q(x) = 4x3 + 3x2 2 by using 3 order
Legendre Polynomials

Solution 39 Use the …rst few set of Legendre polynomialsthe to get

1 = P0 (x) ;
x = P1 (x) ;
1 2P2 (x)+P0 (x)
P2 (x) = 3x2 1 =) x2 = 3 ;
2
1 2P3 (x)+3P1 (x)
P3 (x) = 5x3 3x =) x3 = 5 :
2
Then
2P3 (x)+3P1 (x) 2P2 (x)+P0 (x)
q(x) = 4 5 +3 3 2P0 (x) :

Hence

q(x) = 58 P3 (x) + 2P2 (x) + 12

5 P1 (x) P0 (x) ; for all x 2 [ 1; 1] :

Chebyshev polynomials
As it was earlier stated, Chebyshev polynomials are often useful in approximat-
ing some functions. For this reason we shall examine the nature, properties and
e¢ ciency of the Chebyshev polynomial.

De…nition 40 The Chebyshev polynomials are de…ned in terms of cosine func-

tion as
Tn (x) = cos (n arccos x) ; for 1 x 1; n 0:
If we put arccos x = , (or x = cos ) it becomes

Tn (x) = cos n :

1 Tn (x) 1; is of the orthogonal family of polynomials of degree n and

it has a weighting function

!(x) = p 1 ; 1 < x < 1:

1 x2

Proposition 41 1) For n 1; Tn is a polynomial in x of degree n on the

interval [ 1; 1], with leading coe¢ cient 2n 1 ; Thus the …rst few set of Chebyshev
polynomials can be obtained as follows:

T0 (x) = 1;
T1 (x) = x;
T2 (x) = 2x2 1;
3
T3 (x) = 4x 3x;
4
T4 (x) = 8x 8x2 + 1:

25
 2) Tn has an oscillatory property that in 0 the function has al-
ternating equal maximum and minimum values of Tn (x) = 1 at the n + 1
points
k k
k = n ; k = 0; 1; :::; n; or xk = cos n ; k = 0; 1; :::; n

2) For n 1; the zeros of Tn are at

cos(2j 1)
xj = 2n ; j = 0; 1; :::; n:
n
3) Tn ( x) = ( 1) Tn (x) is an even function on [ 1; 1] if n is even, and an
odd function on [ 1; 1] if n is odd, n 0;
4)The recurrence formula for generating the Chebyshev polynomials Tn (x)
in [ 1; 1] is given by

T0 (x) = 1; T1 (x) = x;
Tn+1 (x) = 2xTn (x) Tn 1 (x); n 1:

5) Thus the orthogonality relation of the Chebyshev polynomials is given as:

8
Z
+1
< 0; if n 6= m
hTn (x); Tm (x)i!(x) = Tnp (x)Tm (x)
dx = ; if n =m=0
1 x2 :
1 2 ; if n = m 6= 0

Proof. 1) We start by proving that Tn is a polynomial of degree n:

By using De Moivre’s formula we get
n
(cos + i sin ) = cos n + i sin n :

Hence
n
Tn (x) = cos n = Re (cos + i sin ) :
p p
By putting x = cos ; then sin = 1 cos2 = 1 x2 ; and hence

n
p n n
X p k
Re (cos + i sin ) = Re x + i 1 x2 = Re Cnk i 1 x2 xn k
:
k=0

From where, it becomes

X 0
p 2k0
2k0
Tn (x) = cos n = Cn2k i 1 x2 xn =
k=2k0 even
X 0 k0 k0 2k0
= Cn2k ( 1) 1 x2 xn =
k0 0
2
= Cn0 xn + Cn2 x2 1 xn 2
+ Cn4 x2 1 xn 4
+ :::;

which proves that Tn is a plolynomial of degree n on the interval [ 1; 1], with

leading coe¢ cient
Cn0 + Cn2 + Cn4 + = 2n 1 :

26
 4) Showing the recurrence formula :
It is easy to see that T0 (x) = 1; T1 (x) = 0; and we have

Tn+1 (x) + Tn 1 (x) = cos (n + 1) + cos (n 1) =

= 2 cos (n ) cos
x=cos
= 2xTn (x):

5) We have

Z+1
(cos(n arccos x))(cos(m arccos x))
hTn (x); Tm (x)i = p
1 x2
dx:
1

By putting = arccos x; it results

1
d = p
1 x2
dx;
x = 1 =) = ;
x = 1 =) = 0:

Hence, for n 6= m we have

Z Z
1
hTn (x); Tm (x)i = cos n cos m d = (cos (n + m) + cos (n m) ) d
2
0 0
1h 1 1
i
= sin (n + m) + m sin (n m) = 0:
2 n+m n
0

If n = m = 0, then
Z
1
hTn (x); Tm (x)i = 2d = :
2
0

If n = m 6= 0, then
Z
1 1 1
hTn (x); Tm (x)i = 2 (cos (2n) + 1) d = 2n sin (2n) + 0
= 2:
2
0

P
n
Remark 42 All polynomial p(x) = am xm de…ned in [ 1; 1] ; can be approx-
m=0
imated with a series of Chebyshev polynomials by writing all polynomial terms
xm as follows
P
m
xm = ak Tk (x):
k=0

Which means that the system fTk gk=0;:::;n forms an orthogonal basis of Pn =
fp a polynomial : deg p ng :

27
Example 43 Find an approximation of p(x) = 2x4 4x3 + x2 2x + 1 by using
4 order Chebyshev polynomials

Solution 44 By using the …ve polynomials Chebyshev, it follows

T2 (x) = 2x2 1;
T3 (x) = 4x3 3x;
4
T4 (x) = 8x 8x2 + 1:

1 = T0 ;
x = T1 ;
T2 + T0
T2 (x) = 2x2 1 =) x2 = ;
2
T3 + 3T1
T3 (x) = 4x3 3x =) x3 = ;
4
T2 +T0
T4 +8 2 T0
T4 (x) = 8x4 8x2 + 1 =) x4 = 8 = T4 +4T2 +3T0
8 :

Thus,

p(x) = 2x4 4x3 + x2 2x + 1 =

= 2 T4 +4T82 +3T0 4 T3 +3T
4
1
+ T2 +T0
2 2T1 + T0 =
1 3
= 4 T4 (x) T3 (x) + 2 T2 (x) 5T1 + 49 T0 (x) ; for all x 2 [ 1; 1] :

Example 45 Convert the …rst 5 terms of the Taylor series expansions for exp x
into Chebyshev polynomials.

Solution 46 We have

exp x = 1 + x + 12 x2 + 16 x3 + 1 4
24 x + ::::
2 3 T4 +4T2 +3T0 4
= T0 + T1 + 12 T2 +T
2
0
+ 61 T3 +3T
4
1
+ 1
24 8 =
81 9 13 1 1
= 64 T0 + 8 T1 + 48 T2 + 24 T3 + 192 T4 :

If we truncate this result after the term T3 (x) we shall obtain

81
exp x = 64 T0 + 98 T1 + 13
48 T2 + 1
24 T3
1
with the principal error as 192 T4 :::
This approximation can as well be regarded as the cubic expansion for exp x:

3.9 Least Squares Approximation

The least squares approach is a technique which is developed to reduce the sum
of squares of errors in …tting the unknown function y = f (x):The Least Squares
Approximation methods can be classi…ed into two, namely the discrete least

28
square approximation and the continuous least squares approximation. The
…rst involves …tting a polynomial function to a set of data points using the least
squares approach, while the latter requires the use of orthogonal polynomials
to determine an appropriate polynomial function that …ts a given function. For
these reasons, we shall treat them separately.

3.9.1 Discrete Least Squares approximation

The basic idea of least square approximation is to …t a polynomial function
pm (x) to a set of data pairs (xk ; yk ); 1 k n having a theoretical solution
y = f (x):
Thus, we want to …nd the coe¢ cients ai ; i = 0; 1; ::; m and the polynomial
P
m
pm (x) = ak xk
k=0

such that
m
!2
2 P
n X
I (a0 ; a1 ; :::; am ) = kf pm k2 = f (xi ) ak xki is minimized.
i=1 k=0

This is a problem from multivariable calculus:

We compute the critical points (a0 ; a1 ; :::; am ) by solving the following system
0 @I(a;b) 1 0 1
@a0 0
B @I(a;b) C B
B C 0 C
rI (a0 ; a1 ; :::; am ) = B @a1 C = B @
C;
@ ::: A ::: A
@I(a;b) 0
@am

where rI (a0 ; a1 ; :::; am ) is the gradient of I.

Thus, ai ; i = 0; 1; :::; n are solutions of the following system
8 Pn
>
> 2 yi a0 + a1 xi + x2i + ::: + am xm =0
>
> i
>
> i=1
>
> P
n
>
> 2 xi yi a0 + a1 xi + x2i + ::: + am xm =0
>
< i=1
i
Pn
>
> 2 x2i yi a0 + a1 xi + x2i + ::: + am xmi =0
>
> i=1
>
> :::
>
>
>
> Pn
>
: 2 x y n
a + a x + x2 + ::: + a xm = 0
i i 0 1 i i m i
i=1

This set of equations are called the Normal Equations of the Least Squares
Method.

Example 47 By using the Least Squares Approximation, …t

(a) a straight line

29
 (b) a parabola
to the given data below
xi 1 2 3 4 5 6
yi 120 90 60 70 35 11
Solution 48 (a) In order to …t a straight line to the set of data above, we
assume the equation of the form
p1 (x) = ao + a1 x:
We have to determine a0 and a1 by minimizing the sum of squares
2 P
n
2
I (a0 ; a1 ) = kf p1 k2 = (yi a0 a1 xi ) :
i=1

Then we have
8
> P
6
>
<
@I(a0 ;a1 )
= 2 (yi a0 a1 xi ) = 0;
@a0
i=1
>
> @I(a0 ;a1 ) P
6
: @a1 = 2 xi (yi a0 a1 xi ) = 0;
i=1

or 8
> P
6 P
6
>
< yi 6a0 a1 xi = 0;
i=1 i=1
>
> P
6 P
6 P
6
: xi yi a0 xi a1 x2i = 0:
i=1 i=1 i=1

Hence we shall need to construct columns for vales of xi yi and x2i in addition
to xi and yi values already given.
Thus the table below shows the necessary columns:
xi yi x2i xi yi
1 120 1 120
2 90 4 180
3 60 9 180
4 70 16 280
5 35 25 175
6 11 36 66
X6
21 386 91 1001
i=1
Thus, we have
386 = 6a0 + 21a1
1001 = 21a0 + 91a1
Solving this system, we obtain
p1 (x) = 134:33 20x
(b) In a similar manner, the parabola can be written as
p2 (x) = a0 + a1 x + a2 x2 :

30
Hence the required normal equations to determine the unknowns a0 ; a1 ; a2 are :
8 6 6 6
>
> X X X
>
> y = 6a + a x + a x2i
>
>
i 0 1 i 2
>
> i=1 i=1 i=1
>
< X 6 6 6 6
X X X
xi yi = a0 xi + a1 xi + a2 x3i 2
>
>
>
> i=1 i=1 i=1 i=1
> 6 6 6 6
> X 2
>
>
X
2
X
3
X
>
: x y
i i = a0 x i + a 1 x i + a 2 x4i
i=1 i=1 i=1 i=1

Thus, we construct the necessary columns in the following Table

xi yi x2i x3i x4i xi yi x2i yi
1 120 1 1 1 120 120
2 90 4 8 16 180 360
3 60 9 27 81 180 540
4 70 16 64 256 280 1120
5 35 25 125 625 175 875
6 11 36 216 1296 66 396
X6
21 386 91 441 2275 1001 3411
i=1
Thus, we have
8
< 386 = 6a0 + 21a1 + 91a2
1001 = 21a0 + 91a1 + 441a2
:
3411 = 91a0 + 441a1 + 2275a2

Solving this system, we obtain the …tted curve

85 5 2
p2 (x) = 136 4 x + 28
x :

3.9.2 Continuous Least Squares approximation

The notion of least squares approximation can be extended beyond the data-
…tting problem. Consider the problem of …nding an approximation to a given
function f in terms of a set of basis functions f'k ; 0 k ng. How do we …nd
the coe¢ cients ak ; k = 0; :::; n in the expansion
n
X
pn (x) = ai 'i (x);
i=0

such that
n Z
X b
2 2
I(a0 ; a1 ; :::; an ) = kf pn k2 = !(x) (f (x) ak 'k (x)) dx
k=0 a

is minimized ?

31
 Using the inner product notation to write I, in the following form
2
I(a0 ; a1 ; :::; an ) = kf pn k2 = hf pn ; f pn i
= hf; f i 2 hf; pn i + hpn ; pn i
X n Xn
2
= kf k2 2 ai hf; 'i i + ai aj 'i ; 'j :
i=0 i;j=0

Applying ordinary calculus to the problem of minimizing I(a0 ; a1 ; :::; an ).

After some manipulations we …nd that the ak are de…ned by simultaneously
solving the set of equations
0 @I 1 0 1
@a0 0
B @I C B 0 C
B @a1 C B C
rI(a0 ; a1 ; :::; an ) = B .. C=B .. C;
@ . A @ . A
@I 0
@an

where
n
X
@I
= 2 hf; 'i i + 2 aj 'i ; 'j ; for all i = 0; ; n:
@ai j=0

which is written as follows

n
X
aj 'i ; 'j = hf; 'i i ; i = 0; 1; :::; n:
j=0

This system can be organized along matrix-vector lines as

0 1
.. 0 1 0 1
B h' 0 ; '0 i h'0 ; '2 i . h'0 ; 'n i C a0 hf; '0 i
B .. CB C B C
B h'1 ; '0 i h'1 ; '2 i
B . h'1 ; 'n i C B a1
C C B hf; '1 i C
B . . .. CB . C=B .. C
B .. .. ..
. . C @ .. A @ . A
@ A
.. a2 hf; 'n i
h'n ; '0 i h'n ; '2 i . h'n ; 'n i

Solving a system of linear equations is a problem that we do not encounter

in the general case.

Example
p 49 Find the least squares quadratic, which best …ts the curve f (x) =
x over the interval [0; 1] :

Solution 50 We need to minimize

Z 1
2 2
I (a0 ; a1 ; a2 ) = kf p2 k2 = (f (x) p2 (x)) dx;
0

32
 where p2 is written in standard basis as follow

p2 (x) = a0 + a1 x + a2 x2 :

By this new approach, we shall …rst of all obtain the normal equations.
Thus we have :
8 R1
@I 2
>
< @a0 = 2R 0 f (x) a0 a1 x a2 x dx = 0
@I 1
rI = 0 () @a1 = R2 0 x f (x) a0 a1 x a2 x2 dx = 0
>
: @I 1 2
@a2 = 0 x f (x) a0 a1 x a2 x2 dx = 0

Integrating, we get
8
< a0 + 21 a1 + 13 a2 = 23
1
a0 + 31 a1 + 14 a2 = 52
: 12 1 1 2
3 a0 + 4 a1 + 5 a2 = 7

Solving these equations simultaneously we get

6 48 4 2
p2 (x) = 35 + 35 x 7x :

1.0
y

0.5

0.0
0.0 0.2 0.4 0.6 0.8 1.0
x
p
p2 (x) with x

Orthogpnal basis
We can avoid it altogether at this point if our basis functions satisfy the orthog-
onality condition
'i ; 'j = 0; for all i 6= j:
In this case, the above system is a diagonal system and we very easily have
hf;'k i
ak = h'k ;'k i ; k = 1; 2; :::; n:

Question : When can we …nd a basis that satis…es the orthogonality con-
dition, and how good is the resulting approximation ?
The answer is that we can always …nd such a basis if we consider polynomial
functions for our basis elements, and the resulting approximations are usually
quite good.

33
3.9.3 Legendre squares approximation
Legendre Polynomials are known to be applicable to least square approxima-
tion of functions. In this sense, we mean that we can follow the least square
approximation technique and adapt this to Legendre polynomial.
For any function f de…ned over [ 1; 1]; using Legendre least squares approx-
imation, the best approximation is given by
n
X
qn (x) = ak Pk (x);
k=0

where the coe¢ cients ak ; k = 0; :::; 3 are given by

hf;Pk i 2k+1
ak = 2k+1 = 2 hf; Pk i ; k = 0; 1; :::; n:
hPk ;Pk i= 2

The Pk are the Legendre polynomials.

Example 51 Let’s construct the cubic least squares approximation to the ex-
ponential function,
f (x) = exp x;
over the interval [ 1; 1], using Legendre polynomials.
The cubic least squares approximation, denoted by q3 ; is

q3 (x) = a0 P0 (x) + a1 P1 (x) + a2 P2 (x) + a3 P3 (x);

where
2k+1
ak = 2 hf; Pk i ; k = 0; 1; 2; 3:
We thus need to compute the following integrals,
Z
1 1 +1 x
a0 = hf; P0 i = e dx = sinh 1;
2 2 1
Z Z +1
3 3 +1
a1 = hf; P1 i = P1 (x)ex dx = 23 xex dx = 3e 1 ;
2 2 1 1
Z Z +1
5 5 +1 1
a2 = hf; P2 i = P2 (x)ex dx = 25 3x2 1 ex dx = 52 e 7e 1 ;
2 2 1 1 2
Z Z +1
7 7 +1 1
a3 = hf; P3 i = P3 (x)ex dx = 27 5x3 3x ex dx = 72 37e 1 5e ;
2 2 1 1 2

so the polynomial approximation is

1 1
1 5(e 7e ) 7(37e 5e)
q3 (x) = sinh 1P0 (x) + 3e P1 (x) + 2 P2 (x) + 2 P3 (x);

which simpli…es to

q3 (x) = 0:99629 + 0:99795x + 0:53672x2 + 0:17614x3 :

34
 3
y

-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0
x
Cubic Legendre least squares to ex

3.9.4 Chebyshev squares approximation

If we have a function f de…ned over [ 1; 1] which we wish to approximate by
using Chebyshev squares approximation, , the best approximation is given by
n
X
f (x) = ak Tk (x):
k=0

How we can …nd the coe¢ cients ak ; k = 0; 1; :::; n?

Multiplying f (x) by Tm (x) we get
n
X
hf (x); Tm (x)i!(x) = ak hTk (x); Tm (x)i!(x) ;
k=0

where
Z
+1
f (x)Tm (x)
hf (x); Tm (x)i!(x) = p
1 x2
dx
1
By making use of the orthogonality property of Tn (x);thus
8
< 0; if m 6= n
hTn (x); Tm (x)i!(x) = ; if m = n = 0
:
2 ; if m = n 6= 0
The only term on the right which doesn’t vanish is the one where m = k,
hf (x);Tm (x)i!(x)
am = hTm (x);Tm (x)i!(x) ; for all m = 0; ; n:

Hence,
hf (x);T0 (x)i!(x) 1
a0 = hT0 (x);T0 (x)i!(x) = hf (x); T0 (x)i!(x) ;
hf (x);Tm (x)i!(x) 2
am = hTm (x);Tm (x)i!(x) = hf (x); Tm (x)i!(x) ; for all m = 1; ; n;

35
 which is equivalent to
Z
+1
1 f (x)
a0 = p
1 x2
dx;
1
Z
+1
2 f (x)Tm (x)
am = p
1 x2
dx; for all m = 1; :::; n:
1

Remark 52 The evaluation of the integral for am will in general have to be

done numerically and in such cases it is obviously important to ensure that the
truncation error is su¢ ciently small or the accuracy available via the Chebyshev
approximation to f (x) will be reduced. In a few special cases, the integral can
be evaluated analytically and the problem of truncation error does not arise.

Example 53 Find the quadratic Chebyshev squares approximation of f (x) =

arccos x in [ 1; 1] :

Solution 54 We have
2
X
f (x) = ak Tk (x); where Tk (x) = cos (k arccos x)
k=0

where
Z
+1
1 arccos x
a0 = p
1 x2
dx;
1
Z
+1
2 arccos x cos(k arccos x)
ak = p
1 x2
dx; k = 1; 2:
1

Putting t = arccos x; and integrating by parts we get

Z
1
a0 = tdt = ;
2
0
Z
2 4
a1 = t cos (t) dt = ;
0
Z
2
a2 = t cos (2t) dt = 0:
0

Thus,
4 4
f (x) = arccos x ' 2 T0 (x) T1 (x) = 2 x:

36
 3
y
2

-1.0 -0.5 0.0 0.5 1.0

x
arccos x with p1 (x)

37