Numerical Analysis : [ MA214 ]

Lecture 1

Instructor: Prof. Tony J. Puthenpurakal

Department of Mathematics
Indian Institute of Technology Bombay
[email protected]

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Textbooks:
1 Elementary Numerical Analysis by S. D. Conte and C. deBoor.
2 Numerical Analysis by Richard L. Burden and J. D. Faires.

Instructions:
80% Attendance Required.
2 Quizes (10% each) : 20%.
Midsem : 30%.
EndSem : 50%
Passing approximately 40% of Max. score.

Note: Last time max score was above 90%. So passing marks ≈ 36%

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Calculators are a must for this course. You need it in tutorials, quizzes,
mid-sem, and end-sem.

All calculations will be done to 4 sig. digit accuracy. (SCI-4 in Casio

Calculators)

Note: Programmable calculators, tablet P.C’s are not allowed in exam.

Note: All angles will be in radians. All logarithms will be to base e.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Introduction

What is Numerical Analysis?

I will explain it through examples
Rb
a f (x)dx exists for example when f : [a, b] → R is continuous.
1

However in most cases it is impossible to compute it.

Examples:-
R1 2
1 I =
1 0 sin(x )dx
R 1 −x 2
2 I =
2 0 e dx
R 41 1
3 I = √ dx
3 0 1−x 3

Not only do we have to approximate the integrals, we also have to do it

with pre-assigned accuracy. For example, approximate I1 upto 10−6 , i.e
|I1 − approx| ≤ 10−6 .

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Interpolation

Suppose you are given a function f : [0, 2] → R at some values

x0 , x1 , . . . , xn .
Problem: approximate f (t) at t ∈ [0, 2] − {x1 , x2 , . . . , xn }
Graphically:

Goal: To find a curve passing through these points mentioned in above

figure.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Methods

There are two standard methods to do this job.

1 Lagrange Interpolation
2 Piecewise Methods
(a) Piecewise Linear
(b) Cubic Spine Interpolation

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Initial Value Differential Equations

dy
= f (x, y ), where y (x0 ) = y0 . (1)
dx

In general not possible to find y exactly.

Example:
dy
= sin(x + y 2 ), where y (0) = 1
dx

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

However for many applications approximate value is enough.
Suppose in the previous example you have to approximate y (1).
All methods will first approximate in between points and then find Y (1).

for example y (0.1) is approximated first. Then using this y (0.2) is

approximated.
y (0.3) is approximated using y (0.2).
y (0.4) is approximated using y (0.3).
..
.
y (1) is approximated using y (0.9).
This creates an additional issue and that is of error propagation.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Eigenvalues and Eigenvectors

Let A be a n × n matrix.

Recall λ ∈ R is said to be an eigenvalue of matrix A if there exists x(6= 0)

such that

Ax = λx

Here, x is called eigen-vector corresponding to λ.

Question: How do you find eigenvalues and eigenvectors ?

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

In applications the size of the matrix is large.
n ≥ 10, 000 is common
n ≥ 1 million for significant % of cases.
So usual method of finding

p(t) = |tI − A|

and then finding roots of p(t) is not feasible.

In practice, matrix A will have a dominant eigenvalue i.e, there exists λ0
s.t.
|λ0 | ≥ |λi |
for all other eigenvalues λi and it is enough for applications to find λ0 .

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Floating Point Arithmetic

n-digit Floating Point number in base β has the form

n = ±(.d1 d2 . . . dn )β β e

(.d1 d2 . . . dn )β → mantissa
e → exponent

For most computers, β = 2

For Calculators β = 10
x = real number, fl(x) = floating point representation of x.
√
Example: 0.5 = 7.071E −1 in 4 significant digits.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Suppose x ∗ is an approximation to x. Then

|x − x ∗ | = Absolute error

|x − x ∗ |
:= relative error, ( provided x 6= 0)
|x|
Problem: We do not know x. So how to find relative error?
x − x∗
α =
x∗
x − x∗ α
= ≈ α, if α is small
x 1+α

Definition:- x ∗ is said to be approximate x to t significant digits if

x − x∗ 
 
−t
 x  ≤ 5 × 10 , we are assuming x 6= 0
 

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Loss of Significant digits
It is not true that if

x ∼ x ∗ and y ∼ y ∗ in 4 sig digits

=⇒

x +y ∼ x∗ + y∗ in 4 sig digits
x −y ∼ x∗ − y∗ in 4 sig digits
xy ∼ x ∗y ∗ in 4 sig digits
x x∗
∼ y∗ in 4 sig digits
y

Things which create loss of significant digits

1 Subtraction of nearly equal quantities
2 division by number which is close to zero
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
Example 1

f (x) = 1 − cosx
cos(0.01) = 1 in 4 sig digits
f (0.01) = 1 − 1 = 0
Actual answer is 5E −5, ( i.e. 5 × 10−5 )
Loss of sig digits arises since cos(0.01) = 1 in 4 sig digits
To avoid this
f (x) = 1 − cosx
1 − cos 2 x
=
1 + cosx
sin2 x
=
1 + cosx
1E −4
f (0.01) = = 5E −5
1+1
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
Example 2

x 2 + 111.11x + 1.2121 = 0

b 2 = 1.235E 4
b 2 − 4ac = 1.234E 4
p
b 2 − 4ac = 1.111E 2 = b in 4 sig digits
therefore √
b 2 − 4ac
−b +
x1 = =0
2a
Again loss of sig digits occur because we are subtracting nearly equal
quantities.
√ √
−b + b 2 − 4ac −b − b 2 − 4ac
x1 = × √
2a −b − b 2 − 4ac
−2c 1.212
= √ =
2
b + b − 4ac 222.1
= 1.091E −2 correct upto 4 sig digits
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
 Example 3

x − sinx 0.01 − 1E −2
f (x) = , so f (0.01) = =0
tanx 1E −2
x − sinx x + sinx
Rewrite f (x) = ×
tanx x + sinx
2 2
x − sin x
=
tanx(x + sinx)
1(E −4) − 1E (−4)
f (0.01) = =0
1(E −2)(0.01 + 1(E −2))
Actual value f (0.01) ≈ 1.667E −5 4 sig digits. So one has to use Taylor
series
x3 x3
sinx ≈ x − and tanx ≈ x −
6 3
x 3 /6 x 2 /6
f (x) ≈ ≈
x − x 3 /3 1 − x 2 /3
f (0.01) = 1.667E −5
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
Error Propagation
Once an error is committed it contaminates subsequent results

Error propagation is studied in terms of two related concepts:

1 condition

2 instability

1. condition:

Condition ↔ sensitivity of f (x) to changes in x

∗
 
 |f (x)−f (x )| 
|f (x)| ∗
= max ∗ : |x − x | small
 |x−x | 
|x|
 0 
 f (x)x 
≈ 
 f (x) 


The larger the condition, the more ill-conditioned the function is said to be.
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
Examples
√
Example 1 : f (x) = x, f 0 (x) = √1
2x
  1 
 f (x)x   √2x x  1
 0
 f (x)  =  √x  = 2
 

So taking square-root is well conditioned, since it actually reduces the

relative error.
10
Example 2 : f (x) = 1−x 2
 0
 f (x)x   2x 2 
  
 f (x)   1 − x 2  , large when |x| is close to 1
 = 

What to do:
put x = 1 − y , then y ∼ 0 if x ∼ 1
10 10 10 5
f (x) = 2
= 2
≈ = , since y 2 ∼ 0
1 − (1 − y ) 2y − y 2y y
5
g (y ) = y  0 
 g (y )y 
 g (y )  = 1
 
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
Example of instability

du1 1
= 9u1 + 24u2 + 5cost − sint
dt 3
du2 1
= −24u1 − 51u2 − 9cost + sint
dt 3
4 2
u1 (0) = and u2 (0) =
3 3
exact solution

1
u1 (t) = 2e −3t − e −39t + cost
3
1
u2 (t) = −e −3t + 2e −39t − cost
3
RK method of order 4 with step size 0.1
u˜1 approximate to u1
u˜2 approximate to u2
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
 t u1 (t) ũ1 (t)
0.1 1.793061 -2.645169
0.2 1.423901 -18.45158
.. .. ..
. . .
0.9 0.3416143 -695332
1.0 0.2796748 -3099671

t u2 (t) ũ2 (t)

0.1 -1.032001 7.844527
0.2 -0.8746809 38.87631
.. .. ..
. . .
0.9 -0.2744088 1390664
1.0 -0.2298877 6199352

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Instability
Instability=sensitivity of the numerical process for the calculation of f (x)
from x to the inevitable rounding error committed in a calculator or a
computer.
√ √
Example:- f (x) = x + 1 − x
 0 
 f (x)x  1
condition =  = √ x √ ≈ 1
f (x)  2 x + 1 x 2
So conditioning is good
f (12345) = 1.111E 2 − 1.111E 2 = 0.0
Actual value = 4.5E −3. So we analyze what goes wrong
x0 = 12345
x1 = f1 (x0 ) = x0 + 1 = 12346
√
x2 = f2 (x1 ) = x1
√
x3 = f3 (x0 ) = x0
f4 (t) = x2 − t
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
Condition of f4 is  0   
 f4 (t)t   t 
 = 
 f4 (t)   x2 − t 
f4 is well conditioned except when t = x2
In our example x2 − x3 ≈ 0.005, x3 − t ≈ 111.11
So condition of f4 ≈ 22, 222 ≈ 40, 000 times condition of f .

What to do
√ √ 1
f (x) = x +1− x=√ √
x +1+ x

1
f (12345) =
1.111E 2 + 1.111E 2
≈ 4.502E −3 correct up to 3 sig digits

Note: It is possible to estimate the effect of instability by considering the

rounding errors once at a time

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Mathematical Preliminaries

Theorem (Intermediate-value theorem for continuous function’s)

Let f : [a, b] −→ R be a continuous function. x1 , x2 ∈ [a, b] and say
f (x1 ) < α < f (x2 ). Then there exists c ∈ [a, b] such that f (c) = α.

Corollary
Let f : [a, b] −→ R be a continuous function. Let x1 , x2 , · · · , xn ∈ [a, b]
and let g1 , g2 , · · · , gn be real numbers all of one sign. Then
n
X n
X
f (xi )gi = f (ξ) gi , for some ξ ∈ [a, b]
i=1 i=1

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Proof (sketch)
We may assume gi ≥ 0. Without loss of generality assume
f (x1 ) = min{f (xi ) : i = 1, 2, · · · , n}
f (xn ) = max{f (xi ) : i = 1, 2, · · · , n}
Then
n
X n
X n
X
f (x1 ) gi ≤ f (xi )gi ≤ f (xn ) gi
i=1 i=1 i=1
Xn
h(x) = f (x) gi is continuous
i=1
n
X
h(x1 ) ≤ f (xi )gi ≤ h(xn )
i=1
So by the intermediate value Theorem there exists ξ such that
n
X
h(ξ) = f (xi )gi
i=1
n
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1
X
Similarly One has the following:-

Corollary
Let g : [a, b] −→ R be non-negative ( or non-positive ) integrable function.
Let f : [a, b] −→ R be continuous. Then
Z b Z b
f (x)g (x)dx = f (ξ) g (x)dx for some ξ ∈ [a, b]
a a

Remark:- The assumption g (x) is of one sign is essential.

Example:- f (x) = g (x) = x, x ∈ [−1, 1]

Z 1 Z 1
2
f (x)g (x)dx = x 2 dx =
−1 −1 3
R1
However −1 g (x)dx =0

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

 Existance of Maximum and Minimum of continuous
function

Theorem
Let f : [a, b] −→ R be continuous. Then there exists α, β ∈ [a, b] such that

f (α) ≤ f (x) ≤ f (β), ∀x ∈ [a, b]

Let us recall:-
Theorem (Rolle’s Theorem)
Let f : [a, b] −→ R be a continuous function and assume f : (a, b) −→ R
is differentiable. If f (a) = f (b), then there exists ξ ∈ (a, b) such that
f 0 (ξ) = 0

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Rolle’s Theorem implies the famous mean value Theorem

Theorem (M.V.T.)
Let f : [a, b] −→ R be a continuous function and assume f : (a, b) −→ R
is differentiable. Then
f (b) − f (a)
= f 0 (ξ) for some ξ ∈ (a, b)
b−a

Proof.
Apply Rolle’s Theorem to

f (b) − f (a)
F (x) = f (x) − f (a) − (x − a)
b−a

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Taylor’s formula with integral remainder

Theorem (Taylor’s formula with integral remainder)

If f (x) has (n + 1) continuous derivatives on [a, b] and c is some point in
[a, b]. Then for all x ∈ [a, b]

f 00 (c)
f (x) = f (c) + f 0 (c)(x − c) + (x − c)2 + · · ·
2
f (n) (c)
··· + (x − c)n + Rn+1 (x)
n!
1 x
Z
where Rn+1 (x) = (x − s)n f n+1 (s)ds
n! c

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Fundamental Theorem of Algebra

Let p(x) = a0 + a1 x + a2 x 2 + · · · + an x n , an 6= 0, n ≥ 1 and all ai ∈ C.

It is easy to see that p(x) has atmost n roots.

However the following is non-trivial to prove

Theorem (FTA)
p(x) has a root, i.e., there exists ξ ∈ C such that p(ξ) = 0

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Measuring how fast sequenses converges

1
Note np −→ 0 for any p ≥ 0
1 1
Intuitively, n −→ 0 more slowly than n2
and so on.

Definition
Let {αn }n≥1 and {βn }n≥1 be two sequences. We say αn is big − oh of βn
and we write
αn = O(βn )
If |αn | ≤ k|βn |, for some constant k and for all n >> 0

Examples:-
{ n1 }, { 100 10
n }, { n −
40
n2
+ e −n }, { n1p } (here p ≥ 1) are all O( n1 ).

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

Definition
Let {αn }n≥1 and {βn }n≥1 be two sequences. We say αn is little − oh of βn
and we write it as αn = o(βn ), if
αn
lim =0
n→∞ βn

Example:
1 { n12 }
1
2 { n log n}
both are o( n1 )

Important Remark:-
1
A convergence order rate of n is much too slow to be useful in calculations.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1

 Example

∞ ∞
π X (−1)i X 2
= =1− 2
4 2i + 1 16j − 1
i=0 j=1
n
P 2
Set αn = 1 − 16j 2 −1
.
j=1
The sequence {αn } is monotonically decreasing. Moreover
π 1
0 ≤ αn − ≤ , n = 1, 2, · · ·
4 4n + 3
to calculate π4 correctly to within 10−6 we would need 106 ≤ 4n + 3 or
roughly n = 250, 000 calculations.
However round of errors in calculation α250,000 will be usually greater that
10−6 . Here
 
π 1
αn = +O
4 n
 
π 1
αn 6= +o
4 n
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 1