Numerical Methods

Class Timings

Thursday & Friday

9:30 AM-10:30 AM

1



 Syllabus

• Mathematical Preliminaries: Sequence of real numbers; limits and continuity of a function and
Intermediate Value Theorem; Mean Value Theorems for Differentiation and Integration; Taylor’s
Theorem; Order of Convergence.

• Error Analysis: Floating-Point Representation; Types of Errors; Loss of Significance; Propagation of

Error, Condition number and stability of a function.

• Finding root of equations: Bisection Method; Regula-falsi, Secant Method; Newton-Raphson

Method; Fixed-point Iteration Method; Rates of Convergence.

• Interpolation by polynomials: Lagrange and Newton Forms of Interpolating Polynomials;

Divided Differences; Error in Polynomial Interpolation;

• Numerical Integration and Differentiation: Rectangle rule, Trapezoidal rule, Simpson’s rule;
Composite Rules; Gaussian Rules; Difference Formulae.

• Numerical Linear Algebra: Gaussian Elimination; Pivoting Strategy; Gauss-Jordon Method, LU

Factorization; Matrix Norms, Condition Number of a Matrix; Solution by Iteration: Gauss-Jacobi,
Gauss-Seidel methods; Newton-Raphson Method in two variables, Eigenvalue problems:
Power Method.

• Numerical Ordinary Differential Equations: Taylor’s Method, Euler’s Method; Modified Euler’s
Methods; Runge-Kutta Methods.

2















 References
1. S. D. Conte and C. de Boor, Elementary Numerical Analysis - An Algorithmic Approach
(3rd edition), McGraw-Hill, 1981.

2. K. Atkinson, An Introduction to Numerical Analysis (2rd edition), John-Wiley-India,

1989.
3. S.S. Sastry, Introductory Methods of Numerical Analysis-Prentice Hall of India.

4. R. L. Burden and J. D. Faires, Numerical Analysis: Theory and Applications, Ceneage

Learning India Pvt. Ltd., 2010.

5. E. Kreyszig, Advanced Engineering Mathematics (8th edition), John-Wiley-India, 1999.







 Chapter-1

Mathematical Preliminaries

 Sequences

Let a and b be two real numbers. We define the following

[a, b] = {x ∈ ℝ : a ≤ x ≤ b} and (a, b) = {x ∈ ℝ : a < x < b}

Definition: A sequence is a function whose mapping is from natural numbers to

Real/Complex numbers. That is,
f : ℕ → ℝ or ℂ

and denoted by (an) or {an}. The terms in a sequence are denoted by

a1, a2, a3, . . . an . . . .

1 1 1 1
Example: (an) = ( ); a1 = 1, a2 = , a3 = , . . . , an = . . . .
n 2 3 n

Note: Concept of convergence of a sequence plays an important role in

Numerical Methods.

 Sequences

For example, an iterative procedure often constructed to approximate a solution of

an equation. We have to look whether approximation goes to exact solution or not?

Convergence of a sequence: Let (an) be a sequence of real numbers and let

a ∈ ℝ. The sequence (an) is said to be converge to a, written as

limn→∞an = a or an → a as n → ∞

if for every ϵ > 0, there exists a natural number N such that

| an − a | < ϵ whenever n ≥ N.

The real number a is called the limit of the sequence (an).

1
Example: 1. Check whether (an) = ( ) converges?
n
1
2. Check whether (an) = ( 2 ) converges?
n
3. Check whether (an) = (n 2) converges?

 Sequences
Sandwich Theorem: Let (an), (bn) and (cn) be sequences of real numbers such that

1. There exists an n0 ∈ ℕ such that every n ≥ n0, the sequences satisfy the inequalities
an ≤ bb ≤ cn and

2. limn→∞an = limn→∞cn = a for some real number a.

Then the sequence (bn) also converges and limn→∞bn = a.

sin n
Example: 1. Check whether (an) = ( ) converges?
n
cos n
2. Check whether (an) = ( ) converges?
n 2

Bounded Sequence: A sequence (an) is said to be a bounded sequence if there exists a real
number M such that
| an | ≤ M for every n ∈ ℕ

Note: Every bounded sequence need not converge. However, it is always possible to find
convergent subsequence.

Example: (an) = ((−1)n)

Bolzano-Weierstrass Theorem: Every bounded sequence (an) has a convergence

subsequence (ank).
7

 Sequences
Monotone Sequences: A sequence (an) of real numbers is said to be

1. An increasing sequence if an ≤ an+1 for every n ∈ ℕ.

2. A strictly increasing sequence if an < an+1 for every n ∈ ℕ.

3. A decreasing sequence if an ≥ an+1 for every n ∈ ℕ.

4. A strictly decreasing sequence if an > an+1 for every n ∈ ℕ.

A sequence (an) is said to be a (strictly) monotonic sequence if it is either (strictly) increasing or

(strictly) decreasing.

Note: Bounded monotonic sequences always converges.

Algebra for limits of sequences: Let (an) and (an) be two sequences. Assume that limn→∞an and
limn→∞bn exists. Then

1. limn→∞(an ± bn) = limn→∞an ± limn→∞bn

2. limn→∞can = climn→∞an, for any number c

3. limn→∞anbn = limn→∞an limn→∞bn

1 1
4. limn→∞ = provided limn→∞an ≠ 0
an limn→∞an
8

 Limits and Continuity for functions

LHL: Let f be a function defined on the left side (or both sides) of a, except possibly at a
itself. Then, we say “the left-hand limit of f(x) as x approaches a, equals l” and denote

limx→a− f(x) = l

if we can make the values of f(x) arbitrarily close to l by taking x to be sufficiently close to a
and x less than a.

RHL: Let f be a function defined on the right side (or both sides) of a, except possibly at a
itself. Then, we say “the right-hand limit of f(x) as x approaches a, equals r” and denote

limx→a+ f(x) = r

if we can make the values of f(x) arbitrarily close to r by taking x to be sufficiently close to a
and x greater than a.

LIMIT: Let f be a function defined on the both sides of a, except possibly at a itself. Then,
we say “the limit of f(x) as x approaches a, equals L” and denote

limx→a f(x) = L
if we can make the values of f(x) arbitrarily close to L by taking x to be sufficiently close to a
and but not equal a.

NOTE: In each of the above definitions, the value of the function f at point a does not play
any role, the function f need not be defined at 9that point a.

Algebra for limits of functions: Let f and g be two functions defined on both sides of a, except
possibly at a itself. Assume that limx→a f (x) and limx→ag(x) exists. Then

1. limx→a( f (x) ± g(x)) = limx→a f (x) ± limx→ag(x)

2. limx→acf (x) = climx→a f (x), for any number c

3. limx→a f (x)g(x) = limx→a f (x) limx→ag(x)

1 1
4. limx→a = provided limx→ag(x) ≠ 0
g(x) limx→ag(x)
NOTE:
Direct substitution property; Polynomials, trigonometric and rational functions

Theorem 1: Let f (x) ≤ g(x) when x is an interval containing a (except possibly at a) and the limits of
f (x) and g(x) both exists as approaches a, then

limx→a f (x) ≤ limx→ag(x)

Theorem 2:(Sandwich Theorem) Let f (x), g(x) and h(x) be given functions such that

1. f (x) ≤ g(x) ≤ h(x) when x is an interval containing a (except possibly at a) and

2. limx→a f (x) = limx→ah(x) = L, then

limx→ag(x) = L .

10

Limit: Let f be a function defined in some open interval that contains a, except possibly at a
itself. Then we say that the limit of f(x) as x approaches a is L and we write

limx→a f(x) = L

if for every ϵ > 0 there is a number δ > 0 such that

| f(x) − L | < ϵ whenever 0 < | x − a | < δ

Continuity: Let f be a function defined in some open interval that contains a. Then we say
that the continuity of f(x) as x approaches a is f(a)and we write

limx→a f(x) = f(a)

if for every ϵ > 0, there exist a number δ > 0 such that

| f(x) − f(a) | < ϵ whenever | x − a | < δ

Or A function f(x) is
1. Continuous from left side of a if limx→a− f(x) = f(a),

2. Continuous from right side of a if limx→a+ f(x) = f(a) and

3. Continuous at a if, limx→a f(x) = f(a)

Theorem: If f(x) and g(x) are continuous at a, then the functions f ± g, cg(x), fg, f /g (provided
g(a) ≠ 0), f ∘ g are all continuous.
11

 Intermediate Value Theorem: Suppose that f is continuous on the closed

interval [a, b] and let N be any number between f(a) and f(b) , where f(a) ≠ f(b) .
Then there exists a point c ∈ (a, b) such that
f(c) = N.

IVT Theorem is often used in the following form in numerical analysis.

Statement: Let f be continuous on the closed interval [a, b] . Let x1, x2, x3, . . . , xn
be points in [a, b] and let g1, g2 g3, . . . , gn be real numbers having same sign.
Then
n n

∑ ∑
f(xi)gi = f(ξ) gi for some ξ ∈ [a, b]
i=1 i=1

Proof: Assume without loss of generality gi ≥ 0, i=1,2,…,n

Let f(x*) = maxi f(xi) and f(x*) = mini f(xi)

Then
n n n n n

∑ ∑ ∑ ∑ ∑
f(x*) gi = f(x*)gi ≤ f(xi)gi ≤ f(x*)gi ≤ f(x*) gi
i=1 i=1 i=1 i=1 i=1

12

Define
n

∑
g(x) = f(x) gi .
i=1

Note that g(x) is continuous function and

n n n

∑ ∑ ∑
g(x*) = f(x*) gi ≤ α = f(xi)gi ≤ f(x*) gi = g(x*)
i=1 i=1 i=1

Thus by IVT, there exist a ξ between x* and x* such that g(ξ) = α.

That is,
n n

∑ ∑
f(ξ) gi = f(xi)gi
i=1 i=1

NOTE: Similarly you can prove for gi ≤ 0 for I=1,2,…,n. (TO DO)

13

 Differentiability: The derivative of a function f at a point a, denoted by f ′(a), is

defined as
′ f(a + h) − f(a)
f (a) = lim
h→0 h

if the limit exists. We say f is differentiable at a.

A function f is said to be differentiable on (c, d) if f is differentiable at every-point

in (c, d).

The derivative of a function f at a point x = a can also be given by

′ f(a) − f(a − h) ′ f(a + h) − f(a − h)

f (a) = lim and f (a) = lim
h→0 h h→0 h

Let x=a+h, h=x-a, then

′ f(x) − f(a)
f (a) = lim
x→a x−a

Note that every differentiable function is continuous

14






 Theorem: Suppose f is differentiable at a. Then there exists a function ϕ such that

f (x) = f (a) + (x − a)f ′(a) + (x − a)ϕ(x)

and lim ϕ(x) = 0.

x→a

Rolle’s Theorem: Let f be a function that satisfies the following three hypothesis.

1. f is continuous on the closed interval [a, b].

2. f is differentiable on the open interval (a, b).

3. f(a)=f(b).

Then there is a number c in the open interval (a, b) such that f ′(c) = 0.

Mean Value Theorem: Let f be a function that satisfies the following hypothesis.

1. f is continuous on the closed interval [a, b].

2. f is differentiable on the open interval (a, b).

f (b) − f (a)
Then there is a number c in the open interval (a, b) such that f ′(c) =
b−a

Or equivalently

(b − a)f ′(c) = f (b) − f (a)

15





 First Mean Value Theorem for Integrals: Suppose f is continuous on the

closed interval [a, b], then there exists a number c in the open interval (a, b) such
that
b

∫a
f(x)dx = f(c)(b − a)

Outline of the proof!

Second Mean Value Theorem for Integrals: Suppose f and g be continuous

on the closed interval [a, b], and let g(x) ≥ 0 for all x ∈ ℝ. Then there exists a
number c ∈ [a, b] such that
b b

∫a ∫a
f(x)g(x)dx = f(c) g(x)dx

Outline of the proof!

16

Taylor’s Theorem:

17