Numerical analysis formulae

February 2021

1 Interpolation
Notation: f : [a, b] → R will be the continuous function to be interpolated.
Pn refers to the vector space of polynomials with degree ≤ n. p ∈ Pn is
the interpolating polynomial (spline) that interpolates f at x0 , x1 , . . . xn . (i.e.
p(xi ) = f (xi ) = fi at all xi .)
kf k = supx∈[a,b] |f (x)| refers to the supremum norm of the function.

1.1 Polynomial interpolation

• Lagrange’s Recipe
n
Y (x − xk )
li (x) =
(xi − xk )
k=0,k6=i
Xn
p(x) = f (xi )li (x)
i=0

Note that
n
X
li (x) = 1 ∀x
i=0

• Uniqueness Theorem
Given n-interpolation points (and value of f at these points), there is a
unique polynomial interpolant from Pn . that can do the interpolation.
• Weierstrass theorem
Given any continuous f : [a, b] → R and any  > 0, there exists a polyno-
mial p such that kf − pk < 

• Estimate on error bound

For any continuous, n + 1 times differentiable f : [a, b] → R ,a polynomial

1
 p that interpolates f at (distinct) x0 , x1 , x2 , . . . xn , and x ∈ [a, b], there
exists ζ ∈ [a, b] such that
n
1 (n+1)
Y
f (x) − p(x) = f (ζ) (x − xi )
(n + 1)! i=0

• Runge’s Example
Equally-spaced polynomial interpolation is a recipe for disaster
 (overfit-

b+a b−a (n−i)π
ting). Chebyshev interpolation points xi = 2 + 2 cos n much
better. Equally spaced splines are fine.
• Newton’s recipe

f [xi ] = f (xi )
f [xi+1 , xi+2 , . . . xj ] − f [xi , xi+1 , . . . xj−1 ]
f [xi , xi+1 , . . . , xj ] =
xj − xi
Xn
p(x) = f [x0 , x1 , . . . xi ](x − x0 )(x − x1 ) . . . (x − xi−1 )
i=0

1.2 Splines
• Basic idea
Make a piecewise polynomial function over each sub-interval through the
interpolation points so that the spline and its degree − 1 derivatives are
continuous at each knot.
• Linear spline
Just join each interpolation point by a line segment.
• Error in linear spline interpolation
For equally space interpolation points with size of segment being h,

h2 00
kf − pk ≤ kf k
8

• Natural Cubic Spline

Spline and 2 derivatives continuous at all interpolation points, Second
derivative at the end-points vanishes.
• Solution for (equally spaced) cubic spline
 
4 1 0 0 ... 0 0 0    
1 4 1 0 . . . 0 0 0 σ1 f0 − 2f1 + f2
0 1 4 1 . . . 0 0 0  σ2   f1 − 2f2 + f3 
6 
   
  ..  = 2  ..
 .  h 
 
 .. .. .. .. . 
. . . . 
σn−1 fn−2 − 2fn−1 + fn
0 0 0 0 ... 0 1 4

2
 σ0 = σn = 0
σi
bi =
2
σi+1 − σi
ai =
6h
fi+1 − fi h
ci = − (σi+1 + 2σi )
h 6
di = f (xi ) = fi
si (x) = ai (x − xi )3 + bi (x − xi )2 + ci (x − xi ) + di ∀x ∈ [xi , xi+1 ]

• Error estimate in cubic splines

kf − pk ≤ h2 kf 00 − p00 k
kf − pk ≤ h4 kf 0000 k - NOT TRUE FOR NATURAL CUBIC SPLINES!

2 Quadrarture
2.1 Basic Idea
Rb
We will estimate the integral If = P a
f (x)dx by the integral of interpolant Ip .
We know that we can write p(x) = f (xi )Li (x) where Li ’s Pdon’t depend on f
n
at all (only on interpolation points). So we can write Ip = i=0 f (xi )Wi where
Rb
wi are some ’weights’ whose value does not depend on f at all.(Wi = a Li (x)dx)

2.2 Newton-Cotes formulae

If we use equally-spaced interpolation points, then we can write a general ex-
pression for Wi = h · wi where wi depends only on number of interpolation
points (but not explicitly their ‘xi ’ value).
 
Z n n
Y t − k
wi =  dt
0 i−k
k=0,k6=i

First few values of weights -

Name n w0 w1 w2 w3 w4
1 1
Trapezoidal 1 2 2
1 4 1
Simpson’s 1/3 2 3 3 3
3 9 9 3
Simpson’s 3/8 3 8 8 8 8
14 64 24 64 14
Milne’s 4 45 45 45 45 45
n
X b−a
If ≈ Ip = h wi f (xi ) xi = a + i · h h=
i=0
n

3
 Properties of weights
• wi = wn−i
Pn Rn
• i=0 wi = 0 1dx = n

• n = n0 rule will give the exact answer if f is a polynomial of degree n0

• n = 2n0 rule will also give exact answer for a degree 2n0 + 1 polynomial.

2.3 Error estimate in Newton-Cotes formulae

Approach - error is atmost the error in interpolation, times the length of interval
Z b n
1 Y
|If − Ip | ≤ kf (n+1) k (x − xi ) dx
(n + 1)! a i=0

• Error in Trapezoidal rule: 1 00

12 kf k(b − a)3

• Error in Simpson’s 1/3 rule: 1 000

196 kf k(b − a)4

2.4 Composite rules

“The splines of quadrature”
Trapezoidal

• If ≈ h( 21 f (a) + f (a + h) + f (a + 2h) + · · · + f (a + (m − 1)h) + 12 f (b)) = Cp1

• |If − Cp1 | ≤ m · 1 00
12 kf kh
3
= b−a 00
12 kf kh
2

Simpsons

• If ≈ h3 (f (a) + 4f (a + h) + 2f (a + 2h) + 4f (a + 3h) + 2f (a + 4h) · · · + 2f (a +

(2m − 2)h) + 4f (a + (2m − 1)h) + f (b)) = Cp2
• |If − Cp2 | ≤ b−a 000
196 kf kh
3

3 Solving Ordinary Differential Equations

• Studying First Order ODE methods is enough -
    
y1 y1
d  
 y 2

  y2 
  
 ..  = f t,  ..  = f (t, y1 , y2 , . . . , yn )
dt  .    . 
yn yn

For higher order ODE, set some of yi0 s to the derivatives of the first order
variables.

4
• Peano’s theorem (existence):
Suppose f : R2 → R is continuous on (−δ, δ) × (y0 − η, y0 + η). Then there
exists an  > 0 and a y : (−, ) → R such that y 0 (t) = f (t, y) y(t =
0) = y0

• Cauchy-Lipschitz theorem (uniquesness)

– We say that f : I → R is Lipschitz Continuous on I if there
exists an L > 0 (called Lipschitz constant) such that ∀x, y ∈ I,
|f (x) − f (y)| < L|x − y|
– Suppose f : R2 → R is continuous on (−δ, δ) × (y0 − η, y0 + η) and
Lipschitz continuous in y variable. Then there exists an  > 0 and a
unique y : (−, ) → R such that y 0 (t) = f (t, y) y(t = 0) = y0
• Integral Equation Representation:
Rt
y 0 = f (t, y) ⇐⇒ y(t) = y0 + 0 f (u, y(u))du

• Philosophy in numerical estimation: Select finitely many mesh points

‘N ’ and a step size ‘h’. Then the nth mesh point is tn = nh. We will es-
timate yn ≈ y(tn ) recursively -
– y0 = y(t0 ) = y(0)
Rh
– yn+1 = yn + 0 f (tn + s, y(tn + s))ds
Here, we can choose to estimate the integral in various ways (we don’t
know y(t) a-priori so can’t hope to obtain the integral exactly)
• Two easy examples:
Euler’s method(explicit):
Rh
0
f (tn + s, y(tn + s))ds ≈ hf (tn , yn )
yn+1 = yn + hf (tn , yn )
Trapezoidal method(implicit):
Rh
0
f (tn + s, y(tn + s))ds ≈ h2 (f (tn , yn ) + f (tn+1 , yn+1 ))
yn+1 = yn + h2 (f (tn , yn ) + f (tn+1 , yn+1 ))
(have to solve this expression for yn+1 )
• Local Truncation error:
Assuming yn is exact (i.e. yn = y(tn ) exactly) what is the error committed
in estimating y(tn+1 ) using our estimate for the integral? In particular,
how does this error scale with h?
Order of local truncation: Assuming yn = y(tn ), if
y(tn+1 ) − yn+1 ∼ O(hp+1 ) we say that the local truncation error is of
order p. We can use Taylor series estimates y(tn+1 ) = y(tn + h) = y(tn ) +
2 n
hy 0 (tn ) + h2 y 00 (tn ) + · · · + hn! y (n) (tn + ζ) (last term is the remainder term)
and compare terms with our method to find the power of the first term
which differs.

5
• Global error:
Notation: yn → yn,h (to explicitly denote dependence of h). We de-
fine en,h = y(tn ) − yn,h . The gloabal error is defined as eglobal,h =
max |en,h |. We say that a method converges if lim eglobal,h = 0.
n=0,1,2,...bT /hc h→0+
n
Further, if eglobal,h ≤ Ch for all h < h0 (i.e. for all small enough h), then
we say the method is of global order n.

• For Euler’s method, the local order is 1 and global order is also 1
C LT
etrunc ≤ Ch2 =⇒ eglobal ≤ (e − 1)h on domain [0, T ]
L
• For Trapezoidal method, the local order is 2 and global order is also 2

Ch2
 
TL
etrunc ≤ Ch3 =⇒ eglobal ≤ exp on domain [0, T ]
L 1 − Lh/2

• In general for multistep methods,local and global orders may be different.

• General Multistep Method Formulation
s s
!
X X
αs yn+s = h βk f (yn+k , tn+k )
k=0 k=0

where h = ti+1 − ti (constant step size)

• Adam’s recipe:

– At each step the previous s values need to be known. Initial s values

should be obtained by a single step method to start off.
– Suppose fn , fn+1 , . . . fn+s−1 are known, then we estimate the func-
tion between tn+s−1 and tn+s by the lagrange interpolant generated
by the n points
Ps Qs−1
– ψ(t) = k=0 ψk (t)fn+k where ψk (t) = j=0,j6=k tt−t k
j −tk

–    
s h s−1
−
Z
1 t tk  
X Y
yn+s − yn+s−1 ≈h  dt fn+k
h 0 tj − tk
k=0 j=0,j6=k
| {z }
βk

6
• Local truncation Order of General s-step Method A general s-step
method is of order p iff -
s
X
αk = 0
k=0
s
X s
X
k m αk = m k m−1 βs for m ≤ p
k=0 k=0
s
X s
X
k p+1 αk = p k p βs
k=0 k=0

• Convergence: Dahlquist’s theorem A method Ps of order p ≥ 1 is

k
convergent iff its first characteristic polynomial α
k=0 k z = 0 lie in
the closed unit disc, with any roots on the unit cicle itself being non-
degenerate.
• Runge Kutta Methods: Basic Skeleton-
ν
X
y1 = y0 + h bi f (ci h, ζi )
i=0
i−1
X
ζi = y0 + aij f (cj h, ζj )
j=1

• To determine optimal aij , bi , ci we need to put the condition that local

truncation error is of highest order.
• We always use c1 = 0 (i.e. ζ1 = y0 )
• ν = 2 Runge Kutta method:
b1 + b2 = 1
a1,2 = c2
1
b2 c2 =
2
• Butcher Tableau representation:
c1 = 0
c2 a21
c3 a31 a32
.. .. ..
. . .
cν aν1 aν2 . . . aν,ν−1
b1 b2 ... bν−1 bν
eg: One possible order-2 RK method is
0
1 1
1 1
2 2

7
4 Solving Systems of Linear Equations
• Simple Systems to solve (Ax = b):
– Diagonal Matrix: If A = diag(a1 , a2 , . . . , an ) then A−1 = diag( a11 , a12 , . . . , a1n ).
x = diag( a11 , a12 , . . . , a1n )b
– Unitary Matrix: In this case, A−1 = AT . So we can directly obtain
x = AT b.
– Upper triangular matrix (lower is similar): Back Substitution
Method: first solve for the equation that looks like an,n xn = bn , then
substitute in an−1,n−1 xn−1 + an−1,n xn = bn−1 and solve for xn−1
and so on, all the way back upto x1 .
• Gaussian Elimination:
– In the simplest case, no pivoting is required, i.e. the diagonal entries
are non-zero throughout the procedure. In this course, we only reduce
A to upper diagonal form and then solve via back-substitution.
(
A i)
– Multipliers: The mij = Aijii where the superscript i denotes that
these values are as calculated at the ith step.
– Pivoting: Exchanging rows before we start elimination so that no
diagonal entry is ever zero.
– Theorem: No pivotining is needed if each k th order submatrix
 
A11 A12 . . . A1k
A21 A22 . . . A2k 
∆k =  .
 
.. .. 
 .. . . 
Ak1 Ak2 . . . Akk

is non-singular.
• LU factorisation

– L should be lower triangular, each diagonal entry is 1. U should be

upper traingular.
– LU factorisation exists whenever A is invertible and is unique. entries
of L are multipliers seen in Gaussian elimination, U is the last step
upper triangular matrix from Gaussian elimination.

• Cholesky Factorisation
– A = BB T where A should be symmetric positive definite, and B
should be lower triangular and with all diagonal entries positive.
– Positive definite: A matrix is positive definite iff all its eigenvalues
are real and positive, iff y T Ay > 0 for all column vectors y.

8
 – If A is positive semidefinite, then the B we get is unique.
– If we try Cholesky factorisation on a matrix with zero or negative
eigenvalue, we will get a diagonal entry of B as either 0 or imaginary
repectively - fast test for positive definite.
• QR factorisation
– We factor A = QR, where Q is orthogonal, i.e. Q−1 = QT (or
equivalently all its columns are orthonormal vectors) and R is upper
triangular.
– This factorisation is unique, if we demand that diagonal entries of R
are positive.
– Columns of Q are obtained as Gram-Scmidt orthonormalisation of
columns of original A.
– Rki = hqk , Ai i if 0 < k < i
Pi−1
Rii = kAi − j=1 hqj , Aj iqj k And all other entries 0 (upper triangu-
lar)
• Norm Equivalence: Two norms k.k1 and k.k2 are considered equivalent
iff ∃m, M > 0 such that mkxk1 ≤ kxk2 ≤ M kxk1 for all x.Note that this
happens to be reflexive,symmetric and transitive relation.
Theorem: in a finite dimensional vector space, all norms are equivalent.
• The lp norm: v
u n
u
p
X
kxklp := t |xk |p
k=0

kxkl∞ = max {|xi |}

1≤i≤n

• Equivalence in the lp norms

1 √
p
√
p kxkl∞ ≤ kxklp ≤ nkxkl∞
n
√
kxkl2 ≤ kxkl1 ≤ nkxkl2

• Matrix Norm: A norm on Rn×n is said to be a matrix norm iff it satisfies

kABk ≤ kAkkBk for all matrices A, B. eg: Frobenius Norm (l2 norm on
Rn×n )
• Subordinate Matrix Norm: For a given norm k.k on Rn and given any
A ∈ Rn×n , we define the subordinate matrix norm as
kAxk
kAksub = sup
x∈Rn kxk

• Properties of Subordinate Matrix Norm:

9
 – Subordinate Matrix Norm is a Matrix norm (satisfies the inequality)
– Subordinate Matrix norm of identity matrix is 1.
– The matrix norm subordinate to lp norm of any unitary matrix is 1.
– If A is unitary and B is arbitrary then kABk2 = kBAk2 = kBk2
– For a diagonal matrix A = diag(a1 , a2 , . . . , an ), kAk2 = max{|ai |}
• Normal Matrices: A† A = AA† , characterisation theorem states that all
normal matrices can be written as U DU † where D is diagonal, with entries
as eigenvalues of original A. Note that hermetian (A = A† )matrices are
normal.
• Thus, for normal matrices, the kAk2 = biggest |λ| where λ iterates over
all eigenvalues. We define ρ(A) = max{|λ| |λ is eigenvalue of A}. Then
kAk2 = ρ(A). Note that ρ(A) is only equal to kAk2 if A is normal, not in
general.

• For any generic matrix norm on A, we can prove that kAk ≥ ρ(A). Sim-
ilarly, we can prove that for any matrix A and for any  > 0, there is a
subordinate matrix norm such that kAk ≤ ρ(A) + 
• We can prove the following results to easily get the subordinate norms-
Pn
– kAk1 = max1≤j≤n { i=0 |Aij |}, i.e. max column sum of modulii.
– kAk2 = ρ(A) (for normal matrices)
Pn
– kAk∞ = max1≤i≤n { j=0 |Aij |} i.e. max row sum norm.

• For any matrix A, A† A is always Hermitian positive semidefinite. So, we

define the singular values of any matrix A as the non-negative square root
of an eigenvalue of A. For a normal matrix, its singular values are the
modulii of its eigenvalues.
• For any general A, kAk2 = largest singular value of A.

• Convergence: A sequence A(k) of matrices converges to A iff

lim kA(k) − Ak = 0.
k→∞

• The following are equivalent -

– lim Ak = 0 (powers)
k→∞

– lim Ak x = 0 for all x

k→∞
– ρ(A) < 1
– kAk < 1 in atleast one subordinate norm.

10
 • Geometric Series of matrices -
∞
X
(I − A)−1 = Ak
k=0

Converges iff ρ(A) < 1, and in this case, the LHS is well defined as well.
• If A is invertible, then all matrices in the open 1
kA−1 k neighbourhood of A
are invertible.
• Condition number of A is defined as kAkkA−1 k ≥ 1.

• If A = A + B and b = b + c where  is small, then the relative error

between the solutions of Ax = b and A x = b is
 
kx − xk kb − bk kA − Ak
= cond(A) + + O(2 )
kxk kbk kAk

• In general, we have the result cond2 (A) = µµmax

min
where cond2 means under
subordinate to l2 norm, and µ are the singular values.

5 Iterative Methods for linear equations

• Basic Notions: Suppose x is the exact solution to theAx = b. Then
consider a function f : Rn → Rn and the generated sequence x(i+1) =
f (x(i) ) with some initial guess for x(0) . We say that this iterative scheme
(characterised by f ) converges iff lim x(n) = x for any choice of initial
n→∞
guess x(0) .
• Error terms e(n) = x(n) − x (what we want to minimise)
Residuals r(n) = Ax(n) − b (what we can practically calculate/minimise)
r(k) = Ae(k)
• Splitting: (M, N ) is said to be a splitting of A if and only if
– A=M −N
– M is easily invertible.
• Basic scheme: For a splitting (M, N ) of A, we consider

x(k+1) = M −1 N x(k) + M −1 b

This converges iff ρ(M −1 N ) < 1 (spectral radius)

• Richardson’s Method/Gradient Descent Use M = 1

α In and N =
M −A
Converges iff |1 − αλi | < 1 for all eigenvalues λi of A.

11
• For a matrix with real eigenvalues (eg:real symmetric) matrix A with
eigenvalues λ1 ≤ λ2 ≤ · · · ≤ λn , the Richardson’s method converges iff all
eigenvalues have same sign and
0 < α < λ2n or λ21 < α < 0

• Under the conditions of the above theorem, the optimal value of α = 2

λ1 +λn
if all eigenvalues are positive.
Proved by minimising ρ(M −1 N ) = max |1 − αλi |
• If A is normal, then the optimal α corresponds to the spectral radius
λn −λ1 cond2 (A)−1
λn +λ1 = cond2 (A)+1 . This is an increasing function of cond2 (A)

• x(k+1) = x(k) + α(b − Ax(k) )

• Jacobi Method: Let D be the matrix diag(A11 , A22 , . . . Ann ) (i.e. diag-
onal entries of A.

M=D
N=D−A
−1
M N = I − D−1 A

• A matrix is said to be row(column)-diagonally dominant if every diagonal

entry is greater (in absolute value) than the sum of every other entry (sum
of modulii) in its row(column). Jacobi method must converge whenever a
matrix is diagonally dominant, since the matrix norm subordinate to l∞
(l1 ) leads to ρ < 1
• x(k+1) = x(k) + D−1 (b − Ax(k) )
• Gauss-Seidel Method We can write A = D + U0 + L0 where U and L
are strictly upper and lower triangular respectively. Then the Gauss-Seidel
method is given as

M = D + L0
N = −U0

Define E = −L0 and F = −U0 , and L = D−1 E, U = D−1 F

• The iteration matrix for Gauss-Seidel is then G = (I − L)−1 U and for
jacobi it is J = L + U.
Theorem: If matrix is strictly row-diagonally dominant, then kGk ≤
kJk ≤ 1 so Gauss-Seidel will converge and faster that jacobi.
• Stein-Rosenberg theorem: If the Jacobi iteration matrix J = L + U is
non-negative, then exactly one of the following holds
– ρ(G) = ρ(J) = 0

12
 – 0 < ρ(G) < ρ(J) < 1
– ρ(G) = ρ(J) = 1
– ρ(G) > ρ(J) > 1
So whenever Jacobi method converges (for J ≥ 0), Gauss Seidel will con-
verge even faster.
• Relaxed Gauss-Seidel Method
1
M= D−E
α
 
1
N= −1 D+F
α
 −1   
−1 1 1
M N = Gα = I−L −1 I+U
α α

• ρ(Gα ) ≥ |1 − α| so 0 < α < 2 is necessary (not sufficient) for relaxed

Gauss Seidel to converge
• Important Lemma: If A = M − N is Hermitian (A = A† ) and further,
if A and M + M † − A are both positive definite, then the real parts of
eigenvalues of A−1 (2M − A) are positive.
• Observe that if H = A−1 (2M − A) then M −1 N = (H − I)(H + I)−1 .
Further, under the conditions of the above lemma, we can prove that
every eigenvalue of (H − I)(H + I)−1 has modulus < 1 so ρ(M −1 N ) < 1
whenever A, M + M † − A are hermitian and positive definite.
• With this theorem we can easily prove that for real symmetric positive
definite matrices A, relaxed Gauss Seidel converges for any value of 0 <
α<2
• Gerschgorin Circle Theorem: Let Dr (z0 ) = {z ∈ C| |z − z0 | ≤ r}.
n
S
Then all eigenvalues of a matrix A must lie in the region DRi (Aii )
P i=1
where Ri = j6=i |Aij | (diagonal removed row sum)

6 Solving Algebraic Equations

• Intermediate Value theorem Given a continuous function f : [a, b] →
R with f (a)f (b) < 0, there exists ξ ∈ (a, b) such that f (ξ) = 0
• Fixed point theorem Given a continuous function f : [a, b] → [a, b],
there exists ξ ∈ [a, b] such that f (ξ) = ξ
• Fixed point iteration method/ Simple Iteration Start with some
initial guess x0 and construct the sequence xk+1 = f (xk ). If this sequence
converges, then it converges to a fixed point of f .

13
• Contraction Mapping A function g : [a, b] → R is said to be a contrac-
tion mapping if, there exists 0 < L < 1 such that for any x, y ∈ [a, b],
|g(x) − g(y)| ≤ L|x − y|. If the range of g is also [a, b] then we know that g
has a fixed point, and if such g is also a contraction, then the fixed point
iteration will converge to the UNIQUE fixed point, for any initial guess
x0 ∈ [a, b]
• Let ξ be a fixed point of g : [a, b] → R which is continuous in some
neighbourhood [ξ−h, ξ+h] and has dervative existing at ξ with |g 0 (ξ)| < 1.
Then g will be a continuous contraction in some neighborhood of ξ and
we can converge to ξ by fixed point iteration starting anywhere in this
neighborhood.However if |g 0 (ξ)| > 1 then we can never converge to ξ by
fixed point iteration method.
• Relaxation iteration To find the zeros of a function f , we can equiv-
alently try to find the fixed points of g(x) = x − λf (x) where λ is an
arbitrary non-zero parameter. (i.e. xk+1 = xk + λf (xk ))
• If we pick λ having the same sign as f 0 (ξ) (f (ξ) = 0) and with small
enough magnitude then by the convergence theorem of fixed point iteration
|1 − λf 0 | < 1 so the relaxation iteration will converge.
• Convergence theorem for relaxation iteration If we have f : [a, b] →
R which has a zero at ξ, and if f 0 is continuous in a neighbourhood of ξ
with f 0 (ξ) 6= 0 then there exists a λ, δ such that the relaxation iteration
with λ converges with any initial guess δ-close to ξ.
• Error sequence The sequence defined as ek = |xk − ξ| for any iteration
method sequence xk is called the error sequence.
If there exists a p ≥ 1 and a non negative constant C such that |ek+1 | <
C|ek |p (with C < 1 if p = 1) then we say that the sequence of iterates
converges with order atleast p.
• Simple iteration for fixed point converges with order 1 with C = L < 1
(contraction constant).
• Improved Relaxation iteration Rather than having a constant λ, we
allow λ = λ(xk ). This will converge whenever |1 − λ0 f − λf 0 | = |1 − λf 0 | <
1 (f (ξ) = 0, so that term drops)
• The smaller that |1 − λf 0 | is, the faster the method will converge. In
1
particular, we can choose λ(xk ) = f 0 (x k)
to make this quantity zero (the
minimum) and this leads to Newtop-Raphson Iteration
f (xk )
xk+1 = xk −
f 0 (xk )

• Let f : [ξ − δ, ξ + δ] → R be a C 2 function where f (ξ) = 0, f 0 (ξ) 6=

0, f ”(ξ) 6= 0 and suppose there exists m > 0 such that |f ”(x)|
|f 0 (x)| ≤ M for all

14
  1
x ∈ [ξ − δ, ξ + δ]. Then for any x0 ∈ [ξ − h, ξ + h] where h = min δ, M
the Newton’s method with initial guess x0 converges with order atleast 2
to ξ.
• Bisection Method Guess an interval [a, b] within which the root should
lie f (a)f (b) < 0. Compute f a+b


2
. If it is zero, we found a root and we are
done.Otherwise, either f (a)f a+b or f (b)f a+b


2 < 0 2 < 0. Accordingly,
halve the interval and iterate with the new interval.
• Secant Method Replace f 0 (xk ) in Newton’s method by the approxima-
tion f (xxkk)−f (xk−1 )
−xk−1

f (xk )(xk − xk−1 )

xk+1 = xk +
f (xk ) − f (xk−1 )

We can show that if f 0 (ξ) 6= 0 then this method is atleast first order with
C = 32

15