BMTE-144

NUMERICAL ANALYSIS
Indira Gandhi National Open University
School of Sciences

Block

2
SOLUTIONS OF LINEAR ALGEBRAIC EQUATIONS
Block Introduction 119
Notations and Symbols 120

UNIT 5
Direct Methods 121

UNIT 6
Inverse of a Square Matrix 169

UNIT 7
Iterative Methods 201

UNIT 8
Eigenvalues and Eigenvectors 247

Miscellaneous Examples and Exercises 285

 Course Design Committee*
Prof. Rashmi Bhardwaj Prof. Meena Sahai
G.G.S. Indraprastha University, Delhi University of Lucknow
Dr. Sunita Gupta Dr. Sachi Srivastava
L.S.R. College, University of Delhi University of Delhi
Prof. Amber Habib
Shiv Nadar University Faculty Members
Gautam Buddha Nagar, U.P School of Sciences, IGNOU
Prof. M. S. Nathawat (Director)
Prof. S. A. Katre
Dr. Deepika
University of Pune, Pune
Mr. Pawan Kumar
Prof. V. Krishna Kumar Prof. Poornima Mital
NISER, Bhubaneswar Prof. Parvin Sinclair
Dr. Amit Kulshreshtha Prof. Sujatha Varma
IISER, Mohali Dr. S. Venkataraman
Dr. Aparna Mehra
I.I.T. Delhi
Prof. Rahul Roy
Indian Statistical Institute, Delhi

* The course design is based on the recommendations of the Programme Expert Committee and the UGC-
CBCS template

Block Preparation Team

Prof. Poornima Mital (Editor) (Retd.) Prof. Deepika
School of Sciences School of Sciences
IGNOU, New Delhi IGNOU, New Delhi

Course Coordinators: Prof. Deepika and Dr. Pawan Kumar

Acknowledgement: To Sh. Santosh Kumar Pal and Sh. Prashant Kumar for the word
processing and to Sh. S. S. Chauhan for preparing CRC of this block. Parts of this block are
based on the course material of the elective course Numerical Analysis (MTE-10) of the
Bachelor’s Degree Programme.

Disclaimer – Any materials adapted from web-based resources in this course are being used for educational purposes
only and not for commercial purpose.

November, 2021
© Indira Gandhi National Open University, 2021
ISBN-978-93-89200-39-3
All right reserved. No part of this work may be reproduced in any form, by mimeograph or any other
means, without permission in writing from the Indira Gandhi National Open University.
Further information on the Indira Gandhi National Open University courses, may be obtained from the
University’s office at Maidan Garhi, New Delhi-110 068 and IGNOU website www.ignou.ac.in.
Printed and published on behalf of the Indira Gandhi National Open University, New Delhi by
118 Prof. Sujatha Varma, Director, School of Sciences.
BLOCK INTRODUCTION
In Block1, we have discussed various numerical methods for finding
approximate roots of a non-linear equation in one unknown. But there are large
number of physical, biological and engineering situations in which we need to
find the solution or the inverse or the eigenvalues and eigenvectors of the
systems of linear equations. These systems arise, both directly in modeling
physical situations and indirectly in the numerical solutions of other
mathematical models. For instance, problems such as determining the potential
in certain electrical networks, stresses in a building frame, flow rates in a
hydraulic systems etc. are all reduced to solving a set of algebraic equations
simultaneously. Linear algebraic systems are also involved in the optimization
theory, least squares fitting of data, numerical solution of boundary value
problems for ordinary and partial differential equations, statistical inference etc.
In this block we shall discuss both direct and iterative methods of solving linear
algebraic system of equations.

This block consists of four units.

In Unit 5, which is the first unit of this block, we begin with a recall of a few
definitions and properties of matrices and determinants which are necessary to
understand the numerical methods of solving linear system of equations. We
shall then discuss some direct methods i.e., the methods which, in the absence
of round-off or other errors, yield the exact solution in a finite number of
elementary arithmetic operations. These methods include Cramer’s rule, direct
methods for solving system of linear equations with special coefficient matrices
such as diagonal matrix, lower-triangular matrix and upper-triangular matrix,
Gauss elimination method and LU decomposition method.
In Unit 6, we shall discuss the method of adjoints, the Gauss-Jordan reduction
method and LU decomposition method for finding the inverse of a non-singular
square matrix.

In Unit 7, we shall discuss three iterative methods namely, the Jacobi iteration
method, the Gauss-Seidel iteration method and successive over relaxation
method for solving the system of linear equation. These methods start with an
initial approximation and by applying a suitable chosen algorithm, lead to
successively better approximations.

In Unit 8, which is the last unit of this block, we shall deal with the problem of
computation of the absolutely largest eigenvalue or smallest eigenvalue or even
all the eigenvalues of a given square matrix along with the corresponding
eigenvectors. More precisely, we shall discuss the power method and the
inverse power method for solving the eigenvalue problems. We shall also
discuss Gershgorin bounds to find the feasible region for the eigenvalues of a
square matrix.
As already mentioned in the course introduction, we shall be using a non-
programmable scientific calculator for doing computations throughout the block. While
attempting the exercises given in this block, you would also need a calculator. We
therefore suggest you to use the non-programmable scientific calculator.

Lastly we remind you to go through the solved examples carefully, and to attempt all
exercises in each unit and all exercises given in miscellaneous examples and
exercises. This will help you to gain some practice over various methods discussed in
this block.
119
 NOTATIONS AND SYMBOLS

A = [a ik ] matrix with the elements a ik

det A = A Determinant of a square matrix A
∞ infinity
ρ Rho
ν Nu
µ Mu
λ Lambda
A Norm of a matrix A
i Imaginary unit, i 2 = −1.
O( x ) The growth function of x. It tends to 0 as x → 0.
l ij element of ith row and jth column of a lower triangular matrix.
u ij element of ith row and jth column of a upper triangular matrix.

120
Unit 5 Direct Methods

UNIT 5

DIRECT METHODS

Structure Page No.

No.

5.1 Introduction 121

Objectives
5.2 Essential Preliminaries 122
5.3 Cramer’s Rule 130
5.4 Direct Methods for Special Matrices 134
5.5 Gauss Elimination Method 138
5.6 Gauss Elimination Method with Pivoting 148
5.7 LU Decomposition Method 154
5.8 Summary 160
5.9 Solutions/Answers 161

5.1 INTRODUCTION
In Block 1, we discussed various numerical methods for finding the
roots of an equation f ( x ) = 0 along with the convergence of these
methods. Another important problem of applied mathematics is to find
the solution of systems of linear equations. Systems of linear equations
arise in a large number of areas, both directly in modelling physical
situations and indirectly in the numerical solution of other mathematical
models. These applications occur in all areas of the physical, biological
and engineering sciences. For instance, in physics, the problem of
steady state temperature in a plate is reduced to solving linear
equations. Engineering problems such as determining the potential in
certain electrical networks, stresses in a building frame, flow rates in a
hydraulic system etc. are all reduced to solving a set of algebraic
equations simultaneously. Linear algebraic systems are also involved
in the optimization theory, least squares fitting of data, numerical
solution of boundary value problems for ordinary and partial differential
equations, statistical inference etc. Hence, the numerical solution of
system of linear algebraic equations play a very important role. 121
Block 2 Solutions of Linear Algebraic Equations
Also, in Block 1, we discussed the direct and indirect methods for
solving non-linear equations in one variable. You may recall that direct
methods are the methods which use a formula and give the exact
results, whereas indirect methods are the iterative methods which are
used to find the approximate solution in successive iterations until the
desired accuracy is achieved. In this unit we will discuss direct methods
for solving the system of linear equations. We will also discuss indirect
methods for solving the system of linear equations in Unit 7.

To understand the numerical methods for solving linear system of

equations it is necessary to have some knowledge of the properties of
matrices. Accordingly, we begin with a quick recall of few definitions
related to matrices in Sec. 5.2 [Ref. BMTE-141]. We will discuss
Cramer’s rule for finding the solution of system of linear algebraic
equations in Sec. 5.3. In Sec. 5.4, we will discuss back substitution
method and forward substitution method for special coefficient matrices.
In Sec. 5.5 and Sec. 5.6, we will discuss Gauss elimination method
without and with pivoting, respectively. At the end, we will discuss LU
decomposition method in Sec. 5.7.

Objectives
After studying this unit, you should be able to:
• define various terms related to matrices;
• obtain the solution of system of linear algebraic equations by using
the direct methods such as Cramer’s rule, Gauss elimination method
and LU decomposition method;
• use the pivoting technique while transforming the coefficient matrix
to upper or lower triangular matrix.

5.2 ESSENTIAL PRELIMINARIES

You might be already familiar with vectors, matrices, determinants and
their properties. We will recall some of the useful definitions from the
course Linear Algebra (BMTE-141) here.

i) Matrix: A rectangular array of (real or complex) numbers of the

form
 a 11 a 12 K a 1n 
a a 22 a 2n 
 21 
 M 
 
a m1 a m2 K a mn 

is called a matrix. The numbers a ij , i = 1, 2,..., m and j = 1, 2,..., n may

be real or complex numbers.

ii) Elements of a matrix: The numbers a 11 , a 12 , K , a mn are the

elements of the matrix A = [a ij ], i = 1, 2, K, m and j = 1, 2, K, n.

iii) Row of a matrix: The horizontal lines are called rows of a matrix.
122
Unit 5 Direct Methods
iv) Column of a matrix: The vertical lines are called columns of a
matrix.

v) Order of a matrix: A matrix with m rows and n columns is called

an m× n matrix (read as m by n matrix) or is called a matrix of
order m by n. The number of rows and columns of a matrix specify
its order. We usually denote matrices by capital letters A, B etc., or
by (a jk ), (b ik ) etc.

vi) Square matrix: If the matrix has the same number of rows and
columns, i.e., if m = n, we call it a square matrix.

vii) Column matrix: If a matrix has only one column it is a column

matrix or column vector. For example, the matrix
 a 11 
a 
A =   is a column matrix.
21

 M 
 
a n1 

viii) Row matrix: If a matrix has only one row it is a row matrix or row
vector. For example, B = [a 11 , a 12 , K, a 1n ] is a row matrix.

We give below some special square matrices A = (a ij ) of order n .

i) Null Matrix: A matrix A = (a ij ) in which a ij = 0 (i, j = 1, 2, K, n ) is

called a null matrix and is denoted by 0 . For example,
0 0
A=  is a 2 × 2 null matrix.
0 0

ii) Diagonal Matrix: A matrix D in which all the non-diagonal

elements vanish i.e., a ij = 0 for i ≠ j is called a diagonal matrix.
For example,
a 11 0 0 
 
D =  0 a 22 0  is a 3 × 3 diagonal matrix.
 
 0 0 a 33 

iii) Identity Matrix: The identity matrix I is a diagonal matrix in

which all the diagonal elements are equal to one. The identity
matrix of order 4 is
1 0 0 0
0 1 0 0
I= .
0 0 1 0
 
0 0 0 1

iv) Lower Triangular Matrix: A square matrix is lower triangular, if

all the elements above the main diagonal vanish i.e., a ij = 0 for
j > i . For example, 123
Block 2 Solutions of Linear Algebraic Equations

 a 11 0 0 
 
L = a 21 a 22 0  is a lower triangular matrix.
 
a 31 a 32 a 33 

v) Upper Triangular Matrix: Upper triangular matrices are the

matrices in which, a ij = 0 for i > j . For example,

a 11 a 12 a 13 
 
U= 0 a 22 a 23  is a upper triangular matrix.
 
 0 0 a 33 

vi) Equality of Matrices: Two matrices A = (a ij ) and B = (b ij ) are

equal iff they have the same number of rows and columns and
there corresponding elements are equal, that is, a ij = b ij for all i, j .
For example,
a b c   − 1 2 1
let A =   and B =  .
d e f   3 4 5
If A = B, then a = −1, b = 2, c = 1, d = 3, e = 4, f = 5.

Let us now discuss some of the operations on matrices.

i) Addition and subtraction of matrices: Addition and subtraction

are defined only for matrices of same order. For two matrices A
and B the sum A + B and the difference A - B are obtained by
adding or subtracting the corresponding elements of A and B,
respectively, i.e., a ij ± b ij .

− 4 6 3 5 − 1 0
For example, if A =   and B =   , then
 0 1 2 3 1 0
1 5 3  − 9 7 3
A+B =  and A − B =  
3 2 2  − 3 0 2

ii) Product of Matrices: Product of an m × n matrix A = (a ij ) and an

n × p matrix B = (b jk ) is an m × p matrix C . C = AB , whose
(i, k ) th entry is
n
c ik =  a ij b jk = a i1 b1k + a i 2 b 2 k + L + a in b nk .
j=1

That is, to obtain the (i, k ) th element of AB , take the ith row of A
and kth column of B, multiply their corresponding elements and
add up all these products. For example, if
1 1 2
2 3 − 1
A=  and B = 2 4 1  , then (1, 2) th element
1 0 2  1 2 1 
124
Unit 5 Direct Methods
of AB is

1
[2, 3, − 1] 4 = 2 × 1 + 3 × 4 + (−1) × 2 = 12 .
2

Note that two matrices A and B can be multiplied only if the

number of columns of A equal the number of rows of B. In the
above example the product BA is not defined because the
number of columns of B is not equal to the number of rows of A.

iii) Transpose of a matrix: The matrix obtained by interchanging the

rows and columns of A is called the transpose of A and is
denoted by A T . If order of the matrix A is m × n , then the order of
its transpose A T is n × m. For example,
 2 3  2 − 1
if A =   then A T =  
− 1 1 3 1 
You may note that the transpose of a row matrix is a column
matrix and vice-versa. For example, if

1
3
 
A =  5  , then AT = [1 3 5 − 2 4].
 
− 2
 4 

iv) Determinant of a Matrix: Determinant is a number associated

with square matrices. Determinant of a square matrix A is
denoted by det ( A) or A .

a a 12 
For a 2 × 2 matrix A =  11
a 21 a 22 
 a 11 a 12  a 11 a 12
det(A ) = det  = = a 11a 22 − a 12 a 21
a 21 a 22  a 21 a 22
 a 11 a 12 a 13 
For a 3 × 3 matrix A = a 21 a 22 a 23 
a 31 a 32 a 33 
a 22 a 23  a 21 a 23  a 21 a 22 
det(A ) = a 11 det  − a det + a det
a 33  a  a a 32 
12 13
a 32  31 a 33   31

A determinant can be expanded about any row or column. The

determinant of an n × n matrix A = (a ij ) expanded about ith row is
given by
det(A) = (−1) i +1 a i1 det(A i1 ) + (−1)1+ 2 a i 2 det(A i 2 ) + L + (−1) i +n a in det(A in ),
where A ij is the (n − 1) × (n − 1) matrix obtained from A by deleting
125
Block 2 Solutions of Linear Algebraic Equations
the ith row and jth column where 1 ≤ i ≤ n and i ≤ j ≤ n. Obviously,
computation is simple if det(A ) is expanded along a row or column
that has maximum number of zeros. This reduces the number of
terms to be computed.
The following example will help you to get used to calculating
determinants.

1 2 6
Example 1: If A = 5 4 1  , calculate det(A ).
7 3 2

Solution: Let us expand by the first row, we have

4 1
| A11 | = = 4 × 2 − 1× 3 = 5,
3 2
5 1
| A12 | = = 5 × 2 − 7 ×1 = 3
7 2
5 4
| A13 | = = 5 × 3 − 4 × 7 = −13
7 3
Thus,
det(A ) = | A | = (−1)1+1 × a 11 × | A11 | + (−1)1+ 2 × a 12 × | A12 | + (−1)1+3 × a 13 × | A13 |

= 1 × 1 × 5 − 1 × 2 × 3 + 1 × 6 × (−3) = 5 − 6 − 78 = −79
***

 4 − 2 3
Example 2: Consider the matrices A = − 2 1 6 and
 1 2 2
 1 −3 0 
B =  − 3 1 0 . Find A T , B T , ( A + B) T , ( AB) T . Also, verify the
 0 0 − 2
following:
i) ( A + B) T = A T + B T
ii) ( AB) T = B T A T

Solution: The transpose of the matrix A is

 4 − 2 1
A =  − 2 1 2
T

 3 6 2
The transpose of the matrix B is
 1 −3 0 
B = − 3 1
T
0 
 0 0 − 2
126
Unit 5 Direct Methods
 5 − 5 3
A + B = − 5 2 6
 1 2 0
 5 − 5 1
( A + B) T = − 5 2 2
 3 6 0
 4 − 2 3  1 − 3 0   10 − 14 − 6 
AB = − 2 1 6  − 3 1 0  = − 5 7 12 
 1 2 2  0 0 − 2 − 5 − 1 − 4
 10 − 5 − 5
( AB) T =  − 14 7 − 1 
 − 6 12 − 4
 4 − 2 1   1 − 3 0  5 − 5 1
i) A T + B T = − 2 1 2 + − 3 1 0 = − 5 2 2 = ( A + B ) T
 3 6 2  0 0 2  3 6 0 
 1 − 3 0   4 − 2 1  10 − 5 − 5
ii) B T A T = − 3 1 0  − 2 1 2 =  − 14 7 − 1 = ( AB) T .
 0 0 − 2  3 6 2  − 6 12 − 4
***

You may now try the following exercises.

 a − b 2a + c  − 1 8 
E1) Find the values of a , b, c and d, if  = .
 2a − b 3c + d   0 17

2 1 3 1 − 2
E2) If A =  3 − 2 1 and B =  2 1 , find AB and BA.
 
− 1 0 1  4 − 2

− 3 − 2 0 2
2 1 0 − 1
E3) If A =  , calculate det(A) .
1 0 1 2
 
2 1 −3 1 

We shall now give some more definitions related to square matrices.

i) Singular and Non-Singular Matrix: If the determinant of a square

matrix A has the value zero, then the matrix A is called a singular
matrix, otherwise, A is called a non-singular matrix.
127
Block 2 Solutions of Linear Algebraic Equations
ii) Inverse of a matrix: The inverse of an n × n non-singular matrix A
is an n × n matrix B having the property
AB = BA = I,
where, I is an identity matrix of order n × n .

The inverse matrix B, if it exists, is denoted by A −1 and is unique.

iii) Matrix of cofactors: For a matrix A = (a ij ) , the cofactor A ij of the

element a ij is given by A ij = (−1) i + j M ij , where M ij (minor) is the
determinant of the matrix of order (n − 1) × (n − 1) obtained from A
after deleting its ith row and the jth column. The matrix A c obtained
from A by replacing each element of A by its cofactor is called a
matrix of cofactors.

iv) Adjoint of a matrix: The transpose of the cofactor matrix A c of A

is called the adjoint of A and is written as adj( A ) . Thus,
adj( A ) = ( A c ) T

v) Matrices of a system of linear equations: Let us consider the

following system of n linear algebraic equations in n unknowns:
a 11 x 1 + a 12 x 2 + L + a 1n x n = b1
a 21 x 1 + a 22 x 2 + L + a 2 n x n = b 2
(1)
M M M
a n1 x 1 + a n 2 x 2 + L + a nn x n = b n

where the coefficients a ij and the constants b i (i = 1, K , n ) are real

and known. This system of equations in matrix form may be
written as
Ax = b
where
 a 11 a 12 K a 1n   x1   b1 
a a 22 K a 2 n   x2  b 
A=  21  , x=   ,b= 
2
(2)
 M M M  M M
     
a n1 a n 2 K a nn  x n  b n 
The matrix A is called the coefficient matrix and has real
elements, the column matrix b is called a constant matrix and
the matrix x is called a variable matrix.

In order to find the solution of system of Eqn. (2), we need to find

the values x i , i = 1, 2, K , n if they exist, satisfying Eqn. (2). Before
we discuss some methods of solving the system of Eqn. (2), we
give few more definitions.

vi) Homogeneous and non-homogeneous systems of linear

equations: The system of Eqn. (2) is said to be homogeneous if
128
Unit 5 Direct Methods
b = 0, that is, all the elements b1 , b 2 , K , b n are zero, otherwise the
system is called non-homogeneous.

In this unit, we shall consider only non-homogeneous systems.

You also know from your knowledge of linear algebra course that the
non-homogeneous system of Eqn. (2) has a unique solution, if the
matrix A is non-singular. You may recall the following theorem (ref.
BMTE-141).

Theorem 1: A non-homogeneous system of n linear equations in n

unknowns has a unique solution if and only if the coefficient matrix A is
non-singular.

Remark 1: You may note that if A is non-singular, then A −1 exists, and

the solution of system (2) can be expressed as x = A −1b . (3)

In case the matrix A is singular, then the system (2) has no solution if
b ≠ 0 or has an infinite number of solutions if b = 0. Here we assume
that A is a non-singular matrix.

vii) Consistency in a system of linear equations: A system of linear

Eqn. (2) is said to be consistent if it has at least one solution. If
no solution exists, then the system is said to be inconsistent.

You may recall from Unit 1, Block 1 that the solution of non-linear
equations f ( x ) = 0 may be obtained using direct methods or iterative
methods, similarly, the methods of solving system (2) may be classified
into two types:

i) Direct Methods: Direct methods are the methods in which, in the

absence of round-off errors, the exact solution can be found in a
finite number of steps. Several direct methods are used to solve a
system of linear equations, such as Cramer’s rule, matrix inversion
method, Gauss elimination method, LU decomposition method,
etc. We will discuss these methods in this unit and the next unit.

ii) Iterative Methods: Starting with an approximate solution vector

x ( 0) , these methods generate a sequence of approximate solution
vectors {x ( n ) } which converge to the exact solution vector x as the
number of iterations n → ∞ . Thus, iterative methods are infinite
processes. Since we perform only a finite number of iterations,
these methods can only find some approximation to the solution
vector x . The examples of these methods are Jacobi iteration
method, Gauss-Seidal iteration method, etc. We shall discuss
iterative methods later in Units 7 and 8.

You may now try the following exercise.

E1) Check whether the following matrices are singular or non-singular.

129
 Block 2 Solutions of Linear Algebraic Equations

1 1 1
i) A = 2 3 1
0 − 1 1

1 5 2
ii) B =  5 4 − 7  .
− 3 0 5 

As we mentioned, herein we shall discuss only the direct methods. You

are familiar with one such method due to the mathematician Cramer
and known as Cramer’s rule. Let us discuss it in the following section.

5.3 CRAMER’S RULE

A simples method which can be used to solve the system of linear
equations is Cramer’s method. This method is due to the Genevan
mathematician Gabriel Cramer (1704-1752).

Let us consider the system of n linear equations in n unknowns

Ax = b, as given in (2).

Let the system be non-homogeneous i.e., b ≠ 0 and d = det(A) ≠ 0.

Gabriel Cramer’s
(1704-1752) a 11 a 12 K a 1n
a 21 a 22 K a 2n
Thus, d = ≠ 0 we multiply by x 1 both the sides and
M M M M
a n1 a n 2 K a nn
get

a 11 a 12 ... a 1n
a 21 a 22 ... a 2 n
x 1d = x 1
M M M M
a n1 a n 2 K a nn

x 1a 11 a 12 ... a 1n
x 1a 21 a 22 ... a 2n [Using the properties of determinant,
= Ref. P4(a), Unit 11, BMTE-141]
M M M M
x 1a n1 a n2 a n3 a nn

x 1a 11 + x 2 a 12 + ... + x n a 1n a 12 L a 1n
x 1a 21 + x 2 a 21 + ... + x n a 2n a 22 L a 2n [Using
= C1 = C1 + x 2 C 2 + x 3 C 3 + ... + x n C n ;
M M
Ref. P4 (b), Unit 11, BMTE-141]
x 1a n1 + x 2 a n 2 + ... + x n a nn a n 2 L a nn

130
Unit 5 Direct Methods
b1 a 12 ... a 1n
b2 a 22 ... a 2 n
=
M M M M
bn a n2 ... a nn

= d 1 (let),

where d1 is the determinant of the matrix obtained from A by

replacing the 1st column of A by the column vector b.

d1
or we can write x 1 = .
d

Similarly, we can obtain other variables

d2
x2 = ,
d

d3
x3 = ,
d
M M
d
xn = n .
d
di
In general x i = , 1 ≤ i ≤ n, (4)
d
where d = det(A ) = | A |≠ 0 and d i is the determinant of the
matrix obtained from A by replacing the ith column of A by the
column vector b.

Let us illustrate the method through the following examples.

Example 3: Solve the system of equations

3x 1 + x 2 + 2 x 3 = 3
2x 1 − 3x 2 − x 3 = −3
x 1 + 2x 2 + x 3 = 4
using Cramer’s rule.
3 1 2
Solution: The coefficient matrix A = 2 − 3 − 1 , and the constant

1 2 1 
3
matrix b = − 3 We have
 4 
3 1 2
d = | A | = 2 − 3 −1 = 8 ≠ 0
1 2 1 131
Block 2 Solutions of Linear Algebraic Equations

3 1 2
d1 = − 3 − 3 − 1 = 8 (first column in A is replaced by the column vector b )
4 2 1
3 3 2
d 2 = 2 − 3 − 1 = 16 (second column in A is replaced by the column vector b )
1 4 1
3 1 3
d 3 = 2 − 3 − 3 = −8 (third column in A is replaced by the column vector b )
1 2 4

Using Eqn. (4), we get the solution

d d d
x 1 = 1 = 1; x 2 = 2 = 2; x 3 = 3 = −1 .
d d d
***

Let us apply Cramer’s rule to a physical situation. Consider the following

example.

Example 4: A person invested ` 10,000 into two accounts. In one of the

accounts he received 8% interest annually and in another account he
received 5% interest annually. At the end of a year, he had a total of
` 10,710 in the two accounts. How much he invested in each account?

Solution: Let the amount invested be ` x and ` y in the first and second
account respectively.
It is given that,
x + y = 10000 (5)
8 5
and x +x× + y + y× = 10710.
100 100

interest in interest in
first account second account

or 108x + 105 y = 1071000 (6)

Thus, Eqns. (5) and (6) represent the system of linear equations. Let us
use Cramer’s rule to find the solution of the given problem. Here we
 1 1   10000 
have A =   and b =   and the variables to be obtained
108 105 1071000
are x and y.

1 1
Now, d = A = = −3 ≠ 0
108 105
10000 1
d1 = = 10000 × 105 − 1071000 × 1 = −21000
1071000 105

132
Unit 5 Direct Methods
1 10000
d2 = = 1 × 1071000 − 100000 × 108 = −9000
108 1071000
Using Eqn. (4), we get
d1 − 21000
x= = = 7000
d −3
d 2 − 9000
and y = = = 3000
d −3

Thus, the person invested ` 7000 in the first account and ` 3000 in the
second account.
***

You may now try the following exercises.

E5) Using Cramer’s rule, solve the following system of equations

3x 1 + 5x 2 = 8
− x1 + 2x 2 − x 3 = 0
3x 1 − 6 x 2 + 4x 3 = 1 .

E6) Using Cramer’s rule solve the following system of equations

x 1 + 2x 2 − 3x 3 + x 4 = −5
x 2 + 3x 3 + x 4 = 6
2x 1 + 3x 2 + x 3 + x 4 = 4
x1 + x 3 + x 4 = 1 .

E7) A company is making health mix of three types H1 , H 2 and H 3 .

Each of which is the combination of the ingredients almonds,
cashews and cranberries in the ratios 1 : 2 : 1, 2 : 1 : 1 and 2 : 0 : 2,
respectively. If the weights of H1 , H 2 and H 3 are 40g, 50g, and
60g, respectively, then find the total amount of each ingredient
used to make these health mix.

While going through the example and attempting the exercises you
must have observed that in Cramer’s method we need to evaluate n + 1
determinants each of order n , where n is the number of equations. If
the number of operations required to evaluate a determinant is
measured in terms of multiplications only, then to evaluate a
determinant of second order, i.e.,
a 11 a 12
= a 11a 22 − a 12 a 21
a 21 a 22

we need two multiplications or (2 − 1) 2! multiplications. To evaluate a

determinant or third order
133
Block 2 Solutions of Linear Algebraic Equations

a 11 a 12 a 13
a 21 a 22 a 23 = a 11a 22 a 33 − a 11a 23 a 32 − a 12 a 21a 33 + a 12 a 23 a 31 + a 13a 21a 32 − a 13 a 22 a 31
a 31 a 32 a 33

we need 12 multiplications or (3 − 1) 3! multiplications. In general, to

evaluate a determinant of nth order we need (n − 1) n! multiplications.

Also, for a system of n equations, Cramer’s rule requires n + 1

determinants each of order n and performs n divisions to obtain
x i , i = 1, 2, K, n . Thus, the total number of operations (multiplications
and divisions) needed to solve a system of n equations, using Cramer’s
rule, becomes
M = total number of multiplications + total number of divisions
= (n + 1) (n − 1) n!+ n

In Table 1, we are giving the values of M for different values of n .

Table 1

Number of Equations Number of Operations

n M = (n + 1)(n - 1) n!+n
2 (2 + 1)(2 − 1)2!+2 = 8
3 (3 + 1)(3 − 1)3!+3 = 51
4 (4 + 1)(4 − 1)4!+4 = 364
5 (5 + 1)(5 − 1)5!+5 = 2885
6 (6 + 1)(6 − 1)6!+6 = 25206
7 (7 + 1)(7 − 1)7!+7 = 241927
8 (8 + 1)(8 − 1)8!+8 = 2540168
9 (9 + 1)(9 − 1)9!+9 = 29030409
10 (10 + 1)(10 − 1)10!+10 = 359251210

From Table 1, you will observe that as n increases, the number of

operations required for Cramer’s rule increases very rapidly. For this
reason, Cramer’s rule is not generally used for n > 4 . Hence, for
solving large systems, we need more efficient methods. In the next
section, we describe some direct methods which depend on the form of
the coefficient matrix.

5.4 DIRECT METHODS FOR SPECIAL MATRICES

Suppose we write the system of n linear equations in n unknown in
matrix notation as Ax = b, where A is the coefficient matrix which is
non-singular, x is the variable matrix and b is the constant matrix.
When we solve such system of linear equations, the solution depends
on the form of matrix A. We will discuss here three special forms of
matrix A namely; diagonal matrix, lower triangular matrix and upper
triangular matrix, for which solution vector x can be obtained directly.
Let us discuss these cases one by one.
134
Unit 5 Direct Methods
Case 1: When A is non-singular diagonal matrix

When A is a non-singular diagonal matrix D, a ij = 0 for i ≠ j, where We denote diagonal

i, j = 1, 2, K, n. In this case, the system of Eqn. Ax = b is of the form matrix by capital letter
D.

a 11 0 L 0   x 1   b1  a 11 x 1 + 0 + L + 0 = b1
0 a 22 0   x   b  0 + a 22 x 2 + L + 0 = b 2
   2 =  2  (7)
 M O M M M M M M M
    
0 L a nn   x n  b n  0 + 0 + L + a nn x n = b n

and det(A) = det(D) = a 11a 22 Ka nn ≠ 0.

Since, the matrix A is non-singular, a ii ≠ 0 for i = 1, 2, K , n and we

obtain the solution as
b1
a 11 x 1 = b1  x 1 =
a 11
b2
a 22 x 2 = b 2  x 2 =
a 22
L L L
bn
a nn x n = b n  x n =
a nn
In general, we write
bi
xi = , a ii ≠ 0, i = 1, 2, K , n
a ii

You may note that in this case we need only n divisions to obtain the
solution vector.

Case 2: When A is non-singular lower triangular matrix

We denote lower
triangular matrix by
In this case, since A is non-singular lower triangular matrix L,
capital letter L.
a ij = 0, j > i. The system of Eqn. Ax = b is of the form

 a 11 0 0 L 0   x 1   b1  a 11 x1 + 0 + 0 + L + 0 = b1
a a 22 0 L 0 x2     b  a 21 x1 + a 22 x 2 + 0 + L + 0 = b 2
 21     2
a 31 a 32 a 33 L 0   x 3  =  b 3   a 31x 1 + a 32 x 2 + a 33 x 3 + L + 0 = b 3 (8)
    
 M M M M  M    M M M M
a n1 a n2 a n3 L a nn   x n   b n  a n1x 1 + a n 2 x 2 + a n 3 x 3 + L + a nn x n = b n

and det(A ) = a 11a 22 K a nn ≠ 0.

You may note here that the first equation of the system (8) contains
only x 1 , the second equation contains only x 1 and x 2 and so on.
Hence, we find x 1 from the first equation, x 2 from the second equation
and proceed in that order till we get x n from the last equation. 135
Block 2 Solutions of Linear Algebraic Equations
Since the coefficient matrix A is non-singular, a ii ≠ 0, i = 1, 2, K, n . We
thus obtain
b
a 11 x 1 = b1  x 1 = 1
a 11
(b − a 21 x 1 )
a 21 x 1 + a 22 x 2 = b 2  x 2 = 2
a 22
(b − a 31 x 1 − a 32 x 2 )
a 31 x 1 + a 32 x 2 + a 33 x 3 = b 3  x 3 = 3
a 33
 n −1 
 b n −  a nj x j 
 
a n1x 1 + a n 2 x 2 + a n 3 x 3 + ... + a nn x n = b n  x n =  j=1 
a nn

In general, we have for any i

 i −1 
 b i −  a ij x j 
 
xi =   , i = 1, 2, K, n
j=1
(9)
a ii

Let us illustrate this case through the following example.

Example 5: Consider the following system of linear equations

5x 1 = 5
− x 1 − 2 x 2 = −7
− x 1 + 3x 2 + 2 x 3 = 5
Find the solution of this system.

Solution: From the first equation of the system we have,

x1 = 1
From the second equation of the system we get,
− 7 + x1
x2 = =3
−2
and from the third equation of the system we have,
5 + x 1 − 3x 2 3
x3 = =− .
2 2
***

Since the unknowns in this method are obtained in the order

x 1 , x 2 , K, x n , this method is called the forward substitution method.

The total number of operations (multiplications and divisions) needed to

obtain the complete solution vector x , using this method is
n (n + 1)
M =1+ 2 +L+ n = .
2

Case 3: When A is non-singular upper triangular matrix.

We denote upper
triangular matrix by
capital letter U. In this case, since A is non-singular upper triangular matrix
U, a ij = 0, j < i. The system of Eqn. Ax = b is of the form
136
Unit 5 Direct Methods
a 11 a 12 a 13 L a 1n   x1   b1  a 11x 1 + a 12 x 2 + a 13 x 3 + L + a 1n x n = b1
0 a 22 a 23 L a 2 n  x  b  0 + a 22 x 2 + a 23 x 3 + L + a 2n x n = b 2
   2  2
0 0 a 33 L a 3n   x 3  =  b3   0 + 0 + a 33 x 3 + L + a 3n x n = b 3
      (10)
 M M M M M  M M M M M M M
 0 0 L L a nn   x n  b n  0 + 0 + 0 + L + a nn x n = b n

and det(A ) = a 11a 22 K a nn .

You may note here that the nth (last) equation contains only x n , the
(n − 1) th equation contains x n and x n −1 and so on. We can obtain x n
from the nth equation, x n −1 from the (n − 1) th equation and proceed in
that order till we get x 1 from the first equation. Since, the coefficient
matrix A is non-singular, a ii ≠ 0, i = 1, 2,K, n and we obtain
bn
a nn x n = b n  x n =
a nn
(b n−1 − a ( n −1) n x n )
a ( n −1)(n −1) x n −1 + a ( n−1) n x n = b n−1  x n −1 =
a ( n −1) ( n −1)
 n 
 b1 −  a 1 j x j 
 
Similarly, x 1 =  
j= 2

a 11
In general, we write
 n 
 b i −  a ij x j 
 
xi =   , i = 1, 2, K, n
j= i +1

a ii

Since the unknowns in this method are determined in the order

x n , x n −1 , K , x 1 , this method is called the back substitution method.
The total number of operations (multiplications and divisions) needed to
obtain the complete solution vector x using this method is again
n (n + 1)
.
2

Let us consider the following example.

Example 6: Solve the linear system of equations

2x 1 + 3x 2 − x 3 = 5
− 2x 2 − x 3 = −7
− 5x 3 = −15

Solution: From the last equation of the system, we have

x3 = 3
From the second equation of the system we have
b − a 23 x 3 (−7 + 3)
x2 = 2 = = 2,
a 22 (−2) 137
Block 2 Solutions of Linear Algebraic Equations
From the first equation of the system, we get
b1 − a 12 x 2 − a 13 x 3 5 − (3)(2) + (1)(3) 5 − 6 + 3
x1 = = = =1
a 11 2 2
***

You may now try the following exercises.

E8) Using forward substitution method, solve the followingsystem of

equations
x1 =1
2x 1 − x 2 =1
3x 1 − x 2 − 2 x 3 =0
4 x 1 + x 2 − 3x 3 + x 4 =3
5x 1 − 2x 2 − x 3 − 2 x 4 + x 5 = 1

E9) Using backward substitution method, solve the following system of

equations
x 1 − 2x 2 + 3x 3 − 4x 4 + 5x 5 = 3
x 2 − 2 x 3 + 3x 4 − 4 x 5 = −2
x 3 − 2x 4 + 3x 5 = 2
x 4 − 2 x 5 = −1
x5 =1

In the above discussion you have observed that the system of Eqn.
Ax = b can be easily solved if the coefficient matrix A has one of the
three forms, diagonal matrix, lower triangular matrix or upper triangular
matrix. Sometimes, if the matrix is not given directly in either of the
three forms, we can transform it to one of these forms. Now, you would
like to know how to reduce the given matrix A into one of these three
forms? One such method which transforms the matrix A to the upper
triangular matrix U is the Gauss elimination method which we shall
describe in the next section.

5.5 GAUSS ELIMINATION METHOD

Gauss elimination is one of the oldest and most frequently used
methods for solving systems of linear equations. It is attributed to the
famous German mathematician, Carl Friedrick Gauss (1777 – 1855).
This method is the generalization of the familiar method of eliminating
one unknown between a pair of simultaneous linear equations. In this
method the matrix A is reduced to the upper triangular matrix form U
by using the elementary row operations. You must have learnt
elementary row operations in your course BMTE-141 (Linear Algebra).

The operation R i ← R i + mR j is an elementary row operation, that

means,
138
Unit 5 Direct Methods
• add to the element of the ith row m times the corresponding
elements of the jth row. The elements of the jth row ( j ≠ i) remain
unchanged.

For example operation R 2 ← R 2 + 3R 3 applied to the matrix

 a 11 a 12 a 13   a 11 a 12 a 13 
a a 22 a 23  results in a + 3a a 22 + 3a 32 a 23 + 3a 33 .
 21  21 31

a 31 a 32 a 33   a 31 a 32 a 33 

Note that to each element of 2nd row is added 3 times the

corresponding elements of the 3rd row. Whereas the elements of the
remaining rows viz; first and third rows remain unchanged.

We frequently use the following three elementary row operations:

i) interchanging any two rows (R i ↔ R j ),

ii) multiplying (or dividing) any row by a non-zero constant

(R i ← mR i ),
iii) adding (or subtracting) a constant multiple of one row to another
row (R i ← R i + m R j ) .

You may note that if any matrix A is transformed into another matrix B
by a series of elementary row operations, we say that A and B are
equivalent matrices. Formally, we have the following definition.

Definition: A matrix B is said to be row equivalent to a matrix A, if B

can be obtained from A by using a finite number of elementary row
operations.

Also, two linear systems Ax = b and A′x = b′ are equivalent and

provide the solution of one as a solution of the other. Thus, if a
sequence of elementary operations on Ax = b produces the new
system A * x = b * then the systems Ax = b and A * x = b * are
equivalent and have the same solution.

To understand the Gauss elimination method let us consider a system

of three equations:
a 11 x 1 + a 12 x 2 + a 13 x 3 = b 1
a 21 x 1 + a 22 x 2 + a 23 x 3 = b 2 (11)
a 31 x 1 + a 32 x 2 + a 33 x 3 = b 3

As you know that the purpose of Gauss elimination method is to reduce

Ax = b to an equivalent system A *x = b* , where A * is an upper
triangular matrix. To do so, we eliminate a 21 , a 31 and a 32 in two stages. In
the first stage, we eliminate a 21 and a 31 and in the second stage, we
eliminate a 32 .

For this, let a 11 ≠ 0 . 139

Block 2 Solutions of Linear Algebraic Equations
In the first stage of elimination we multiply the first equation of
−a 
system (11) by m 21 =  21  and add to the second equation. Then
 a 11 
−a 
multiply the first equation by m 31 =  31  and add to the third equation.
 a 11 
This eliminates x 1 from the second and third equations.

The new system called the first derived system then becomes
a 11 x 1 + a 12 x 2 + a 13 x 3 = b1

a (221) x 2 + a (231) x 3 = b (21) (12)

a (321) x 2 + a (331) x 3 = b (31)

where,
a 21 a
a (221) = a 22 − a 12 , a (231) = a 23 − 21 a 13
a 11 a 11
a a
b (21) = b 2 − 21 b1 , a 32 (1)
= a 32 − 31 a 12
a 11 a 11
(1) a a
a 33 = a 33 − 31 a 13 , b (31) = b 3 − 31 b 1 (13)
a 11 a 11

In the second stage of elimination we multiply the second equation of

 − a (1) 
system (12) by m 32 =  (132)  provided a (221) ≠ 0 and add to the third
 a 22 
equation. This eliminates x 2 from the third equation. The new system
called the second derived system becomes
a 11 x 1 + a 12 x 2 + a 13 x 3 = b1

a (221) x 2 + a (231) x 3 = b (21) (14)

a (332 ) x 3 = b3( 2)
where

( 2) (1) a (321) (1) ( 2) (1) a (321)

a 33 =a 33 − (1)
a , b
23 3 =b 3 − (1)
b (21) (15)
a 22 a 22

You may note here that the system of Eqn. (14) is an upper triangular
system and can be solved using the back substitution method provided
( 2)
a 33 ≠ 0 . The situation a (332) = 0 shall be discussed later in Example 9.

Let us illustrate the method through an example.

Example 7: Solve the following linear system

2x 1 + 3x 2 − x 3 = 5
4x 1 + 4x 2 − 3x 3 = 3 (16)
− 2x 1 + 3x 2 − x 3 = 1

140 using Gauss elimination method.

Unit 5 Direct Methods
Solution: In the first stage of elimination we eliminate x 1 from the
−4
second and third equations of the system (16). For this we add = −2
2
− (−2)
times the first equation to the second equation and add = 1 times
2
the first equation to the third equation.

We obtain the new system as

2x 1 + 3x 2 − x 3 = 5
− 2x 2 − x 3 = −7 (17)
6x 2 − 2 x 3 = 6
In the second stage of elimination we eliminate x 2 from the third
−6
equation of system (17). For this use add = 3 times the second
(−2)
equation to the third equation, and get
2x 1 + 3x 2 − x 3 = 5
− 2x 2 − x 3 = −7 (18)
− 5x 3 = −15

In system (18) the coefficient matrix is in upper triangular form.

Now to find the solution of the system, we use back substitution

method, and get
x 3 = 3.
−7+3
x2 = =2
−2
5 − 3(2) + 3
and x 1 = =1
2
Thus, x 1 = 1, x 2 = 2 and x 3 = 3.
***

You may observe that we can write the above procedure more
conveniently in matrix form. Since the arithmetic operations we have
performed here affect only the elements of the matrix A and the vector
b, , we consider the augmented matrix i.e., [ A | b] (the matrix A
augmented by the vector b ) and perform the elementary row operations
on the augmented matrix as shown below:
 a 11 a 12 a 13 b1 
[A | b] = a 21 a 22 a 23 b 2  R 2 ← R 2 − 21 R 1
a
  a 11
a 31 a 32 a 33 b 3 
a
R 3 ← R 3 − 31 R 1
a 11
a 11 a 12 a 13 b1 
≈  0 a (221) a (231) b (21)  (symbol ≈ means
(1) equivalent to)
 0 a (321) a (331) b 3(1)  R ← R − a 32 R
3 3 2
a (221)
141
Block 2 Solutions of Linear Algebraic Equations

a 11 a 12 a 13 b1 
≈  0 a (221) a (231) b (21)  (19)
 0 a ( 2)
33 b (32) 

which is in the desired form where, a (221) , a (231) , a 32

(1)
, a (331) , b (21) , b (31) , a (332) , b (32)
are given by Eqns. (13) and (15).

Let us once again consider the linear system of equations considered in

Example 7. In matrix form various steps involved in the solution of the
system can be written as
 2 3 − 1  x 1  5
 4 4 − 3  x  = 3
   2  
 − 2 3 − 1  x 3  1

The augmented matrix is

 2 3 − 1 5
[A | b] =  4 4 − 3 3 R 2 ← R 2 − 4 R 1
2
− 2 3 − 1 1 (−2)
R3 ← R3 − R1
(2)
2 3 − 1 5 
≈ 0 − 2 − 1 − 7 
 
0 6 − 2 6  R ← R − 6 R
3 3 2
(−2)
2 3 − 1 5 
≈ 0 − 2 − 1 − 7 
0 0 − 5 − 15

Hence, the solution is x 3 = 3, x 2 = 2 and x 1 = 1.

Now, going back to system (19), the diagonal members of the upper
triangular matrix are given special name as given in the following
definition.

Definition: The diagonal elements a 11 , a (221) and a (332 ) which are used as
divisors are called pivots.

You might have observed here that for a linear system of order 3, the
elimination was performed in 3 − 1 = 2 stages. In general for a system
Ax = b, of n equations given by Eqn. (2) the elimination is performed in
(n − 1) stages. At the ith stage of elimination, we eliminate x i , starting
from (i + 1) th row upto the nth row. Sometimes, it may happen that the
elimination process stops in less than (n − 1) stages. But this is
possible only when no equations containing the unknowns are left or
when the coefficients of all the unknowns in remaining equations
become zero. Thus, if the process stops at the rth stage of elimination
142 then we get a derived system of the form
Unit 5 Direct Methods
a 11 x 1 + a 12 x 2 + L + a 1n x n = b1
a (221) x 2 + L + a 2(1n) x n = b 2(1)
M
( r −1) ( r −1)
a rr x r + L + a (rnr −1) x n = b r (20)
( r −1)
0=b r +1

M M
0 = b n( r −1)
where r ≤ n and a 11 ≠ 0, a (221) ≠ 0, K , a (rrr −1) ≠ 0 .

In the solution of system of linear equations we can thus expect two

different situations
1) r=n 2) r<n

Let us now illustrate these situations through examples.

Example 8: Using Gauss elimination method, solve the following

system of equations:
4 x1 + x 2 + x 3 = 4
x1 + 4 x 2 − 2 x 3 = 4
− x1 + 2 x 2 − 4 x 3 = 2

Using Gauss elimination method.

Solution: The augmented matrix is

 4 1 1 4
[ A | b] =  1 4 − 2 4 R 2 ← R 2 − R 1
1
  4
− 1 2 − 4 2 1
R 3 ← R 3 + R1
4
4 1 1 4

 
≈ 0 15 − 9 3
4 4
  3
0 9 − 15 3 R 3 ← R 3 − R 2
 4 4  5

4 1 1 4 

 
≈ 0 15 − 9 6
4 4
 
0 0 − 12 6
 5 5 

Using back substitution method, we get

6
5 1
x3 = =−
− 12 2
5
143
Block 2 Solutions of Linear Algebraic Equations

9 1
3+ − 
4 2 1
x2 = =
15 2
4
1  1
4 −   − − 
x1 =  2   2  =1
4
1 1
Hence, x 1 = 1, x 2 = and x 3 = − .
2 2

You may observe that in this case r = n = 3 and the given system of
equations has a unique solution. Also, the coefficient matrix A in this
case is non-singular as det ( A ) = A = −36 ≠ 0.
***

Let us look at another example.

Example 9: Solve the system of equations

3x 1 + 2 x 2 + x 3 = 3
2x 1 + x 2 + x 3 = 0
6x 1 + 2 x 2 + 4 x 3 = 6
using Gauss elimination method. Does the solution exist?

Solution: The augmented matrix is

3 2 1 3
[A | b] = 2 1 1 0 R 2 ← R 2 − R 1
2
3
6 2 4 6 R ← R − 2R
3 3 1

3 2 1 3
 1 − 2
≈ 0 − 1 
3 3
0 − 2 2 0  R 3 ← R 3 − 6R 2

3 2 1 3
 1 − 2
≈ 0 − 1 
3 3
0 0 0 12 

In this case, you can see that r < n and the elements b1 , b (21) and b 3( 2)
are all non-zero.

Since we cannot determine x 3 from the last equation, the system has
no solution. In such a situation we say that the equations are
inconsistent, which means that the system does not have a solution.
Also, note that det(A) = 0 i.e., the coefficient matrix is singular.
***

We now consider a situation in which not all b' s are non-zero.

144 Example 10: Solve the system of equations

Unit 5 Direct Methods
16 x 1 + 22x 2 + 4 x 3 = −2
4x 1 − 3x 2 + 2 x 3 = 9
12 x 1 + 25x 2 + 2x 3 = −11
using Gauss elimination method.

Solution: The augmented matrix is

16 22 4 − 2 
[A | b] =  4 − 3 2 9  R 2 ← R 2 − R 1
1
  4
12 25 2 − 11 3
R 3 ← R 3 − R1
4
 
16 22 4 − 2 
 
≈  0 − 17 1 19
2 
2
 
 0 17 − 1 − 19 
 2 2  R 3 ← R 3 + R 2

16 22 4 − 2
 
≈  0 − 17 1 19 
2 2
 
 0 0 0 0 

Now in this case r < n and elements b1 , b (21) are non-zero, but b (32) is
zero. Also, the last equation is satisfied for any value of x 3 . Thus, we
get
x 3 = any value

2  19 
x2 = −  − x3 
17  2 
1
x1 = (−2 − 22x 2 − 4x 3 )
16

Hence, the system of equations has infinitely many solutions.

Note that in this case also det(A) = 0 .
***

The conclusions derived from Examples 8, 9 and 10 are true for any
system of linear equations. We now summarise these conclusions as
follows:

i) If r = n, then the system of Eqn. (2) has a unique solution which

can be obtained using the back substitution method. Moreover,
the coefficient matrix A in this case, is non-singular.

ii) If r < n, and all the elements b (rr+−11) , b (rr+−21) , K , b (nr −1) are non-zero,
then the system has no solution. In this case, we say that the
system of equations is inconsistent. 145
Block 2 Solutions of Linear Algebraic Equations

iii) If r < n, and all the elements b (rr+−11) , b (rr+−21) , K , b (nr −1) , if present, are
zero, then the system has infinite number of solutions. In this
case, the system has only r linearly independent rows.

In both the cases (ii) and (iii) above, the matrix A is singular.

Now we estimate the number of operations (multiplication and division)

in the Gauss elimination method for a system of n linear equations in n
unknowns as follows:

Number of divisions in stages of eliminations

1st step of elimination (n − 1) divisions

2nd step of elimination (n − 2) divisions
………………………………………….
(n − 1) th step of elimination 1 division
∴ Total number of divisions = (n − 1) + (n − 2) + L + 1
= sum of (n − 1) natural numbers
n (n − 1)
=  (n − 1) = (20)
2

Number of multiplications in stages of eliminations

1st step of elimination n (n − 1) multiplications

2nd step of elimination (n − 1) (n − 2) multiplications
……………………………………………………….
(n − 1) th step of elimination (2) (1) multiplications

∴ Total number of multiplications = n (n − 1) + (n − 1) (n − 2) + L + 2.1

=  n (n − 1)
= n2 − n
= sum of squares of first n natural
numbers – sum of first n natural
numbers
n (n + 1) (2n + 1) n (n + 1)
= −
6 2
1
= n (n + 1) (n − 1)
3
1
= n (n 2 − 1) (21)
3

Number of multiplications and divisions in back substitution

Also the back substitution adds n divisions (one division at each step)
and the number of multiplications added are

(n − 1) th equation 1 multiplication
(n − 2) th equation 2 multiplications
……………………………………….
1st equation (n − 1) multiplication
146
Unit 5 Direct Methods
n (n − 1)
∴ Total multiplications =  (n − 1) =
2
Total operations (multiplication and division) added by back substitution
n (n − 1) n (n + 1)
= +n = . (22)
2 2
You can verify these results for n = 3 from Eqns. (12) and (14).

Thus, from Eqns. (20), (21) and (22) we find the total number of
operations needed to find the solution vector x using the Gauss
elimination method. Thus, we need
n (n − 1) 1 n
M= + n (n 2 − 1) + (n + 1)
2 3 2
n
= [ 2 n 2 + 6 n − 2]
6
n3 n
= + n 2 − operations, which is of O(n 3 ) Big O notation (with a
3 3 capital letter O, not a zero)
defines the asymptotic
Thus we may say that the total number of operations needed on an behavior of functions. It
shows you how fast a
1
average is n 3 (approximately). Thus, we find that Gauss elimination function grows or declines.
3 For example, if we write
method needs much lesser number of operations compared to the x2
Cramer’s rule. ex = 1+ x + + O( x 3 );
2
for x ≥ 0, this means that
You may now try a few exercises. the error is smaller in
absolute value than some
3
constant times x if x is
E10) Use Gauss elimination method to solve the system of equations close enough to 0.

x1 + 2x 2 + x 3 = 3
3x1 − 2 x 2 − 4 x 3 = −2
2x1 + 3x 2 − x 3 = −6

E11) Use Gauss elimination method to solve the system of equations

3x1 + 18x 2 + 9 x 3 = 18
2x1 + 3x 2 + 3x 3 = 117
4x1 + x 2 + 2 x 3 = 283

E12) Solve the system of equations

1 2 − 3 1  x 1   − 5
0 1 3 1 x   6 
  2 =  
2 3 1 1 x3   4 
     
1 0 1 1 x 4   1 
using Gauss elimination method.

E13) Using the Gauss elimination method show that the system of
equations
147
Block 2 Solutions of Linear Algebraic Equations

 3 2 − 1 − 4  x 1   10 
1 − 1 3 − 1   x   − 4
   2 =  
2 1 − 3 0   x 3   16 
     
0 − 1 8 − 5 x 4   3 

are inconsistent.

E14) Use Gauss elimination method to solve the following system of

equations Ax = b, where A is a tridiagonal matrix:

 2 −1 0 0 0   x1  1
− 1 2 − 1 0 0   x 2  0

 0 − 1 2 − 1 0   x 3  =  0
    
0 0 − 1 2 − 1  x 4  0
 0 0 0 − 1 2   x 5  1

It is clear from above that you can apply Gauss elimination method to a
system of equations of any order. However, what happens if one of the
diagonal elements i.e., the pivots in the triangularization process
vanishes? Then the method will fail. In such situations we modify the
Gauss elimination method and this procedure is called pivoting. We will
discuss Gauss elimination method with pivoting in the following section.

5.6 GAUSS ELIMINATION METHOD WITH

PIVOTING

In the elimination procedure the pivots a 11 , a (221) , K , a (nnn −1) are used as
divisors. If at any stage of the elimination one of these pivots say
a ii(i−1) , (a 11
( 0)
= a 11 ) , vanishes then the elimination procedure cannot be
continued further. Also, it may happen that the pivot a (iii−1) , though not
zero, may be very small in magnitude compared to the remaining
elements in the ith column. Using a small number as a divisor may lead
to the growth of the round-off error. In such cases, the multipliers
 − a ((ii −−12))i − a ((ii −−32)) i 
 e.g. , (i −1)  will be larger than one in magnitude. The use of
 a ( i −1)
a ii 
 ii 
large multipliers will lead to magnification of errors both during the
elimination phase and during the back substitution phase of the
solution. To avoid this we rearrange the remaining rows (ith row upto
nth row) so as to obtain a non-vanishing pivot or to make it the largest
element in magnitude in that column. The strategy is called pivoting.
The pivoting is of the two types:
i) partial pivoting
ii) complete pivoting.

We will discuss these one by one.

148
Unit 5 Direct Methods
Partial Pivoting

In the first stage of elimination, the first column is searched for the
largest element in magnitude and this largest element is then brought at
the position of the first pivot by interchanging the first row with the row
having the largest element in magnitude in the first column. In the
second stage of elimination, the second column is searched for the
largest element in magnitude among the (n − 1) elements leaving the
first element and then this largest element in magnitude is brought at
the position of the second pivot by interchanging the second row with
the row having the largest element in the second column. This
searching and interchanging of rows is repeated in all the n − 1 stages
of the elimination. Thus we follow the following steps to find the pivot.

Step 1: In the first stage of elimination, searching for the largest

element in magnitude among the members of the first column.
Let this be a i1 .
a i1 = max { a 11 , a 21 , a 31 , ... a n1 }.

Step 2: Interchanging of 1st and ith rows by performing the elementary

row operation R 1 ↔ R i .

Step 3: In the second stage of elimination, searching the element

a j2 , which is the largest element in magnitude among the
members of second column leaving the first element of the
column.
{
a (j12) = max a (221) , a 32
(1)
, a (421) ,..., a (n12) }
Step 4: Interchanging of rows 2 and j by R 2 ↔ R j .

Step 5: Repeat searching and interchanging (n − 1) times in the

subsequent stages of elimination.

In general, for i = 1, 2, K, n, searching j such that

{ }
a (jii −1) = max a ii(i −1) , a (( ii −+11))i , a (( ii−+12))i , ..., a (nii−1) ,

and interchanging rows i and j .

So, partial pivoting includes searching for the largest element in

magnitude in each column as well as interchanging of rows at each
stage of elimination.

Complete Pivoting

Here in the first stage of elimination, we search the entire matrix A for
the largest element in magnitude and bring it at the position of the first
pivot. In the second stage of elimination we search the square matrix of
order n − 1 (leaving the first row and the first column) for the largest
element in magnitude and bring it to the position of second pivot and so
on. This requires at every stage of elimination not only the 149
Block 2 Solutions of Linear Algebraic Equations
interchanging of rows but also interchanging of columns. The following
steps are performed.

Step 1: In the first stage of elimination, searching for the largest

element in magnitude among all the elements of the matrix
A. Let this be a ij .
{ }
a ij = max a ij ,1 < i, j ≤ n

Step 2: Interchanging the first row and i th row and then interchanging
first column and j th column i.e. R 1 ↔ R i and C1 ↔ C j .

Step 3: In the second stage of elimination, searching for the largest

element in magnitude among all the elements leaving the first
row and first column and use this element as second pivot.
{ }
a ij(1) = max a (ij1) ; 2 ≤ i, j ≤ n .

Step 4: Interchanging the second row and i th row and then

interchanging the second column and j th column.

Step 5: Repeat searching and interchanging (n − 1) times in the

subsequent stages of elimination.

In general,
{ }
a ij( k ) = max a ij( k ) ; i, j = k + 1, k + 2,..., n , k = 0, 1, K, (n − 1).

Complete pivoting is much more complicated than the partial pivoting

and is not often used.

Let us now understand the pivoting procedure through examples.

Example 11: Solve the system of equations

x1 + x 2 + x 3 = 6
3x 1 + 3x 2 + 4x 3 = 20
2x 1 + x 2 + 3x 3 = 13
using Gauss elimination method a) without pivoting b) with partial
pivoting and c) complete pivoting.
Solution: a) Let us first attempt to solve the system without pivoting.
We have
1 1 1 6 
[A | b] = 3 3 4 20 R 2 ← R 2 − 3R 1
2 1 3 13  R 3 ← R 3 − 2R 1

1 1 1 6 
≈ 0 0 1 2
0 − 1 1 1 
150
Unit 5 Direct Methods
(1)
Note that in the above matrix the second pivot that is a has the value 22
zero and the elimination procedure cannot be continued further unless,
pivoting is used.
b) Let us now use the partial pivoting.
Since a 21 = max { a 11 , a 21 , a 31 } therefore, interchanging R 1 ↔ R 2 ,
we get
 3 3 4 20
[ A | b ] = 1 1 1 6  R 2 ← R 2 − R 1
1
  3
 2 1 3 13  2
R 3 ← R 3 − R1
3
 
3 3 4 20 
 
≈ 0 0 − 1 − 2 
 3 3
0 − 1 1 − 1 
 3 3 

Since a (221) = 0, we again perform partial pivoting. As

(1)
a 32 {
= max a (221) , a (321) } leaving the first element, we interchange
R 2 ↔ R 3 , and obtain
 
3 3 4 20 
 
[A | b] ≈ 0 − 1 1 − 1 
3 3
 
0 0 − 1 − 2 
 3 3 

You may observe here that the resultant matrix is in triangular

form and no further elimination is required. Using back
substitution method, we obtain the solution
x 3 = 2, x 2 = 1, x 1 = 3 .

c) Let us now solve the system using complete pivoting.

1 1 1 6 
 
[A | b] =  3 3 4 20
 
 2 1 3 13 

As the largest element in magnitude among the elements of A is

a 23 = 4, therefore, interchange R 2 ↔ R 1 and C3 ↔ C1.
Interchanging R 2 ↔ R 1 , first, we get

 3 3 4 20
 
[ A | b ] ≈ 1 1 1 6 
 
 2 1 3 13 
and then interchanging C3 ↔ C1 , we get 151
Block 2 Solutions of Linear Algebraic Equations

 4 3 3 20 [You may note that the

  1 moment we interchange
[ A | b ] ≈ 1 1 1 6  R 2 ← R 2 − R 1 columns C 3 and C1 , the
  4
 3 1 2 13  R ← R − 3 R order of variables x3 and
3 3 1
4 x 1 is also interchanged.
Therefore now becomes
  x 3 , x 2 , x1 ]
4 3 3 20 
 
≈ 0 1 1 1

4 4
 
0 − 5 − 1 − 2 
 4 4 

Leaving the first row and first column, a (321) is the largest element in
magnitude in the given matrix. Thus, a (321) is the second pivot.
Interchanging R 2 ↔ R 3 (there is no need to interchange columns
as already the pivot is in the second column), we get
 
4 3 3 20 
 
≈ 0 − 5 − 1 − 2
4 4
  1
0 1 1 1 R3 ← R3 + R2
 4 4  5
 
4 3 3 20 
 
≈ 0 − 5 − 1 − 2
4 4
 
0 0 1 3 
 5 5 

The resultant matrix is in upper triangular form and no further

elimination is required. Using back substitution method, we get
x 1 = 3, x 2 = 1, x 3 = 2
***

In this unit, henceforth, we shall be using partial pivoting only and refer
to it by pivoting.

Let us consider another example.

Example 12: Solve the system of equations

0.0003x 1 + 1.566 x 2 = 1.569
0.3454 x 1 − 0.436 x 2 = 3.018 (21)
using Gauss elimination method with and without pivoting and compare
them with the exact solution. Assume that the numbers in arithmetic
calculations are rounded to four significant digits. The exact solution of
the system (21) is x 1 = 10, x 2 = 1 .

152 Solution: Without pivoting

Unit 5 Direct Methods
a 21 0.3454
m 21 = − =− = −1151.0 (rounded-off to four decimal places)
a 11 0.0003
a (221) = −0.436 − 1.566 × 1151
= −0.436 − 18020 − 1802.436
= −1802.0
(1)
b 2 = 3.018 − 1.569 ×1151.0
= 3.018 − 1806.0
= −1803.0
Thus, we get the system of equations
0.0003x 1 + 1.566 x 2 = 1.569
− 1802.0x 2 = −1803.0
which gives
1803.0
x2 = = 1.001
1802.0
1.569 − 1.566 × 1.001 1.569 − 1.568
x1 = =
0.0003 0.0003
= 3.333

which is highly inaccurate compared to the exact solution.

With Pivoting

We interchange the first and second equation of system (21) and get
0.3454 x 1 + 0.436 x 2 = 3.018
0.0003x 1 + 1.566 x 2 = 1.569
we obtain
a
m 21 = − 21 = −0.0009 (rounded-off to four decimal places)
a 11
a (221) = 1.566 − 0.0009 × (0.436)
= 1.566 − 0.0004
= 1.5656
(1)
b 2 = 1.569 − 3.018 × (0.0009)
= 1.569 − 0.0027
= 1.5683
Thus, we get the system of equations
0.3454 x 1 − 0.436 x 2 = 3.018
1.5656x 2 = 1.5683
which gives
x 2 = 1.0017
3.018 + 0.4367 3.4547
x1 = = = 10.0020
0.3454 0.3454

which is almost close to the exact solution.

***

We now make the following remark about pivoting.

Remark 2: If the matrix A is diagonally dominant i.e., 153

Block 2 Solutions of Linear Algebraic Equations
n
| a ii | ≥  | a ij | , then no pivoting is needed. See Example 8 in
j=1
j≠ i

which A is diagonally dominant.

Now, here is an exercise for you.

E15) Solve the system of equations

0.729 x + 0.81y + 0.9z = 0.6867
x + y + z = 0.8338
1.331x + 1.21y + 1.1z = 1.000
using Gauss elimination method with and without pivoting.
Round-off the numbers in arithmetic calculations to four
significant digits. The exact solution is x = 0.2245, y = 0.2814,
z = 0.3279 .

We shall now describe the triangularization method which is also a

direct method for the solution of system of equations.

In this method the matrix of coefficients of the linear system being

solved is factored into the product of two triangular matrices. This
method is frequently used to solve a large system of equations. We
shall discuss the method in the next section.

5.7 LU DECOMPOSITION METHOD

Let us consider the system of Eqn. (2), where A is a non-singular
matrix. We first write the matrix A as the product of a lower triangular
matrix L and an upper triangular matrix U in the form
A = LU (22)
or in matrix form we write
 a 11 a 12 K a 1n   l 11 0 K 0   u 11 u 12 K u 1n 
a a 22 K a 2 n  l 21 l 22 K 0 0 u 22 K u 2 n 
 21 =   (23)
 M   M  M 
    
a n1 a n2 K a nn  l n1 l n2 K l nn   0 0 K u nn 

The left side matrix A has n 2 elements, whereas L and U have

n (n + 1)
1+ 2 +L+ n = elements each. Thus, we have n 2 + n unknowns
2
in L and U which are to be determined. On comparing the
corresponding elements on two sides in Eqn. (23), we get n 2 equations
in n 2 + n unknowns and hence n unknowns are undetermined. Thus,
we get a solution in terms of these n unknowns i.e., we get a n
parameter family of solutions. In order to obtain a unique solution we
either take all the diagonal elements of L as 1, or all the diagonal
elements of U as 1. These two types of LU decomposition methods are
154
Unit 5 Direct Methods
given by different mathematicians, hence are named after them as
given below:

i) Crout LU decomposition method: For u ii = 1, i = 1, 2, K, n , the

method is called the Crout LU decomposition method.

ii) Doolittle LU decomposition method: For l ii = 1, i = 1, 2, K, n , the

method is called the Doolittle LU decomposition method.

Usually Crout LU decomposition method is used unless it is

specifically mentioned. We shall now explain the method for n = 3 with
u ii = 1, i = 1, 2, 3 in the following steps:

Step 1: Write A = LU as
 a 11 a 12 a 13   l 11 0 0 1 u 12 u 13 
a a 22 a 23  = l 21 l 22 0  0 1 u 23 
 21 
a 31 a 32 a 33  l 31 l 32 l 33  0 0 1 
or
 a 11 a 12 a 13   l 11 l 11u 12 l 11u 13 
a a 22 a 23  = l 21 l 21u 12 + l 22 l 21u 13 + l 22 u 23 
 21
a 31 a 32 a 33  l 31 l 31u 12 + l 32 l 31u 13 + l 32 u 23 + l 33 

Step 2: Comparing the elements of the first column, we obtain

l 11 = a 11 , l 21 = a 21 , l 31 = a 31 (24)
i.e., the first column of L is determined.

Step 3: Comparing the remaining elements of the first row, we get

a 12
l 11u 12 = a 12  u 12 =
l 11
a 13
l 11u 13 = a 13  u 13 = (25)
l 11
Hence, the first row of U is determined.

Step 4: Comparing the elements of the second column leaving the first
element, we get
l 21 u 12 + l 22 = a 22  l 22 = a 22 − l 21u 12

l 31 u 12 + l 32 = a 32  l 32 = a 32 − l 31 u 12 (26)
Thus, the second column of L is determined.

Step 5: Comparing the remaining elements of the second row, we get

(a 23 − l 21 u 13 )
l 23 u 13 + l 22 u 23 = a 23  u 23 = (27)
l 22
and the second row of U is determined.
155
Block 2 Solutions of Linear Algebraic Equations
Step 6: Comparing the remaining elements of the third column, we get
l 31 u 13 + l 32 u 23 + l 33 = a 33  l 33 = a 33 − l 31u 13 − l 32 u 23 (28)

Step 7: Once we have obtained the elements of the matrices L and U,

we write the system of equations Ax = b in terms of L and U in the
form
LUx = b (29)
The system (29) may be further written as the following two systems
U x=y (30)
L y =b (31)

Step 8: Now, we first solve the system (31), i.e.,

L y =b
using the forward substitution method to obtain the solution vector y.

Step 9: Using this y, we solve the system (30), i.e.,

U x = y,
using the backward substitution method to obtain the solution vector x .

The number of operations for this method remains the same as that in
the Gauss-elimination method. We are leaving it for you to verify
yourself (see E16).

You must have observed that in this method, we alternate between

getting a column of L and a row of U in that order. If instead of
u ii = 1, i = 1, 2, K , n , we take l ii = 1, i = 1, 2, K, n , then we alternate
between getting a row of U and a column of L in that order in the
subsequent steps.

Thus, it is clear from Eqns. (24) – (28) that we can determine all the
elements of L and U provided the non-singular matrix A is such that
 a 11 a 12 
a 11 ≠ 0,  ≠0.
a 21 a 22 

In general, for the system of Eqn. (2), we obtain the elements of L and
U using the relations
j−1
l ij = a ij −  l ik u kj , i ≥ j
k =1

 j−1

 a ij −  l ik u kj 
u ij =  , i ≥ j
k =1

l ii
u ii = 1
Also, det(A) = l 11l 22 , K, l nn .
156
Unit 5 Direct Methods
Thus, we can say that every non-singular matrix A can be written as
the product of a lower triangular matrix and an upper triangular matrix if
all the principal minors of A are non-singular, i.e., if
 a 11 a 12 a 13 
 a 11 a 12 
a 11 ≠ 0,   ≠ 0, a 21 a 22 a 23  ≠ 0, K| A | ≠ 0
a 21 a 22 
a 31 a 32 a 33 

We now illustrate this method through an example.

Example 13: Use Doolittle LU decomposition method to solve the

system of equations
x1 + x 2 + x 3 = 1
4x 1 + 3x 2 − x 3 = 6
3x 1 + 5x 2 + 3x 3 = 4

Solution: We perform various steps of the LU decomposition method.

Step 1: For Doolittle LU decomposition method, l ii = 1, i = 1, 2, 3 , we use

A = LU and get
1 1 1   1 0 0 u 11 u12 u 13 
 4 3 − 1 = l 1 0  0 u 22 u 23 
   21
 3 5 3  l 31 l 32 1  0 0 u 33 

1 1 1   u 11 u 12 u 13 
   
 4 3 − 1 = l 21u 11 l 21u 12 + u 22 l 21 u 13 + u 23 
   
3 5 3  l 31 u 11 l 31u 12 + l 32 u 22 l 31 u 13 + l 32 u 23 + u 33 

Since, we are taking l ii = 1, i = 1, 2, 3, we would compare the elements of

row and column alternatively, on both sides.

Step 2: On comparing the elements of first row, we get

u11 = 1, u12 = 1, u13 = 1 .

Step 3: On comparing the remaining elements of first column, we get

l 21 u 11 = 4  l 21 = 4,
l 31u11 = 3  l 31 = 3.

Step 4: On comparing the remaining elements of second row, we get

l 21u 12 + u 22 = 3  u 22 = −1,
l 31u12 + l 32 u 22 = 5  l 32 = −2.

Step 5: On comparing the remaining elements of second column, we

get
l 31 u 12 + l 32 u 22 = 5  l 32 = −2 157
Block 2 Solutions of Linear Algebraic Equations
Step 6: On comparing the remaining elements of third row, we get
l 31u 13 + l 32 u 23 + u 33 = 3  u 33 = −10

Step 7: We obtain A = LU, where

1 0 0 1 1 1 
L = 4 1 0 ; U = 0 − 1 − 5 
  
3 − 2 1 0 0 − 10

Step 8: Now from the system

L y =b
or
 1 0 0  y 1   1 
 4 1 0  y  =  6 
   2  
 3 − 2 1  y 3  4
we get
y 1 = 1, y 2 = 2, y 3 = 5

Step 9: From the system

U x=y
or
1 1 1   x 1  1 
0 − 1 − 5   x  = 2
   2  
0 0 − 10  x 3  5
we get
1 1
x 3 = − , x 2 = . x1 = 1 .
2 2
***

Example 14: Use the Crout LU decomposition method to solve the

system of linear equations given in Example 13.

Solution: We perform various steps of Crout LU decomposition

method.

Step 1: For Crout LU decomposition method, u ii = 1, i = 1, 2, 3, we have

A = LU and get
1 1 1   l 11 0 0  1 u12 u 13 
 4 3 − 1 = l  
   21 l 22 0  0 1 u 23 
 3 5 3  l 31 l 32 l 33  0 0 1 

1 1 1   l 11 l 11 u 12 l 11u 13 
   
 4 3 − 1 = l 21 l 21 u 12 + l 22 l 21u 13 + l 22 u 23 
   
 3 5 3  l 31 l 31 u 12 + l 32 l 31 u 13 + l 32 u 23 + l 33 

Since, we are taking u11 = 1, i = 1, 2, 3, we would compare the elements

158 of column and row alternatively, on both sides.
Unit 5 Direct Methods
Step 2: On comparing the elements of first column, we get
l 11 = 1, l 21 = 4, l 31 = 3

Step 3: On comparing the remaining elements of first row, we get

l 11u12 = 1  u12 = 1
l 11u 13 = 1  u 13 = 1

Step 4: On comparing the remaining elements of second column, we

get
l 21u 21 + l 22 = 3  l 22 = −1
l 31u 12 + l 32 = 5  l 32 = 2

Step 5: On comparing the remaining elements of second row, we get

l 21 u 13 + l 22 u 23 = −1  u 23 = −5

Step 6”On comparing the remaining elements of third columns, we get

l 31u 13 + l 32 u 23 + l 33 = 3  l 33 = −10

Step 7: We obtain A = LU, where

1 0 0  1 1 1
L =  4 − 1 0  and U = 0 1 5
 
 3 2 − 10 0 0 1

Step 8: Now from the system

Ly = b
or
1 0 0   y1   1 
 4 − 1 0   y  = 6
   2  
 3 2 − 10  y 3   4
We get
1
y1 = 1, y 2 = −2, y 3 = −
2

Step 9: From the system

Ux = y
or
 
 1 1 1  x 1   1 
 0 1 5  x  =  − 2 
   2  
0 0 1  x 3  − 1 
 2
we get
1 1
x 3 = − , x 2 = , x1 = 1
2 2
***

You may now try the following exercises.

159
Block 2 Solutions of Linear Algebraic Equations

E16) Verify that the number of operations in LU decomposition method

is same as that of the Gauss elimination method.

E17) Use Doolittle LU decomposition method to solve the system of

equations given in E11.

E18) Use Crout LU decomposition method to solve the system of

equations given in E14.

We now end this unit by giving a summary of what we have covered in

it.

5.8 SUMMARY
In this unit we have covered the following:

1. For a system of n equations

Ax = b (see Eqn. (2))
in n unknowns, where A is an n × n non-singular matrix, the
methods of finding the solution vector x may be broadly classified
into two types: (1) direct methods and (ii) iterative methods.

2. Direct methods produces the exact solution in a finite number of

steps provided there are no round-off errors. Cramer’s rule is one
such method. This method gives the solution vector as
di
xi = i = 1, 2, K , n
d
where d = | A | and d i is the determinant of the matrix obtained
from A by replacing the ith column of A by the column vector b. .
Total number of operations required for Cramer’s rule in solving a
system of n equations are
M = (n + 1) (n − 1)n!+ n
Since the number M increases very rapidly. Cramer’s rule is not
used for n > 4 .

3. For larger systems, direct methods become more efficient if the

coefficient matrix A is in one of the forms D (diagonal), L (lower
triangular) or U (upper triangular). The solution of such system of
equations are given as
b
• If A is D , then x i = i , i = 1, 2, 3,..., n .
a ii
• If A is L , we use forward substitution method.
• If A is U , we use backward substitution method.

4. Gauss elimination method is another direct method for solving

large systems (n > 4). In this method the coefficient matrix A is
160
Unit 5 Direct Methods
reduced to the form U by using the elementary row operations.
The solution vector x is then obtained by using the back
substitution method. For large n , the total number of operations
1
required in Gauss elimination method are n 3 (approximately).
3

5. In Gauss elimination method if at any stage of the elimination any

of the pivots vanishes or become small in magnitude, elimination
procedure cannot be continued further. In such cases pivoting is
used to obtain the solution vector x . Pivoting is of two types:
• Partial pivoting includes searching for the largest element in
magnitude in each column as well as interchanging of rows in
each stage of elimination.
• Complete pivoting includes searching of largest element in
magnitude of the matrix and interchanging of corresponding
columns and rows in each stage of elimination.
6. Every non-singular matrix A can be written as the product of a
lower triangular matrix and an upper triangular matrix, by the LU
decomposition method, if all the principal minors of A are non-
singular. The, LU decomposition method, which is a modification
of the Gauss elimination method can be used to obtain the
solution vector x. . The method is further divided into two methods.
i) For u ii = 1, i = 1, 2, 3,...n , the method is called Crout LU
decomposition method.
ii) For l ii = 1, i = 1, 2, ....n , the method is called Doolittle LU
decomposition method.

5.9 SOLUTIONS/ANSWERS
E1) Equating the corresponding elements both sides, we get
a − b =1
2a − b = 0
2a + c = 8
3c + d = 17
 a = 1, b = 2, c = 6, d = −1.

2 1 3 1 − 2 16 − 12
E2) AB =  3 − 2 1 2 1  =  3 − 10

− 1 0 1 4 − 2  3 0 

The product BA cannot be found as number of column of B is not

equal to number of rows of A.

E3) det(A ) = 8

E4) i) det(A ) = 0, thus, A is a singular matrix.

161
Block 2 Solutions of Linear Algebraic Equations
ii) det(B) = 45 ≠ 0, thus, B is a non-singular matrix.

3 5 0
E5) d = − 1 2 − 1 = 11
3 −6 4
5 0 8
d1 = 2 − 1 0 = 11
6 4 1
3 8 0
d 2 = − 1 0 − 1 = 11
3 1 4
3 5 8
d3 = −1 2 0 = 11
3 −6 1
Thus, x 1 = x 2 = x 3 = 1

1 2 −3 1
0 1 3 1
E6) d = = 20
2 3 1 1
1 0 1 1
5 2 −3 1
6 1 3 1
d1 = =0
4 3 1 1
1 0 1 1
1 5 −3 1
0 6 3 1
d2 = = 20
2 4 1 1
1 1 1 1
1 2 5 1
0 1 6 1
d3 = = 40
2 3 4 1
1 0 1 1
1 2 −3 5
0 1 3 6
d4 = = −20
2 3 1 4
1 0 1 1
Thus, x 1 = 0, x 2 = 1, x 3 = 2, x 4 = −1

E7) Let x , y, z be the amount (in g) of almond, cashews and

cranberries, respectively. It is given that
162
Unit 5 Direct Methods
1 2 1
x + y + z = 40 or x + 2 y + z = 160
4 4 4
2 1 1
x + y + z = 50 or 2x + y + z = 200
4 4 4
2 2
x + z = 60 or 2x + z = 120
4 4
We can write the matrix form as
1 2 1 x  160 
  
Ax = b where A = 2 1 1 x =  y  and b = 200

1 0 1  z  120 

d = A = −2 ≠ 0
160 2 1
d1 = 200 1 1 = −120
120 0 1

1 160 1
d 2 = 2 200 1 = −40
1 120 1

1 2 160
d 3 = 2 1 200 = −120
1 0 120

d1 − 120
x= = = 60
d −2
d − 40
y= 2 = = 20
d − 20
d − 120
z= 3 = = 60
d −2

Thus, the amount of almond, cashews and cranberries are 60g,

20g and 60g, respectively.

E8) x 1 = 1
2 ×1 − x 2 = 1  x 2 = 1
3 ×1 − 1 − 2 x 3 = 0  x 3 = 1
4 ×1 + 1 − 3 ×1 + x 4 = 3  x 4 = 1
5 × 1 − 2 × 1 − 1 − 2 × 1 + x 5 = 1  x 5 = 1.

E9) x 5 = 1
x 4 − 2 × 1 = −1  x 4 = 1
x 3 − 2 ×1 + 3 ×1 = 2  x 3 = 1
x 2 − 2 × 1 + 3 × 1 − 4 × 1 = −2  x 2 = 1
x 1 − 2 ×1 + 3 ×1 − 4 ×1 + 5 ×1 = 3  x 1 = 1
163
Block 2 Solutions of Linear Algebraic Equations

1 2 1 3
E10) [A | b] =  3 − 2 4 − 2 R 2 ← R 2 − 3R 1

 2 3 − 1 − 6 R 3 ← R 3 − 2R 1

1 2 1 3 
≈ 0 − 8 − 7 − 11

0 − 1 − 3 − 12 R ← R − 1 R
3 3 2
8
 
1 2 1 3 
 
≈ 0 − 8 − 7 − 11
 17 85 
0 0 − 8 − 8 
 
The given system of equations is reduced to
x1 + 2x 2 + x 3 = 3
− 8x 2 − 7 x 3 − 11
17 85
− x3 = −
8 8
Hence, x 3 = 5, x 2 = −3, x 1 = 4.

3 18 9 18 
  2
E11) [A | b] = 2 3 3 117  R 2 ← R 2 − R 1
  3
4 1 2 283 R ← R − R 4
3 3 1
3
3 18 9 18 
≈ 0 − 9 − 3 105 

0 − 23 − 10 259 R ← R − 23 R
3 3 2
9
 
3 18 9 18 
 
≈ 0 − 9 − 3 105 
 7 28 
0 0 − 3 − 3 
 
Thus, the reduced system of equations is
3x 1 + 18x 2 + 9x 3 = 18
− 9 x 2 − 3x 3 = 105
7 28
− x3 = −
3 3
Thus, solution of the system is x 3 = 4, x 2 = −13, x 1 = 72.

1 2 − 3 1 − 5
 
0 1 3 1 6
E12) [A | b] =  
2 3 1 1 4  R 3 ← R 3 − 2R 1
 
1 0 1 1 1  R 4 ← R 4 − R 1
164
Unit 5 Direct Methods
1 2 − 3 1 − 5
0 1 3 1 6 
≈
0 0 10 0 20 
 
0 − 2 4 0 6  R 4 ← R 4 + 2R 2
1 2 − 3 1 − 5
0 1 3 1 6 
≈
0 0 10 0 20 
 
0 0 10 2 18  R 4 ← R 4 − R 3
1 2 − 3 1 − 5
0 1 3 1 6 
≈
0 0 10 0 20 
 
0 0 0 2 − 2

Thus, solution of the system is x 4 = −1, x 3 = 2, x 2 = 1, x 1 = 0

3 2 −1 − 4 10 
 
0 − 5 10 1 − 22 
3 3 3 3
E13) Final derived system [A | b] ≈ 
0 0 −3 13 54 
 5 5 
 
0 0 0 0 29 

We cannot determine x 4 from the last equation. Hence, the

system of linear equations is inconsistent.

2 −1 0 0 0 1
 
0 3 −1 0 0 1 
 2 2
 4 −1 0 1 
E14) Final derived system [A | b] ≈ 0 0
3 3
 
0 0 0 5 −1 1 
4 4

0 0 0 0 6 6
 5 5 
Solution of the system is x 5 = x 4 = x 3 = x 2 = x1 = 1

E15) Solution without pivoting:

Using m 21 = 1.3717
m 31 = 1.8258 and m 32 = 2.42
The final derived system is
0.7290 0.8100 0.9000 0.6867 
 0. 0 − 0.1110 − 0.2346 − 0.1082

 0.0 0.0 0.0244 − 0.008 

The solution is
x = 0.2245, y = 0.2814, z = 0.3279 165
Block 2 Solutions of Linear Algebraic Equations
Solution with pivoting:

Interchanging first and the third row and using

m 21 = 0.7513
m 31 = 0.5477
and m 32 = 0.6171
the final derived system is
1.331 1.210 1.100 1.000 
 0.0 0.1473 0.2975 0.1390 

 0.0 0. 0 − 0.0100 − 0.003280

The solution is x = 0.2246, y = 0.2812, z = 0.3280 .

E16) Let us consider a system of n linear equations in n unknown and

find the number of operations in LU decomposition method with
l ii = 1, i = 1, 2, K, n.
1 0 L 0  u 11 u12 L u 1n 
l 1 L 0  0 u 22 L u 2n 
 21  
Let A = l 31 l 32 L 0  0 0 L u 3n  = LU
  
 M M M M  M M M M 
l n1 l n2 L l nn   0 0 L u nn 

Number of operations for U :

• No operation is needed for u ii , i = 1, 2, K n.
• u 2 j for j = 2, 3, K , n requires (n − 1) subtractions and (n − 1)
multiplications.
• u 3 j for j = 3, 4, K, n requires (n − 2) subtractions and (n − 2)
multiplications.
• u nn requires 1 subtraction and 1 multiplication.
Total operations = 2 (n − 1) − 2
= n (n − 1) − 2

Number of operation for L :

• l j1 for j = 1, 2, K, n requires n divisions.

• l j2 for j = 2, 3, K, n requires n divisions and (n − 1)

subtractions.
Total operations = n (n − 1) + (n − 1) + (n − 2) + L + 2 ⋅1
=  n (n − 1)
n (n + 1)(2n + 1) n (n + 1)
= −
6 2
2
n (n − 1)
=
3
166
Unit 5 Direct Methods
Number of operations in back substitution
n (n + 1)
=  (n − 1) + n =
2

Number of operations in forward substitution =  (n − 1) + n

n (n + 1)
=
2

Total operations in LU decomposition method

n (n 2 − 1)
= n (n − 1) − 2 + + n (n + 1)
3
n3 n
= + 2n 2 − − 2 , which is of O(n 3 ).
3 3

E17) Let A = LU with l ii = 1, i = 1, 2, 3

 3 18 9   1 0 0 u 11 u 12 u 13 
 2 3 3 = l 1 0 0 u 21 u 22 
   21 
 4 1 2 l 31 l 32 0  0 0 u 33 

We multiply both the matrices of RHS and compare elements both

the sides and get
  3 18 9 
1 0 0  
   
L = 2 1 0 ; U = 0 − 9 − 3 
3  
4 23 1 7
0 0 − 3 
 3 9 
T
 28 
The system Ly = b gives y = 10 105 − 
 3

The system Ux = y gives x = [72 − 13 4]

T

E18) Let A = LU with u ii = 1, i = 1, 2, K, 5

 2 − 1 0 0 0   l 11 0 0 0 0  1 u 12 u 13 u 14 u 15 
     
 − 1 2 − 1 0 0  l 21 l 22 0 0 0  0 1 u 23 u 24 u 25 
     
 0 − 1 2 − 1 0  = l 31 l 32 l 33 0 0  0 0 1 u 34 u 35 
     
 0 0 − 1 2 − 1 l 41 l 42 l 43 l 44 0  0 0 0 1 u 44 
     
 0 0 0 − 1 2  l l 42 l 43 l 44 l 45  0 0 0 0 1 
   41 

We multiply both the matrices on RHS and compare elements

both the sides and get
167
Block 2 Solutions of Linear Algebraic Equations

2 0 0 0 0
 
− 1 3
0 0 0
 2 
 −1 4
0 0
L = 0 3 
 
0 0 −1 5 0 
4
 
0 0 0 −1 6 
 5 

 1 
1 − 2 0 0 0 
 
0 1 2
− 0 0 
 3 
U=
0 0 1 −
3
0 
 4 
 
0 0 0 1 − 4
5
 
0 0 0 0 1 

T
1 1 1 1 
The system Ly = b gives y =  1
2 3 4 5 
The system Ux = y gives x = [1 1 1 1 1]T .

168
Unit 6 Inverse of a Square Matrix

UNIT 6

INVERSE OF A SQUARE MATRIX

Structure Page No.

No.
6.1 Introduction 169
Objectives
6.2 The Method of Adjoints 170
6.3 The Gauss-Jordan Reduction Method 174
6.4 LU Decomposition Method 183
6.5 Summary 186
6.6 Solutions/Answers 187

6.1 INTRODUCTION
In the previous unit, you have studied the Gauss elimination method
with and without pivoting. We also discussed there the LU
decomposition method for solving systems of algebraic equations
Ax = b , when A is a n × n non-singular matrix. Matrix inversion is
another problem associated with the problem of finding solutions of a
linear system. If the inverse matrix A −1 of the coefficient matrix A is
known then the solution vector x can be obtained from x = A −1b . In
general, inversion of matrices for solving system of equations should be
avoided whenever possible. This is because, it involves greater amount
of work and also it is difficult to obtain the inverse accurately in many
problems. However, there are two cases in which the explicit
computation of the inverse is desirable. Firstly, when several systems
of equations, having the same coefficient matrix A but different right
hand side b, have to be solved. Then computations are reduced if we
first find the inverse matrix and then find the solution. Secondly, when
the elements of A −1 themselves have some special physical
significance. For instance, in the statistical treatment of the fitting of a
function to observational data by the method of least squares, the
elements A −1 give information about the kind and magnitude of errors
in the data.
In this unit, we shall study a few important methods for finding the
inverse of a non-singular square matrix. We begin the unit by discussing
the method of adjoints to find the inverse of a matrix in Sec. 6.2. In Sec. 169
Block 2 Solutions of Linear Algebraic Equations
6.3, we will discuss Gauss-Jordan reduction method with and without
pivoting to find an inverse of a matrix. Finally, we will discuss LU
decomposition method of finding the inverse of a matrix in Sec. 6.4.

Now we will list the objectives of this unit. After going through the unit,
please read the list again and make sure that you have achieved the
objectives.

Objectives
After studying this unit, you should be able to:
• obtain the inverse of a matrix using the method of adjoints for n < 4 ;
• obtain the inverse of a matrix by the Gauss-Jordan method and LU
decomposition method;
• obtain the solution of a system of linear equations using the inverse
of a matrix.

6.1 THE METHOD OF ADJOINTS

You may recall from the course BMTE-141 (Linear Algebra), that the
inverse of a square matrix A is another matrix, which is represented by
A −1 . Inverse of a matrix on multiplication with matrix itself gives the
multiplicative identity, i.e., AA −1 = I = A −1 A, where I is the identity
matrix. Inverse of a matrix can only be calculated for non-singular
matrix. The matrix for which inverse matrix can be found is known as
invertible matrix. In this section, we will discuss the adjoint method to
find the inverse of a matrix.

You may also recall from the course BMTE-141, that the transpose of
the matrix of the cofactors of elements of A is called the adjoint matrix
and is denoted by adj (A ) .

Formally, we have the following definition.

Definition: The transpose of the cofactor matrix A c of A is called the

adjoint of A and is written as adj (A ) .
Thus,
adj( A ) = ( A c ) T

The inverse of a matrix can be calculated using the adjoint of a matrix.

We obtain the inverse matrix A −1 of A from
1
A −1 = adj(A ) (1)
det(A)
This method of finding the inverse of a matrix is called the method of
adjoints.

Note that det(A ) in Eqn. (1) must not be zero and therefore the matrix
A must be non-singular.

We shall not be going into the details of the method here.

170
Unit 6 Inverse of a Square Matrix
However, we mention the steps involved in finding the solution of the
system of linear equations using the method of adjoints.

Let us consider the system of equations of the form Ax = b.

Step 1: Find adj ( A);

1
Step 2: Find A −1 using A −1 = adj ( A), det(A ) ≠ 0.
det(A)
Step 3: Write the system Ax = b in the form x = A −1b, and obtain the
solution vector x.

We now illustrate the method through examples.

Example 1: Find A −1 for the matrix

5 8 1 
A = 0 2 1 
4 3 − 1

and hence, solve the system of equations

Ax = b (2)
for
2 1 1 
i) b = − 1 , ii) b = 0 , iii) b = − 2
 3  0  3 

Solution: Since det(A ) = −1 ≠ 0 , therefore the matrix A is invertible.

Thus, we obtain the inverse of the matrix A stepwise.

Step 1: We obtain the cofactor matrix A c from A by replacing each

element of A by its cofactor as follows:
 2 1 0 1 0 2 
 − 
 3 −1 4 −1 4 3 
 
 8 1 5 1 5 8 
− −
Ac =  3 −1 4 −1 4 3
 
 
 8 1 5 1 5 8 
 − 
 2 1 0 1 0 2 
 

 − 5 4 − 8
A =  11 − 9 17 
c

 6 − 5 10 
− 5 11 6 
∴ adj( A) = (A ) =  4 − 9 − 5
c T

 − 8 17 10 
171
Block 2 Solutions of Linear Algebraic Equations
1
Step 2: Now A −1 = adj(A ) [using Eqn. (1)]
det(A)

 − 5 11 6   5 − 11 − 6 
1 
∴ A =−1
 4 − 9 − 5 = − 4 9 5 
(−1)
 − 8 17 10   8 − 17 − 10

Step 3: Having obtained A −1 we now solve the system of equations for

the given three forms of the vector b, one by one.

Writing the system Ax = b in the form x = A −1b, we get

 5 − 11 − 6  2  3 
i) x = A b = − 4
−1
9 5  − 1 = − 2 .
   
 8 − 17 − 10  3   3 

 5 − 11 − 6  1  5 
ii) x = A b = − 4
−1
9 5   0 =  − 4
   
 8 − 17 − 10 0  8 

 5 − 11 − 6  1  9 
iii) x = A b = − 4
−1
9 5   − 2 = − 7 
   
 8 − 17 − 10  3   12 

***

We now take up an example in which the given matrix A is lower

triangular and we shall show that its inverse is also a lower triangular
matrix.

Example 2: Find A −1 for the matrix

1 0 0
A = 2 3 0 .
4 5 6

Solution: We have det(A) = 18 ≠ 0 . Thus A −1 exists.

Now
 3 0 2 0 2 3
 − 
 5 6 4 6 4 5  18 − 12 − 2
 0 0 1 0 1 0   
c 
A = − − =  0 6 − 5 
  
 5 6 4 6 4 5  
 0 0 1 0 1 0  0 0 3 
 − 
 3 0 2 0 2 3 
172
Unit 6 Inverse of a Square Matrix
 
1 0 0
 18 0 0  
c T
1   
− 12 6 0 = − 3
Adj ( A) ( A ) 2 2
0
∴ A −1 = = =  3
det (A ) det (A ) 18
 − 2 − 5 3  1 
− −
5 1
 9 18 6 

Thus, A −1 is again a lower triangular matrix.

***

Similarly, we can illustrate that the inverse of an upper triangular matrix

is again upper triangular matrix. Consider the following example.

Example 3: Find A −1 for the matrix

1 2 3
A = 0 4 5
0 0 6

Solution: Since, det(A ) = 24 ≠ 0, A −1 exists.

We obtain
 4 5 0 5 0 4
 − 
 0 6 0 6 0 0
 24 0 0
 2 3 1 3 1 2 
A c = − −  =  − 12 6 0
 0 6 0 6 0 0
 − 2 − 5 4
 2 3 1 3 1 2 
 4 −
 5 0 5 0 4 

 1 1
 24 − 12 − 2 1 − 2 − 12 
adj ( A) (A c ) T 1    
∴ A −1 = = = 0 6 − 5  = 0 1 − 5 
det ( A ) det ( A) 24    4 24 
 0 0 4  0 0 1 
 6 
which is again an upper triangular matrix. You may note that the
inverse of an upper triangular matrix is an upper triangular matrix.
***

You may now try the following exercises.

E1) Solve the system of equations

3x 1 + x 2 + 2 x 3 = 3
2x 1 − x 2 − x 3 = 1
x 1 − 2 x 2 + x 3 = −4
using the method of adjoints.

E2) Solve the system of equations 173

Block 2 Solutions of Linear Algebraic Equations

2 3 4 1   x 1  3
1 2 0 1   x 2  2
 =
2 3 1 − 1  x 3  1 
    
1 − 2 − 1 4   x 4  5

using the method of adjoints.

The method of adjoints provides a systematic procedure to obtain the

inverse of a given matrix and for solving systems of linear equations.
To obtain the inverse of an n × n matrix, using this method, we need to
evaluate one determinant of order n, n determinants each of order n − 1
and perform n 2 divisions. In addition, if this method is used for solving
a linear system we also need matrix multiplication. The number of
operations (multiplications and divisions) needed, for using this method,
increases very rapidly an n increases. For this reason, this method is
not used when n > 4 .

For large n , there are methods which are efficient and are frequently
used for finding the inverse of a matrix and solving linear systems. We
shall now discuss these methods.

6.3 THE GAUSS-JORDAN REDUCTION

METHOD
This method is a variation of the Gauss elimination method. In the
Gauss elimination method, using elementary row operations, we
transform the matrix A to an upper triangular matrix U and obtain the
solution by using back substitution method. In Gauss-Jordan reduction
method not only the elements below the diagonal but also the elements
above the diagonal of A are made zero at the same time. In other
words, we transform the matrix A to a diagonal matrix D. This diagonal
matrix may then be reduced to an identity matrix by dividing each row
by its pivot element. Alternately, the diagonal elements can also be
made unity at the same time when the reduction is performed. This
transforms the coefficient matrix into an identity matrix. For doing so, we
start with the coefficient matrix A and the constant matrix b and write
these two matrices separated by a vertical line in the form [A | b]. The
matrix [ A | b] is called augmented matrix. We perform elementary row
operations according to Gauss-Jordan reduction method and reduce
this augmented matrix to the form [I | d], where I and d are the identity
matrix and solution vector transformed from A and b, respectively,
after elementary row operations. Thus, on completion of the Gauss-
Jordan method, we have
Gauss
[A | b] → [ I | d] (3)
Jordan
The solution is then given by

x i = d i , i = 1, 2, K, n (4)
174
Unit 6 Inverse of a Square Matrix
In this method also, we use elementary row operations that are used
in the Gauss elimination method. We apply these operations both
below and above the diagonal in order to reduce all the off-diagonal
elements of the matrix to zero. We can also use pivoting in the Gauss-
Jordan reduction method. Pivoting can be used to make the pivot non-
zero or to make it the largest element in magnitude in that column as
discussed in Unit 5. We illustrate the method through an example.

Example 4: Solve the system of equations

x1 + x 2 + x 3 = 1
4x 1 + 3x 2 − x 3 = 6
3x 1 + 5x 2 + 3x 3 = 4
using Gauss-Jordan method without or with pivoting.

Solution: Let us first solve the given system of equations without

pivoting.
The augmented matrix
1 1 1 1 
[A | b] = 4 3 − 1 6 R 2 ← R 2 − 4R 1
3 5 3 4 R 3 ← R 3 − 3R 1
1 1 1 1
≈ 0 − 1 − 5 2

0 2 0 1 R 2 ← − R 2
1 0 0 1 
≈ 0 1 5 − 2 R 1 ← R 1 − R 2
0 2 0 1  R 3 ← R 3 − 2R 2
1 0 − 4 3 
≈ 0 1 5 − 2
0 0 − 10 5  R ← − 1 R
3 3
10
1 0 − 4 3 
 
 
≈ 0 1 5 − 2
 R ← R 1 + 4R 3
1 1
 0 0 1 −
2  R ← R − 5R
2 2 3

 
1 0 0 1 
 
≈ 0 1 0 1 
2
 
0 0 1 − 1 
 2 

The matrix form obtained after row operations is [I | d]. Thus, the
solution is given by
1 1
x 1 = 1, x 2 = and x 3 = −
2 2 175
Block 2 Solutions of Linear Algebraic Equations
We now solve the given system of equations using Gauss-Jordan
reduction method with partial pioviting.
The augmented matrix
1 1 1 1 
[A | b] = 4 3 − 1 6
3 5 3 4 R 1 ↔ R 2
4 3 − 1 6 
≈ 1 1 1 1  R 2 ← R 2 − R 1
1
  4
3 5 3 4 3
R 3 ← R 3 − R1
4
 
4 3 − 1 6 
 
≈ 0 1 5 − 1 
 4 4 2
0 11 15 − 1  R1
 4 4 2  R 1 ←
4
 
4 3 − 1 6 
 11 15 
≈ 0 4 4 − 1  1
2 R3 ← R3 − R2
  11
0 1 5 1  12
 4 4 − 2 R ← R − R2
 1 1
11
 − 56 72 
4 0 11 11 
 11 15 1

≈ 0 −  56
 4 4 2  R 1 ← R1 + R3
10
0 0 10
−
5
33
 11 11  R 2 ← R 2 − R3
8
 
4 0 0 4 
 
≈ 0 11 0 11
 R
8 R1 ← 1 , R 2 ←
4
 4
 R2
4 11
0 0 10 5  11
 11 11  R 3 ←
10
R3

 
1 0 0 1 
 
≈ 0 1 0 12 
 
0 0 1 − 1 
 2 

The matrix obtained is of the form [I | d], where d is the solution vector.
Thus, we obtain
1 1
x 1 = 1, x 2 = , x 3 = − .
2 2
***
176
Unit 6 Inverse of a Square Matrix
The method can be easily extended to a general system of n equations
in n unknowns. Just as we calculated the number of operations needed
for Gauss elimination method in Unit 5, in the same way you can verify
that the total number of operations needed for this method is
1 n2
M = n3 + + n , which is of O(n 3 ).
2 2

You may try an exercise.

E3) Verify that the total number of operations needed for the Gauss-
1 n2
Jordan reduction method is n 3 + + n.
2 2

E4) Use Gauss-Jordan reduction method without or with pivoting to

solve the following system of linear equations:
x1 + x 2 + x 3 + x 4 = 1
2x 1 + 4 x 2 + 3x 3 − x 4 = 6
− 2x 1 + 3x 2 + 5x 3 + 3x 4 = 4
3x 1 + x 2 − x 3 + 2 x 4 = 7.

E5) Solve the system of equations

x1 + 2x 2 + x 3 = 0
2x1 + 2 x 2 + 3x 3 = 3
− x1 − 3x 2 = 2
using the Gauss-Jordan method with pivoting.

Clearly this method requires more number of operations compared to

the Gauss elimination method. We, therefore, do not use this method
generally for solving system of equations but is very commonly used for
finding the inverse matrix. This is done by augmenting the matrix A by
the identity matrix I of the order same as that of A. Using elementary
row operations on the augmented matrix [A | I ] we reduce the matrix A
to the form I and in the process the matrix I is transformed to A −1 .

That is
Gauss
[A | I ] → [I | A −1 ] (5)
Jordan

We now illustrate the method through examples.

Example 5: Find the inverse of the matrix

3 1 2
A = 2 − 3 − 1
1 − 2 1 

using the Gauss-Jordan method.

177
Block 2 Solutions of Linear Algebraic Equations
Solution: We have

3 1 2 1 0 0
[A | I ] = 2 − 3 − 1 0 1 0

1 − 2 1 0 0 1 R ← 1 R
1 1
3

 1 2 1 
1 3 3 3
0 0
 
≈ 2 − 3 − 1 0 1 0
  R 2 ← R 2 − 2R 1
1 − 2 1 0 0 1
  R 3 ← R 3 − R1

 1 2 1 
1 3 3 3
0 0
 7

≈ 0 − 11 − −
2
1 0
 3 3 3 
0 − 7 1
−
1
0 1 R ← − 3 R
 3 3 3  2 2
11

 1 2 1 
1 3 3 3
0
0
 7 3

2
≈ 0 1 − 0 1
 11 11 11  R1 ← R1 − 3 R 2
0 − 7 1
−
1
0 1 7
 3 3 3  R 3 ← R 3 + R 2
3

 5 3  1
1 0 11 11
0
11
 7 2
 3
≈ 0 1 0
−
 11 11  11
0 0 20 1
−
7
1 R ← 11 R
 11 11 11  3 3
20

 5 3 1 
1 0 11 11 11
0
 7 2 3

≈ 0 1 − 0 5
 11 11 11  R 1 ← R 1 − 11 R 3
0 0 1 1
−
7 11 
7
 20 20 20  R 2 ← R 2 − R3
11
 1 1 1
1 0 0 4 4
−
4
 3 1 7

≈ 0 1 0 − −  = [I | A −1 ]
 20 20 20 
0 0 1 1
−
7 11 
 20 20 20 

Thus, we obtain

178
Unit 6 Inverse of a Square Matrix
1 1 1 
4 −
4 4
 
A −1 =  3 −
1
−
7

 20 20 20 
1 −
7 11 
 20 20 20 

***

Example 6: Find the inverse of the matrix

2 0 0 0
 
1 1 0 0
A= 
2
2 0 −3 0
 
1 − 7 − 17
55 
 2 3 

using the Gauss-Jordan method.

Solution: Here we have

2 0 0 0 1 0 0 0
 
1 1 0 0 0 1 0 0
[A | I ] =  
2
2 0 −3 0 0 0 1 0
 
1 − 7 − 17 55 1
0 0 0 1 R 1 ← R 1
 2 3  2
 1 
1 0 0 0
2
0 0 0
 
1 1 0 0 0 1 0 0 R 2 ← R 2 − R 1
≈ 2 
2 0 −3 0 0 0 1 0 R 3 ← R 3 − 2R 1
 
1 − 7 − 17 55 0 0 0 1 R 4 ← R 4 − R 1
 2 3 
 1 
1 0 0 0
2
0 0 0
 
0 1 0 0 −1 1 0 0
≈ 2 2 
0 0 − 3 0 −1 0 1 0
 
0 − 7 − 17 55 − 1 0 0 1 R 2 ← 2R 2
 2 3 2 
 1 
1 0 0 0
2
0 0 0
 
0 1 0 0 −1 2 0 0
≈ 
0 0 − 3 0 −1 0 1 0
 7 55  7
0 − 2 − 17 3 − 2
1
0 0 1 R 4 ← R 4 + 2 R 2
 
179
Block 2 Solutions of Linear Algebraic Equations

 1 
1 0 0 0
2
0 0 0
 
0 1 0 0 − 1 2 0 0
≈ 
0 0 −3 0 − 1 0 1 0
 55  1
0 0 − 17 − 4 7 0 1 R 3 ← − 3 R 3
 3 
 1 
1 0 0 0
2
00 0
0 1 0 0 − 1 2 0 0
≈ 1 1

0 0 1 0 3
0 −
3
0 
 
0 R ← R 4 + 17R 3
0 − 17 55
− 4 7 0 1 4
 3 
 1 
1 0 0 0
2
0 0 0
0 1 0 0 −1 2 0 0
≈ 1 1

0 0 1 0 3
0 −
3
0
  3
0  R4 ← R4
0 0 55 5
7 − 17 1 55
 3 3 3 

 1 
1 0 0 0
2
0 0 0
 
0 1 0 0 −1 2 0 0
−1
≈ 1 1  = [I | A ]
0 0 1 0
3
0 −
3
0
 
0 0 0 1 1 21
−
17 3
 11 55 55 55 

Hence,
1 
2 0 0 0
− 1 2 0 0
A =1
−1 
 3 0 − 13 0
 
 1 21 − 17 3
 11 55 55 55 

is the inverse of the given lower triangular matrix. You may again
note that inverse of a lower triangular matrix is again a lower triangular
matrix.
***

Let us now consider the problem of finding the inverse of an upper

triangular matrix.

Example 7: Find the inverse of the matrix

180
Unit 6 Inverse of a Square Matrix
 3 1
1 2 2
2
 
0 1 − 4 1
A=
 
0 0 1 2
3
 
0 0 0 1 

using the Gauss-Jordan method.

Solution: The augmented matrix

 3 
1
1 2 2 1 0 0 0
2
 
0 1 − 4 1 0 1 0 0
[A | I ] = 
 
0 0 1 2 0 0 1 0
3
  R1 ← R 1 − 3 R 2
0 0 0 1 0 0 0 1 2
 3 
1 0 8 −1 1 − 0 0
2
 
0 1 −4 1 0 1 0 0
≈
 
0 0 1 2
0 0 1 0  R 1 ← R 1 − 8R 3
3
 
0 0 0 1 0 0 0 1 R 2 ← R 2 + 4R 3

 19 3 
1 0 0 − 1 − − 8 0
3 2 19
  R1 ← R1 + R 4
11
0 1 0 0 1 4 0 3
≈ 3  R ← R − 11 R
0 0 1 2
0 0 1 0
2 2
3
4

 3  2
  R3 ← R3 − R4
0 0 0 1 0 0 0 1 3
 3 19 
1 0 0 0 1 − −8 
2 3
 
11
0 1 0 0 0 1 4 − 
≈ 3  = [I | A −1 ]
0 0 1 0 0 0 1 −
2
 3
 
0 0 0 1 0 0 0 1 
Hence,
 3 
19
1 − 2 −8 
3
 
11
0 1 4 − 
A −1 =  3
0 0 1 −
2
 3
 
0 0 0 1 

which is the inverse of the given upper triangular matrix.

*** 181
Block 2 Solutions of Linear Algebraic Equations
Note that in Example 2, 3, 6 and 7, the inverse of a lower/upper
triangular matrix is again a lower/upper triangular matrix.

You may now try the following exercises.

E6) In Examples 6 and 7 verify that

AA −1 = A −1A = I .

E7) Find the inverse of the matrix

 2 −1 0 0 
− 1 2 − 1 0 
A= 
 0 − 1 2 − 1
 
 0 0 −1 2 
using the Gauss-Jordan method. Also, solve the system of
equations:
2x1 − x 2 = 2
− x1 + 2 x 2 − x 3 = −1
− x 2 + 2x 3 − x 4 = −1
− x 3 + 2x 4 = 2

E8) Find the inverse of the matrix

 1 0 0
 
 
A =  2 1 0
 1 
− 1 2 1
using the Gauss-Jordan method.

E9) Find the inverse of the matrix

 1 2 − 1
A =  2 1 0 
− 1 1 2 

using the Gauss-Jordan method. And hence, solve the system of

equations Ax = b for
1  0
   
i) b = 3 ii) b = 2
2  
 1  1 

You may recall that in Unit 5 we discussed the LU decomposition

method. Using this method we can factorize any non-singular square
matrix A into the product of a lower triangular matrix L and an upper
triangular matrix U. That is, we can write
A = LU (6)
182
Unit 6 Inverse of a Square Matrix
In the next section we shall discuss how form (6) can be used to find the
inverse of non-singular square matrices.

6.4 LU DECOMPOSITION METHOD

Let us consider Eqn. (6) and take the inverse on both the sides. If we
use the fact that the inverse of the product of matrices is the product of
their inverses taken in reverse order (ref. BMTE-141, Linear Algebra),
then we obtain
A −1 = (LU ) −1 = U −1L−1 (7)
We can now find the inverses of U and L separately and obtain the
inverse matrix A −1 from Eqn. (7).

You may recall the LU decomposition method discussed in Unit 5 that

we can take either u ii = 1, i = 1, 2, K, n or l ii = 1, i = 1, 2, K, n. We use
u ii = 1, i = 1, 2, K, n in Crout’s LU decomposition method and
l ii = 1, i, 1, 2, K, n in Doolittle’s LU decomposition method.

Remark 1: It may appear to you that finding an inverse of a matrix by

this method is a lengthy process. But, in practice, this method is very
useful because of the fact that here we deal with triangular matrices and
triangular matrices are easily invertible. It involves only forward and
backward substitutions.

Let us consider an example to understand how the method works.

Example 8: Find the inverse of the matrix

3 1 2
A = 2 − 3 − 1
1 − 2 1 

using LU decomposition method with u ii = 1, i = 1, 2, 3.

Solution: We write
3 1 2   l 11 0 0  1 u12 u 13 
    
A = LU  2 − 3 − 1 = l 21 l 22 0  0 1 u 23 
    
1 − 2 1  l 31 l 32 l 33  0 0 1 
3 1 2   l 11 l11u12 l 11u13 
   
 2 − 3 − 1 = l 21 l 21u12 + l 22 l 21u13 + l 22 u 23  (8)
   
1 − 2 1  l 31 l 31u12 + l 32 l 31u13 + l 32 u 23 + l 33 

We now compare the coefficients on both sides of Eqn. (8) in columns

and rows alternatively

On comparing the elements of first column:

l 11 = 3, l 21 = 2, l 31 = 1
183
Block 2 Solutions of Linear Algebraic Equations
On comparing the remaining elements of first row:
1
l 11u12 = 1  u12 =
3
2
l 11u 12 = 2  u 13 =
3
On comparing the remaining elements of second column:
2 11
l 21u 12 + l 22 = −3  l 22 = −3 − =−
3 3
1 7
l 31u12 + l 32 = −2  l 32 = −2 − = −
3 3
On comparing the remaining elements of second row:
l 21u13 + l 22 u 23 = −1  u 23 = 7
11

On comparing the remaining elements of third column

20
l 31u13 + l 32 u 23 + l 33 = 1  l 33 =
11
Thus, we have
3 0 0  
  1 1 2
2 − 11 0   3 3
L= 3  and U =  7
 7 20  0 1
11 
1 − 3 11 
 
  0 1

0
Now since, L is a lower triangular matrix, L−1 is also a lower triangular
matrix. Let us assume that
 l′11 0 0
L = l′21 l′22 0 
−1 
l′31 l′32 l′33 

Using the identity LL−1 = I , we have

3 0 0
   l′11 0 0  1 0 0
2 − 11 0    
LL−1 =  3  l′21 l′22 0  = 0 1 0
 7 20     
1 − 3 11 
l′31 l′32 l 33  0 0 1
′  
 
 
 3l′11 0 0  1 0 0
 11 11   
  2l′11 − 3 l′21 − l′
3 22
0  = 0 1 0 (9)
   
l′ − 7 l′ + 20 l′ l′  0 0 1 

7 20 20
− l′ + 11 l′32
 11 33 
11
3 21 11 31 3 22

On comparing the elements of the diagonal both the sides of the

Eqn. (9)
20
l′11 = 1 , l′22 = − 3 , l′33 =
3 11 11
184
Unit 6 Inverse of a Square Matrix
Also, on comparing the remaining elements, we get

l′ = 0  l′21 = 6 = 2
11
2 l′11 −
3 21 33 11

7 20
l′11 − l′21 + l′ = 0  l′31 = 1
3 11 31 20

7 20
− l′ + l′ = 0  l′32 = − 7
3 22 11 32 20

1 
3 0 0
 3

∴ L−1 =  2 −
11
0
 11 
1 − 7
20 
 20 20 11 

Similarly, since U is an upper triangular matrix, U −1 is also an upper

triangular matrix. Using UU −1 = I , we obtain by backward substitution.
 1 2  1 5
1 3 3  1 − 3 − 11 
   
7
U = 0 1 11  and U −1 = 0 1 − 7 
   11 
0 0 1  0 0 1 
   

Therefore, we have from Eqn. (7)

 1 5 1 
1 − 3 − 11   3 0 0
  
A −1 = U −1L−1 = 0 1 − 7   11 2
−
3
11
0
 11  
0 0 1 1 −
7 11 
   20 20 20 

1 1 1 
4 − 
4 4
3 1 7

= − − 
 20 20 20 
1 − 7 11 
 20 20 20 

***

And now a few exercises for you.

E10) Find the inverse of the matrix

5 8 1 
A = 0 2 1 
4 3 − 1

using the LU decomposition method with l 11 = l 22 = l 33 = 1. 185

Block 2 Solutions of Linear Algebraic Equations
E11) Find the inverse of the matrix
3 1 2
A = 2 − 1 − 1

1 − 2 1 

using the LU decomposition method with u11 = u 22 = u 33 = 1. Also,

use the inverse to find the solution of the following system of linear
operations:
3x 1 + x 2 + 2x 3 = 3
2x 1 − x 2 − x 3 = 0
x 1 − 2x 2 + x 3 = 0

E12) Find the inverse of the matrix given in E7) using the LU
decomposition method with l ii = 1, i = 1, 2, 3, 4.

E13) Find the inverse the matrix

1 0 5 2
 −1 4 1 0 
A=
3 0 4 1
 
− 2 1 1 3
ùsing the LU decomposition method with u ii = 1, i = 1, 2, 3, 4. Also,
use the inverse to find the solution of the following system of linear
equations:
x 1 + 5x 3 + 2 x 4 = 8
− x 1 + 4x 2 + x 3 = 4
3x 1 + 4 x 3 + x 4 = 8
− 2x 1 + x 2 + x 3 + 3x 4 = 3.

We now end this unit by giving a summary of what we have covered in

it.

6.5 SUMMARY
In this unit we have covered the following:

1. Using the method of adjoints, the inverse of a given non-singular

matrix A can be obtain from
1
A −1 = adj( A), det(A ) ≠ 0.
det(A)
Since the number of operation in the adjoint method to find the
inverse of a n × n non-singular matrix A increases rapidly as n
increases, the method is not generally used for n > 4 .
186
Unit 6 Inverse of a Square Matrix
2. For large n , the Gauss-Jordan reduction method, which is an
extension of the Gauss elimination method can be used for
finding the inverse matrix and solve the linear system
Ax = b
using the Gauss-Jordan method.
a) the solution of system Ax = b given by Eqn. (2) can be
obtained by using elementary row operations

[A | b] reduced
  to
→ [ I | d]

b) the inverse matrix A −1 can be obtained by using elementary

row operations

[A | I ] reduced
  to
→[I | A −1 ]

3. For larger n , another useful method of finding the inverse matrix

A −1 is LU decomposition method. Using this method any non-
singular matrix A is first decomposed into the product of a lower
triangular matrix L and an upper triangular matrix U. That is
A = LU

U −1 and L−1 can be obtained by backward and forward

substitutions. Then the inverse can be found from
A −1 = U −1L−1 .

a) for Crout LU decomposition method, u ii = 1, i = 1, 2, K, n.

b) for Doolittle LU decomposition method, l 11 = 1, i = 1, 2, K, n.

6.6 SOLUTIONS/ANSWERS

E1) Since, det(A ) = −18 ≠ 0, therefore A −1 exists.

 − 3 − 3 − 3
A c = − 5 1 7 
 1 7 − 5
1 5 1
6 −
18 18 
adj(A ) ( A ) c T  
A −1 = = = 1 −
1
−
7
;
det(A) det(A)  6 18 18 
1 −
7 5 
 6 18 18 
The solution vector x is
1
x = A −1b =  2 
− 1

E2) Since, det(A ) = −55 ≠ 0, therefore A −1 exists.

187
Block 2 Solutions of Linear Algebraic Equations
 5 0 − 15 − 5 
 
 32 − 33 14 − 21
Ac =  ;
− 39 11 7 17 
 
 − 19 11 2 − 3 
 1 32 39 19 
 − 11 − 55 55 55 
 
3 1 1
c T  0 − − 
adj ( A ) ( A ) 5 5 5
A −1 = = = ;
det(A ) det(A)  3 − 14 − 7 −
2
 11 55 55 55 
 1 21 3 
 11 − 17 
 55 55 55 
1
 
0
x = A −1b =  
0
 
1

E3) No. of divisions

1st step of elimination : n
2nd step of elimination : (n − 1)
……………………………….
nth step of elimination : 1
∴ Total number of divisions = n + (n − 1) + L + 1
n (n + 1)
= n =
2
No. of multiplication
1st step: elimination in 2nd equation : n
elimination in 3rd equation : n
………………………………
elimination in nth equation : n
∴ Total of 1st step = n (n − 1)
2nd step: elimination in 1st equation : (n − 1)
elimination in 3rd equation : (n − 1)
……………………………………
elimination in nth equation : (n − 1)
∴ Total of 2nd step = (n − 1) (n − 1)
3rd step: elimination in 1st equation = (n − 2)
elimination in 2nd equation = (n − 2)
elimination in 4th equation = (n − 2)
……………………………………..
elimination in nth equation = (n − 2)
∴ Total of 3rd step = (n − 1) (n − 2)
(n − 1)th step: elimination in 1st equation = 1
elimination in 2nd equation = 1
………………………………..
188
elimination in nth equation = 1
Unit 6 Inverse of a Square Matrix
∴ Total of (n − 1) th step = 1.(n − 1)
Total multiplication
= n (n − 1) + (n − 1) (n − 1) + (n − 1) (n − 2) + L + 1(n − 1)
= (n − 1) [n (−1) + (n + 2) + L + 1]
(n − 1) n (n + 2)
= (n − 1) n =
2
We also need n divisions to find the solution vector
n (n + 1) (n − 1) n(n + 2)
∴ Total operations = + +n
2 2
n3 n2
= + +n.
2 2

E4) We first apply Gauss-Jordan method without pivoting. The

augmented matrix
1 1 1
1 1
 
2 4 3 − 1 6 R 2 ← R 2 − 2R 1
[A | b] =  
− 2 3 5 3 4 R 3 ← R 3 + 2R 1
 
 3 1 − 1 2 7 R 4 ← R 4 − 3R 1

1 1 1 1 1
 
0 2 1 −3 4
≈ 
0 5 7 5 6
  1
0 − 2 − 4 − 1 4 R 2 ← R 2
2
1 1 1 1 1
 
1 3
0 1 − 2 R 1 ← R 1 − R 2
2 2
≈ 
0 5 7 5 6 R 3 ← R 3 − 5R 2
 
0 − 2 − 4 − 1 4 R 4 ← R 4 + 2R 2

 1  5
1 0 − 1
2 2
 
0 1 1 −3 2 
≈ 2 2 
0 0 9 25
− 4
 2 2  2
0 R ← R3
0 − 3 − 4 8 
3
 9

 1  5
1 0 − 1
2 2
  1
0 1 1 − 3 2  R1 ← R1 − 2 R 3
≈ 2 2  1
0 0 1 25
−
8  R 2 ← R1 − R 3
 9 9 2
0 0 − 3 − 4 8  R 4 ← R 4 + 3R 3

189
Block 2 Solutions of Linear Algebraic Equations

 10 5
1 0 0
9
−
9
 
0 1 0 − 26 22

9 9 
≈
0 0 1 25
−
8
 9 9 3
 13 16  R 4 ← R4
0 0 0  13
 3 3 

 10 5
1 0 0
9
−
9
  R ← R − 10 R
0 1 0 − 26 22
 1 1
9
4
9 9 
≈ 26
0 0 1 25
−
8 R2 ← R2 + R4
 9 9 9
 25
16  R 3 ← R 3 − R4
0 0 0 1  9
 13 

 
1 0 0 0 − 25 
13
 
0 1 0 0 6 
= 
0 0 1 0 − 56 
13
 
0 0 0 1 16 
 13 

Thus, the solution is

25 56 16
x 1 = − , x 2 = 6, x 3 = − , x 4 = .
13 13 13

Now we will apply the same method but with pivoting. The
augmented matrix
 1 1 1 1 1
 
 2 4 3 − 1 6
[A | b] =  
− 2 3 5 3 4
 
 3 1 − 1 2 7 R 1 ↔ R 4
3 1 −1 2 7
 
2 4 3 − 1 6
≈ 
− 2 3 5 3 4
  1
 1 1 1 1 1  R 1 ← 3 R 1
 1 1 2 7
1 3
−
3 3 3
 
2 4 3 − 1 6  R 2 ← R 2 − 2R 1
≈
 
− 2 3 5 3 4  R 3 ← R 3 + 2R 1
 
 1 1 1 1 1  R 4 ← R 4 − R 1
190
Unit 6 Inverse of a Square Matrix
 1 1 2 7 
1 3 − 3 3 3 
 10 11 7 4

0 − 
3 3 3 3 
≈
0 11 13 13 26 
 3 3 3 3 
 4 1 R ↔ R
0 23 − 4 2 3
 3 3 3

 1 1 2 7 
1 3 −
3 3 3 
 
0 11 13 13 26

3 3 3 3 
≈
0 10 11
−
7 4 
 3 3 3 3  3
 2 4 1 4 R2 ← R2
0 3 −  11
 3 3 3

 1 1 2 7 
1 3 −
3 3 3 
  1
13 13 26 R 1 ← R 1 − R 2
0 1  3
11 11
≈ 11 
10
0 10 11
−
7 4  R3 ← R3 − R2
 3 3 3 3  3
 2
2 4 1 4 R ← R4 − R2
0 3 −  4 3
 3 3 3

 17 
1 0 −8 3
11 11 11 
 
13 13 26
0 1 
11 11 11 
≈
0 0 −3 −
69
−
72 
 11 11 11  11
 6 5 32  R 3 ← − R 3
0 0 − −  3
 11 11 11 

 
1 0 −8 3 17

11 11 11 8
 13 13

26 R 1 ← R 1 + R3
0 1  11
≈ 11 11 11  13
0 0 1 23 24  R 2 ← R 2 − R3
11
 
0 6 5 32  R ← R − 6 R
0 − −
 11 11 11  3 4
11
3

1 0 0 19 
17
 
0 1 0 − 26 − 26
≈ 
0 0 1 23 24 
  1
0 0 0 − 13 − 16  R 4 ← − 13 R 4

191
Block 2 Solutions of Linear Algebraic Equations

1 0 0 19  17
 
0 1 0 − 26 − 26 R ← R − 17R
≈  4 1 4

0 0 1 23 24  R ← R + 26R
  2 2 4
16
0 0 0 1  R ← R − 23R
 13  3 3 4

 
1 0 0 0 − 25 
13
 
0 1 0 0 6 
≈ 
0 0 1 0 − 56 
13
 
0 0 0 1 16 
 13 

Thus, the solution is

25 56 16
x 1 = − , x 2 = 6, x 3 = − , x 4 = .
13 13 13

E5) The augmented matrix

1 2 1 0
 
[A | I] =  2 2 3 3
 
− 1 − 3 0 2 R 1 ↔ R 2

 3 3
1 1
2 2
 
≈1 2 1 0  R 2 ← R 2 − R1
 
− 1 − 3 0 2 R 3 ← R 3 + R1
 
 3 3 
1 1 2 2 
 
≈ 0 1 − 1 −
3

 2 2
0 − 2 3 7  R 2 ↔ R3
 2 2 

 3 3 
1 1
2 2 
 3 7

≈ 0 −2 − 
 2 2
1
0 1 −
1
−
3 R2 ← − R2
 2 2  2
 3 3 
1 1
2 2 
 
≈ 0 1 −3 −
7

 4 4
R 1 ← R1 − R 2
0 1 −
1
− 
3
 2 2  R 3 ← R 3 − R 2
192
Unit 6 Inverse of a Square Matrix
 9 13 
1 0 4 4 
 
≈ 0 1 − 3 −
7

 4 4
0 0 1 1 
 4 4  R 3 ← 4R 3

 9 13 
1 0 4 4 
 
≈ 0 1 − 3 −
7
 9
 4 4  R1 ← R1 − R 3
4
0 0 1 1 3
  R 2 ← R 2 + R 3
4

1 0 0 1 
 
≈ 0 1 0 − 1 = [I | d]
 
0 0 1 1 

Hence, the solution is

x 1 = 1, x 2 = −1, x 3 = 1.

E6) In Example 6

2 0 0 0 1 
2 0 0 0  1 0 0 0
     
1 1 0 0 − 1 2 0 0  0 1 0 0
2
AA −1 =   1 = =I
2 0 −3 0  3 0 −3
1
0  0 0 1 0
     
1 − 7 − 17
55 
 1 21 − 17 3  0 0 0 1
 2 3  
 11 55 55 55 

and
1  2 0 0 0  1
2 0 0 0 0 0 0
     
− 1 2 0 0 0 1 0 0  0 1 0 0
2
A −1A =  1   = =I
3 0 −3
1
0 2 0 −3 0  0 0 1 0
     
1 − 7 55 
3  0 0 0 1 
 1 21 − 17 3 − 17
 2
 11 55 55 55 

Similarly, for Example 7.

 
1 1 − 3 19 
3 −8 1 0 0 0
1 2 2 2   2 3 
    
−  0 1 0 0
11
0 1 − 4 1  0 1 4
AA −1 =   3= =I
 2 
 0 0 1 2  0 0 1 0
3  0 0 1 −
3  
 
0 0 0 1  0 0  0 0 0 1
 0 1  
193
Block 2 Solutions of Linear Algebraic Equations
 3 19 
1 2 − 8 3  1 3 2 1  1 0 0 0
  2 2

−1 0 1 4 − 3  0 1 − 4 1  0
11  1 0 0
A A=  = =I
0 0 1 − 2  0 0 1 2
 0 0 1 0
 3 
3
 0 0 0 1

0 0 0 1 
0 0 0 1  

E7) The augmented matrix

 2 −1 0 0 1 0 0 0
 
− 1 2 − 1 0 0 1 0 0
[ A | I] =  
 0 −1 2 − 1 0 0 1 0
  1
 0 0 − 1 2 0 0 0 1 R1 ← R 1
2
 1 1 
1 −2 0 0 2
0 0 0
 
−1 2 −1 0 0 1 0 0
≈
 
 0 −1 2 −1 0 0 1 0
 
 0 0 −1 2 0 0 0 1 R 2 ← R 2 + R 1
 1 1 
1 − 2 0 0 2 0 0 0
 
3 1
 1 0 0
≈ 0 2 − 1 0 2 
0 − 1 2 − 1 0 0 1 0
 R ← 2R
0 0 − 1 2 0 0 0 1
2
3
2

 1 
1 − 2 0 0 1 0 0 0
2
 
2 1 2
 
≈ 0 1 − 3 0 3 3
0 0
 1
0 − 1 2 − 1 0 0 1 0  R 1 ← R 1 + R2
2
 
0 0 − 1 2 0 0 0 1  R 3 ← R 3 + R 2
 
1 0 −1 0 2
3
1
3
0 0
3
 
0 1 − 23 0 1
3
2
3
0 0 
≈ 
0 0 4
− 1 13 2
1 0
 3 3  3
  R3 ← 4 R3
0 0 −1 2 0 0 0 1
 
1 0 −1 0 2 1
0 0
3 3 3
  R ← R + 1R
0 1 −2 0 1 2
0 0 1 1
3
3
≈ 3 3 3  2
0 0 1 −
3 1 1 3
0  R 2 ← R 2 + R3
 4 4 2 4  3
 
0 0 −1 2 0 0 0 1 R 4 ← R 4 + R 3
194
Unit 6 Inverse of a Square Matrix
 
1 0 0 −1 3 1 1
0
4 4 2 4
 
0 1 0 −1 1
1 1
0
2 2 2
≈ 
0 0 1 −3 1 1 3
0
 4 4 2 4  4
 5 1 1 3  R4 ← R4
0 0 0 1 5
 4 4 2 4 
 
1 0 0 −1 3 1 1
0
4 4 2 4
  1
0 1 0 −1 1
1 1
0  R1 ← R 1 + 4 R 4
2 2 2
≈  1
0 0 1 −3 1 1 3
0 R 2 ← R 2 + 2 R 4
 4 4 2 4 
 3
1 2 3 4 R3 ← R3 + R4
0 0 0 1  4
 5 5 5 5
 4 3 2 1
1 0 0 0
5 5 5 5
 3 6 4 2

0 1 0 0 
5 5 5 5
≈ = [I | A −1 ]
0 0 1 0 2 4 6 3
 5 5 5 5
 1 2 3 4
0 0 0 1 
 5 5 5 5
Hence,
4 3 2 1
5 5 5 5
 
3 6 4 2
5 5 5 5
A −1 = 
2 4 6 3
5 5 5 5
1 2 3 4
5 5 5 5
 
The system of equation Ax = b  x = A −1b, if A −1 ≠ 0.
4 3 2 1
5 5 5 5  2  1
    
3 6 4 2
 − 1 0
5 5 5 5
Thus, x =   = 
2 4 6 3  − 1 0
5 5 5 5    
1 2 3 4   2  1
5 
 5 5 5
Hence, x1 = 1, x 2 = 0, x 3 = 0, x 4 = 1.

E8) The augmented matrix

1 0 0 1 0 0
 
 
[A | I] =  2 1 0 0 1 0 R 2 ← R 2 − 2R 1
 1 
− 1 2
1 0 0 1 R 3 ← R 3 + R 1
 195
Block 2 Solutions of Linear Algebraic Equations

1 0 0 1 0 0
 
 
≈ 0 1 0 − 2 1 0
 1  1
0 2 1 1 0 1 R 3 ← R 3 − 2 R 2

1 0 0 1 0 0
 
 
≈ 0 1 0 − 2 1 0 = [I | A −1 ]
 1 
0 0 1 2 − 2 1


Thus, we obtain
1 0 0
 
 
A −1 =  − 2 1 0
 1 
 2 − 2 1


E9) The Augmented matrix

 1 2 − 1 1 0 0
 
[ A | I ] =  2 1 0 0 1 0  R 2 ← R 2 − 2R 1
 
− 1 1 2 0 0 1  R 3 ← R 3 + R 1

1 2 − 1 1 0 0
 
≈ 0 − 3 2 − 2 1 0
  1
0 3 1 1 0 1 R 2 ← − R 2
3
1 2 − 1 1 0 0
 
≈ 0 1 − 2 2
−
1
0 R 1 ← R 1 − 2R 2
 3 3 3 
0 3 1 1 0 1 R 3 ← R 3 − 3R 2

 1 1  2
1 0 3 −
3
0
3
 
≈ 0 1 − 2 2
−
1
0
 3 3 3  1
0 0 3 − 1 1 1 R 3 ← R 3
  3

 1 1 2 
1 0 3 −
3 3
0
 2

≈ 0 1 − 2 −
1
0 1
 3 3 3  R1 ← R 1 − 3 R 3
0 0 1 −
1 1 1 2
 3 3 3  R 2 ← R 2 + R 3
3
196
Unit 6 Inverse of a Square Matrix
 2 5 1
1 0 0 − 9 9 − 9 
 
≈ 0 1 0 4 − 1 2  = [I | A −1 ]
 9 9 9 
0 0 1 − 1 1 1 
 3 3 3 
Hence,
 2 5 1
− 9 9 − 9 
 
A −1 =  94 − 19 29 
 
− 1 1 1 
 3 3 3 

The solution of equations in the form Ax = b is x = A −1b.

 2 5 1 1
− 9 9 − 9   1   2 
    
i) x =  94 − 19 92   32  =  12 
    
− 1 1 1   1   1 
 3 3 3   2 
1
Hence, x1 = x 2 = x 3 = .
2
 2 5 1
 − 9 9 − 9  0  1
    
ii) x =  94 − 19 92   2 = 0
    
− 1 1 1  1  1
 3 3 3 

Hence, x1 = 1, x 2 = 0, x 3 = 1.

E10) Let A = LU
5 8 1   1 0 0 u11 u12 u13 
    
 0 2 1  = l 21 1 0  0 u 22 u 23 
    
 4 3 − 1 l 31 l 32 1  0 0 u 33 
After multiplying both the matrices on RHS and comparing
corresponding elements of LHS and RHS, we get
1 0 0 5 8 1 
   
   
L = 0 1 0 and U = 0 2 1 
4 17   1
 5 − 10 1 0 0 − 10 
Now the inverse of the matrices L and U are
1 1 4 
0 0  − −6
  5 5
   1

−1
L = 0 1 0 and U −1 =  0 5 
 2 
 4 17 
−
 5 10 1   0 0 − 10 
 
197
Block 2 Solutions of Linear Algebraic Equations

 5 − 11 − 6 
Thus, A −1 = U −1L−1 =  − 4 9 5 
 8 − 17 − 10

E11) Let A = LU
3 1 2   l11 0 0  1 u12 u13 
    
  2 − 1 − 1 = l 21 l 22 0  0 1 u 23 
    
1 − 2 1  l 31 l 32 l 33  0 0 1 

After multiplying both the matrices on RHS and comparing

corresponding elements of LHS and RHS, we get

   1 2
3 0 0 1 3 3
   
7
L = 2 − 5 0  and U = 0 1 
3  5
 
1 − 7 18  0
0 1
 3 5  
Now, the inverse of the matrices L and U are
1   1 1
3 0 0 1 − 3 −
5
   
L−1 =  2 −
3
0  and U −1 = 0 1 −
7
5 5   5
3 7 5 0 0 1 
−
18 18 18   
1 5 1
 6 18 − 18 
 
Thus, A −1 = U −1L−1 =  1 − 1 − 7 
6 18 18 
1 − 7 5 
 6 18 18 

Now let us use A −1 to solve the given system of equations in the

form Ax = b. The solution vector
1 5 1 1 
6 −
18 18  3 2
    1 
x = A −1b =  1 −
1
−
7
 0 =  
6 18 18    2
1 7 5  0  1 
−
 6 18 18   2 
1 1 1
Thus, the solution is x1 = , x 2 = , x3 = .
2 2 2
E12) Let A = LU , then

 2 −1 0 0 1 0 0 0 u11 u12 u13 u14 

 − 1 2 − 1 0  l 1 0 0 0 u 22 u 23 u 24 
  =  21   
 0 − 1 2 − 1 l 31 l 32 1 0 0 0 u 33 u 34 
     
0 0 − 1 2  l 41 l 42 l 43 1 0 0 0 u 44 
198
Unit 6 Inverse of a Square Matrix
We multiply both the matrices of RHS and compare the
corresponding elements of RHS and LHS. We get

1 0 0 0 2 − 1 0 0 
   
− 1 1 0 0 0 3 − 1 0 
2
 2   
L=  and U = 
0 −
2
1 0 0 0 4
− 1
 3   3 
 3   5 
0 0 − 1 0 0 0 4 
 4  

Now, the inverse of the matrices L and U are

1 0 0 0 1 1 1 1
1  2 3 4 5
 1 0 0  1

2 2
2  0 
3 2 5
−1
L = 1 and U = 
−1 
2
1 0 0 3 3
3 3  0
1   4 5
4
1 3
1  4
 2 4  0 0 0
5 


4 3 2 1
5 5 5 5
 
3 6 4 2

5 5 5 5
Now, A −1 = U −1L−1 =  
2 4 6 3
5 5 5 5
1 2 3 4
5 5 
 5 5

E13) Let A = LU , then

1 0 5 2  l 11 0 0 0  1 u12 u13 u14 

     
 −1 4 1 0 l 21 l 22 0 0  0 1 u 23 u 24 
 =   
3 0 4 1  l 31 l 32 l 33 0  0 0 1 u 34 
     
 − 2 1 1 3 l 41 l 42 l 43 l 44  0 0 0 1 

We multiply both the matrices of RHS and compare the

corresponding elements of RHS and LHS, we obtain
1 0 00 1 0 5 2
   
−1 4 0 0 0 1 3 1
L=  and U =  2 2
3 0 − 11 0  0
  0 1 5
 11 
− 2 1 19 24  0
 2 11   0 0 1 

Now, the inverse of matrices L and U are

199
Block 2 Solutions of Linear Algebraic Equations
 1 0 0 0  3 
 1  1 0 −5
1 11 
 0 0  
 4 4  0 1 −3 2

L−1 =  3 1  and U −1 =  2 11

0 − 0
 11 11  0 0 1 − 
5
 37 11 19 11   11 
 − 96 −  0 0 0 1 
 96 48 24  

 15 1 9 1 
− 32 −
32 16 8 
 
− 11 11 5 1

48 48 24 12
Hence, A = U L = 
−1 −1 −1 
 43 5
−
13
−
5
 96 96 48 24 
 37 11 19 11 
− 96 −
24 
 96 48

 15 1 9 1 
 − 32 −
32 16 8  8 1
    
− 11 11 5 1 
4 1
48 48 24 12
The solution vector x = A b = 
−1    =  .
 43 5
−
13
−
5 8 1
 96 96 48 24    
 37 11 19 11  3 1
− 96 −
24 
 96 48

Thus, x1 = 1, x 2 = 1, x 3 = 1, x 4 = 1.

200
Unit 7 Iterative Methods

UNIT 7

ITERATIVE METHODS

Structure Page No.

7.1 Introduction 201
Objectives
7.2 The General Iteration Method 202
7.3 The Jacobi Iteration Method 206
7.4 The Gauss-Seidel Iteration Method 218
7.5 The Successive Over Relaxation (SOR) Method 228
7.6 Summary 235
7.7 Solutions/Answers 236

7.1 INTRODUCTION
In the previous two units, you have studied direct methods for solving
linear system of equations Ax = b, A being n × n non-singular matrix.
Direct methods provide the exact solution in a finite number of steps
provided exact arithmetic is used and there is no round-off error. Also,
direct methods are generally used when the matrix A is dense or
filled, that is, there are few zero elements, and the order of the matrix is
not very large say n < 50 .

Iterative methods, on the other hand, start with an initial approximation

and by applying a suitably chosen algorithm, lead to successively better
approximations. Even if the process converges, it would give only an
approximate solution. These methods are generally used when the
matrix A is sparse and the order of the matrix A is very large say
n > 50 . Sparse matrices have very few non-zero elements. In most
cases these non-zero elements lie on or near the main diagonal giving
rise to tri-diagonal, five diagonal or band matrix systems. It may be
noted that there are no fixed rules to decide when to use direct
methods and when to use iterative methods. However, when the
coefficient matrix is sparse or large, the use of iterative methods is
ideally suited to find the solution which take advantage of the sparse
nature of the matrix involved. 201
Block 2 Solutions of Linear Algebraic Equations
We will begin the unit by discussing the general iteration methods in
Sec. 7.2. In Sec. 7.3 we shall discuss Jacobi iteration method. The
Gauss-Seidel iteration method which is a modified form of the Jacobi
method will be discussed in Sec. 7.4. Finally in Sec. 7.5, we shall
discuss the successive over relaxation (SOR) method which is a
generalised form of the Gauss-Seidel method. These iterative methods
are frequently used for solving linear system of equations.

Now we will list the objectives of this unit, please read this list again and
make sure that you have achieved the objectives.

Objectives
After studying this unit, you should be able to:
• obtain the solution of system of linear equations, Ax = b, when the
matrix A is large or sparse, by using the iterative method viz;
Jacobi method or the Gauss-Seidel method or the successive over
relaxation method;
• check convergence of these iterative methods;
• obtain the rate of convergence and the approximate number of
iterations needed for the required accuracy of these iterative
methods.

7.2 THE GENERAL ITERATION METHOD

In iteration methods as we have already mentioned, we start with some
initial approximate solution vector x ( 0) and generate a sequence of
approximate solution vectors {x ( k ) } which converge to the exact solution
vector x as k → ∞ . If the method is convergent, each iteration
produces a better approximation to the exact solution. We repeat the
iterations till the required accuracy is obtained. Therefore, in an
iterative method the amount of computation depends on the desired
accuracy whereas in direct methods the amount of computation is fixed.
The number of iterations needed to obtain the desired accuracy also
depends on the initial approximation, closer the initial approximation to
the exact solution, faster will be the convergence.

Consider the system of equations

Ax = b (1)
where A is an n × n non-singular matrix.

Writing the system in expanded form, we get

a 11 x 1 + a 12 x 2 + L a 1n x n = b1
a 21 x 1 + a 22 x 2 + L a 2n x n = b 2
(2)
.................................................
a n1x 1 + a n 2 x 2 + L + a nn x n = b n

We assume that the diagonal coefficients a ii ≠ 0, (i = 1, K, n ) . If some of

a ii = 0 , then we rearrange the equations so that this condition holds.
We then rewrite system (2) as
202
Unit 7 Iterative Methods
1 b
x1 = − (a 12 x 2 + a13 x 3 + L + a 1n x n ) + 1
a 11 a 11
1 b
x2 = − (a 21x1 + a 23 x 3 + L + a 2n x n ) + 2
a 22 a 22
...........................................................................
1 b
xn = − (a n1x1 + a n 2 x 2 + L + a n ( n −1) x n−1 ) + n (3)
a nn a nn

In matrix form, system (3) can be written as

x = Hx + c , where
 a a a   b1 
 0 − a12 − a13 L − a1n   a 11 
11 11 11
   
 − a 21 0
a
− a 23 L −a
a 2n   b2 
H =  a 22 22 22  and c =  a 22  (4)
 M M M M M   M 
   
− a n1 a
− an2 L −
a n , ( n −1)
0   bn 
 a nn nn a nn   a nn 

To solve system (3) we make an initial guess x ( 0) of the solution vector

and substitute into the RHS of Eqn. (3). The solution of Eqn. (3) will
then yield a vector x (1) , which hopefully is a better approximation to the
solution than x ( 0) . We then substitute x (1) into the RHS of Eqn. (3) and
get another approximation, x ( 2 ) . We continue in this manner until the
successive iterations x ( k ) have converged to the required number of
significant figures.

In general we can write the iteration method for solving the linear
system of Eqn. (1) in the form
x ( k +1) = Hx ( k ) + c, k = 0, 1, 2, K (5)
where x ( k ) and x ( k +1) are the approximations to the solution vector x at
the kth and the (k + 1) th iterations, respectively. H is called the
iteration matrix and depends on A . c is a column vector and depends
on both A and b . The matrix H is generally a constant matrix.

When the method (5) is convergent, then

lim x ( k ) = lim x ( k +1) = x
k →∞ k →∞

and we obtain from Eqn. (5)

x = Hx + c (6)
If we define the error vector at the kth iteration as
ε (k ) = x (k ) − x (7)
then subtracting Eqn. (6) from Eqn. (5), we obtain
ε ( k +1) = Hε ( k ) (8)
Thus, we get from Eqn. (8) 203
Block 2 Solutions of Linear Algebraic Equations
ε ( k ) = H ε ( k −1) = H 2 ε ( k − 2) = K = H k ε ( 0) (9)

where ε ( 0) is the error in the initial approximate vector. Thus, for the
convergence of the iterative method, we must have

lim ε (k) = 0
k→∞

independent of ε (0) .

Before we discuss the above convergence criteria, let us recall the

following definitions from the course linear algebra, BMTE-141.

Definition: For a square matrix Α of order n , and a number λ the

value of λ for which the vector equation Αx = λx has a non-trivial
solution x ≠ 0 , is called an eigenvalue or characteristic value of the
matrix Α .

Definition: The largest eigenvalue in magnitude of Α is called the

spectral radius of Α and is denoted by ρ( Α ).

The eigenvalues of the matrix Α are obtained from the characteristic

equation
det(Α − λI ) = 0

which is an nth degree polynomial in λ . The roots of this polynomial

are the eigenvalues of Α and are denoted by λ1 , λ 2 , K , λ n . Therefore,
ρ( A ) denotes the we have
special radius of A i.e., ρ( Α ) = max{ | λ i | }, i = 1, 2, K , n. (10)
i
the largest absolute
value of its eigenvalue.
We now state a theorem on the convergence of the iterative methods.

Theorem 1: An iteration method of the form (5) is convergent for

arbitrary initial approximate vector x (0) if and only if ρ(H) < 1 .

We shall not be proving this theorem here as its proof makes use of
advanced concepts from linear algebra and is beyond the scope of this
course.

We define the rate of convergence as follows:

Definition: The number ν = − log10 ρ(H ) is called the rate of

convergence of an iteration method.

Obviously, smaller the value of ρ(H ) , larger is the value of ν.

Definition: The method is said to have converged to m significant

digits if max | ε (i k ) | ≤ 10 − m , that is, largest element in magnitude, of the
i

error vector ε ( k ) ≤ 10 − m . Also the number of iterations k that will be

needed to make max | ε (i k ) | ≤ 10 − m is given by
i
204
Unit 7 Iterative Methods
m
k= (11)
ν

Therefore, the number of iterations that are required to achieve the

desired accuracy depends on ν. For a method having higher rate of
convergence, lesser number of iterations will be needed for a fixed
accuracy and fixed initial approximation.

There is another convergence criterion for iterative methods which is

based on the norm of a matrix.

The norm of a square matrix Α of order n can be defined in the same

way as we define the norm of an n-vector by comparing the size of Αx
with the size of x (an n-vector) as follows: || A || Denotes the norm
of A.
|| Αx ||2
i) || Α || 2 = max based on the euclidean vector norm, where
|| x || 2
|| x ||2 = | x1 |2 + | x 2 |2 +L + | x n | 2
and
|| Αx ||∞
ii) || Α ||∞ = max , based on the maximum vector norm, where
|| x ||∞
|| x ||∞ = max {| x i | }
1≤i≤ n

In (i) and (ii) above, the maximum is taken over all (non-zero) n-vectors.
The most commonly used norms is the maximum norm || Α ||∞ , as it is
easier to calculate. It can be calculated in any of the following ways:
Α ∞
= max  a ik (maximum absolute column-sum)
k
i
or,
|| Α ||∞ = max  | a ik | (maximum absolute row-sum)
i
k

The norm of a matrix is a non-negative number which in addition to the

property
|| ΑB || ≤ || Α || || B ||

satisfies all the properties of a vector norm, viz.,

a) || Α || ≥ 0 and || Α || = 0 iff Α = 0
b) || αΑ || = | α | || Α || , for all numbers α
c) || Α + B || ≤ || Α || + || B ||

where Α and B are square matrices of order n .

We now state a theorem which gives the convergence criterion for

iterative methods in terms of the norm of a matrix.

Theorem 2: The iteration method of the form (5) for the solution of
system (1) converges to the exact solution for any initial vector, if
|| H || < 1 . 205
Block 2 Solutions of Linear Algebraic Equations
Also note that
|| H || ≥ ρ(H) .

This can be easily proved by considering the eigenvalue problem

Αx = λ x .

Then || Αx || = || λx || = | λ | || x ||
or | λ | || x || = || Αx || ≤ || Α || || x ||
i.e., | λ | ≤ || Α || since || x || ≠ 0

Since this result is true for all eigenvalues, we have

ρ( Α ) ≤ || Α || .

The criterion given in Theorem 2 is only a sufficient condition, it is not

necessary. Therefore, for a system of equations for which the matrix H
n n
is such that either max  | h ik | < 1 or max  | h ik | < 1 the iteration always
k i
i =1 k =1
converges, but if the condition is violated it is not necessary that the
iteration diverges.

There is another sufficient condition for convergence as follows:

Theorem 3: If a matrix Α is strictly diagonally dominant that is,

n
| a ii | >  | a ij |, i = 1, 2, K, n
j=1
j≠ i

then the iteration method (5) converges for any initial approximation
x ( 0) .

If no better initial approximation is known, we generally take x ( 0) = 0 .

We shall mostly use the criterion given in Theorem 1, which is both
necessary and sufficient.

For using the iteration method (5), we need the matrix H and the vector
c which depend on the matrix Α and the vector b . The well-known
iteration methods are based on the splitting of the matrix Α in the form
Α = D+L+U (12)
where D is the diagonal matrix, L and U are respectively the lower and
upper triangular matrices with zero diagonal elements. Based on the
splitting (12), we now discuss three iteration methods of the form (5)
namely, the Jacobi iteration method, the Gauss-Seidel iteration method
and successive over relaxation (SOR) method in this unit.

7.3 THE JACOBI ITERATION METHOD

The Jacobi iteration method is the simplest iteration method to find the
Carl Jacobi (1804-1851) approximate solution of system of linear equations. This method was
invented by a German mathematician Carl Gustav Jacobi (1804-1851).
206
Unit 7 Iterative Methods
We write the system of Eqns. (1) in the form (2), viz.,
a 11 x 1 + a 12 x 2 + La 1n x n = b1
a 21 x 1 + a 22 x 2 + L a 2 n x n = b 2
.................................................
a n1x 1 + a n 2 x 2 + L + a nn x n = b n

We assume that a 11 , a 12 , K, a nn are pivot elements and

a ii ≠ 0, i = 1, 2, K, n . If any of the pivots is zero, we can interchange the
equations to obtain non-zero pivots (partial pivoting).

Note that, A being a non-singular matrix, it is possible for us to make

all the pivots non-zero. It is only when the matrix A is singular that
even complete pivoting may not lead to all the non-zero pivots.

We rewrite system (2) in the form (3) and define the Jacobi iteration
method as
1
x (1k +1) = − (a 12 x (2k ) + a 13 x (3k ) + L + a 1n x (nk ) − b1 )
a 11
1
x (2k +1) = − (a 21 x (2k ) + 2a 23 x 3( k ) + L + a 2 n x (nk ) − b 2 )
a 22
M
1
x (nk +1) = − (a n1 x1( k ) + a n 2 x (2k ) + L + a n , n −1x (nk−)1 − b n )
a nn

 
1 n
or, x ( k +1)
=−  a ij x j − b i  , i = 1, 2, K , n , k = 0,1, K
(k)
(13)
i

a ii j=1 
 i≠ j 

The method (13) can be put in the matrix form as

 1 
 x 1( k +1)   a 11 0 L 0   0 a 12 L a 1n   x 1( k )   b1  
        
 x (2k +1)  0 1
L 0  a 21 0 L a 2 n   x (2k )   b 2  
  = −   −   
a 22   
 M   M M M M   M M M M   M   M 
         
 x ( k +1)  0 1   (k ) 
a n1 a n 2 L 0   x n  b n  
 
 n  
0 L a
nn 

or
x ( k +1) = − D −1 (L + U ) x ( k ) + D −1b, k = 0, 1, 2, K (14)

where
0 0 L 0 0
a 11 L 0  a 0
0
0 0 L 0
a 22 L 0   21 
D= , L = a 31 a 32 L 0 0
M M L M   
 
L a nn   M M L M M
0 0
 a n1 a n 2 L a n ( n −1) 0

207
Block 2 Solutions of Linear Algebraic Equations
0 a 12 a 13 L a 1n 
0 0 a 23 L a 2n 
 
and U =  M M M L M 
 
0 0 0 L a ( n −1) n 
0 0 0 L 0 

The method (14) is of the form (5), where

H = − D −1 (L + U ) and c = D −1b (15)

For computational purpose, we obtain the solution vector x ( k +1) at the

(k + 1) th iteration, element by element using Eqn. (13). For large n , we
rarely use the method in its matrix form as given by Eqn. (14).

In this method in the (k + 1) th iteration we use the values, obtained at

the kth iteration viz., x 1( k ) , x (2k ) , K , x (nk ) on the right hand side of Eqn. (13)
and obtain the solution vector x ( k +1) . We then replace the entire vector
x ( k ) on the right side of Eqn. (13) by x ( k +1) to obtain the solution at the
next iteration. In other words, each of the equations is simultaneously
changed by using the most recent set of x-values. It is for this reason
this method is also known as the method of simultaneous
displacement.

For the better understanding of the method and its convergence let us
now solve a few examples.

Example 1: Perform four iterations of the Jacobi method for solving the
system of equations
− 8 1 1   x1   1 
 1 −5 1   x  = 16 (16)
   2  
 1 1 − 4  x 3   7 

with x (0 ) = 0 , the exact solution is x = [−1 − 4 − 3]T .

Solution: The Jacobi method when applied to the system of Eqn. (16)
becomes
1
x 1( k +1) = [ x (2k ) + x (3k ) − 1]
8
1
x (2k +1) = [ x 1( k ) + x 3( k ) − 16] (17)
5
1
x (3k +1) = [x 1( k ) + x (2k ) − 7],
4
where
k = 0, 1, 2, K

Starting with x (0) = [0 0 0]T we perform the following iterations:

208
Unit 7 Iterative Methods
Iteration 1: Putting k = 0 in Eqn. (17), we get
1
x 1(1) = [0 + 0 − 1] = −0.125
8
1
x (21) = [0 + 0 − 16] = −3.2
5
1
x 3(1) = [0 + 0 − 7] = −1.75
4
Iteration 2: Putting k = 1 in Eqn. (17), we get
1
x 1( 2) = [−3.2 − 1.75 − 1] = −0.7438
8
1
x (22) = [−0.125 − 1.75 − 16] = −3.5750
5
1
x 3( 2) = [−0.125 − 3.2 − 7] = −2.5813
4
Iteration 3: Putting k = 2 in Eqn. (17), we get
1
x 1(3) = [−3.5750 − 2.5813 − 1] = −0.8945
8
1
x (23) = [−0.7438 − 2.5813 − 16] = −3.8650
5
1
x 3(3) = [−0.7438 − 3.5750 − 7] = −2.8297
4
Iteration 4: Putting k = 3 in Eqn. (17), we get
1
x 1( 4) = [−3.8650 − 2.8297 − 1] = −0.9618
8
1
x (24) = [−0.8945 − 2.8297 − 16] = −3.9448 (18)
5
1
x 3( 4) = [−0.8945 − 3.8650 − 7] = −2.9399
4

Thus, after four iterations we get the solution vector as

x (4) = [−0.9618 − 3.9448 − 2.9399]T , which is reasonably close to the
given exact solution vector x = [−1 − 4 − 3]T . We find that after each
iteration, we get better approximation to the exact solution.
***

Example 2: Jacobi method is used to solve the system of equations

 4 − 1 1  x 1   7 
 4 − 8 1  x  = − 21 (19)
   2  
 − 2 1 5  x 3   15 

Determine the rate of convergence of the method and the number of

iterations needed to make max | ε (i k ) | ≤ 10 −2 .
i

Perform these number of iterations starting with initial approximation

x (0) = [1 2 2]T and compare the result with the exact solution [2 4 3]T . 209
Block 2 Solutions of Linear Algebraic Equations
Solution: The Jacobi method when applied to the system of Eqn. (19),
gives the iteration matrix H as follows:

a 11 0 0  4 0 0
D =  0 a 22 0  = 0 − 8 0,
 0 0 a 33  0 0 5

0 0 0  0 0 0
L = a 21 0 0 =  4 0 0
a 31 a 32 0 − 2 1 0
and
0 a 12 a 13  0 − 1 1
U = 0 0 a 23  = 0 0 1
0 0 0  0 0 0

Now, we obtain the matrices H and c using the relations given in

Eqn. (15).
H = − D −1 ( L + U )
1 
 4 0 0   0 − 1 1
   
= −0 − 1 0  4 0 1
 8   
 0 0 1  − 2 1 0
 5 

 1 1   1 1
 0 − 4 4  0 4 −
4
   
= − − 1 0 − 1  =  1 0 1

 2 8  2 8 
− 2 1 0  2 − 1 0 
 5 5   5 5 
 b1   
 a11   7 
  4
and c =  a 2  =  21 
b
 22   
b  8
 3  3
 33   
a

The eigenvalues of the iteration matrix H are the roots of the

characteristic equation
det(H − λI ) = 0

 1 1
− λ 4 − 4 
 
  1 −λ 1  = 0
 2 8

 2 − 1 − λ
 5 5 
210
Unit 7 Iterative Methods
3
 λ3 − =0
80

All the three eigenvalues of the matrix H are equal and they are equal
to
λ = 0.3347

The spectral radius is

ρ(H ) = max λ i = 0.3347 (20)
i

We obtain the rate of convergence as

ν = − log10 ρ(H ) = − log10 (0.3347) = 0.4753

The number of iterations needed for the required accuracy is given by

m
k=
ν
2 (k )
= [ m = 2, as we require max ε i ≤ 10 −2 ] (21)
ν
≈5

Let us now perform 5 iterations the Jacobi method to the system of

Eqn. (19), we have
x ( k +1) = Hx ( k ) + c, k = 0, 1, 2, K

 1 1 7
0 4 −
4 4
   
 x ( k +1) = 1 0 1
 x(k ) +  21 , k = 0, 1, 2, K (22)
2 8
 8
2 − 1 0  
 5 5   3 

Starting with the initial approximation x ( 0) = [1 2 2]T , we get the

following iterations:
Iteration 1: Putting k = 0 in Eqn. (22), we get
 1 1 7
0 4 −
4  1   4 
 
1  
 
= 1 0  2 +  21  = 1.75 [ 3.375 3.0]
T
x (1)
2 8
  8  
2 − 1 2
0  
 5 5   3 

Iteration 2: Putting k = 1 in Eqn. (22), we get

 1 1 7
0 4 −
4   1.75   4 
 1 
  
= 1 0  3.375 +  21  = 1.8437
[ 3.875 3.0 25]
T
x ( 2)
2 8
   8
2 − 1   3.0   
0
 5 5   3  211
Block 2 Solutions of Linear Algebraic Equations
Iteration 3: Putting k = 2 in Eqn. (22), we get

 1 1 7
0 4 −
4  1.8437  4 
 1 
  
= 1 0   3.875  +  21  = 1.9625
[ 3.925 2.9625]
T
x ( 3)
2 8
  8
2 − 1  3.025 
0  
 5 5   3 

Iteration 4: Putting k = 3 in Eqn. (22), we get

 1 1 7
0 4 −
4  1.9625   4 
 1 
  
= 1 0   3.925  +  21 = 1.9906
[ 3.9766 3.0000]
T
x ( 4)
2 8
  8
2 − 1  2.9625
0  
 5 5   3 

Iteration 5: Putting k = 4 in Eqn. (22), we get

 1 1 7
0 4 −
4  1.9906   4 
 1 
  
= 1 0  3.9766 +  21  = 1.9941
[ 3.9953 3.0009]
T
x (5)
2 8
  
8
2 − 1 3.0000
0  
 5 5   3 

which is the solution vector after five iterations. Thus, you can see that
the solution vector obtained after five iterations is quite close to the
exact solution [2 4 3]T .
***

Remark 1: The matrix formulation of the Jacobi method as in Eqns. (5)

and (15) is used whenever we want to check whether the iterations
converges or to find the rate of convergence of the method. Otherwise,
to find the solution, iterations can be performed on the given system of
equations after putting them either in the form (13), (as in Example 1) or
in the form (14), (as in Example 2).

Example 3: Perform four iterations of the Jacobi method for solving the
system of equations

 2 −1 0 0   x 1  1
 − 1 2 − 1 0   x  0
   2 =   (23)
 0 − 1 2 − 1  x 3  0
     
0 0 − 1 2   x 4  1

with x ( 0) = [0.5 0.5 0.5 0.5]T . What can you say about the solution
obtained if the exact solution is x = [1 1 1 1]T ?

Solution: The Jacboi method when applied to the system of Eqn. (23)
becomes
212
Unit 7 Iterative Methods
1
x 1( k +1) = [1 + x (2k ) ]
2
1
x (2k +1) = [ x 1( k ) + x (3k ) ]
2
1
x 3( k +1) = [ x (2k ) + x (4k ) ]
2
1
x (4k +1) = [1 + x (3k ) ] ,
2
where, k = 0, 1, 2, K

In matrix notation, we can write the system as

 1  1
0 2
0 0
2
1 1
  
 0 0
x ( k +1) =
2 2
 x ( k ) +  0  (24)
0 1
0 1
0
 2 2
1
 1  2
0 0 0  
 2 
where, k = 0, 1, 2, K

We start with x (0) = [0.5 0.5 0.5 0.5]T and perform following iterations:

Iteration 1: Putting k = 0 in Eqn. (24), we get

 1 
 0 2 0 0  0.5  1 
1 1
   0
 0 0  0.5  0 
2 2
x (1) =     +   = [0.75 0.5 0.5 0.75]T
 0 1 0 1  0.5  0 
 2 2    
    1
 0 0 2 0     2 
1 0 .5
 
Iteration 2: Putting k = 1 in Eqn. (24), we get
 1 
 0 2 0 0  0.75  1 
1 1
  0
 0 0   0. 5   0 
2 2
x ( 2) =    +   = [0.75 0.625 0.625 0.75]T
 0 1 0 1   0. 5   0 
 2 2    
   0.75  1 
0 0 2 0 
1
  2
Iteration 3: Putting k = 2 in Eqn. (24), we get
 1 
 0 2 0 0   0.75   1 
1 1
  0
 0 0  0.625  0 
2 2
x ( 3) =    +   = [0.8125 0.6875 0.6875 0.8125]T
0 1
0 1  0.625  0 
 2 2  
   0.75   1 
0 0 2 0   2
1
    213
Block 2 Solutions of Linear Algebraic Equations
Iteration 4: Putting k = 3 in Eqn. (24), we get
 1 
 0 2 0 0  0.8125  1 
1 1
  0
 0 0  0.6875  0 
2 2
x ( 4) =    +   = [0.8438 0.75 0.75 0.8438]T
 0 1 0 1  0.6875  0 
 2 2  
  0.8125  1 
0 0 2 0    2
1
   

You may notice here that the solution is improving after each iteration.
Also, the solution vector obtained after four iterations is not a good
approximation to the exact solution x = [1 1 1 1]T . This shows that we
require a few more iterations to get a good approximation.
***

Example 4: Find the spectral radius of the iteration matrix when the
Jacobi method is applied to the system of equations
1 0 2   x 1   − 1
0 1 − 2  x  =  5 
   2  
1 − 1 1   x 3  − 3

Verify that the iterations do not converge to the exact solution

x = [1 3 − 1]T . You may start with x ( 0) = 0.

Solution: Here, the matrices D, L and U are

1 0 0  0 0 0 0 0 2 
D = 0 1 0 , L = 0 0 0 and U = 0 0 − 2,
0 0 1 1 − 1 0 0 0 0 

The iteration matrix H = − D −1 (L + U ) becomes

1 0 0 0 0 2   0 0 − 2
H = − 0 1 0 0 0 − 2 =  0 0 2 
  
0 0 1 1 − 1 0  − 1 1 0 

 b1   1 
 a 11   − 
   1
and c =  a 2  =  5  = [−1 5 − 3]T .
b
 22   1 
 b   3
 3  − 1 
 33   
a

The eigenvalues of H are the roots of the characteristic equation

det (H − λI ) = 0. . This gives us

− λ (λ2 − 4) = 0
214
Unit 7 Iterative Methods
i.e., λ = 0, ± 2
∴ ρ(H) = max{λ i } = 2 > 1 .
i

Thus, the condition in Theorem 1 is violated. The iteration method

does not converge.

We now perform few iterations and see what happens actually. Taking
x ( 0) = [0 0 0] and using the Jacobi method
T

 0 0 − 2  − 1
x =  0 0 2  x +  5 

( k +1)  (k)
[ref. Eqn. (5)] (25)
− 1 1 0  − 3
where k = 0, 1, 2, K .

We start with x ( 0) = [0 0 0]T and obtain the following iterations:

Iteration 1: Putting k = 0 in Eqn. (25), we get

 0 0 − 2 0  − 1
    
x (1) =  0 0 2  0 +  5  = [−1 5 − 3]T
    
− 1 1 0  0 − 3

Iteration 2: Putting k = 1 in Eqn. (25), we get

 0 0 − 2   − 1  − 1
    
x ( 2) =  0 0 2   5  +  5  = [5 − 1 3]T
    
 − 1 1 0   − 3 − 3

Iteration 3: Putting k = 2 in Eqn. (25), we get

 0 0 − 2   5   − 1
    
x ( 3) =  0 0 2  − 1 +  5  = [−7 11 − 9]T
    
− 1 1 0   3  − 3

Iteration 4: Putting k = 3 in Eqn. (25), we get

 0 0 − 2   − 7   − 1
    
x ( 4) =  0 0 2   11  +  5  = [17 − 13 15]T
    
 − 1 1 0   − 9  − 3

Iteration 5: Putting k = 4 in Eqn. (25), we get

 0 0 − 2  17   − 1
    
x (5) =  0 0 2  − 13 +  5  = [−31 35 − 33]T
    
− 1 1 0   15  − 3
215
Block 2 Solutions of Linear Algebraic Equations
and so on, which shows that the iterations are diverging fast. You may
also try to obtain the solution with other initial approximations.
***

Now try the following exercise.

E1) Perform five iterations of the Jacobi method for solving the system
of equations given in Example 4 with x (0) = [1 1 1]T .

Let us now consider an example to show that the convergence criterion

given in Theorem 3 is only a sufficient condition. That is, there are
system of equations which are not diagonally dominant but, the Jacobi
iteration method converges.

Example 5: Perform iterations of the Jacobi method for solving the

system of equations:
1 1 1   x 1  3
0 2 0   x  = 2
   2  
0 3 − 1  x 3  1

with x (0) = [0 1 1]T . What can you say about the solution obtained if the
exact solution is x = [0 1 2]T ?

Solution: The Jacobi method when applied to the given system of

equations becomes
x 1( k +1) = [3 − x (2k ) − x 3( k ) ]

x 2( k +1) = 1
x 3( k +1) = [−1 + 3x (2k ) ] ,
where, k = 0, 1, 2, K

0 1 1  3
The matrix notation is x ( k +1)
= 0 0 0 x +  1 
  (k )
(26)
0 − 3 0  − 1
where k = 0, 1, 2, K .

We start with x ( 0) = [0 1 1]T and perform the following iteration:

Iteration 1: Putting k = 0 in Eqn. (26), we get

 0 1 1   0  3 
    
x (1) = 0 0 0 1 +  1  = [1 1 2]T
    
0 − 3 0 1  − 1

Iteration 2: Putting k = 1 in Eqn. (26), we get

216
Unit 7 Iterative Methods
0 1 1 1   3 
    
x ( 2) = 0 0 0 1  +  1  = [0 1 2]T
    
0 − 3 0 2 − 1

Iteration 3: Putting k = 1 in Eqn. (26), we get

0 1 1 0  3 
    
x (3) = 0 0 0 1  +  1  = [0 1 2]T
    
0 − 3 0 2 − 1

You may notice here that the coefficient matrix is not diagonally
dominant but the iterations converge to the exact solution after only two
iterations.
***

And now a few exercises for you.

E2) Perform four iterations of the Jacobi method for solving the system
of equations
5 2 2  x 1   1 
 2 5 3  x  =  − 6 .
   2  
 2 1 5  x 3   − 4

with x ( 0) = [0 0 0] . Exact solution is x = [1 − 1 − 1]T

T

E3) Perform four iterations of the Jacobi method for solving the system
of equations
 
 2 −1 0  3
2  x1   2 
  
− 3 2 − 1  x2 = 0 
 2 2    
 0  x 3   1 
−3 2   2 
 2 

with x ( 0) = [0 0 0] . The exact solution is x = [1 1 1]T .

T

E4) Perform four iterations of the Jacobi method for solving the system
of equations
5 − 1 − 1 − 1  x 1   − 4
− 1 10 − 1 − 1  x   12 
A=  2 =  
− 1 − 1 5 − 1 x 3   8 
     
− 1 − 1 − 1 10   x 4   34 

with x (0) = [0 0 0 0] . The exact solution is x = [1 2 3 4]T .

T

E5) Set up the Jacobi method in matrix form for solving the system of
equations 217
Block 2 Solutions of Linear Algebraic Equations
 1 1 1
1 0 −
4
− 
4  x1  2
 1 1
   1
0 1 − −  x 2   2 
4 4
   = 
− 1 −
1
1 0  x 3   1 
 4 4    2
 1 1   x 4   1 
− 4 − 0 1  2
 4   
and perform four iterations. Exact solution is x = [1 1 1 1]T . Take
x (0) = [0 0 0 0] .
T

E6) Jacobi method is used to solve the system of equations

 2 0 1  x 1   1 
 0 3 2  x  =  1 
   2  
1 2 4  x 3  − 1

Determine the rate of convergence of the method and the number

of iterations needed to make max | ε i( k ) | ≤ 10 −2 . Perform four
i

iterations starting with x = [0 0 0]T , and compare the result with

(0 )

the exact solution [1 1 − 1]T .

We have already mentioned that iterative methods are usually applied

to large linear systems with a sparse coefficients matrix. For sparse
matrices, the number of non-zero entries is small, and hence the
number of arithmetic operations to be performed per step is less.
However, iterative methods may not always converge, and even when
they converge, they may require a large number of iterations.

We shall now discuss the Gauss-Seidel method which is a simple

modification of the method of simultaneous displacements and has
improved rate of convergence.

7.4 THE GAUSS-SEIDEL ITERATION METHOD

Carl Friedrich Gauss Gauss-Seidel iteration method is improved form of Jacobi method. This
(1777-1855) method is named after the German mathematicians Carl Friedrich
Gauss (1777-1855) and Philipp Ludwig von Seidel (1821-1896).

Consider the system of Eqn. (2) written in form (3). For this system of
equations, we define the Gauss-Seidel method as:
1
x 1( k +1) = − (a 12 x 2( k ) + a 13 x 3( k ) + L + a 1n x n( k ) − b1 )
a 11
1
x 2( k +1) =− (a 21 x 1( k +1) + a 23 x 3( k ) + L + a 2n x n( k ) − b 2 ) (27)
a 22
M
1
Philipp Ludwig Von Seidel x (nk+1) = − (a n1x 1( k +1) + a n 2 x (2k+1) + L + a n ( n−1) x n( k−+11) − b n )
(1821-1896) a nn
218
Unit 7 Iterative Methods
or
1  i −1 n 
x i( k +1) = −  ij j 
( k +1)
a x + a ij x (jk ) − b i  , i = 1, 2, K , n and k = 0,1, 2, ...
a ii  j=1 j= i +1 

You may notice here that in the first equation of system (27), we
substitute the initial approximation ( x 2(0 ) , x 3( 0) , K , x (n0) ) on the right hand
side. In the second equation, we substitute ( x 1(1) , x 3( 0) , K , x n(0) ) on the
right hand side. In the third equation, we substitute
( x 1(1) , x (21) , x 4(0 ) , K , x (n0) ) on the right hand side. We continue in this
manner until all the components have been improved. At the end of this
first iterations, we will have an improved vector ( x 1(1) , x (21) , K , x (n1) ) . The
entire process is then repeated. In other words, the method uses an
improved component as soon as it becomes available. It is for this
reason the method is also called the method of successive
displacements.

We can also write the system of Eqns. (27) as follows:

a 11x 1( k+1) = −a 12 x (2k ) − a 13 x (3k ) − L a 1n x (nk ) + b1
a 21 x1( k +1) + a 22 x (2k+1) = −a 23 x (3k ) − L − a 2n x (nk ) + b 2
M
a n1x1( k +1) + a n 2 x (2k +1) + L + a nn x (nk+1) = b n
In matrix form, this system can be written as
(D + L )x ( k +1) = − Ux ( k ) + b (28)
where D is the diagonal matrix
a 11 0 L 0 
0 a 22 L M 
D= 
 M M L M 
 
0 0 L a nn 

and L and U, are respectively the lower and upper triangular matrices
with the zeros along the diagonal and are of the form
0 0 L 0 0 0 a 12 a 13 K a 1n 
a  0 0
 21
0 K 0 0
  a 23 K a 2 n 
L = a 31 a 32 0 0 0 and U = 0 0 0 K M 
   
 M M M M M M K a ( n−1) n 
a n1 a n 2 K a n ( n−1) 0 0 K K K 0 


From Eqn. (28), we obtain

x ( k +1) = −(D + L) −1 Ux ( k ) + (D + L) −1 b (29)
which is of the form (5) with
H = −(D + L) −1 U and c = (D + L) −1 b .
It may again be noted here, that if A is diagonally dominant then the
iterations always converge. 219
Block 2 Solutions of Linear Algebraic Equations
Gauss-Seidel method will generally converge if the Jacobi method
converges, and will converge at a faster rate. For a symmetric matrix
A, it can be shown that

ρ (Gauss-Seidel iteration method) = [ρ (Jacobi iteration method )]2

Hence, the rate of convergence of the Gauss-Seidel method is

twice the rate of convergence of the Jacobi method. This result is
usually true even when A is not symmetric.

We shall illustrate this fact through examples.

Example 6: Gauss-Seidel Method is used to solved the system of

equations which was solved using Jacobi method in Example 2.
Consider the system of linear equations
 4 − 1 1  x 1   7 
 4 − 8 1  x  = − 21 (30)
   2  
 − 2 1 5  x 3   15 

Determine the rate of convergence of the method and the number of

{ }
iterations needed to make max | ε (i k ) | ≤ 10 −2 . Compare the rate of
i
convergence of this with the rate of convergence using Jacobi method
in Example 2.

Perform the number of iterations required to achieve the given accuracy

starting with x (0) = [1 2 2]T and compare the results with the exact
solution x = [2 4 3]T .

Solution: The Gauss-Seidel method (29) when applied to the system of

Eqn. (30) gives the iteration matrix H as follows:

4 0 0  0 0 0 0 − 1 1
D = 0 − 8 0, L =  4 0 0 and U = 0 0 1.
   
0 0 5 − 2 1 0 0 0 0

H = − (D + L) −1 U
−1
4 0 0 0 − 1 1
= −  4 − 8 0
 0 0 1
  (31)
− 2 1 5 0 0 0

Let us find the inverse (D + L) −1. Since the inverse of a lower triangular
matrix is also a lower triangular matrix let
−1
 l 11 0 0 4 0 0
   
M = l 21 l 22 0  =  4 − 8 0
   
l 31 l 32 l 33  − 2 1 5

220 then
Unit 7 Iterative Methods
4 0 0  l 11 0 0  1 0 0
    
 4 − 8 0 l 21 l 22 0  = 0 1 0
    
 − 2 1 5 l 31 l 32 l 33  0 0 1
4l 11 0 0  1 0 0

 4l 11 − 8l 21 − 8l 22 0  = 0 1 0
 − 2l 11 + l 21 + 5l 31 l 22 + 5l 32 5l 33  0 0 1

On comparing corresponding elements both the side, we get

1
4l 11 = 1  l 11 =
4
1
4l 11 − 8l 21 = 0  l 21 =
8
1
− 8l 22 = 1  l 22 = −
8
3
− 2l 11 + l 21 + 5l 31 = 0  l 31 =
40
1
l 22 + 5l 32 = 0  l 32 =
40
1
5l 33 = 1  l 33 =
5
1 
4 0 0
 
∴ M =  1 − 1 0
8 8

3 1 1
 40 40 5 
Hence,
 1 
− 4 0 0 0 − 1 1
   
H=− 1 1
0 0 0 1 [Using Eqn. (31)]
 4 8
  
− 3 − 1 1  0 0 0
 40 40 5 

 1 1
0 4 − 4 
 
= 0 1 − 1 
4 8
 
0 3 − 1 
 40 10 

The eigenvalues of the iteration matrix H are the roots of the

characteristic equation
1 1
−λ
4 4
1 1
det (H − λI) = 0 −λ − =0
4 8
3 1
0 − −λ
40 10
221
Block 2 Solutions of Linear Algebraic Equations
We have
3 2 1
− λ3 + λ + λ=0
20 64
which gives
λ = 0, 0.2207, − 0.0707.

Therefore, we have
ρ(H ) = max { λ i } = 0.2207
i

The rate of convergence of the method is given by

ν = − log10 (ρ(H )) = − log10 (0.2207) = 0.6561

We compare the rate of convergence of Jacobi method and Seidel

method for the given system of equations. We observe that the rate of
convergence of Jacobi method is 0.4753 (ref. Example 2) and rate of
convergence of Gauss Seidel method is 0.6561. Thus, the Gauss Seidel
method converges faster than Jacobi method.

The number of iterations needed for obtaining the desired accuracy is

given by
m 2 (k)
k= = ≈ 3 [ m = 2 as ε i ≤ 10 −2 ]
ν 0.6561

We can also compare the number of iterations to make max ε i( k ) ≤ 10 −2.

i
To achieve this accuracy, the number of iterations needed in Jacobi
method is 5 (ref. Example 2) whereas in Seidel method we need 3
iterations. Thus, we require less number of iterations in Seidel method
than Jacobi method to achieve the same accuracy for the given system
of equations.

The Gauss-Seidel method when applied to the system of Eqn. (30)

becomes
1
x 1( k +1) = [7 − x (3k ) + x (2k ) ]
4
1
x (2k+1) = − [−21 − 4x 1( k+1) − x 3( k ) ] (32)
8
1
x 3( k +1) = [15 + 2 x 1( k +1) − x (2k +1) ]
5
where k = 0, 1, 2, K .

We get the successive iterations using x (0) = [1 2 2] as given below.

T

Iteration 1: Putting k = 0 in Eqn. (32), we get

x (1) = [1.75 3.75 2.95]T

Iteration 2: Putting k = 1 in Eqn. (32), we get

x ( 2 ) = [1.95 3.9688 2.9863]T

Iteration 3: Putting k = 2 in Eqn. (32), we get

x ( 3) = [1.9956 3.9961 2.9990]T
222
Unit 7 Iterative Methods

which is an approximation to the exact solution after three iterations.

Thus, you can see that result obtained after three iterations is quite
close to the exact solution [2 4 3]T .
***

Example 7: Perform four iterations (rounded-off to four decimal places)

using the Gauss-Seidel method for solving the system of equations
− 8 1 1   x1   1 
 1 −5 1   x  = 16 (33)
   2  
 1 1 − 4  x 3   7 

with x ( 0) = [0 0 0] . The exact solution is x = [−1 − 4 − 3]T . Compare

T

the results with those obtained in Example 1 after 4 iterations.

Solution: The Gauss-Seidel method as given in Eqn. (27), for the given
system is
1
x 1( k +1) = [ x (2k ) + x (3k ) − 1]
8
1
x (2k +1) = [ x 1( k +1) + x 3( k ) − 16] (34)
5
1
x (3k +1) = [ x1( k +1) + x (2k +1) − 7],
4
where k = 0, 1, 2, K .

Starting with x (0 ) = [0 0 0]T , we now perform the iterations of the

method.

Iteration 1: Putting k = 0 in Eqn. (34), we obtain

1
x 1(1) = [0 + 0 − 1] = −0.125
8
1
x (21) = [−0.125 + 0 − 16] = −3.225
5
1
x (31) = [−0.125 − 3.225 − 7] = −2.5875
4

Iteration 2: Putting k = 1 in Eqn. (34), we obtain

1
x 1( 2) = [−3.225 − 2.5875 − 1] = −0.8516
8
1
x (22) = [−0.8516 − 2.5875 − 16] = −3.8878
5
1
x 3( 2) = [−0.8516 − 3.8878 − 7] = −2.9349
4

Iteration 3: Putting k = 2 in Eqn. (34), we obtain

1
x 1(3) = [−3.8878 − 2.9349 − 1] = −0.9778
8
1
x (23) = [−0.9778 − 2.9349 − 16] = −3.9825 223
5
Block 2 Solutions of Linear Algebraic Equations
1
x 3(3) = [−0.9778 − 3.9825 − 7] = −2.9901
4

Iteration 4: Putting k = 3 in Eqn. (34), we obtain

1
x 1( 4) = [−3.9825 − 2.9901 − 1] = −0.9966
8
1
x (24) = [−0.9966 − 2.9901 − 16] = −3.9973
5

1
x 3( 4) = [−0.9966 − 3.9973 − 7] = −2.9985
4

Thus after 4 iterations we get the solution vector

x ( 4) = [−0.9966 − 3.9973 − 2.9985]T , which is a good approximation to
the exact solution x = [−1 − 4 − 3]T with maximum absolute error 0.0034 .
Comparing with the results obtained in Example 1, we find that the
values of x i , i = 1, 2, 3 obtained here are better approximation to the
exact solution than the one obtained in Example 1.
***

Example 8: Use the Gauss-Seidel method for solving the following

system of equations.
 2 −1 0 0   x1  1 
 − 1 2 − 1 0   x  0
   2 =   (35)
 0 − 1 2 − 1  x 3  0
    
0 0 − 1 2   x 4  1 

with x ( 0) = [0.5 0.5 0.5 0.5]T . Compare the results with those obtained
in Example 3 after four iterations. The exact solution is x = [1 1 1 1]T .

Solution: The Gauss-Seidel method, when applied to the system of

Eqn. (35) becomes
1
x 1( k +1) = [1 + x (2k ) ]
2
1
x (2k +1) = [ x 1( k +1) + x (3k ) ]
2
1
x 3( k +1) = [ x (2k +1) + x (4k ) ] (36)
2
1
x (4k+1) = [1 + x 3( k +1) ],
2
where, k = 0, 1, 2, K

Starting with the initial approximation x (0) = [0.5 0.5 0.5 0.5]T , we obtain
the following iterations:

Iteration 1: Putting k = 0 in Eqn. (36), we get

x (1) = [0.75 0.625 0.5625 0.7813]T
224
Unit 7 Iterative Methods
Iteration 2: Putting k = 1 in Eqn. (36), we get
x ( 2 ) = [0.8125 0.6875 0.7344 0.8672]T

Iteration 3: Putting k = 2 in Eqn. (36), we get

x (3) = [0.8438 0.7891 0.8281 0.9141]T

Iteration 4: Putting k = 3 in Eqn. (36), we get

x ( 4 ) = [0.8946 0.8614 0.8878 0.9439]T

In Example 3, the result obtained after four iterations by the Jacobi

method is x ( 4) = [0.8438 0.75 0.75 0.8438]T . Thus, the maximum error in
the solution of given system after four iterations of Jacobi method is
0.25 and that of Seidel method is 0.1386. Hence, after 4 iterations
maximum error in Seidel method is less than that of Jacobi method.
***

Example 9: If Jacobi’s method and Seidel method are used to solve the
system of equation
1 2 − 3  x 1   6 
 − 3 1 − 1  x  =  8 
   2  
 2 − 1 1   x 3  9

which of the methods converge faster?

Solution: Let us find the rate of convergence of each of the methods.

Jacobi Method:

The iteration matrix H is as follows:

1 0 0 0 0 0  0 2 − 3
D = 0 1 0 , L =  − 3 0 0 and U = 0 0 1 
   
0 0 1  2 − 1 0 0 0 0 

− 1 0 0 0 2 − 3
H = − D (L + U ) =  0 − 1 0 
−1  − 3 0
 1 
 0 0 − 1  2 − 1 0 

 0 −2 3 
=  3 0 − 1
 − 2 1 0 

The eigenvalues of the iteration matrix H are given by

| H − λI | = 0  −λ3 − 13λ + 5 = 0  λ = 0.3804, − 0.1902, − 0.1902
The spectral radius ρ(H ) = 0.3804 < 1
Hence, the Jacobi method converges. 225
Block 2 Solutions of Linear Algebraic Equations
Seidel Method:
0 − 2 3
The iteration matrix H = −(D + L) U = 0 − 6 8 .
−1

0 − 2 2
The eigenvalues are given by
| H − λI | = 0  −λ (λ + 2) 2 = 0  λ = 0, − 2, − 2
The spectral radius ρ(H ) = 2 > 1
Hence, the Seidel method does not converge.
***

Example 10: Consider the following system of linear equations

 1 − k   x1   b1 
 − k 1   x  = b ,
   2  2
where k is a constant. For what value(s) of k, the Jacobi and the
Gauss-Seidel methods converge. Also, show that
ρ (Gauss-Seidel) = [ρ (Jacobi) ]2 .

Solution: We find the convergence of the Jacobi method.

The iteration matrix is given as follows:
1 0  0 0 0 − k 
D=  , L=  and U =  
0 1 − k 0 0 0 
H = − D −1 (L + U)
0 k 
= .
k 0 
The eigenvalues of the iteration matrix H are the roots of the
characteristic equation det(H − λI ) = 0
−λ k
 =0
k −λ

 λ2 − k 2 = 0
 λ = ± k.
Thus, ρ(H ) = | k | or ρ (Jacobi) = | k | .
The method converges if ρ(H ) < 1, i.e., | k | < 1 or − 1 < k < 1 . So, using
Theorem 1, we conclude that the Jacobi method would converge, if
− 1 < k < 1.

We now check the convergence of Gauss-Seidel method.

The iteration matrix H is

H = −(D + L ) −1 U
 1 0  0 − k 
= −  
k − 1 0 0 
226
Unit 7 Iterative Methods
0 k 
= 2
0 k 
The eigenvalues of the iteration matrix H are the roots of the
characteristic equation det(H − λI ) = 0
−λ k
 2
=0
0 k −λ

 −λ ( k 2 − λ ) = 0
 λ = 0, k 2 .
Thus, ρ(H ) = | k 2 | = k 2 or ρ (Gauss-Seidel) = k 2 .

The method converges if ρ(H ) < 1, i.e., k 2 < 1 or 0 < k < 1. So, using
Theorem 1, we conclude that the Gauss-Seidel method would
converge, if 0 < k < 1.
Also, we observe that
ρ (Jacobi) = | k |

and ρ (Gauss-Seidel) = | k 2 | = | k |2 .
Thus, [ρ (Jacobi) ]2 = ρ (Gauss-Seidel).
***

Remark 2: As in the case of the Jacobi method, the matrix formulations

of the Gauss-Seidel method are used whenever we want to check
whether the iterations converges or to find the rate of convergence of
the method. If we wish to iterate and find solutions of the systems, we
shall use the equation form of the method.

You may now attempt the following exercises.

E7) Perform four iterations of the Gauss-Seidel method for solving the
system of equations given in E2. Also, compare the results with
exact solution [1 − 1 − 1]T .

E8) Perform four iterations of the Gauss-Seidel method for solving the
system of equations given in E3 using x ( 0) = 0. Also, compare the
result with exact solution [1 1 1]T .

E9) Set up the matrix formulation of the Gauss-Seidel method for

solving the system of equations with x ( 0) = 0 given in E5. Perform
four iterations of the method.

E10) Gauss-Seidel method is used to solve the system of equations

given in E6. Determine the rate of convergence and the number
of iterations needed to make max | ε i( k ) | ≤ 10 −2 . Perform four
i
( 0)
iterations starting with x = 0 and compare the results with the
exact solution x = [1 1 − 1]T .
227
Block 2 Solutions of Linear Algebraic Equations
E11) Consider the system of linear equations given in E6. Find the
relation between the rate of convergences found in E6 using
Jacobi method and in E10 using Gauss Seidel method.

The iterative methods which you studied so far are Jacobi method and
Gauss Seidel method. Both these methods converge when the
coefficient matrix has strictly either diagonally (row/column) dominant. In
the following section, we will discuss an iterative method called
successive over relaxation method. This method is a generalisation of
the Gauss Seidel method.

7.5 SUCCESSIVE OVER RELAXATION (SOR)

METHOD
So far, we discussed two iterative methods namely; Jacobi and Gauss-
Seidel methods to solve Ax = b. Both the methods converge when the
coefficient matrix A is strictly diagonally row/column dominant. Some
systems may not be of diagonally row/column dominant. In such
systems of linear equations we use the generalised method known as
successive over relaxation method.

David M. Young Jr.

Successive over relaxation method is the generalisation of the Gauss-
(1923-2008) Seidel method.

This method was developed by the two American mathematicians David

M. Young Jr. (1923-2008) and Stanley P. Frankel (1919-1978). In this
method, we use the relaxation factor w and rewrite the system of
equations Ax = b as
x = x + w (b − Ax),
 x = x − wAx + wb.

The method converges, if I − wA < 1.

Thus, the iteration scheme of the method is
Stanley P. Frankel.
(1919-1978) x ( k +1) = x ( k ) − wAx ( k ) + wb.

Therefore, we define the SOR method as:

w
x 1( k +1) = (1 − w )x 1( k ) −
a 11
(
a 12 x (2k ) + a 13 x 3( k ) + ... + a 1n x (nk ) − b1 )
w
x (2k +1) = (1 − w )x (2k ) −
a 22
(
a 12 x 1( k +1) + a 23 x (3k ) + ... + a 2 n x (nk ) − b 2 ) (37)

M
w
x (nk +1) = (1 − w )x (nk ) −
a nn
(
a n1x 1( k +1) + a n 2 x (2k +1) + ... + a n , n−1 x (nk−+11) − b n )
or in general
w  i−1 n 
x (i k+1) = (1 − w ) x i( k ) −  a ij x j +  a ij x j − b i  , i = 1,2,3,..., n, and
( k +1) (k)

a ii  j=1 j=i +1 
k = 0,1, 2,K (38)
228
Unit 7 Iterative Methods
where w is a constant called the relaxation factor. You may notice
that this method reduces to Gauss-Seidal method, if w = 1.

If we take the terms with ( x j ) k+1 from RHS of Eqn. (38) to the LHS, we
can write it in the matrix form as
(D + wL )x ( k +1) = (1 − w )Dx ( k ) − w Ux ( k ) + wb (39)
where D, L and U are diagonal matrix, lower triangular matrix and
upper triangular matrix, respectively with zero diagonal elements. From
Eqn. (39), we obtain
x ( k +1) = (D + wL ) −1 (1 − w )Dx ( k ) − (D + wL) −1 w U x ( k ) + (D + wL) −1 wb
x ( k +1) = (D + wL ) −1 [(1 − w )D − wU ] x ( k ) + (D + wL ) −1 wb (40)
which is of the form (5), with
H = (D + wL) −1 [(1 − w )D − w U ] and c = (D + wL) −1 wb. (41)

You may note that this method depends on the value of the relaxation
factor w. Different values of w in Eqn. (41) yield different forms of the
method.

1) When 0 < w < 1, the method is called under relaxation.

2) When w = 1, the method becomes Gauss-Seidel method.

3) When 1 < w < 2, the method is over relaxation, that is SOR.

4) When w > 2, the method does not converge.

The convergences of the SOR method also depends on the value of w.

To attain the maximum convergence the value of w depends on the
spectral radius of Jacobi method. The optimum value of relaxation
factor is
2
 µ 
wo = 1+   , (42)
 1 + 1 − µ2 
 
where µ = ρ (Jacobi).

The spectral radius of the SOR method is

ρ(H ) = w o − 1 (43)

The rate of convergence of the SOR method is

ν = − log10 (ρ(H))
 ν = − log10 ( w o − 1) (44)

We consider a few examples to illustrate the method.

Example 11: Perform four iterations for the following system of

equations using SOR method.
229
Block 2 Solutions of Linear Algebraic Equations
3x 1 − x 2 + x 3 = −1
− x 1 + 3x 2 − x 3 = 7
x 1 − x 2 + 3x 3 = −7

You may use w = 1.25, the initial approximation is x ( 0) = [0 0 0] .

T

Compare the result with the exact solution x = [1 2 − 2]T .

Solution: The SOR method as given in Eqn. (37) for the given system
is
w
x 1( k +1) = (1 − w )x 1( k ) + (− 1 + x (2k ) − x 3( k ) )
3
w
(
x 2( k +1) = (1 − w )x 2( k ) +
3
)
7 + x 1( k +1) − x 3( k ) (45)
w
(
x 3( k +1) = (1 − w )x 3( k ) +
3
)
− 7 − x1( k +1) − x (2k+1)
where, k = 0, 1, 2, K

Starting with x (0) = [0 0 0] and w = 1.25, we now perform the

T

following iterations:

Iteration 1: Putting k = 0 in Eqn. (45), we get

w
x1(1) = (1 − w )x 1(0) + ( 3
)
− 1 + x 2( 0) − x 3(0 )
1.25
= (1 − 1.25) ⋅ 0 + (−1 + 0 − 0) = −0.41667
3
w
x 2(1) = (1 − w ) x (20 ) + (7 + x (20 ) − x 3( 0 ) )
3
1.25
= ( −0.25) 0 + (7 − 0.41667 + 0) = 2.7431
3
w
x 3(1) = (1 − w ) x 3(0 ) + (− 7 + x1(1) − x (21) )
3
1.25
= ( −0.25)0 + ( −7 + 0.41667 + 2.7431) = −1.6001
3
Hence, x (1) = [− 0.41667 2.7431 1.6001]
T

Iteration 2: Putting k = 1 in Eqn. (45), we get

x1(2) = 1.4972
x 2( 2) = 2.1880
x 3( 2 ) = −2.2288
Thus, x ( 2) = [1.4972 2.1880 − 2.2288]
T

Iteration 3: Putting k = 2 in Eqn. (45), we get

x (3) = [1.0494 1.8782 − 2.0141]
T

Iteration 4: Putting k = 3 in Eqn. (45), we get

x ( 4 ) = [0.9428 2.0007 − 1.9723]
T

After four iterations the solution obtained is a reasonably good

approximation to the exact solution x = [1 2 − 2] .
T

230
***
Unit 7 Iterative Methods
Example 12: Successive over relaxation method is used to solve the
system of equations as given in Example 2.
 4 − 1 1  x1   7 
 4 − 8 1  x  = − 21 (46)
   2  
 − 2 1 5  x 3   5 

Obtain the optimal value of w and use it to determine the rate of

convergence of the method and the number of iterations needed to
make max ε i( k ) ≤ 10 −2. Perform these number of iterations of the SOR
i

method starting with x (0 ) = [1 2 2] . Also, compare the result with

T

exact solution x = [2 4 3] .
T

Solution: To find the optimal value of w , we use spectral radius of

Jacobi method. From Example 2,
ρ (Jacobi) = 0.3347
Thus we substitute µ = 0.3347 in Eqn. (42) and get
2
 0.3347 
wo = 1+  
 1 + 1 − (0.3347) 2 
 
≈ 1.03

Therefore, the spectral radius for the SOR method is given by

ρ(H ) = w o − 1 = 1.03 − 1 = 0.03

The rate of convergence of the SOR method is given by

ν = − log10 (ρ(H )) = − log10 (0.03) = 1.5228

The number of iterations needed for obtaining the desired accuracy is

given by
m 2
k= = ≈ 1.3133 [m = 2 as ε i( k ) ≤ 10 −2 ]
ν 1.5228

The SOR method when applied to the system of Eqn. (46) becomes
1
x 1( k +1) = (1 − w ) ⋅ x 1( k ) + w ⋅ (7 + x (2k ) − x 3( k ) )
4
−1
x 2( k +1) = (1 − w ) x 2( k ) + w ⋅ ( −21 − 4 x1( k +1) − x 3( k ) ) (47)
8
1
x 3( k +1) = (1 − w ) x 3( k ) + w ⋅ (15 + 2x1( k +1) − x 3( k +1) ),
5
where k = 0, 1, 2, K .

The successive iterations using w = 1.03 and x (0) = [1 2 2] are as

T

follows.

Iteration 1: Putting k = 0 in Eqn. (47), we get

[1.7725 3.8141 2.9746]T 231
Block 2 Solutions of Linear Algebraic Equations

Iteration 2: Putting k = 1 in Eqn. (47), we get

[1.9855 3.9845 2.9897]T
which is an approximation to the exact solution after two iterations.
Thus, you can see that result after two iterations is quite close to the
exact solution [2 4 3]T .
***

Remark 3: In Example 2, 6 and 10, you may note that we found the rate
of convergence of the system of linear equations using Jacobi, Gauss-
Seidel and SOR methods, respectively. When we compare the spectral
radius, we get
ρ (Jacobi) = 0.3347
ρ (Gauss-Seidel) = 0.2207
ρ (SOR) = 0.1391
i.e., ρ (Jacobi) > ρ (Gauss-Seidel) > ρ (SOR).

On comparing the rate of convergence of these methods, we get

ν (Jacobi) = 0.47753
ν (Gauss-Seidel) = 0.6561
ν (SOR) = 0.8566 [with w = 1.03]
i.e., ν (Jacobi) < ν (Gauss-Seidel) < ν (SOR)w =1.03

Example 13: Perform four iterations of the SOR method for solving the
following system of equations as given in Example 8:
 2 −1 0 0   x 1  1
 − 1 2 − 1 0   x  0
   2 =   (48)
 0 − 1 2 − 1  x 3  0
    
0 0 − 1 2   x 4  1

with x (0) = [0.5 0.5 0.5 0.5] and w = 1.25. Compare the results with
T

those obtained in Example 3 and Example 8 after 4 iterations. Also,

compare the result with exact solution x = [1 1 1 1]T .

Solution: The SOR method, when applied to the system of Eqn. (48)
becomes
1
x 1( k +1) = (1 − w ) x1( k ) + w ⋅ (1 − x 2( k ) )
2
1
x 2( k +1) = (1 − w ) x 2( k ) + w ⋅ ( x1( k +1) + x 3( k ) ) (49)
2
1
x 3( k +1) = (1 − w ) x 3( k ) + w ⋅ ( x 2( k +1) + x 4( k ) )
2
1
x 4( k +1) = (1 − w ) x 4( k ) + w ⋅ (1 − x 3( k +1) )
2
where k = 0, 1, 2, K

Starting with the initial approximation x (0) = [0.5 0.5 0.5 0.5] and
T

w = 1.25, obtain the following iterations:

232
Unit 7 Iterative Methods
Iteration 1: Putting k = 0 in Eqn. (49), we get
x 1(1) = 0.8125
x (21) = 0.6953
x 3(1) = 0.6221
x (41) = 0.8888

Iteration 2: Putting k = 1 in Eqn. (49), we get

x 1( 2 ) = 0.8564
x (22 ) = 0.7502
x 3( 2 ) = 0.8689
x (42 ) = 0.9459

Iteration 3: Putting k = 2 in Eqn. (49), we get

x 1( 3) = 0.8798
x (23) = 0.9054
x 3( 3) = 0.9398
x (43) = 0.9759

Iteration 4: Putting k = 3 in Eqn. (49), we get

x 1( 4 ) = 0.9709
x (24) = 0.9678
x 3( 4 ) = 0.9799
x (44) = 0.9935

Thus after the fourth iteration we get the solution vector

x (4) = [0.9709 0.9678 0.9799 0.9935]T , with maximum absolute error
0.0322 . However, the result is reasonable close to the exact solution
x = [1 1 1 1]T . Comparing with the results obtained in Example 3 and
Example 8, we find that the values of x i , i = 1, 2, 3, 4 obtained here are
better approximation to the exact solution that the one obtained in
Example 3 and Example 8.
***

Example 14: Solve the following system of equations

3x1 + x 2 + 2 x 3 = 6
− x1 + 4x 2 + 2x 3 = 5 (50)
2x1 + x 2 + 4 x 3 = 7
using SOR method with (i) w = 2.5 and (ii) w = 1.1 starting with
x ( 0) = [0 0 0]T . Also, compare the results obtained in (i) and (ii) with the
exact solutions x = [1 1 1]T .

Solution: The iterated values for the given system of equations (50) are
1
x 1( k +1) = (1 − w ) x1( k ) + w ⋅ (6 − x 2( k ) − 2 x 3( k ) )
3 233
Block 2 Solutions of Linear Algebraic Equations
1
x (2k +1) = (1 − w ) x 2( k ) + w ⋅ (5 + x1( k +1) − 2 x 3( k ) ) (51)
4
1
x 3( k +1) = (1 − w ) x 3( k ) + w ⋅ (7 − 2 x1( k +1) − x (2k +1) )
4
where, k = 0, 1, 2, K .

i) Taking w = 2.1 and starting with x ( 0) = [0 0 0]T , we get the

following iterations:

Iteration 1: Putting k = 0 in Eqn. (51), we get

x (1) = [4 4.5 − 2.75]T .

Iteration 2: Putting k = 1 in Eqn. (51), we get

x ( 2) = [0.667 1.0833 5.0417]T .

Iteration 3: Putting k = 2 in Eqn. (51), we get

x (3) = [−4.1111 − 5.6806 5.4097]T .

Iteration 4: Putting k = 3 in Eqn. (51), we get

x ( 4) = [4.6852 5.1134 − 9.1516]T .

After 4 iterations, it is clear that the method does not converge as it is

not close to the exact solution x = [1 1 1]T .

ii) Let us now perform 4 iterations with w = 1.1 . Again starting with
x ( 0) = [0 0 0]T we get the following iterations.

Iteration 1: Putting k = 0 in Eqn. (51), we get

x (1) = [2.2 1.98 0.1705]T .

Iteration 2: Putting k = 1 in Eqn. (51), we get

x ( 2) = [1.129 1.3937 0.9038]T .

Iteration 3: Putting k = 2 in Eqn. (51), we get

x (3) = [0.9133 0.9897 1.0601]T .

Iteration 4: Putting k = 3 in Eqn. (51), we get

x ( 4) = [0.9683 0.9593 1.0226]T .

After 4 iterations with w = 1.1, we observe that the solution vector

x ( 4) = [0.9683 0.9593 1.0226]T is reasonable close to the exact solution
x = [1 1 1]T . You may note that the method does not converge for
w = 2.1 > 2 and converges for w = 1.1 < 2.
***

Now, you may try the following exercise.

234
Unit 7 Iterative Methods

E12) Perform four iterations of the SOR method for solving the system
of equations given in E2. Use w = 1.2, and start with
x ( 0) = [0 0 0]T . Also, compare the result with the exact solution
x = [1 − 1 − 1]T .

E13) The SOR method is used to solve the system of equations given in
E6. Obtain the optimal value of w and use it to determine the rate
of convergence and the number of iterations needed to make
max ε i( k ) ≤ 10 −2. Perform four iterations with x ( 0) = [0 0 0]T , and
i

compare the result with exact solution x = [1 1 − 1]T . Also, compare

the maximum error obtained in E6, and E10 using Jacobi method
and Gauss-Seidel method, respectively with the maximum error in
solution using the SOR method.

E14) Perform four iterations of the SOR method for solving the system
of equations given in E3. Use w = 1.25 and start with
x ( 0) = [0 0 0]T . Also, compare the result with exact solution
x = [1 1 1]T .

E15) Perform four iterations of the SOR method for solving the system
of equations given in E4. Use w = 1.5 and start with
x = [0 0 0 0]T .

E16) Set up the matrix formulation of the SOR method for solving the
system of equations given in E5. Perform four iterations of the
SOR method. Use w = 1.75 and start with x = [0 0 0 0]T .

We now end this unit by giving a summary of what we have covered in

it.

7.5 SUMMARY
In this unit we have covered the following:

1. Iterative methods for solving linear system of equations

Ax = b (see Eqn. (1))
where A is an n × n , non-singular matrix. Iterative methods are
generally used when the system is large and the matrix A is
sparse. The process is started using an initial approximation and
lead to successively better approximations.

2. General iterative method for solving the linear system of Eqn. (1)
can be written in the form
x ( k +1) = Hx ( k ) + c, k = 0, 1, K (see Eqn. (5))
where x ( k ) and x ( k +1) are the approximations to the solution vector
x at the kth and the (k + 1) th iterations respectively. H is the
235
Block 2 Solutions of Linear Algebraic Equations
iteration matrix which depends on A and is generally a constant
matrix. c is a column vector and depends on both A and b .

3. Iterative method of the form given in 2) above converges for any

initial vector, if || H || < 1 , which is a sufficient condition for
convergence. The necessary and sufficient condition for
convergence is ρ(H ) < , where ρ(H) is the spectral radius of H .

4. In the Jacobi iteration method or the method of simultaneous

displacements.
H = − D −1 (L + U) and c = D −1b
where D is a diagonal matrix, L and U are respectively the lower
and upper triangular matrices with zero diagonal elements.

5. In the Gauss-Seidel iteration method or the method of

successive displacements
H = −(D + L ) −1 U and c = (D + L) −1 b .

6. In the SOR method

H = (D + wL) −1 [(1 − w )D − w U ] and c = (D + wL) −1 wb. .

7. If the matrix A in Eqn. (1) is strictly diagonally dominant then the

Jacobi and Gauss-Seidel methods converge. Gauss-Seidel
method converges faster than the Jacobi method. And, SOR
method converges faster than the Gauss-Seidel method.

8. The iterative method we discussed are summarised in the

following table.

Name Method
of the
Method
Jacobi  
1 n 
x ( k +1)
i =   a ij x j − b i , i = 1, 2, ..., n and k = 0,1,...
(k)

a ii  j=1 
 i≠ j 
Gauss-
1  i −1 n 
Seidel x i( k +1) =   a ij x (jk ) −  a ij x (jk +1) −b i , i = 1, 2, ..., n and k = 0, 1, 2,...
a ii  j=1 j=1 
SOR
w i −1 n 
x i( k +1) = (1 − w ) x i( k ) −   a ij x (jk +1) −  a ij x (jk ) −b i , i = 1, 2, ..., n
a ii  j=1 j=1+1


and k = 0,1, 2,...

7.6 SOLUTIONS/ANSWERS
 0 0 − 2  − 1
E1) The Jacobi’s method gives x ( k +1)
=  0 0 2  x +  5  , where
  (k )

− 1 1 0  − 3

k = 0, 1, 2, K .
236
Unit 7 Iterative Methods
( 0) T
We start with x = [1 1 1] and get the following iterations:

Iteration 1: Putting k = 0,
x (1) = [−3 7 − 3]T

Iteration 2: Putting k = 1,
x ( 2) = [5 − 1 7]T

Iteration 3: Putting k = 2,
x (3) = [−15 19 − 9]T

Iteration 4: Putting k = 3,
x ( 4) = [17 − 13 31]T

Iteration 5: Putting k = 4,
x (5) = [−63 67 − 33]T
Iterations do not converge.

E2) The Jacobi’s method gives

1
x 1( k +1) = [1 − 2x 2( k ) − 2x 3( k ) ]
5
1
x (2k +1) = [ −6 − 2 x1( k ) − 3x 3( k ) ]
5
1
x 3( k +1) = [ −4 − 2 x1( k ) − x 2( k ) ]
5
where k = 0, 1, 2, K

We start with x ( 0) = [0 0 0]T and perform the following iterations:

Iteration 1: Putting k = 0
x (1) = [0.2 − 1.2 − 0.8]T

Iteration 2: Putting k = 1
x ( 2 ) = [1.0 − 0.8 − 0.64]T

Iteration 3: Putting k = 2
x (3) = [0.776 − 1.216 − 1.04]T

Iteration 4: Putting k = 3
x ( 4) = [1.1024 − 0.8864 − 0.8672]T
After 4 iterations, the solution vector
x ( 4) = [1.1024 − 0.8864 − 0.8672]T is reasonably close to the exact
solution vector x = [1 − 1 − 1]T .

E3) The Jacobi’s method gives

1
x 1( k +1) = [1.5 + 0.5x 2( k ) ]
2 237
Block 2 Solutions of Linear Algebraic Equations
1
x (2k +1) = [1.5x1( k ) + 0.5x 3( k ) ]
2
1
x 3( k +1) = [0.5 + 1.5x 2( k ) ]
2
where k = 0, 1, 2, K .

We start with x ( 0) = [0 0 0]T and perform the following iterations:

Iteration 1: Putting k = 0
x (1) = [0.75 0.0 0.25]T

Iteration 2: Putting k = 1
x ( 2) = [0.75 0.625 0.4375]T

Iteration 3: Putting k = 2
x (3) = [0.9063 0.625 0.7188]T

Iteration 4: Putting k = 3
x ( 4) = [0.9062 0.8594 0.7188]T

After 4 iterations, the solution vector

x ( 4) = [0.9062 0.8594 0.7188]T is not very close to the exact
solution vector x = [1 1 1]T . It requires few more iterations.

E4) The Jacobi’s method gives

1
x 1( k +1) = [−4 + x 2( k ) + x 3( k ) + x (4k ) ]
5
1
x (2k +1) = [12 + x 1( k ) + x 3( k ) + x (4k ) ]
10
1
x 3( k +1) = [8 + x1( k ) + x (2k ) + x (4k ) ]
5
1
x 3( k +1) = [34 + x1( k ) + x (2k ) + x 3( k ) ]
10

where k = 0, 1, 2, K .

We start with x ( 0) = [0 0 0 0]T and get the following iterations:

Iteration 1: Putting k = 0
x (1) = [−0.8 1.2 1.6 3.4]T

Iteration 2: Putting k = 1
x ( 2 ) = [0.44 1.62 2.36 3.6]T

Iteration 3: Putting k = 2
x ( 3) = [0.716 1.84 2.732 3.842]T
238
Unit 7 Iterative Methods
Iteration 4: Putting k = 3
x ( 4 ) = [0.8828 1.9290 2.8796 3.9288]T

After 4 iterations, the solution vector

x ( 4) = [0.8828 1.9290 2.8796 3.9288]T is reasonably close to the
exact solution [1 2 3 4]T .

E5) We apply Jacobi’s method and get

 1 1 1
0 0
4 4 2
 1 1
 1
0 0   
4 4 2
x ( k +1) =  x (k) + 
1 1
0 0 1
4 4  2
1 1  1
4 0 0 2
 4   
where k = 0, 1, 2, K

We start with x ( 0) = 0 and perform the following iterations:

Iteration 1: Putting k = 0
x (1) = [0.5 0.5 0.5 0.5]T

Iteration 2: Putting k = 1
x ( 2 ) = [0.75 0.75 0.75 0.75]T

Iteration 3: Putting k = 2
x ( 3) = [0.875 0.875 0.875 0.875]T

Iteration 4: Putting k = 3
x ( 4 ) = [0.9375 0.9375 0.9375 0.9375]
T

After 4 iterations, the solution vector

x ( 4) = [0.9375 0.9375 0.9375 0.9375]T is reasonably close to the
exact solution [1 1 1 1]T .

E6) The iteration matrix H is given as follows:

2 0 0  0 0 0 0 0 1 
D = 0 3 0, L = 0 0 0 and U = 0 0 2
   
0 0 4 1 2 0 0 0 0
Thus,
 1
0 0 −
2
 2

H = D-1 (L + U ) =  0 0 − 
3
 
− 1 −
1
0
 4 2 
239
Block 2 Solutions of Linear Algebraic Equations
Eigenvalues of H are given by
11 11
| H − λI | = 0  −λ3 + λ = 0  λ = 0, ± .
24 24
11
ρ(H ) = = 0.6770 (spectral radius)
24
ν = log10 ρ(H) = 0.1694 (rate of convergence)
m 2
k= = ≈ 12 (number of iterations)
ν 0.1694

The Jacobi’s method gives

1
x 1( k +1) = [1 − x 3( k ) ]
2
1
x (2k +1) = [1 − 2 x 3( k ) ]
3
1
x 3( k +1) = [ −1 − x1( k ) − 2x (2k ) ]
4
where k = 0, 1, 2, K .

We start with x ( 0) = 0 and perform the following iterations:

Iteration 1: Putting k = 0
x (1) = [0.5 0.3333 − 0.25]T

Iteration 2: Putting k = 1
x ( 2) = [0.625 0.500 − 0.5417]T

Iteration 3: Putting k = 2
x ( 3) = [0.7709 0.6945 − 0.6563]T

Iteration 4: Putting k = 3
x ( 4 ) = [0.8282 0.7709 − 0.7900]T

You can see that the solution vector

x (4) = [0.8282 0.7709 − 0.7900]T is not much close to the exact
solution [1 1 − 1]T . It is tending to the exact solution but require
more iterations.

E7) The Gauss-Seidel method gives

1
x 1( k +1) = [1 − 2x 2( k ) − 2x 3( k ) ]
5
1
x (2k +1) = [ −6 − 2 x1( k +1) − 3x 3( k ) ]
5
1
x 3( k +1) = [ −4 − 2x 1( k +1) − x 2( k +1) ]
5
where k = 0, 1, 2, K .
240
Unit 7 Iterative Methods
( 0)
Starting with x = 0 , we get the following iterations:

Iteration 1: Putting k = 0
x (1) = [0.2 − 1.28 − 0.624]T

Iteration 2: Putting k = 1
x 2 = [0.9616 − 1.2102 − 0.9426]T

Iteration 3: Putting k = 2
x 3 = [1.0611 − 1.0589 − 1.0127]T

Iteration 4: Putting k = 3
x ( 4 ) = [1.0286 − 1.0038 − 1.0107]T

You can see that the solution vector

x (4) = [1.0286 − 1.0038 − 1.0107]T is quite close to the exact solution
[1 − 1 − 1]T .

E8) The Gauss-Seidel method gives

1
x 1( k +1) =
2
[
1.5 + 0.5x 2( k ) ]
1
x (2k +1) =
2
[
1.5x1( k +1) + 0.5x 3( k ) ]
1
x 3( k +1) =
2
[
0.5 + 1.5x (2k +1) ]
where k = 0, 1, 2, K .

Starting with x ( 0) = 0, we get the following iterations:

Iteration 1: Putting k = 0
x (1) = [0.75 0.5625 0.6719]T

Iteration 2: Putting k = 1
x ( 2 ) = [0.8906 0.8359 0.8769]T

Iteration 3: Putting k = 2
x ( 3) = [0.9590 0.9385 0.9539]T

Iteration 4: Putting k = 3
x ( 4 ) = [0.9846 0.9769 0.9827]T

You can see that after 4 iterations the solution vector

x ( 4) = [0.9846 0.9769 0.9827]T is quite close to the exact solution
[1 1 1]T . 241
Block 2 Solutions of Linear Algebraic Equations
E9) The iteration matrix H is as follows:
0 0 0 0  1
1 0 0 0 0 0 −1 − 
0 0 0 0  4 4
0 1 0 0    
D=  , L = − 1 − 1 0 0 and U = 0 0 −1 1
− .
0 0 1 0  4 4   4 4

   1 1  0 0 0 0 
0 0 0 1 − 4 − 4 0 0 0
 0 0 0 
Thus
 1 0 0 0
 
 0 1 0 0
(D + L) −1 =  1 1 
1 0
4 4 
 1 1 0 1
 4 4 
 1 1
0 0 4 4 
 
0 0 1 1 
4 4
H = −( D + L) −1 U =  
0 0 1 1 
 8 8
 1 1
0 0 8 8 
 
T
1 1 3 3
−1
c = ( D + L) b =  
2 2 4 4
The matrix formulation for the Gauss-Seidel method is
 1 1 1
0 0
4 4 2
 1 1
 1
0 0   
4 4 2
x ( k +1) =  x(k ) +  (ref. Eqn. (29))
0 0 1 1 3
 8 8 4
 1 1 3
0 0
8  4
 8  
where k = 0, 1, 2, K .

We start with x ( 0) = 0 and get the following iterations:

Iteration 1: Putting k = 0
x (1) = [0.5 0.5 0.75 0.75]T

Iteration 2: Putting k = 1
x ( 2 ) = [0.875 0.875 0.9375 0.9375]T

Iteration 3: Putting k = 2
x ( 3) = [0.9688 0.9688 0.9844 0.9844]T

Iteration 4: Putting k = 3
x ( 4 ) = [0.9922 0.9922 0.9961 0.9961]T
242
Unit 7 Iterative Methods
E10) The iteration matrix H is as follows:
2 0 0  0 0 0 0 0 1 
  
D = 0 3 0, L = 0 0 0 and U = 0 0 2

0 0 4 1 2 0 0 0 0
1 
 2 0 0
 1

(D + L ) −1 =  0 0
3
 
− 1 − 1 1 
 8 6 4 

 1
0 0 − 2 
 
H = (D + L) −1 U = 0 0 − 2 
3
 
0 0 11 
 24 
Eigenvalues of H are given by
11 11
| H − λI | = 0  −λ3 + λ = 0  λ = 0, 0,
24 24
Thus,
11
ρ( H ) = (Spectral radius)
24
ρ(H ) = 0.4583
ν = − log10 ρ(H ) = 0.3388 (Rate of convergence)
2
k= ≈ 6 (Number of iterations)
v

We apply Gauss-Seidel method with x ( 0) = 0 and get the following

iterations:
Iteration 1: x (1) = [0.5 0.3333 − 0.5417]T
Iteration 2: x ( 2 ) = [0.7709 0.6945 − 0.7900]T
Iteration 3: x ( 3) = [0.8950 0.8600 − 0.9038]T
Iteration 4: x ( 4 ) = [0.9519 0.9359 − 0.9559]T

After 4 iterations, the solution vector

x ( 4) = [0.9519 0.9359 − 0.9559]T is good approximation to the
exact solution x = [1 1 − 1]T .

E11) The spectral radius of the system using Jacobi method is

11
ρ J (H J ) = (ref. E6)). The spectral radius of the system using
24
11
Gauss-Seidel method is ρS ( H S ) = (ref. E10)). It is clear that
24
ρS ( H S ) = [ρ J ( H J )]2 . Since the system under consideration has
symmetric coefficient matrix, therefore it shows that the rate of
243
Block 2 Solutions of Linear Algebraic Equations
convergence of Gauss-Seidel method is twice the rate of
convergence of Jacobi method.

E12) The iteration scheme of the SOR method for the given system of
equations is
1
x 1( k +1) = (1 − w ) x1( k ) + w ⋅ (1 − 2x (2k ) − 2 x 3( k ) )
5
1
x (2k +1) = (1 − w ) x 2( k ) + w ⋅ ( −6 − x 1( k +1) − 3x 3( k ) )
5
1
x 3( k +1) = (1 − w ) x 3( k ) + w ⋅ (−4 − 2 x1( k +1) − x (2k +1) )
5
where k = 0, 1, 2, K .

Taking w = 1.2 and starting with x 0) = [0 0 0] , we get the

T

following iterations:

Iteration 1: Putting k = 0
x (1) = [0.24 − 1.5552 − 0.702]
T

Iteration 2: Putting k = 1
x (2) = [1.2754 − 1.2358 − 1.135 2]
T

Iteration 3: Putting k = 2
x (3) = [1.123 − 0.9145 − 1.0525]
T

Iteration 4: Putting k = 3
x ( 4 ) = [0.9596 − 0.9599 − 0.9797]
T

It is clear that the solution vector x ( 4) = [0.9596 − 0.9599 − 0.9797]T

is reasonably close to the exact solution x ( 0) = [1 − 1 − 1]T .

E13) The iteration matrix H is as follows:

2 0 0  0 0 0  0 0 1 
D = 0 3 0 , L = 0 0 0 and U = 0 0 2
   
0 0 4 1 2 0 0 0 0

From E6, spectral radius of Jacobi method = 0.6770 = µ .

2
 µ 
Now, w o = 1 +  
1 + 1 − µ2 
 
≈ 1.15
ρ(H ) = 0.15 (Spectral radius)
ν = 0.8239 (Rate of convergence)
244
Unit 7 Iterative Methods
2
k= ≈ 2 (number of iterations)
0.8239

Taking w = 1.15 and starting with x (0 ) = [0 0 0]T , we get the

following iteration:

Iteration 1: x (1) = [0.575 0.3833 − 0.6732]T

Iteration 2: x ( 2) = [0.8759 0.842 − 0.9225]T

Iteration 3: x (3) = [0.975 0.9643 − 0.9836]T

Iteration 4: x ( 4) = [0.9945 0.9928 − 0.9967]T

After 4 iterations, the solution vector

x ( 4) = [0.9945 0.9928 − 0.9967]T is reasonably close to the exact
solution x = [1 1 − 1]T . Let us now compare the maximum error
after 4 iterations using the Jacobi method, Gauss-Seidel method
and the SOR method, we get the summarised values given in
Table 1.

Table 1

Iteration Iterated values x (i ) using Iterated values x (i ) using Iterated values x (i ) using
(i) Jacobi method Gauss-Seidel method SOR method
1. [0.5 0.3333 − 0.25]T [0.5 0.3333 − 0.5417]T [0.575 0.3833 − 0.6732]T
2. [0.625 0.500 − 0.5417]T [0.7709 0.6945 − 0.7900]T [0.8759 0.842 − 0.9225]T
3. [0.7709 0.6945 − 0.6563]T [0.8950 0.8600 − 0.9038]T [0.975 0.9643 − 09836]T
4. [0.8282 0.7709 − 0.7900]T [0.9519 0.9359 − 0.9559]T [0.9945 0.9928 − 0.9967]T

After 4 iterations
The maximum error in Jacobi method = 0.2291
The maximum error in Gauss-Seidel method = 0.0641
The maximum error in SOR method = 0.0055

This shows that the Gauss-Seidel method converges faster than

Jacobi method and the SOR method converges faster than
Gauss-Seidel method.

E14) w = 1.25, x ( 0) = [0 0 0]
T

Iteration 1: x (1) = [0.938 0.879 1.136]

T

Iteration 2: x ( 2) = [0.978 1.052 1.015]

T

Iteration 3: x (3) = [1.022 1.012 1.008]

T

Iteration 4: x ( 4) = [0.998 0.998 0.996]

T 245
Block 2 Solutions of Linear Algebraic Equations
After 4 iterations, the solution vector x ( 4) is reasonably close to the
exact solution vector x = [1 1 1]T

E15) w = 1.5, x ( 0) = [0 0 0 0]
T

Iteration 1: x (1) = [− 1.2 1.62 2.526 5.542]

T

Iteration 2: x ( 2) = [2.306 2.546 4.255 3.695]

T

Iteration 3: x (3) = [0.796 1.839 2.171 3.973]

T

Iteration 4: x ( 4) = [0.797 1.922 3.322 4.02]

T

E16) w = 1.75, x ( 0) = [0 0 0 0]
T

Iteration 1: x (1) = [0.875 0.875 1.641 1.641]

T

Iteration 2: x ( 2) = [1.654 1.654 1.092 1.092]

Iteration 3: x (3) = [0.59 0.59 0.572 0.572]

T

Iteration 4: x ( 4) = [0.933 0.933 1.262 1.262]

T

246
 UNIT 8

EIGENVALUES AND
EIGENVECTORS

Structure Page No.

8.1 Introduction 247

Objectives
8.2 The Eigenvalue Problem 248
8.3 The Power Method 257
8.4 The inverse Power Method 263
8.5 Gershgorin Bounds 270
8.6 Summary 274
8.7 Solutions/Answers 276

8.1 INTRODUCTION
In Unit 7, you have seen that eigenvalues of the iteration matrix play a
major role in the study of convergence of iterative methods for solving
linear system of equations. Eigenvalues are also of great importance in
many physical problems. The stability of an aircraft is determined by the
location of the eigenvalues of a certain matrix in the complex plane. The
natural frequencies of the vibrations of a beam are actually eigenvalues
of a matrix. Thus, the computation of the absolute largest eigenvalue or
smallest eigenvalue, or even all the eigenvalues of a given matrix is an
important problem.

For a given system of equations of the form

Ax = λx (1)
or
(A − λI) x = 0 (2)
the values of the parameter λ , for which the system of Eqn. (2) has a
non-zero solution, are called the eigenvalues of A. Corresponding to
these eigenvalues, the non-zero solutions of Eqn. (2) i.e., the vectors x,
are called the eigenvectors of A. The problem of finding the
eigenvalues and the corresponding eigenvectors of a square matrix A
Block 2 Solutions of Linear Algebraic Equations
is known as the eigenvalue problem. In this unit, we shall discuss the
eigenvalue problem. In Sec. 8.2, we will begin by discussing a few
basics of eigenvalue problem. We shall give you some definitions and
properties related to eigenvalues of a matrix. In Sec. 8.3 we will discuss
the power method to find the largest eigenvalue in magnitude and
corresponding eigenvector of a given matrix A. In Sec. 8.4, we will
discuss inverse power method to find the smallest eigenvalue in
magnitude and its corresponding eigenvector of a given matrix. Finally,
in Sec. 8.5, we will find the region in which all the eigenvalues of a
matrix lie and obtain a bound called the Gershgorin bound on the
eigenvalues.

Objectives
After studying this unit, you should be able to:
• solve simple eigenvalue problems;
• obtain the largest eigenvalue in magnitude and the corresponding
eigenvector of a given matrix by using the power method;
• obtain the smallest eigenvalue in magnitude and an eigenvalue
closest to any chosen number along with the corresponding
eigenvector of a given matrix by using the inverse power method;
• obtain the bounds on the eigenvalues of a given matrix.

8.2 THE EIGENVALUE PROBLEM

In the previous three units, we were concerned with the non-
homogeneous system of linear equations, Ax = b . We know that this
system has a unique solution iff the matrix A is non-singular. But, if the
vector b = 0 , then the system reduces to the homogeneous system
Ax = 0 (3)

If the coefficient matrix A, in Eqn. (3) is non-singular, then system has

only the zero solution, x = 0 . For the homogeneous system (3) to have
a non-zero solution, the matrix A must be singular and in this case the
solution is not unique (ref. BMTE-144, Linear Algebra).

The homogeneous system of Eqn. (2) will have a non-zero solution only
when the coefficient matrix (A − λI) is singular, that is,
det(A − λI) = 0 (4)

If the matrix A is an n × n matrix then Eqn. (4) gives a polynomial of

degree n in λ . This polynomial is called the characteristic equation of
A . The n roots λ1 , λ 2 , K, λ n of this polynomial are the eigenvalues of
A . For each eigenvalue λ i , there exists a vector x i (the eigenvector)
which is the non-zero solution of the system of equations
(A − λ i ) xi = 0 (5)

The eigenvalues have a number of interesting properties. We shall now

state and prove a few of these properties which we shall be using
248
Unit 8 Eigenvalues and Eigenvectors
frequently. In the discussion, we shall be using various properties of
determinant of a matrix for which you may refer the course BMTE-141.
P1: A matrix A is singular if and only if it has a zero eigenvalue.

Proof: If A has a zero eigenvalue then

det(A − 0I ) = 0 [using Eqn. (4)]

 det(A) = 0
 A is singular.

Conversely, if A is singular then

det(A ) = 0
 det(A − 0I) = 0
 0 is an eigenvalue of the matrix A

Hence the result.

P2: A matrix A and its transpose matrix A T have the same

eigenvalues.

Proof: If λ is an eigenvalue of A then

det(A − λI) = 0
 det(A − λI ) T = 0 [Since, if det(A ) = 0, then det(A T ) = 0]
[ (
 det(A T − λI T ) = 0 Q det(A ± B) T = det A T ± B T )]
 det(A − λI ) = 0 [∴ I = I ]
T T

 λ is an eigenvalue of A T

Hence the result.

However, the eigenvectors of A and A T are not the same.

P3: If the eigenvalues of a matrix A are λ1 , λ 2 , K, λ n then the

eigenvalues of A m , m any positive integer, are λm1 , λm2 , K, λmn . Also
both the matrices A and A m have the same set of eigenvectors.

Proof: Since λ i (i = 1, 2, K, n ) are the eigenvalues of A, we have

Ax = λ i x, i = 1, 2, K, n (6)
Premultiplying Eqn. (6) by A on both sides, we get
A 2 x = Aλ i x = λ i ( Ax) = λ2i x (7)

which implies that λ21 , λ22 , K, λ2n are the eigenvalues of A 2 .

Further, A and A 2 have the same eigenvectors. Premultiplying
Eqn. (7) (m − 1) times by A on both sides the general result
follows.
1 1 1
P4: If λ1 , λ 2 , K, λ n are the eigenvalues of A, then , , K, are
λ1 λ 2 λn
the eigenvalues of A −1. Also both the matrices A and A −1 have
the same set of eigenvectors. 249
Block 2 Solutions of Linear Algebraic Equations
Proof: Since λ i (i = 1, 2, K, n ) , are the eigenvalues of A, we have
Ax = λ i x, i = 1, 2, K, n (8)

Premultiplying Eqn. (8) on both sides by A −1 , we get

A −1Ax = λ i A −1x
which gives
x = λ i A −1x
1
or A −1x = x
λi
and hence the result.

P5: If λ1 , λ 2 , K, λ n are the eigenvalues of A, then λ i − q, i = 1, 2, K, n

are the eigenvalues of A − qI for any real number q. Both the
matrices A and A − qI have the same set of eigenvectors.

Proof: Since λ i is an eigenvalues of A, we have

Ax = λ i x, i = 1, 2, K, n (9)
Subtracting q x from both sides of Eqn. (9), we get
Ax − qx = λ i x − qx
which gives
( A − qI ) x = ( λ i − q ) x
and the result follows.

P6: If λ i , i = 1, 2, K , n are the eigenvalues of A then

1
, i = 1, 2, K, n are the eigenvalues of ( A − qI ) −1 for any real
λi − q
number q. Both the matrices A and ( A − qI ) −1 have the same set of
eigenvectors.

P6 can be proved by combining P4 and P5. We leave the proof to

you.

E1) Prove P6.

We now give you a direct method of calculating the eigenvalues and

eigenvectors of a matrix.
Example 1: Find the eigenvalues of the following matrices.
1 0 0 1 0 0
a) 
A = 0 2 0 ; b) A = 2 3 0
0 0 3 4 5 6
1 2 3
c) A = 0 4 5
0 0 6
250
Unit 8 Eigenvalues and Eigenvectors
Solution: a) Using Eqns. (4), we obtain the characteristic equations as
1− λ 0 0
det(A − λI ) = 0 2−λ 0 =0
0 0 3−λ

which gives (1 − λ) (2 − λ) (3 − λ) = 0 .

And hence the eigenvalues of A are λ1 = 1, λ 2 = 2, λ 3 = 3 .

1− λ 0 0
b) det(A − λI ) = 2 3−λ 0 =0
4 5 6−λ

which gives (1 − λ) (3 − λ) (6 − λ) = 0

Eigenvalues of A are λ1 = 1, λ 2 = 3, λ 3 = 6 .

1− λ 2 3
c) det(A − λI ) = 0 4−λ 5 =0
0 0 6−λ

Therefore, (1 − λ) (4 − λ) (6 − λ) = 0 .

Eigenvalues of A are λ1 = 1, λ 2 = 4, λ 3 = 6 .
***

Remark: Observe that in Example 1 (a), the matrix A is diagonal and

in parts (b) and (c), it is lower and upper triangular, respectively. In
these cases the eigenvalues of A are the diagonal elements. This is
true for any diagonal, lower triangular or upper triangular matrix.
Formally, we give the result in the following theorem.

Theorem 1: The eigenvalues of a diagonal, lower triangular or an upper

triangular matrix are the diagonal elements themselves.

Let us consider another example, where we also find the eigenvectors

corresponding to the eigenvalues obtained.

Example 2: Find the eigenvalues and the corresponding eigenvectors

of the following matrices.
2 2 1 2 1 − 2
a) A=  ; b) A =   c) A =  
1 3 0 1  2 1 

Solution: a) Using Eqns. (4), we obtain the characteristic equation as

2−λ 2
A − λI = =0,
1 3− λ
which gives the polynomial 251
Block 2 Solutions of Linear Algebraic Equations
2
λ − 5λ + 4 = 0
i.e., (λ − 1) (λ − 4) = 0

The matrix A has two distinct real eigenvalue λ1 = 1, λ 2 = 4. To obtain

the corresponding eigenvectors we solve the system of Eqns. (5) for
each value of λ.

For λ = 1 , we obtain the system of equations

x 1 + 2x 2 = 0
x 1 + 2x 2 = 0
which reduces to a single equation
x 1 + 2x 2 = 0

Taking x 2 = k , we get x 1 = −2k , k being arbitrary non-zero constant.

Thus, the eigenvector is of the form
 x 1   − 2k  − 2
x  =  k  = k  1 
 2    

For λ = 4 , we obtain the system of equations

− 2x1 + 2x 2 = 0
x1 − x 2 = 0
which reduces to a single equation
x1 − x 2 = 0

Taking x 2 = k , we get x 1 = k and the corresponding eigenvector is

 x1  1
x  = k 1
 2 

Note: In practice we usually omit k and say that [−2 1]T and [1 1]T are
the eigenvectors of A corresponding to the eigenvalues λ = 1 and λ = 4
respectively. Moreover, the eigenvectors in this case are linearly
independent.

b) The characteristic equation in this case becomes

(λ − 1) 2 = 0 .
Therefore, the matrix A has a repeated real eigenvalue. The
eigenvector corresponding to λ = 1 is the solution of the system of
Eqn. (5), which reduces to a single equation
x2 = 0
Taking x 1 = k, we obtain the eigenvector as
x 1  1 
x  = k 0
 2  
252
Unit 8 Eigenvalues and Eigenvectors
Note that in this case of repeated eigenvalues, we got linearly
independent eigenvectors.

c) The characteristic equation in this case becomes

λ2 − 2λ + 5 = 0
which gives two complex eigenvalues λ = 1 ± 2i .

The eigenvector corresponding to λ = 1 + 2i is the solution of the

system of Eqn. (5). In this case we obtain the following equations
ix 1 + x 2 = 0
x 1 − ix 2 = 0
which reduces to the single equation
x 1 − ix 2 = 0

Taking x 2 = k , we get the eigenvector

 x1  i 
 x  = k 1
 2 

Similarly, for λ = 1 − 2i , we obtain the eigenvector

x1  − i
x  = k  1 
 2  

In the above problem you may note that corresponding to complex

eigenvalues, we get complex eigenvectors. Let us now consider
an example of 3 × 3 matrices.
***

Example 3: Determine the eigenvalues and the corresponding

eigenvectors for the matrices
 2 −1 0   6 −2 2 
a)  
A =  − 1 2 − 1 ; b) A =  − 2 3 − 1
 0 − 1 2   2 − 1 3 

Solution: a) The characteristic equation in this case becomes

2−λ −1 0
−1 2−λ −1 = 0
0 −1 2−λ

which gives the polynomial

(2 − λ) (λ2 − 4λ + 2) = 0

Therefore, the eigenvalues of A are = 2, 2 + 2 and 2 − 2 .

The eigenvector of A corresponding to λ = 2 is the solution of the

system of Eqn. (5), which reduces to 253
Block 2 Solutions of Linear Algebraic Equations
x2 = 0
x1 + x 3 = 0

Taking x 3 = k , we obtain the eigenvector

x 1  − 1
x  = k 0
 2  
 x 3   1 

The eigenvector of A corresponding to λ = 2 + 2 is the solution of the

system of equations
− 2 −1 0   x 1  0
   x  = 0
 −1 − 2 −1   2   (10)
 0 −1 − 2   x 3  0


To find the solution of system of Eqn. (10), we use Gauss elimination

1
method. Performing R 2 ← R 2 − R 1 , we get
2
− 2 − 1 0   x 1  0
 
− 1   x 2  = 0
−1
 0
 2 
 x  0
 0 − 1 − 2   3   

Again performing R 3 ← R 3 − 2 R 2 , we get

− 2 −1 0  x 1  0
 −1   x  = 0
 0 − 1  2  
 2 
 0 0 0   x 3  0

which give the equations

− 2 x1 − x 2 = 0

− x2 − 2 x3 = 0

Taking x 3 = k , we obtain the eigenvector

x1  1 
x  = k  
 2  2
 x 3  1 
 

Similarly, corresponding to the eigenvalue λ = 2 − 2 , the eigenvector

is the solution of system of equations
− 2 −1 0  x 1  0
   x  = 0
 −1 − 2 − 1  2  
 0 −1 2   x 3  0

254
Unit 8 Eigenvalues and Eigenvectors
Using the Gauss elimination method, the system reduces to the
equations
2 x1 − x 2 = 0

x 2 − 2 x3 = 0

Taking x 3 = k, we obtain the eigenvector

x1  1 
x  = k  
 2  2 .
 x 3  1 
 

b) The characteristic equation in this case becomes

(λ − 8) (λ − 2) 2 = 0

Therefore, the matrix A has the real eigenvalues 8, 2 and 2. The

eigenvalue 2 is repeated two times.
The eigenvector corresponding to λ = 8 is the solution of system
of Eqn. (5), which reduces to
x1 + x 2 − x 3 = 0
2x 1 + 5x 2 + x 3 = 0 (11)

2 x1 − x2 − 5 x3 = 0

Subtracting the last equation of system (11) from the second

equation we obtain the system of equations
x1 + x 2 − x 3 = 0
x2 + x3 = 0

Taking x 3 = k , the eigenvector is

x 1  2 
x  = k − 1
 2  
 x 3  1 

The eigenvector corresponding to λ = 2 is the solution of system

of Eqns. (5), which reduces to a single equation.
2x 1 − x 2 + x 3 = 0 (12)

we can take any values for x 1 and x 2 which need not be related
to each other. The two linearly independent solutions can be
written as
1   0
k 0  or k 1 
− 2 1 
255
Block 2 Solutions of Linear Algebraic Equations
Note that in Eqn. (12), it is not necessary that we always assign
values to x 1 and x 2 . We can assign values to any of the two
variables and obtain the corresponding value of the third variable.
***

On the basis of Examples 2 and 3, we can make in general, the

following observations:

For a given n × n matrix A, the characteristic Eqn. (4) is a polynomial of

degree n in λ . The n roots of this polynomial λ1 , K, λ n , called the
eigenvalues of A may be real or complex, distinct or repeated. Then,
i) For distinct, real eigenvalues, we obtain linearly independent
eigenvectors. (Examples 2(a), 3(a)).
ii) For a repeated eigenvalue, there may or may not be linearly
independent eigenvectors. (Example 2(b), 3(b)).
iii) For a complex eigenvalue, we obtain a complex eigenvector.
iv) An eigenvector is not unique. Any non-zero multiple of it is again
an eigenvector.

You may try some exercises now.

Determine the eigenvalues and the corresponding eigenvectors of the

following matrices.

1 2 2
 
E2) A =  2 3 2
2 2 1 


− 15 4 3
E3) A =  10 − 12 6

 20 − 4 2

− 2 2 − 3 
E4) A =  2 1 − 6
 1 − 2 0 

2 − 1 − 1
E5) A = 3 − 2 1 
0 0 1 

In the examples considered so far, it was possible for us to find all the
roots of the characteristic equation exactly. But this may not always be
possible. This is particularly true for n > 3. In such cases, some
iterative method like Newton-Raphson method may have to be used to
find a particular eigenvalue or all the eigenvalues from the characteristic
256
Unit 8 Eigenvalues and Eigenvectors
equation. However, in many practical problems, we do not require all
the eigenvalues but need only a selected eigenvalue. For example,
when we use iterative methods for solving a non-homogeneous system
of linear equation Ax = b, we need to know only the largest eigenvalue
in magnitude of the iteration matrix H, to find out whether the method
converges or not. One iterative method, which is frequently used to
determine the largest eigenvalue in magnitude (also called the
dominant eigenvalue) and the corresponding eigenvector for a given
square matrix A is the power method. In this method we do not find
the characteristic equation. This method is applicable only when all the
eigenvalues are real and distinct. If the magnitude of two or more
eigenvalues is the same then the method converges slowly.

8.3 THE POWER METHOD

The power method as we mentioned is used to find the largest
eigenvalue in magnitude and corresponding eigenvector of a square
matrix A.

Let us consider the eigenvalue problem

Ax = λx

Let λ 1 , λ 2 , K, λ n be the n real and distinct eigenvalues of A such

that
| λ1 | > | λ 2 | > K > | λ n |
Therefore, λ1 is the largest or the dominant eigenvalue of A .

In this method, we start with an arbitrary non-zero vector y ( 0) (not an

eigenvector), and form a sequence of vectors (y ( k ) ) such that

y ( k +1) = Ay ( k ) , k = 0, 1, K (13)

In the limit as k → ∞, y ( k ) converges to the eigenvector corresponding

to the dominant eigenvalue of the matrix A. We can stop the iteration
when the largest element in magnitude in y ( k +1) − y ( k ) is less than the
predefined error tolerance. For simplicity, we usually take the initial
vector y ( 0) with all its elements equal to one.

Note that in the process of multiplying the matrix A with the vector y ( k ) ,
the elements of the vector y ( k +1) may become very large. To avoid this,
we normalize (or scale) the vector y ( k ) at each step by dividing y ( k ) ,
by its largest element in magnitude. This will make the largest element
in magnitude in the vector y ( k +1) as one and the remaining elements
less than one. Vector for which scaling has been done is called a
scaled vector otherwise, it is unscaled.

If y ( k ) represents the unscaled vector and v ( k ) the scaled vector then,

we have the power method.
y ( k +1) = Av ( k ) (14) 257
Block 2 Solutions of Linear Algebraic Equations
1 ( k +1)
v ( k +1) = y , k = 0, 1, K (15)
m k +1

with v ( 0) = y (0) and m k +1 being the largest element in magnitude of

y ( k +1) . We then obtain the dominant eigenvalue by taking the limit

 y (k+1) 
λ1 = lim  r (k)  , r = 1, 2,...., n.
 (16)
k →∞
 vr 
where r represents the rth component of that vector. Obviously, there
are n ratios of numbers. As k → ∞ all these ratios tend to the same
value, which is the largest eigenvalue in magnitude i.e., λ1 . The
iteration is stopped when the magnitude of the difference of any two
ratios is less than the prescribed tolerance.

The corresponding eigenvector is then v ( k +1) obtained at the end of the

last iteration performed.

We can summarise the steps to be performed in each iteration as

follows:

Step 1: Compute y ( k +1) = Av ( k )

Step 2: Compute m k +1 = max y ( k+1) { }

1 (k +1)
Step 3: Compute v ( k +1) = y
m k +1

Step 4: Repeat Steps 1, 2 and 3 in successive iterations with

k = 0, 1, 2, K to achieve the desired accuracy.

We now illustrate the method through the following examples.

Example 4: Find the dominant eigenvalue and the corresponding
eigenvector correct to two decimal places of the matrix
 2 −1 0 
A =  − 1 2 − 1
 0 − 1 2 

using the power method with initial vector [1 1 1]T .

Solution: We perform the following iterations:

Iteration 1: y (0 ) = v (0) = [1 1 1]
T

Using Eqn. (14), we obtain

 2 −1 0  1
  
1

y (1) = Av ( 0) =  − 1 2 − 1 1 = 0
  
 
 0 − 1 2  1 1 
258
Unit 8 Eigenvalues and Eigenvectors
Now m1 = max y , y , y
i
{ (1)
1
(1)
2
(1)
3 }= max{1,0,1} = 1
1
1 (1) 1  
y = 0 = [1 0 1] .
T
and v (1) =
m1 1
1

 2 − 1 0  1   2 
Iteration 2: y (2)
= Av (1)
= − 1 2 − 1 0 = − 2
 0 − 1 2  1   2 
m 2 = max { 2 , − 2 , 2 } = 2
2
1 ( 2) 1 ( 2 ) 1 
=  − 2 = [1 − 1 1]
T
v ( 2) = y = y
m2 2 2 
 2 

 2 −1 0   1 
  
Iteration 3: y (3) = Av ( 2) = − 1 2 − 1 − 1 = [3 − 4 3]T
  
 0 − 1 2   1 
m3 = 4
3
1 1 
v ( 3) = y ( 3) = − 4 = [0.75 − 1 0.75]T
4 4 
 3 

 2 − 1 0  0.75
   
Iteration 4: y ( 4) = Av (3) = − 1 2 − 1  − 1  = [2.5 − 3.5 2.5]T
   
 0 − 1 2  0.75
m 4 = 3.5
 2.5 
1 ( 4) 1  
v ( 4) = y =  − 3.5 = [0.7143 − 1 0.7143]T
3. 5 3.5  
 2.5 

Iteration 5:
 2 − 1 0  0.7143
  
y (5) = Av ( 4) = − 1 2 − 1  − 1  = [2.4286 − 3.4286 2.4286]T
  
 0 − 1 2  0.7143
m5 = 3.4286
 2.4286 
1 1  
v (5) = y ( 5) =  − 3.4286 = [0.7083 − 1 0.7083]T
3.4286 3.4286  
 2.4286  259
Block 2 Solutions of Linear Algebraic Equations
Iteration 6:
 2 − 1 0  0.7083
  
y ( 6) = Av (5) =  − 1 2 − 1  − 1  = [2.4166 − 3.4166 2.4166]T
  
 0 − 1 2  0.7083
m 6 = 3.4166
 2.4166 
1 1  
v ( 6) = y ( 6) = − 3.4166 = [0.7073 − 1 0.7073]T
3.4166 3.4166  
 2.4166 

Iteration 7:
 2 − 1 0  0.7073
  
y ( 7) = Av (6) = − 1 2 − 1  − 1  = [2.4146 − 3.4146 2.4146]T
  
 0 − 1 2  0.7073
m 7 = 3.4146
 2.4146 
1 1  
v ( 7) = y (7 ) = − 3.4146 = [0.7071 − 1 0.7071]T
3.4146 3.4146  
 2.4146 
(7)
(y r )
After seven iterations, the ratios (6)
for r = 1, 2, 3 are
(v r )
2.4146 − 3.4146 2.4146
, , = 3.4138, 3.4146, 3.4138 , respectively.
0.7071 −1 0.7071

The maximum difference or the error in these ratios is 0.0008 . Hence,

the dominant eigenvalue can be taken as 3.414 and the corresponding
eigenvector is [0.7071 − 1 0.7071]T .

Note that the exact dominant eigenvalue of A as obtained in

Example 3 a) was 2 + 2 = 3.4142 and the corresponding eigenvector
was [1 − 2 1]T which can also be written as
T
 1 1  T
 2 − 1 2  = [0.7071 − 1 0.7071] .
 
***

Example 5: Perform six iterations of the power method to find the

dominant eigenvalue for the following matrix:
− 15 4 3
 
A =  10 − 12 6
 
 20 − 4 2

Use initial vector [1 1 1]T . Also, compare the result with the eigenvalue
260 obtained in E3.
Unit 8 Eigenvalues and Eigenvectors
( 0) (0) T
Solution: Starting with y =v = [1 1 1] we perform the following
iterations:

Iteration 1: y ( 0) = v ( 0) = [1 1 1]T .

Using Eqn. (14), we obtain

 − 15 4 3 1 − 8
   
y (1) = Av ( 0) =  10 − 12 6  1 =  4 
   
 20 − 4 2 1  18 

Now m1 = max{| −8 |, | 4 |, | 18 |} = 18

 − 8 − 0.4444
1 (1) 1    
and v (1) = y =  4  =  0.2222 
m1 18    
 18   1 

− 15 4 3 − 0.4444  10.5556 
    
Iteration 2: y ( 2) = Av (1) =  10 − 12 6   0.2222  =  − 1.1111
    
 20 − 4 2  1   − 7.7778

Now, m 2 = 10.5556

 10.5556   1 
1 ( 2) 1    
and v ( 2) = y =  − 1.1111 =  − 0.1053
m2 10.5556    
− 7.7778 − 0.7368

 − 15 4 3  1   − 17.6316
    
Iteration 3: y (3) = Av ( 2) =  10 − 12 6  − 0.1053 =  6.8421 
    
 20 − 4 2  − 0.7368  18.9474 

Now m 3 = 18.9474

− 17.616 − 0.9306
1 ( 3) 1    
and v (3) = y =  6.8421  =  0.3611 
m3 18.9474    
18.9474   1 

 − 15 4 3  − 0.9306  18.4028 
    
Iteration 4: y ( 4) = Av ( 3) =  10 − 12 6  0.3611  =  − 7.6389 
    
 20 − 4 2  1  − 18.0556

Now m 4 = 18.4028
261
Block 2 Solutions of Linear Algebraic Equations
 18.4028   1 
1 ( 4) 1    
and v ( 4) = y =  − 7.6389  = − 0.4151
m4 18.4028    
 − 18.0556 − 0.9811

− 15 4 3  1  − 19.6038
    
Iteration 5: y (5) = Av ( 4) =  10 − 12 6 − 0.4151 =  9.0943 
    
 20 − 4 2 − 0.9811  19.6981 
Now m 5 = 19.6981
− 19.6038 − 0.9958
1 (5) 1    
and v (5) = y =  9.0943  =  0.4617 
m5 19.6981    
 19.6981   1 

 − 15 4 3  − 0.9958  19.7749 
    
Iteration 6: y ( 6) = Av (5) =  10 − 12 6  0.4617  =  − 9.4923
    
 20 − 4 2  1   − 19.751
Now m 6 = 19.7749
 19.7749   1 
1    
and v (6) = − 9.4923 =  − 0.48 
19.7749    
 − 19.751  − 0.9988

After six iteration, the ratios are

y (r6) 19.7749 − 9.4923 − 19.751
( 6)
for r = 1, 2, 3 = , ,
vr 1 − 0.48 − 0.9988
= 19.7749, 19.7756, 19.7747

Therefore, the maximum difference or the error in these ratios is 0.0009.

Hence, dominant eigenvalue can be taken as 19.7749 and the
corresponding eigenvector is [1 − 0.48 − 0.9988]T .

You may note that the exact dominant eigenvalue of A as obtained is

E3 was − 20 ( 20 in magnitude) and the corresponding eigenvector is
T T
 1   1 
 − 1 2 1 , which can also be written as 1 − 2 − 1 and is reasonably
   
close to [1 − 0.48 − 0.9988]T .
***

You may now try the following exercises.

E6) Using four iterations of the power method and taking the initial
vector y (0 ) with all its elements equal to one, find the dominant
262
Unit 8 Eigenvalues and Eigenvectors
eigenvalue and the corresponding eigenvector of the following
matrices.
 2 −1 0 0 
1 2 2  
  − 1 2 − 1 0 
i) A= 2 3 2 ii) A= .
   0 − 1 2 − 1
2 2 1   
  0 0 − 1 2 

E7) Perform four iterations to find the dominant eigenvalue in

magnitude for the matrix given in E2 using power method starting
with initial vector [1 1 1]T . Also, compare the results with the
eigenvalues obtained there.

You must have realised that an advantage of the power method is that
the eigenvector corresponding to the dominant eigenvalue is also
generated at the same time. Usually, for most of the methods of
determining eigevalues, we need to do separate computations to obtain
the eigenvector.

In some problems, the most important eigenvalue is the eigenvalue of

least magnitude. We shall discuss now the inverse power method
which gives the least eigenvalue in magnitude.

8.4 THE INVERSE POWER METHOD

We first note that if λ is the smallest eigenvalue in magnitude of A,
1
then is the largest eigenvalue in magnitude of A −1 . The
λ
corresponding eigenvectors are the same. If we apply the power
method to A −1 , we obtain the dominant eigenvalues in magnitude of
A −1 and the corresponding eigenvector. This eigenvalue is then the
smallest eigenvalue in magnitude of A and the eigenvector is same.
Since power method is applied to A −1 , it is called the inverse power
method.

Consider the method

y ( k +1) = A −1v ( k ) , k = 0, 1, 2, K (17)
1 ( k +1)
v ( k +1) = y with v ( 0) = y ( 0)
m k +1

where y (0 ) is an arbitrary nonzero vector different from the eigenvector

of A. However, in algorithm (17) we need not find A −1 to find the
smallst eigenvalue (in magnitude) of A. Alternately, we write Eqn. (17)
as
Ay ( k +1) = v ( k )
1 ( k +1)
v ( k +1) = y , k = 0, 1, 2, K (18)
m k +1 263
Block 2 Solutions of Linear Algebraic Equations
We now need to solve a system of equations for y ( k+1) , which can be
obtained using any of the method discussed in the previous units. The
largest eigenvalue of A −1 is again given by
( k +1)
(y r )
µ = lim (k)
; r = 1, 2, ...n.
k →∞ (vr )

The corresponding eigenvector is v ( k +1) .

We can summarise the steps to be performed in each iteration as

follows:

Step 1: Use A y ( k +1) = v ( k +1) , k = 0, 1, 2, K to find y ( k +1) . Any method can

be used to find y (k +1) .

Step 2: Compute m k +1 = max{y (i k+1) }.

i

1
Step 3: Compute v ( k +1) = k +1
y ( k +1) .
m

Step 4: Repeat Step 1, 2 and 3 in successive iterations with

k = 0, 1, 2, K to achieve the desired accuracy.

We now illustrate the method through the following examples.

Example 6: Find the smallest eigenvalue in magnitude and the

corresponding eigenvector of the matrix
 2 −1 0 
 
A = − 1 2 − 1
 
 0 − 1 2 

using four iterations of the inverse power method using initial vector
[1 1 1]T . .

Solution: Taking v ( 0) = [1 1 1]T , we get the following iterations:

Iteration 1: Ay (1) = v ( 0)
or
 2 − 1 0   y1  1
 − 1 2 − 1  y  = 1 (19)
   2  
 0 − 1 2   y3  1

For solving the system of Eqn. (19), we use the LU

decomposition method. We write
 2 −1 0   l 11 0 0 1 u 12 u 13 
A = − 1 2 − 1 = LU = l 21
  l 22 0  0 1
 u 23  (20)
 0 − 1 2  l 31 l 32 l 33  0 0 1 
264
Unit 8 Eigenvalues and Eigenvectors
comparing the coefficients on both the sides of Eqn. (20),
we obtain
  1 − 1 0 
 2 0 0  2 
 3   2
A = LU = − 1 0  0 1 − 
 2  3
 0 −1 4  0 0 1 
 3   

Solving Lz = v ( 0)
And then Uy (1) = z , we obtain
T
3 3
y =  2  = [1.5 2.0 1.5]T
(1)

2 2
m1 = 2.0
1 (1)
∴ v (1) = y = [0.75 1.0 0.75]T
m1

Iteration 2: Ay ( 2) = v (1)
Solving Lz = v (1)
and Uy ( 2) = z, we obtain
y ( 2 ) = [1.25 1.75 1.25]T
m 2 = 1.75
1 (2)
v ( 2) = y = [0.7143 1 0.7143]T
m2

Iteration 3: Ay (3) = v ( 2)
y (3) = [1.2143 1.7143 1.2143]T
m3 = 1.7143
1 ( 3)
v ( 3) = y = [0.7083 1 0.7083]T
m3

Iteration 4: Ay ( 4) = v (3)
y ( 4) = [1.2083 1.7083 1.2083]T
m 4 = 1.7083
1 (4)
v ( 4) = y = [0.7073 1 0.7073]T
m4

( 4)
(y r )
After four iterations, the ratios (3)
for r = 1, 2, 3, K are
(vr )
1.2083 1.7083 1.2083
, , = 1.7059, 1.7083, 1.7059 , respectively. The
0.7073 1 0.7073
maximum error in these ratios is 0.0024. Hence, the dominant
1
eigenvalue of A −1 can be taken as 1.7083. Therefore, = 0.5882
1.7083

265
Block 2 Solutions of Linear Algebraic Equations
is the smallest eigenvalue of A in magnitude and the corresponding
eigenvector is given by [0.7073 1 0.7073]T .

Note that the smallest eigenvalue in magnitude of A as calculated in

Example 3 a) was 2 − 2 = 0.5858 and the corresponding eigenvector
T
T  1 1 
was [1 2 1] which can be written as  1  or
 2 2
[0.7071 1 0.7071]T .
***

Example 7: Find the smallest eigenvalue in magnitude and the

corresponding eigenvector of the matrix
− 15 4 3
 
A =  10 − 12 6 
 
 20 − 4 2

Use initial vector [1 1 1]T and perform four iterations of the inverse
power method. Also, compare the result with the smallest eigenvalue
obtained in E3.

Solution: We start with v ( 0) = [1 1 1]T , we get the following iterations:

Iteration 1: A y (1) = v ( 0)
 − 15 4 3  y1  1
   
 10 − 12 6  y 2  = 1 (21)
   
 20 − 4 2  y 3  1
We solve the system of Eqn. (21), and obtain
y (1) = [0.0400 0.1299 0.3599]T .
m1 = 0.3599
v (1) = [0.1111 0.3609 1]T .

Iteration 2: On solving A y ( 2) = v (1) , we obtain

y ( 2) = [0.0528 0.0986 0.1694]T
Now m 2 = 0.1694
and v ( 2) = [0.3116 0.5820 1]T .

Iteration 3: On solving A y (3) = v ( 2) , we obtain

y (3) = [0.0484 0.0988 0.214]T
Now m 3 = 0.214
1 ( 3)
and v (3) = y = [0.2261 0.4616 1]T .
m3

Iteration 4: On solving A y ( 4) = v (3) , we obtain

y ( 4) = [0.0508 0.1011 0.1945]T
266
Unit 8 Eigenvalues and Eigenvectors
Now m 3 = 0.1945
1 ( 4)
and v ( 4) = y = [0.2611 0.5197 1]T .
m4

After four iterations the ratios are

( y (r4 ) ) 0.0508 0.1011 0.1945
( 4)
for r = 1, 2, 3 = , ,
(vr ) 0.2611 0.5197 1
= 0.1937, 0.1945, 0.1945

The maximum error in these ratio is 0.008. Hence, the dominant

1
eigenvalue of A −1 can be taken as 1.1945. Therefore, = 5.1413 is
0.1945
the smallest eigenvalue (in magnitude) of A and the corresponding
eigenvector is given by [0.2611 0.5197 1]T .

Note that the smallest eigenvalue (in magnitude) of A as calculated in

E3 is 5 and the corresponding eigenvector is [0.25 0.5 1]T . Thus, we
see that the calculated eigenvalue and the corresponding eigenvector
are reasonably close to the values as obtained in E3.
***

You may now try the following exercise.

E8) Find the smallest eigenvalue in magnitude and the corresponding

eigenvector of the matrix
2 2
A= 
1 3
with v ( 0) = [−1 1]T , using four iterations of the inverse power
method.

E9) Perform four iterations to find the smallest eigenvalue in

magnitude for the matrix given in E2 using inverse power method
starting with initial vector [1 1 1]T . Also, compare the results with
eigenvalues obtained there.

The inverse power method can be further generalized to find some

other selected eigenvalues of A. For instance, one may be interested to
find the eigenvalue of A which is nearest to some chosen number q.
You know from P6 of Sec. 8.2 that the matrices A and A − qI have the
same set of eigenvectors. Further, for each eigenvalue λ i of A, λ i − q is
the eigenvalue of A − qI .

We can therefore use the iteration

y ( k +1) = ( A − qI ) −1 v ( k ) (23) 267
Block 2 Solutions of Linear Algebraic Equations
with scaling as described in Eqns. (14) – (16). We determine the
dominant eigenvalue µ of ( A − qI ) −1 using the procedure given in Eqn.
(18) i.e.,
( A − qI ) y ( k+1) = v ( k )
1
v ( k +1) = y ( k +1) (24)
m k +1
Using P6, we have the relation
1
µ= , where λ is an eigenvalue of A .
λ−q
1
i.e., λ = +q (25)
µ
1
Now since µ is the largest eigenvalue in magnitude of ( A − qI) −1 ,
µ
must be the smallest eigenvalue in magnitude of A − qI . Hence, the
1
eigenvalue + q of A is closest to q .
µ

Example 8: Find the eigenvalue of the matrix A , nearest to 3 and also

the corresponding eigenvector using four iterations of the inverse power
method where
 2 −1 0 
A = − 1 2 − 1 .
 0 − 1 2 

Take the initial approximation vector as v ( 0) − [1 1 1]T .

Solution: In this case the nearest value of the eigenvalue q = 3 . Thus

we have
− 1 − 1 0 
A − 3I =  − 1 − 1 − 1
 0 − 1 − 1

To find y ( k+1) , we need to solve the system

− 1 − 1 0 
 − 1 − 1 − 1 y ( k+1) = v ( k ) (26)
 
 0 − 1 − 1

and normalise y ( k +1) as given in Eqn. (24).

Iteration 1: Starting with v ( 0) = [1 1 1]T and using the Gauss elimination

method to solve the system (26), we obtain
y (1) = [0 − 1 0]T
m1 = 1
268
Unit 8 Eigenvalues and Eigenvectors
1 (1)
v (1) = y = [0 − 1 0]T
m1

Iteration 2: Ay ( 2) = v (1)

y ( 2 ) = [1 − 1 1]T
m2 = 1
1 (2)
v ( 2) = y = [1 − 1 1]T
m2

Iteration 3: Ay (3) = v ( 2)

y ( 3) = [2 − 3 2]T
m3 = 3
1 ( 3)
y = [0.6667 − 1 0.6667]
T
v ( 3) =
m3

Iteration 4: Ay ( 4) = v (3)

y ( 4) = [1.6667 − 2.3333 1.6667 ]

T

m 4 = 2.333
1 ( 4)
y = [0.7143 − 1 0.7143]
T
v ( 4) =
m4

After four iterations, the ratios are

( 4)
(y r ) 1.6667 − 2.3333 1.6667
( 3)
for r = 1, 2, 3 , = , , .
(vr ) 0.7143 −1 0.7143
= 2.5, 2.3333, 2.5

The maximum error in these ratios is 0.1667. Hence the dominant

eigenvalue µ of ( A − 3I) −1 can be taken as 2 ⋅ 5. Thus the eigenvalue λ
of A closest to 3 as given by Eqn. (25) is
1
λ= +3
µ
1
= + 3 = 3.4
2⋅5
and the corresponding eigenvector is v ( 4) = [0.7143 − 1 0.7143]T .

Note that the eigenvalue of A closest to 3 as obtained Example 3 a)

was 2 + 2 = 3.4142 . The eigenvector corresponding to this eigenvalue
was [0.7071 − 1 0.7071]T .
***

And now a few exercises for you.

E10) Find the eigenvalue which is nearest to − 1 and the corresponding

eigenvector for the matrix 269
Block 2 Solutions of Linear Algebraic Equations
2 2
A= 
1 3
with v ( 0) = [−1 1]T , using four iterations of the inverse power
method.

E11) Using four iterations of the inverse power method, find the
eigenvalue which is nearest to 5 and the corresponding
eigenvector for the matrix
3 2
A=  (exact eigenvalues are = 1 and 6 )
3 4
with v ( 0) = [1 1]T .

So far, we have discussed the methods of finding eigenvalues and the

corresponding eigenvectors of a square matrix. We also obtained the
smallest and greatest eigenvalue, in magnitude of the matrix and the
corresponding eigenvectors. In the following section, we will estimate
the eigenvalues of a given matrix and obtain their bounds called the
Gershgorin Bounds.

8.5 GERSHGORIN BOUNDS

The eigenvalues of a given matrix can also be estimated. That is, for a
given matrix A, we can find the region in which all its eigenvalues lie.
This can be done as follows:

Let λ i be an eigenvalue of A and x i be the corresponding eigenvector,

i.e.,

Ax i = λi xi (27)
or
a11 x i ,1 + a 12 x i , 2 + L + a 1n x i , n = λ i x i , 1
a 21x i , 1 + a 22 x i , 2 + L + a 2 n x i, n = λi x i , 2
M M M M
a k1x i , 1 + a k 2 x i , 2 + L + a kn x i , n = λ i x i, k
(28)
M M M M
a n1x i , 1 + a n 2 x i , 2 + L + a nn x i , n = λ i x i , n

Let | x i , k | be the largest element in magnitude of the vector

[x i , 1 , x i , 2 , K , x i , n ]T . Consider the kth equation of the system (28) and
divide it by x i , k . We then have

x  x  x 
a k1  i , 1  + a k 2  i , 2  + L + a kk + L + a kn  i, n  = λi (29)
 x i, k   x i, k   x i, k 
     

Taking the magnitude on both sides of Eqn. (29), we get

270
Unit 8 Eigenvalues and Eigenvectors
x i, 1 x i, 2 x i, n
| λ i | ≤ | a k1 | + | a k2 | + L + | a kk | + L + | a kn |
x i, l x i, k x i, k

≤ | a k1 | + | a k 2 | + L + | a kk | + L + | a kn | (30)

x i, j
Since ≤ 1 for j = 1, 2, K , n .
x i, k

Since eigenvalues of A and A T are same (Ref. P2), Eqn. (30) can also
be written as
| λ i | ≤ | a1k | + | a 2 k | + L + | a kk | + L + | a nk | (31)

since | x i , k | , the largest element in magnitude, is unknown, we

approximate Eqns. (30) and (31) by
n 
| λ | ≤ max  | a ij | (maximum absolute row sum) (32)
 j=1 
i

and
n 
| λ | ≤ max  | a ij | (maximum absolute column sum) (33)
j
 i=1 

We can also rewrite Eqn. (29) in the form

 x i ,1  x  x 
λ i − a kk = a k1   + a k 2  i, 2  + L + a kn  i , n 
 x i, k   x i, k   x i, k 
     
and taking magnitude on both sides, we get
n
| λ i − a kk | ≤  | a ij | (34)
j=1
i≠ j

Again, since A and A T have the same eigenvalues Eqn. (34) can be
written as
n
| λ i − a kk | ≤  | a ij | (35)
i =1
i≠ j

A square matrix A is
Note that since the eigenvalues can be complex, the bounds (32), (33), symmetric if A = A .
T

(34) and (35) represent circles in the complex plane. If the eigenvalaues
are real, then they represent intervals. For example, when A is
symmetric then the eigenvalus of A are real.

Again in Eqn. (34), since k is not known, we replace the circle by the
union of the n circles.
n
| λ − a ii | ≤  | a ij |, i = 1, 2, K , n . (36)
j=1
i≠ j

Similarly from Eqn. (35), we have that eigenvalues of A lie in the union
of circles
271
Block 2 Solutions of Linear Algebraic Equations
n
| λ − a ii | ≤  | a ij | , j = 1, 2, K, n . (37)
i =1
i≠ j

The bounds derived in Eqns. (32), (33), (36) and (37) for eigenvalues
are all independent bounds. Hence the eigenvalues must lie in the
intersection of these bounds. The circles derived above are called the
Gershgorin circles and the bounds are called the Gershgorin
bounds.
Let us now consider the following examples:

Example 9: Estimate the eigenvalues of the matrix

 1 − 1 2
A =  2 1 3
 − 1 3 2

using the Gershgorin bounds. Also, draw the rough sketch of the region.

Solution: The eigenvalues of A lie in the following regions:

i) Maximum absolute row sums:

First row absolute sum = 1 + 1 + 2 = 4

Second row absolute sum = 2 + 1 + 3 = 6
Thirst row absolute sum = 1 + 3 + 2 = 6
Hence, the maximum row sum = | λ | ≤ max{4, 6, 6} = 6 (38)

ii) Maximum absolute column sums:

First column absolute sum = 1 + 2 + 1 = 4

Second column absolute sum = 1 + 1 + 3 = 5
Third column absolute sum = 2 + 3 + 2 = 7
Hence, the maximum column sum = | λ | ≤ max{4, 5, 7} = 7 (39)

iii) Union of the circles [using (36)]:

λ − a 11 ≤ ( a12 + a13 ) | λ − 1 | ≤ 3

λ − a 22 ≤ ( a 21 + a 23 ) | λ − 1 | ≤ 5

λ − a 33 ≤ ( a 31 + a 32 ) | λ − 2 | ≤ 4

Since all the eigenvalues lie in the union of circles, therefore,

union of circles is | λ − 1 | ≤ 5 (40)

iv) Union of the circles [using (35)]

λ − a 11 ≤ ( a 21 + a 31 ) | λ − 1 | ≤ 3

λ − a 22 ≤ ( a 12 + a 32 ) | λ − 1 | ≤ 4

λ − a 33 ≤ ( a 13 + a 23 ) | λ − 2 | ≤ 5
272
Unit 8 Eigenvalues and Eigenvectors
Since all the eigenvalues lie in the union of circles, therefore,
union of circles is | λ − 2 | ≤ 5 (41)

The eigenvalaues lie in all the circles (38), (39), (40) and (41) i.e., in the
intersection of these circles as shown by shaded region in Fig. 1.

Fig. 1
***

Let us find the Gershgorin bounds for a symmetric matrix in the

following example.

Example 10: Estimate the eigenvalues of the symmetric matrix

 1 −1 2 
 
A = − 1 1 2
 
 2 2 − 2

by the Gershgorin bounds.

Solution: Since the given matrix is symmetric matrix, therefore, rows

and column are identical. Thus we find the maximum absolute row
sums and union of the circles row-wise. Thus, the eigenvalues lie in the
following regions:

i) Maximum absolute row sums: | λ | ≤ max{4, 4, 6} = 6

ii) Union of the circles:

a) | λ −1 | ≤ 3
b) | λ −1 | ≤ 3
c) |λ+2| ≤ 4 273
Block 2 Solutions of Linear Algebraic Equations
Since A is symmetric, it has real eigenvalues. Therefore, the
eigenvalues lie in the intervals

i) | λ | = 6  −6 ≤ λ ≤ 6

ii) Union of the circles:

a) − 3 ≤ λ − 1 ≤ 3 i.e., − 2 ≤ λ ≤ 4
b) − 4 ≤ λ + 2 ≤ 4 , i.e., − 6 ≤ λ ≤ 2
union of (a) and (b) is − 6 ≤ λ ≤ 4 .

Intersection of (i) and (ii) is − 6 ≤ λ ≤ 4 . Hence, the eigenvalues of A lie

in the interval − 6 ≤ λ ≤ 4 .

Note that in Example 10, since the matrix A is symmetric, the bounds
(32) and (33) are same and also the bounds (36) and (37) are same.
***
You may now try the following exercise.

E12) Estimate the eigenvalues of the matrix A given in Example 3(a)

and 3(b), using the Gershgorin bounds.

E13) Draw the rough sketches of the estimation of eigenvalues of the

following matrices:
3 − 2 1  4 3 6
   
a) 5 6 − 4 b) 2 − 2 1
   
 2 − 2 3   3 1 2

using Gershogrin bounds.

We now end this unit by giving a summary of what we have covered in

it.

8.6 SUMMARY
In this unit we have covered the following:

1. For a given system of equations of the form

Ax = λx (see Eqn. (1))
the values of λ for which Eqn. (1) has a nonzero solution are
called the eigenvalues and the corresponding nonzero solutions
(which are not unique) are called the eigenvectors of the matrix A .

2. The following are the steps involved in solving an eigenvalue

problem
i) Find the nth degree polynomial (called the characteristic
equation) in λ from det(A − λI ) = 0 .
274
Unit 8 Eigenvalues and Eigenvectors
ii) Find the n roots λ i , i = 1, 2, K , n of the characteristic
equation.
iii) Find the eigenvectors corresponding to each λ i .

3. For n ≥ 3, it may not be possible to find the roots of the

characteristic equation exactly. In such cases, we use some
iterative method like Newton-Raphson method to find these roots.
However,

i) when only the largest eigenvalue in magnitude is to be

obtained, we use the power method. In this method we
obtain a sequence of vectors {y ( k ) } , using the iterative
scheme
y ( k+1) = A y ( k ) , k = 0, 1, K
which in the limit as k → ∞ , converges to the eigenvector
corresponding to the dominant eigenvalue of the matrix A .
The vector y ( 0) is an arbitrary non-zero vector (different from
the eigenvectors of A ).

ii) we use the inverse power method with the iteration scheme
y ( k+1) = ( A − qI) −1 v ( k )
i.e., ( A − qI) y ( k +1) = v ( k ) , k = 0, 1, 2, K
where y ( 0) = v ( 0) is an arbitrary non-zero vector (not an
eigenvector)

a) with q = 0 , if only the least eigenvalue of A in

magnitude and the corresponding eigenvector are to be
obtained and

b) with any q, if the eigenvalue of A , nearest to some

chosen number q and the corresponding eigenvector
are to be obtained.

4. The estimation of the eigenvalues of a given matrix using

Gershgorin bounds is obtained using following bounds:
n 
i) The maximum absolute row sum is | λ | ≤ max  | a ij |
 j=1 
i

n 
ii) The maximum absolute column sum is | λ | ≤ max  | a ij |
i
 i=1 
n
iii) Union of circles | λ − a ii | ≤  | a ij |, i = 1, 2, K , n
j=1
i≠ j

n
iv) Union of circles | λ − a ii | ≤  | a ij | , j = 1, 2, K, n
i =1
i≠ j
275
Block 2 Solutions of Linear Algebraic Equations
The intersection of bounds obtained in (i), (ii), (iii) and (iv) is the
region in which eigenvalues lie.

5. The eigenvalues of a symmetric matrix are real and represent

intervals.

8.7 SOLUTIONS/ANSWERS
E1) Since λ i (i = 1, 2, K, n ) are the eigenvalues of A, we have
Ax = λ i x, i = 1, 2, K, n.
On subtracting q x both the sides, we get
Ax − q x = λx − q x
 ( A − qI )x = (λ − q )x

Multiplying ( A − q I) −1 both the sides, we get

( A − qI ) −1 ( A − qI )x = ( A − qI ) −1 (λ i − q )x
1
 x = (A − qI) −1 x.
( λ i − q)

E2) Characteristic equation: | A − λI | = 0  −λ3 + 5λ2 − λ − 5 = 0

eigenvalues: − 1, 1, 5

eigenvectors: [−1, 0, 1]T for λ = −1

[1 − 2 1]T for λ = 1

[1 2 1]T for λ = 5

E3) Characteristic equation: | A − λI | = 0  λ3 + 25λ2 + 50λ − 1000 = 0

eigenvalues: − 20, − 10, 5
T
 1 
eigenvectors:  − 1 1 for λ = −20
 2 
[−1 − 2 1]T for λ = −10
T
1 1 
 4 2 1 for λ = 5
 

E4) Characteristic equation: | A − λI | = 0  λ3 + λ2 − 15λ − 27 = 0

eigenvalues: − 3, 1 − 10 , 1 + 10
T
 3 
eigenvectors: 0 1 for λ = −3
 2 
T
 − 1 + 10 2(−1 + 10 ) 
 1 for λ = 1− 10 .
 3 3 
276
Unit 8 Eigenvalues and Eigenvectors
T
 1 + 10 2(1 + 10 ) 
− − 1 for λ = 1+ 10
 3 3 

E5) Characteristic equation: | A − λI | = 0  λ3 − λ2 − λ + 1 = 0

eigenvalues: − 1, 1, 1

eigenvectors: [1 / 3 1 0]T for λ = −1

[1 1 0]T for λ = 1

[1 1 0]T for λ = 1.

E6) i) Iteration 1: y (1) = [4.4142 5.8284 4.4142]T ; m1 = 5.8284

v (1) = [0.7574 1 0.7574]T

Iteration 2: y ( 2) = [3.6864 5.1422 3.6864]T ; m 2 = 5.1422

v ( 2) = [0.7169 1 0.7169]T

Iteration 3: y (3) = [3.5649 5.0276 3.5649]T ; m 3 = 5.0276

v (3) = [0.7090 1 0.7090]T

Iteration 4: y ( 4) = [3.5412 5.0054 3.5412]T ; m 4 = 5.0053

v ( 4) = [0.7075 1 0.7075]T

(y (4) ) r
After 4 iterations the ratios for r = 1, 2 and 3 are given
( v ( 3) ) r
by 4.9946, 5.0054, 4.9946 respectively. The maximum error in
these ratios is 0.0108. Thus the dominant eigenvalue of A
can be taken as 5.00 and the corresponding eigenvector is
[0.7075 1 0.7075]T

ii) Iteration 1: y (1) = [1 0 0 1]T ; m1 = 1

v (1) = [1 0 0 1]T

Iteration 2: y ( 2) = [2 − 1 − 1 2]T ; m 2 = 2

v ( 2) = [1 − 0.5 − 0.5 1]T

Iteration 3: y (3) = [2.5 − 1.5 − 1.5 2.5]T ; m 3 = 2.5

v (3) = [1 − 0.6 − 0.6 1]T

Iteration 4: y ( 4) = [2.6 − 1.6 − 1.6 2.6]T ; m 4 = 2.6

v ( 4) = [1 − 0.6154 0.6154 1]T

277
Block 2 Solutions of Linear Algebraic Equations
( y (r4) ) r
After 4 iterations the ratios for r = 1, 2, 3, 4 are given
( v (r3) ) r
by 2.6, 2.6667, 2.6667, 2.6, respectively. The maximum error
in these ratios is 0.0667. Thus the dominant eigenvalue of A
can be taken as 2 and the corresponding eigenvector is
[1 − 0.6154 − 0.6154 1]T .

E7) The coefficient matrix given in E2 is

1 2 2
 
A= 2 3 2
 
2 2 1 
 
We use the power method with y ( 0) = v (0) = [1 1 1]T and get the
following iterations:

Iteration 1: y (1) = A v ( 0) = [4.4142 5.8284 4.4142]T

m1 = 5.8284
1 (1)
v (1) = y = [0.7574 1 0.7574]T
m1

Iteration 2: y ( 2) = A v (1) = [3.6863 5.1421 3.6863]T

m 2 = 5.1421
1 (2)
v ( 2) = y = [0.7169 1 0.7169]T
m2

Iteration 3: y (3) = A v ( 2) = [3.5649 5.0276 3.5649]T

m 3 = 5.0276
1 ( 3)
v ( 3) = y = [0.7091 1 0.7091]T
m3

Iteration 4: y ( 4) = A v (3) = [3.5414 5.0055 3.5414]T

m 4 = 5.0055
1 ( 4)
v ( 4) = y = [0.7075 1 0.7075]T .
m4

After four iterations the ratios are

y (r4 ) 3.5414 5.0055 3.5414
( 4)
for r = 1, 2, 3 = , ,
νr 0.7075 1 0.7075
= 5.0055, 5.0055, 5.0055.

Therefore, the maximum error in these ratios is 0. Hence, the

dominant eigenvalue can be taken as 5.0055 and the
278
Unit 8 Eigenvalues and Eigenvectors
T
corresponding eigenvector is [0.7075 1 0.7075] .

You may note that the exact eigenvalue of A as obtained in E2 is

5 and the corresponding eigenvector is [1 2 1]T which can also
T
 1 1  T
be written as  1  = [0.7075 1 0.7075] which is
 2 2
reasonable close to [0.7075 1 0.7075]T .

E8) Iteration 1: Starting with v ( 0) = [−1 1]T and solving Ay (1) = v ( 0) , we

get
y (1) = [− 1.25 0.75] ; m1 = 1.25
T

v (1) = [− 1 0.60]
T

Iteration 2: On solving A y ( 2) = v (1) , we get

y ( 2) = [− 1.05 0.55] ; m 2 = 1.05

T

v ( 2) = [− 1 0.5238]
T

Iteration 3: On solving A y (3) = v ( 2) , we get

y (3) = [− 1.0119 0.5119] ; m 3 = 1.0119

T

v (3) = [− 1 0.5176]
T

Iteration 4: On solving A y ( 4) = v (3) , we get

y ( 4) = [− 1.0088 0.5088] ; m 4 = 1.0088

T

v ( 4) = [− 1 0.5043]
T

After 4 iterations, the ratios are

( y (r4) ) r
for r = 1, 2 = −1.0088, 1.0089 .
( v (r3) ) r

The maximum error in these ratios is 0.0001. Hence the dominant

eigenvalue of A −1 can be taken as 1.0088. The smallest
1
eigenvalue of A is therefore = 0.9912.
1.0088

The corresponding eigenvector is [− 1 0.5043] .

T

E9) Starting with [1 1 1]T , we get the following iterations:

Iteration 1: On solving Ay (1) = v ( 0) , we get

y (1) = [0.3171 0.0343 0.3171]
T

m1 = 0.3171

v (1) = [1 0.1082 1]
T
279
Block 2 Solutions of Linear Algebraic Equations
= [0.5693 − 0.5007 0.5693]
( 2) T
Iteration 2: y
m 2 = 0.5693

v ( 2) = [1 − 0.8795 1]
T

Iteration 3: y (3) = [0.8486 − 1.0932 0.8486]

T

m 3 = 1.0932

v (3) = [0.7763 − 1 0.7763]

T

Iteration 4: y ( 4) = [0.7485 − 1.039 0.7485]

T

m 4 = 1.039

v ( 4) = [0.7204 − 1 0.7204]
T

After four iterations, the ratios are

( y (r4 ) ) 0.7485 − 1.039 0.7485
( 3)
for r = 1, 2, 3 = , ,
(vr ) 0.7204 −1 0.7204
= 1.0390, 1.039, 1.0390

The maximum error in these ratios is 0. Hence the dominant

1
egienvalue of A −1 can be taken as 1.039. Therefore, = 0.9625
1.039
is the smallest eigenvalue (in magnitude) of A and the
corresponding eigenvector is [0.7204 − 1 0.7204]T . This
eigenvalue is reasonably close to the eigenvalue obtained in E2.

3 2
E10) It is given that q = −1. Thus, A + qI is [A + I] =  .
1 4

Starting with v ( 0) = [−1 1]T , we get the following iterations:

3 2 (1)  − 1
Iteration 1: Solving   y =   , we get
1 4  1 

y (1) = [− 0.6 0.4] ; m1 = 0.6

T

v (1) = [− 1 0.6667]
T

Iteration 2: y ( 2) = [− 0.5333 0.3] ; m 2 = 0.5333

T

v ( 2) = [− 1 0.5625]
T

Iteration 3: y (3) = [− 0.5125 0.2688] ; m 3 = 0.5125

T

v (3) = [− 1 0.5244]
T

280
Unit 8 Eigenvalues and Eigenvectors
= [− 0.5049 0.2573] ; m 4 = 0.5049
( 4) T
Iteration 4: y

v ( 4) = [− 1 0.5097]
T

( y (r4) )
After 4 iterations, the ratios for r = 1, 2 are 0.5049, 0.4907,
( v (r3) )
respectively. The maximum error in these ratio is 0.0142. Hence
the dominant eigenvalues of (A + I) −1 can be taken as µ = 0.5 .
The eigenvalue of A which is nearest to − 1 is obtained from
1 1
λ= −1 = −1 = 1
µ 0.5

The corresponding eigenvector is [− 1 0.5097 ] .

T

− 2 2 
E11) [A − 5I] =  
 3 − 1
Starting with v ( 0) = [1 1]T , we get the following iterations:

 − 2 2  (1) 1
Iteration 1: Solving   y =   , we get
 3 − 1 1

y (1) = [0.75 1.25] ; m1 = 1.25

T

v (1) = [0.6 1]
T

Iteration 2: y ( 2) = [0.65 0.95] ; m 2 = 0.95

T

v ( 2) = [0.6842 1]
T

Iteration 3: y (3) = [0.6711 1.0132] ; m 3 = 1.0132

T

v (3) = [0.6623 1]
T

Iteration 4: y ( 4) = [0.6656 0.9968] ; m 4 = 0.9968

T

v ( 4) = [0.6664 1]
T

( y ( 4) ) r
After 4 iterations, the ratios for r = 1, 2 are 1.005, 0.9968 ,
( v (3) ) r
respectively. The maximum error in these ratios is 0.0082 . Hence
the dominant eigenvalue of ( A − 5I) −1 can be taken as µ = 0.99 .
The eigenvalue of A which is nearest to 5 is obtained from
1
λ= +5
µ
1
= + 5 = 6.0101
0.99
The corresponding eigenvector is [0.664 1] .
T
281
Block 2 Solutions of Linear Algebraic Equations
E12) a) The given matrix A is a symmetric matrix. Thus the
eigenvalue of A lie in the following regions:

i) | λ | ≤ max{3, 4, 3} = 4

ii) Union of the circles:

a) | λ − 2 | ≤1

b) |λ−2| ≤ 2

c) | λ−2 | ≤1

Since A is symmetric, it has real eigenvalues.

Therefore, the eigenvaluaes lie in the intervals

i) | λ | ≤ 4  −4 ≤ λ ≤ 4

ii) union of circles:

a) −1 ≤ λ − 2 ≤ 1  1 ≤ λ ≤ 3

b) −2≤λ−2≤ 20≤λ ≤4

union of (a) and (b) is 0 ≤ λ ≤ 4 .

Intersection of (i), (ii) is 0 ≤ λ ≤ 4. Hence the

eigenvalues of A lie in the interval 0 ≤ λ ≤ 4 .

b) A is a symmetric matrix. Thus, the eigenvalues of A lie in

the intervals

i) | λ | ≤ {10, 6, 6} | λ | ≤ 10  −10 ≤ λ ≤ 10

ii) Union of circles

a) | λ − 6 | ≤ 4  −4 ≤ λ − 6 ≤ 4  2 ≤ λ ≤ 10

b) | λ − 3 | ≤ 3, and
| λ − 3 | ≤ 3  −3 ≤ λ − 3 ≤ 3  0 ≤ λ ≤ 6

union of (a) and (b) is 0 ≤ λ ≤ 10 .

Intersection of (i), (ii) is 0 ≤ λ ≤ 10. Hence the

eigenvalues of A lie in the interval 0 ≤ λ ≤ 10.

E13) a) i) Maximum absolute row sums

| λ | ≤ max{6, 15, 7} = 15

ii) Maximum absolute column sums

| λ | ≤ max{10, 10, 8} = 10
282
Unit 8 Eigenvalues and Eigenvectors
iii) Union of circles
| λ − 3 | ≤ 4,
| λ − 6 | ≤ 9,
| λ −3| ≤ 5

Thus, union is | λ − 6 | ≤ 9

iv) Union of circles

| λ − 3 | ≤ 7,
| λ − 3 | ≤ 4,
| λ − 3 | ≤ 5,
Thus, union is | λ − 3 | ≤ 7.

The intersection of circles | λ | ≤ 15, | λ | ≤ 10, | λ − 6 | ≤ 9 and

| λ − 3 | ≤ 7 is the shaded region of Fig. 2.

Fig.2

b) i) Maximum absolute row sums

| λ | ≤ max{13, 5, 6} = 13

ii) Maximum absolute column sums

| λ | ≤ max{9, 6, 9} = 9

iii) Union of circles | λ − 4 | ≤ 9, | λ + 2 | ≤ 3 and | λ − 2 | ≤ 4 is

| λ − 4 | ≤ 9.

iv) Union of circles | λ − 4 | ≤ 5, | λ + 2 | ≤ 4 and | λ − 2 | ≤ 8 is

| λ − 2 | ≤ 8.
283
Block 2 Solutions of Linear Algebraic Equations
The intersection of circles | λ | ≤ 13, | λ | ≤ 9, | λ − 4 | ≤ 9 and
| λ − 2 | ≤ 8 is the estimation of bounds for eigenvalues. This
region is given in Fig. 3.

Fig.3

284
Block 2 Miscellaneous Examples and Exercises

MISCELLANEOUS EXAMPLES AND EXERCISES

The few examples and exercises, given below cover the concepts and
processes you have studied in this block. Studying the examples, and
solving the exercises, will give you a better understanding of the
underlying concepts concerned. Trying exercises on your own will also
give you more practice and confidence in solving such problems.

Example 1: Solve the following systems of equations by Cramer’s rule.

i) 3x 1 + 2 x 2 + x 3 = 5 ii) 7.6 x1 + 0.5x 2 + 2.4 x 3 = 1.9
2x1 + 5x 2 + x 3 = −3 2.2 x1 + 9.1x 2 + 4.4x 3 = 9.7
2x1 + x 2 + 3x 3 = 11 − 1.3x1 + 0.2 x 2 + 5.8x 3 = −1.4

Solution: i) We use Cramer’s rule to find the values of x 1 , x 2 and x 3 .

3 2 1
d= 2 5 1
2 1 3
5 1 2 1 2 5
= 3× − 2× + 1×
1 3 2 3 2 1
= 3 × (15 − 1) − 2 × (6 − 2) + 1 × (2 − 10)
= 42 − 8 − 8
= 26 ≠ 0
5 2 1

d1 = − 3 5 1

11 1 3
= 52
3 5 1
d2 = 2 − 3 1
2 11 3
= −52
3 2 5
d3 = 2 5 − 3
2 1 11
= 78
d1 52
∴ x1 = = = 2,
d 26
d − 52
x2 = 2 = = −2
d 26
d 3 78
x3 = = =3
d 26 285
Block 2 Solution of Linear Algebraic Equations
ii) Using the Cramer’s rule, we get

7. 6 0.5 2.4
d = 2.2 9.1 4.4
− 1.3 0.2 5.8

9.1 4.4 2.2 4.4 2.2 9.1

= 7.6 × − 0.5 × + 2.4 ×
0.2 5.8 − 1.3 5.8 − 1.3 0.2

= 394.44 − 9.24 + 29.448

= 414.648 ≠ 0

1. 9 0.5 2.4
d1 = 9.7 9.1 4.4
− 1.4 0.2 5.8

= 102 .632

7. 6 1.9 2.4
d 2 = 2.2 9.7 4.4
− 1.3 − 1.4 5.8

= 462 .152

7.6 0.5 1.9

d 3 = 2. 2 9.1 9. 7

− 1. 3 0. 2 − 1. 4

= −93.02

d1 102.632
∴ x1 = = = 0.2475
d 414.678
d 2 462.152
x2 = = = 1.1145
d 414.648
d 3 − 93.02
x3 = = = −0.2243
d 414.648
***

Example 2: Solve the following system of equations using Gauss

elimination method.
1 1
i) x1 + x 2 + x 3 = 1 ii) 4x + y + z = 4
2 3 x + 4 y − 2z = 4
1 1 1 3x + 2 y − 4z = 6
x1 + x 2 + x 3 = 0
2 3 4
1 1 1
x1 + x 2 + x 3 = 0
3 4 5
286
Block 2 Miscellaneous Examples and Exercises
iii) x1 − 4x 2 − x 4 =6 iv) 1.14x − 2.15x 2 − 5.11x 3 = 2.05
x 1 + x 2 + 2 x 3 + 3x 4 = −1 0.42 x1 − 1.13x 2 + 7.05x 3 = 0.80
2 x 1 + 3x 2 − x 3 − x 4 = −1 − 0.71x 1 + 0.81x 2 − 0.02 x 3 = −1.07
x 1 + 2 x 2 + 3x 3 − x 4 =3

Solution: i) The argument matrix is

 1 1 1
1 2 3 
  1
[A | b] =  1 1 1
0 R 2 ← R 2 − × R 1
2 3 4  2
1 1 1  R ← R − 1×R
3 4 5 0 3 3 1
 3

 1 1 
1 2 3
1 
 
1 1
≈ 0 − 1
 12 12 2
0 1 4
− 1 R3 ← R3 − R2
 12 45 3 

 1 1 
1 2 3
1 
 
1 1
≈ 0 − 1
 12 12 2
 1 1 
0 0 180 6 

Now using back substitution method, we get

1 1
x3 =
180 6
 x 3 = 30

1 1 1
x 2 + x3 = −
12 12 2
 x 2 = −36
1 1
x1 + x2 + x3 = 1
2 3
 x1 = 9

Thus, x 1 = 9, x 2 = −36 and x 3 = 30

ii) The argument matrix [A | b] is

4 1 1 4
  1
[A | b] = 1 4 −2 4 R 2 ← R 2 − × R 1
  4
3 2 −4 6 R ← R − 3 × R
3 3 1
4
287
Block 2 Solution of Linear Algebraic Equations

4 1 1 4

 15 9

≈ 0 4
− 3
4
  1
0 5
− 19 3 R 3 ← R 3 − 3 × R 2
 4 4 

4 1 1 4
 
15 9
≈ 0 4
− 3
4
 
0 0 −4 2

We use back substitution method and get

1 1
x = 1, y = and z = −
2 2

iii) The argument matrix is

1 −4 0 −1 6
 
1 1 2 3 − 1 R 2 ← R 2 − R 1
[A | b] =  
2 3 −1 −1 − 1 R 3 ← R 3 − 2 × R 1
 
1 2 3 −1 3  R 4 ← R 4 − R 1

1 −4 0 −1 6 
 
0 5 2 4 −7
≈ 
0 11 −1 1 − 13 R 3 ← R 3 − 2.2 × R 2
 
0 6 3 0 − 3  R 4 ← R 4 − 1.2 × R 2

1 −4 0 −1 6
 
0 5 2 4 − 7
≈ 
0 0 − 5. 4 − 7. 8 2. 4
 
0 0 0.6 − 4.8 5.4 R 4 ← R 4 + 0.1111 × R 3

1 −4 0 −1 6 
 
0 5 2 4 −7 
≈ 
0 0 − 5.4 − 7. 8 2.4 
 
0 0 0 − 5.6667 5.6667

We use back substitution method, and get

x 1 = 1, x 2 = −1, x 3 = 1 and x 4 = −1
288
Block 2 Miscellaneous Examples and Exercises
iv) The argument matrix is

 1.14 − 2.15 − 5.11 2.05 

[A | b] =  0.42 − 1.13 7.05 0.8  R 2 ← R 2 − 0.3684 × R 1
 − 0.71 0.81 − 0.02 − 1.07 R 3 ← R 3 + 0.6228 × R 1

1.14 − 2.15 − 5.11 2.05 

≈  0 − 0.3379 8.9326 0.0447 
 0 − 0.529 − 3.2025 0.2068 R 3 ← R 3 − 1.5657 × R 2

1.14 − 2.15 − 5.11 2.05 

≈  0 − 0.3379 8.9326 0.0447 
 0 0 − 17.1882 0.1367 

We use back substitution method and get

x 1 = 1.1163, x 2 = −0.3427 and x 3 = −0.008
***

Example 3: Find the inverse of the matrix

11 3 − 1
2 5 5 

 1 1 1 

using the Gauss-Jordan method and solve the following system of

equations.
11x1 + 3x 2 − x 3 = 15
2x1 + 5x 2 + 5x 3 = −11

x1 + x 2 + x 3 = 1

Solution: The argument matrix is

11 3 − 1 1 0 0
[A | I ] =  2 5 5 0 1 0
 1 1 1 0 0 1 R ← 1 R
1 1
11
 3 −1 1 
1 11 11 11
0 0
≈ 2 5 5 0 1 0 R 2 ← R 2 − 2 × R 1
 
1 1 1 0 0 1 R 3 ← R 3 − R 1
 
 3 1 1 
1 11
−
11 11
0 0
 
49 57
≈ 0 − 2 1 0
11 11 11
  11
0 8 12
−
1
0 1  R2 ← × R2
 11 11 11  49
289
Block 2 Solution of Linear Algebraic Equations
 3 1 1 
1 11
−
11 11
0 0
  3
57 11
≈ 0 1 − 2 0  R1 ← R1 − × R 2
49 49 49
  11
0 8 12
−
1
0 1 R 3 ← R 3 − × R 2
8
 11 11 11  11
 20 5 3 
1 0 −
49 49
−
49
0
 
57 11
≈ 0 1 − 2 0 
49 49 49
  49
0 0 12
−
3
−
8
1  R3 ← R3
 49 49 49  12

 20 5 3 
1 0 −
49 49
−
49
0
  20
57 11
≈ 0 1 − 2 0  R1 ← R1 + R3
49 49 49
  49
0 0 1 −
1
−2
49 
R ← R2 −
57
R3
 4 3 12  2 49
 1 5 
1 0 0 0 −
3 3 
 
≈ 0 1 0 1
4
1 19
−  = I|A
4
[ ]-1

 
0 0 1 −
1
−2
49 
 4 3 12 

 1 5 
0 −
3 3 
 
Hence, A −1 =  14 1 −
19
4

 
− 1 −2
49 
 4 3 12 

Let us now solve the system of equations using A −1.
Ax = b  x = A -1b
 1 5 
 x1   0 −
3 3   15 
    
  x 2  =  14 1 −
19
4
 − 11
    
 x 3  − 1 −2
49   1 
 4 3 12 

  1 5 
 (0) (15) +  −  (−11) +   (1)  16 
  3 3   3 
1  19 
=   (15) + (1) (−11) +  −  (1) =  − 12
 4  4    23 
   
 1  2  49 
  −  (15) +  −  (−11) +   (1)   3 
  4  3  12  
16 23
i.e. x1 = , x 2 = −12, x 3 = .
3 3

290 ***
Block 2 Miscellaneous Examples and Exercises
Example 4: Find the inverses of the following matrices using Guass
Jordan method with partial pivoting.

0 1 2 − 1 2 0
   
i) A = 3 5 1 ii) A=1 0 5
   
6 8 9  3 8 7

0 1 2 1 0 0
Solution: i) [A | I ] = 3 5 1 0 1 0
6 8 9 0 0 1  R 1 ↔ R 3

6 8 9 0 0 1
≈ 3 5 1 0 1 0
0 1 2 1 0 0 R ← R 1
1
6
 4 3 1
1 3 2
0 0 6
 
≈ 3 5 1 0 1 0
 
0 1 2 1 0 0  R 2 ← R 2 − 3R 1
 

 4 3 1 
1 3 2
0 0 6 
  4
≈ 0 1 − 72 0 1 − 12  R 1 ← R1 − R 2
  3
0 1 2 1 0 
0 R3 ← R3 − R2
 

 37 4 5 
1 0 6
0 −
3 6 
 
≈ 0 1 − 72 0 1 − 12 
  2
0 0 11
1 −1 1  R3 ← R3
 2 2 
 11

 37 5 
1 0 6 0 − 43 6 
  37
≈ 0 1 − 72 0 − 12  R 1 ← R 1 − R 3
1
  6
0 0 1 2 − 2 1 
R ← R2 + R3
7
 11 11 11  2
 2

 3 
1 0 0 − 37
33
7
− 33 11 
 
≈ 0 1 0 7
11
4
11
2
− 11 [
 = I | A −1 ]
 
0 0 1 2 2
− 11 1 
 11 11 

291
Block 2 Solution of Linear Algebraic Equations
 37 7 3 
− 33 − 33
−1
0 1 2 11 
   
A −1 = 3 5 1  =  11 7 4 2
− 11
11
   
6 8 9   2 2
− 11 1 
 11 11 

ii) Initial Matrix is
− 1 2 0 1 0 0
 
[A | I] =  1 0 5 0 1 0
 
 3 8 7 0 0 1 R 1 ← − R 1
1 − 2 0 −1 0 0
 
≈ 1 0 5 0 1 0 R 2 ← R 2 − R 1
 
3 8 7 0 0 1 R 3 ← R 3 − 3R 1
1 − 2 0 −1 0 0
 
≈ 0 2 5 1 1 0
  1
0 14 7 3 0 1 R 2 ← R 2
2
1 −2 0 −1 0 0
 5

1 1
≈ 0 1 0  R 1 ← R 1 − ( − 2) R 2
2 2 2
 
0 14 7 3 0 1 R 3 ← R 3 − 14R 2
1 0 5 0 1 0
 5

1 1
≈ 0 1 0
2 2 2
  1
0 0 − 28 −4 − 7 1  R 3 ← − R 2
28
 
1 0 5 0 0 0 
 5 1 1

≈ 0 1 0  R 1 ← R 1 − 5R 3
2 2 2
  5
0 0 1 1 1
− 28 1  R2 ← R2 − R3
 7 4  2
 5 
1 0 0 − 57 − 14
28 
 
≈ 0 1 0 1
7
− 18
55
[
5  = I | A −1 ]
 
0 0 1 1 1 1
− 28
 7 4 
 5 5 
− 1 0
−1
− 7 − 14
2 28 
   
5
A −1 =  1 0 5 =  17 − 18 56

   
 3 8 7   1 1 −
1
 7 4 28 

***
292
Block 2 Miscellaneous Examples and Exercises
Example 5: Use Doolittle LU decomposition method to solve the
following system of equations
x1 + x 2 + x 3 = 3

2x 1 − x 2 + 3x 3 = 16
3x1 + x 2 − x 3 = −3

Solution: Let A = LU
1 1 1 1 0 0  u11 u12 u13 
    
 2 − 1 1  = l 21 1 0  0 u 22 u 23 
    
 3 1 − 1 l 31 l 32 1  0 0 u 33 
1 1 1  u 11 u 12 u13 
   
  2 − 1 1 = l 21u11 l 21u12 + u 22 l 21u13 + u 23 
   
 3 1 1  l 31u11 l 31u 12 + l 32 u 22 l 32 u 31 + l 32 u 23 + u 33 

We compare the corresponding elements of LHS and RHS and get

14 2
u11 = 1, u12 = 1, u13 = 1, u 22 = −3, u 23 = 1, u 33 = − , l 21 = 2, l 31 = 3, l 32 =
3 3
Thus, we have

1 0 0 1 1 1 
   

L = 2 1 0 and U = 0

−3 1 

 
3 2
1   14 
 3  0 0 − 
3

Now, Ax = B,
 LUx = B [Q A = LU ]
Let Ux = y , then Ly = B

1 0

0
  y1   3 
     
2 1 0  y 2  =  16 
     
 3
2
1  y 3  − 3
3 
y1 = 3
2 y1 + y2 = 16
2
3 y1 + y 2 + y3 = − 3
3
We use forward substitution method and get
56
y1 = 3, y 2 = 10, y 3 −
3
293
Block 2 Solution of Linear Algebraic Equations
Now using Ux = y , we get

1 1 1   x1   3 
   
     
0 −3 − 1  ×  x 2  =  10 
     56 
0 0 − 10 x  −
3   3   3 

x1 + x 2 + x 3 = 3
− 3x 2 − x 3 = 10
10 56
− =−
3 3

We now use back substitution method and get

28 26 13
x3 = , x 2 = − and x 3 =
5 5 5
***

Example 6: Use Crout LU decomposition method to solve the following

system of linear equations.
3x1 + 2x 2 = 7
2x1 + 3x 2 + 2x 3 = 14

2x 2 + 3x 3 + 2x 4 = 21
2x 3 + 3x 4 = 18

Solution: Let A = LU
 l 11 0 0 0 1 u12 u 13 u14 
   
l 21 l 22 0 0 0 1 u 23 u 24 
where L =   and U =  
l 31 l 32 l 33 0 0 0 1 u 34 
   
l 41 l 42 l 43 l 44  0 0 0 1 
Now A = LU gives
3 2 0 0  l 11 l 11u12 l12 u13 l 14 u14 
2  
3 2 0 l 21 l 21u12 + l 22 l 21u13 + l 22 u 23 l 21u14 + l 22 u 24 
 = 
0 2 3 2 l 31 l 31u12 + l 32 l 31u13 + l 32 u 23 + l 33 l 31u14 + l 32 u 24 + l 33 u 34 
   
0 0 2 3 l 41 l 41u12 + l 42 l 41u13 + l 42 u 23 + l 43 l 41u14 + l 42 u 24 + l 43 u 34 + l 44 

On comparing the corresponding elements in LHS and RHS, we obtain

2
l 11 = 3, u12 = , u13 = 0, u14 = 0,
3
5 6
l 21 = 2, l 22 = , u 23 = , u 24 = 0,
3 5
3 10
l 31 = 0, l 32 = 2, l 33 = , u 34 = ,
5 3
294
Block 2 Miscellaneous Examples and Exercises
11
l 41 = 0, l 42 = 0, l 43 = 2, l 44 = −
3

∴ LU decomposition for A is

3 0 0 0   2 
  1 3
0 0
2 5
3
0 0   
6
  0 1 5
0
A=
0 2 3
0   
 5  0 0 1 10 
   3
0 − 11  0 1 
0 2
 3  0 0

Now, Ax = b, and A = LU  LUx = b

Let Ux = y , then Ly = b

3 0 0 0 
   y1   7 
2 5
0 0     

3
  y 2  14 
  = 
0 2 3
0   y 3  21
 5 
     
0 0 2 − 11  y 4  18
 3

We now use forward substitution method and get

7 28 49
y1 = , y 2 = , y3 = , y 4 = 4
3 5 3

We now use Ux = y and get

 2  7
1 3
0 0  x1   3 
     
0 1 6
0  x 2   28 
 5
  =5
0 10   x 3   49 

0 1 3   3
   x 4   
0 0 0 1 4

We now use back substitution method and get

x 4 = 4, x 3 = 3, x 2 = 2, x 1 = 1.
You may note that the coefficient matrix A is known as tridiagonal
matrix.

***

Example 7: Solve the following system of equations by (i) Gauss-

Jordan and (ii) Gauss-Seidal methods with initial approximation as
[2 1 − 1]T correct to four decimal places.
295
Block 2 Solution of Linear Algebraic Equations
9 x1 + 2x 2 + 4 x 3 = 20
x 1 + 10 x 2 + 4 x 3 = 6
2x 1 − 4x 2 + 10 x 3 = −15

Solution: i) We use Gauss Jordan method and obtain

9 2 4 20 

[A | B] = 1 10 4 6 
 2 − 4 10 − 15 R ← 1 R
1 1
9
 2 4 20 
1 9 9 9 
 
≈ 1 10 4 6  R 2 ← R 2 − R1
 
 2 − 4 10 − 15 R 3 ← R 3 − 2 × R 1
 
 2 4 20 
1 9 9 9 
 
88 32 34
≈ 0 9

9 9
  9
0 − 40 82
−
175  R 2 ← R2
 9 9 9  88
 2 4 20 
1 9 9 9 
  2
4 17
≈ 0 1 11
 R 1 ← R1 − × R 2
44
  9
0 − 40 82
−
175 R ← R + ×R 40
 9 9 9  3

3
9
2

 4 47 
1 0 11 22 
 
4 17
≈ 0 1 11 44

  11
0 0 118 − 195  R 3 ← R3
 11 11 
 118
 4 47 
1 0 11 22 
  4
4 17
≈ 0 1 11 44
 R1 ← R 1 − × R 3
  11
0 0 1 − 195  R ← R − 4 × R
 118  2

2
11
3

 323 
1 0 0 118 
 
233
≈ 0 1 0 
236
 
0 0 1 − 195 
 118 

The solution using Gauss Jordan elimination method is

323 233 195
x1 = = 2.7373, x 2 = = 0.9873 and x 3 = − = −1.6525
118 236 118
296
Block 2 Miscellaneous Examples and Exercises
ii) We now use Gauss Seidal method to solve the system of
equations.
From the given system of equations, we get
1
x 1( k+1) = (20 − 2 x (2k ) − 4 x 3( k ) )
9
1
x (2k +1) = (6 − x1( k ) − 4x 3( k+1) ) (1)
10
1
x 3( k +1) = (−15 − 2 x1( k +1) + 4x (2k +1) )
10
where k = 0,1, 2,...

We perform following iterations using x (0 ) = [2 − 1] .

T
1

Iteration 1: Put k = 0 in Eqns. (1).

1 1
x 1(1) = [20 − 2(1) − 4(−1)] = [22] = 2.4444
9 9
1 1
x (21) = [6 − (2.4444) − 4(−1)] = [7.5556] = 0.7556
10 10
1 1
x 3(1) = [−15 − 2(2.4444) − 4(0.7556)] = [−16.8667] = −1.6867
10 10

Iteration 2: Put k = 1 in Eqns. (1)

1 1
x 1( 2 ) = [20 − 2(0.7556) − 4(−1.6867)] = [25.2356] = 2.804
9 9
1 1
x (22 ) = [6 − (2.804) − 4(−1.6867)] = [9.9427] = 0.9943
10 10
1 1
x 3( 2 ) = [−15 − 2(2.804) − 4(0.9943)] = [−16.6308] = −1.6631
10 10

Iteration 3: Put k = 2 in Eqns. (1)

1 1
x 1( 3) = [20 − 2(0.9943) − 4(−1.6631)] = [24.6638] = 2.7404
9 9
1 1
x (23) = [6 − (27404) − 4(−1.6631)] = [9.9119] = 0.9912
10 10
1 1
x 3( 3) = [−15 − 2(2.7404) − 4(0.9912)] = [−16.5161] = −1.6516
10 10

Iteration 4: Put k = 3 in Eqns. (1)

1 1
x 1( 4) = [20 − 2(0.9912) − 4(−1.6516)] = [24.6241] = 2.736
9 9
1 1
x (24 ) = [6 − (2.736) − 4(−1.6516)] = [9.8704] = 0.987
10 10
1 1
x 3( 4 ) = [−15 − 2(2.736) − 4(0.987)] = [−16.5238] = −1.6524
10 10
297
Block 2 Solution of Linear Algebraic Equations
Iteration 5: Put k = 4 in Eqns. (1)
1 1
x 1( 5) = [20 − 2(0.987) − 4(−1.6524)] = [24.6355] = 2.7373
9 9
1 1
x (25) = [6 − (2.7373) − 4(−1.6524)] = [9.8723] = 0.9872
10 10
1 1
x 3( 5) = [−15 − 2(2.7373) − 4(0.9872)] = [−16.5256] = −1.6526
10 10

Iteration 6: Put k = 5 in Eqns. (1)

1 1
x 1( 6) = [20 − 2(0.987) − 4(−16526)] = [24.6358] = 2.7373
9 9
1 1
x (26) = [6 − (2.7373) − 4(−1.6526)] = [9.8729] = 0.9873
10 10
1 1
x 3( 6) = [−15 − 2(2.7373) − 4(0.9873)] = [−16.5254] = −1.6525
10 10

Thus the solution of the system of linear equations correct to three

decimal places is

x 1 = 2.737
x 2 = 0.987
x 3 = −1.653

We summarise all the iterations in Table 1.

Table 1

Number of x1(k) x(k)

2
x (k)
3
Iteration
(k − 1)
1 2.4444 0.7556 − 1.6867
2 2.804 0.9943 − 1.6631
3 2.7404 0.9912 − 1.6516
4 2.736 0.987 − 1.6524
5 2.7373 0.9872 − 1.6526
6 2.737 0.987 − 1.653

We compare the results obtained in part (i) and (ii) and observe
that they are reasonably close.
***

Example 8: Perform four iterations to solve the following systems of

equations using Gauss-Jacobi and Gauss-Seidal methods. Also,
compare the results.
10 x1 + 15x 2 + 3x 3 = 14
− 30 x1 + x 2 + 5x 3 = 17
x 2 + x 2 + 4x 3 = 3

You may use x (0) = [1 1] .

T
1
298
Block 2 Miscellaneous Examples and Exercises
Solution: i) First, we use Jacobi method and get
1
x 1( k+1) = (17 − x 2( k ) − 5x 3( k ) )
− 30
1
x (2k +1) = (14 − 10x 2( k ) − 3x 3( k ) ) (2)
15
1
x 3( k +1) = (3 − x1( k ) − x (2k ) )
4
where k = 0,1, 2,...

We apply Jacobi method with x (0) = [1 1] and obtain the following

T
1
iteration:

Iteration 1: Put k = 0 in Eqns. (2)

1 1
x 1( k ) = [17 − (1) − 5(1)] = [11] = −0.3667
− 30 − 30
1 1
x (21) = [14 − 10(1) − 3(1)] = [1] = 0.0667
15 15
1 1
x 3(1) = [3 − (1) − (1)] = [1] = 0.25
4 4

Iteration 2: Put k = 1 in Eqns. (2)

1 1
x 1( 2) = [17 − (0.0667) − 5(0.25)] = [15.6833] = −0.5228
− 30 − 30
1 1
x (22 ) = [14 − 10(−0.3667) − 3(0.25)] = [16.9167] = 1.1278
15 15
1 1
x 3( 2 ) = [3 − (−0.3667) − (0.0667)] = [3.3] = 0.825
4 4

Iteration 3: Put k = 2 in Eqns. (2)

1 1
x 1( 3) = [17 − (1.1278) − 5(0.825)] = [11.7472] = −0.3916
− 30 − 30
1 1
x (23) = [14 − 10(−0.5228) − 3(0.825)] = [16.7528] = 1.1169
15 15
1 1
x 3( 3) = [3 − (−0.5228) − (1.1278)] = [2.395] = 0.5988
4 4

Iteration 4: Put k = 3 in Eqns. (2)

1 1
x (14) = [17 − (1.1169) − 5(0.5988)] = [12.8894] = −0.4296
− 30 − 30
1 1
x (24 ) = [14 − 10(−0.3916) − 3(0.5988)] = [16.1195] = 1.0746
15 15
1 1
x 3( 4) = [3 − (−0.3916) − (1.1169)] = [2.2747] = 0.5687
4 4 299
Block 2 Solution of Linear Algebraic Equations
We summarise all the four iterations in Table 2.

Table 2

Number of Iteration x1(k) x(k)

2
x (k)
3
(k − 1)
1 − 0.3667 0.0667 0.25
2 − 0.5228 1.1278 0.825
3 − 0.3916 1.1169 0.5988
4 − 0.4296 1.0746 0.5687

After four iterations, the solution is

x 1 = −0.4296
x 2 = 1.0746
x 3 = 0.5687

ii) We now use Gauss Seidel method for the given system of
equations and get
1
x 1( k+1) = (17 − x 2( k ) − 5x 3( k ) )
− 30
1
x (2k+1) = (14 − 10 x1( k +1) − 3x 3( k ) ) (3)
15
1
x 3( k+1) = (3 − x1( k+1) − x (2k +1) )
4
where k = 0,1, 2,...

We use x ( 0 ) = [1 1 1]T and get the following iterations:

Iteration 1: Put k = 0 in Eqns. (3).

1 1
x 1(1) = [17 − (1) − 5(1)] = [11] = −0.3667
− 30 − 30
1 1
x (21) = [14 − 10(−0.3667) − 3(1)] = [14.6667] = 0.9778
15 15
1 1
x 3(1) = [3 − (−0.3667) − (0.9778)] = [2.3889] = 0.5972
4 4

Iteration 2: Put k = 1 in Eqns. (3).

1 1
x 1( 2) = [17 − (0.9778) − 5(0.5972)] = [13.0361] = −0.4345
− 30 − 30
1 1
x (22) = [14 − 10(−0.4345) − 3(0.5972) = [16.5537] = 1.1036
15 15
1 1
x 3( 2) = [3 − (−0.4345) − (1.1036) = [2.331] = 0.5827
4 4

300
Block 2 Miscellaneous Examples and Exercises
Iteration 3: Put k = 2 in Eqns. (3).
1 1
x 1( 3) = [17 − (1.1036) − 5(0.5827)] = [12.9827] = −0.4328
− 30 − 30
1 1
x (23) = [14 − 10(−0.4328) − 3(0.5827)] = [16.5794] = 1.1053
15 15
1 1
x 3( 3) = [3 − (−0.4328) − (1.1053)] = [2.3275] = 0.5819
4 4

Iteration 4: Put k = 3 in Eqns. (3).

1 1
x 1( 4) = [17 − (1.1053) − 5(0.5819)] = [12.9854] = −0.4328
− 30 − 30
1 1
x (24 ) = [14 − 10(−0.4328) − 3(0.5819)] = [16.5829] = 1.1055
15 15
1 1
x 3( 4 ) = [3 − (−0.4328) − (1.1055)] = [2.3273] = 0.5818
4 4

We summarise all the iterated values in Table 3.

Table 3

Number of Iteration x1(k) x(k)

2
x (k)
3
(k + 1)
1 − 0.3667 0.9778 0.5972
2 − 0.4345 1.1036 0.5827
3 − 0.4328 1.1053 0.5819
4 − 0.4328 1.1055 0.5818

Thus, the Solution using Gauss Seidel method is

x 1 = −0.4328
x 2 = 1.1055
x 3 = 0.5818

On comparing the iterated values after four iterations in Table 2

and Table 3, we observe that Gauss Seidel method gives the
correct values upto three decimal places and Jacobi method does
not give accuracy upto that place. So, we can conclude that the
Seidel method converges faster than Jacobi method.
***

Example 9: Use Gauss elimination method to solve the following

system of linear equations
2.5x1 + 5.2x 2 = 6.2
1.251x 1 + 2.605x 2 = 3.152
and get your answer to 4 significant decimal places. Also, perform four
iterations to improve the solution by Gauss Seidel method.

Solution: We apply Gauss elimination method for the given system of

linear equations. For this, we get
301
Block 2 Solution of Linear Algebraic Equations
 2.5 5.2 6.2 
[A | B ] =  
1.251 2.605 3.152 R 2 ← R 2 − 0.5004 × R1

 2.5 5.2 6.2 

≈ 
 0 0.0029 0.0495
2.5x1 + 5.2x 2 = 6.2
0.0029x 2 = 0.00495

We now use back substitution method and get

0.0495
x2 = = 16.9589
0.0029
81.9863
and x 1 = − = −32.7945
2.5

Thus, x 1 = −32.7945 and x 2 = 16.9589

We now apply Gauss Seidel method with initial values as x ( 0) =

[− 32 16]T to the given system of linear equations and obtain
1
x 1( k+1) = (6.2 − 5.2x (2k ) )
2.5
1
x (2k+1) = (3.152 − 1.251x1( k +1) ) (4)
2.605
where k = 0,1, 2,...

Iteration 1: Put k = 0 in Eqns. (4)

1 1
x 1(1) = [6.2 − 5.2(16)] = [−77] = −30.8
2.5 2.5
1 1
x (21) = [3.152 − 1.251(−30.8)] = [41.6828] = 16.0011
2.605 2.605

Iteration 2: Put k = 1 in Eqns. (4)

1 1
x 1( 2) = [6.2 − 5.2(16.0011)] = [−77.0056] = −30.8022
2.5 2.5
1 1
x (22) = [3.152 − 1.251(−30.8022)] = [41.6856] = 16.0021
2.605 2.605

Iteration 3: Put k = 2 in Eqns. (4)

1 1
x 1( 3) = [6.2 − 5.2(16.0021)] = [−77.0112] = −30.8045
2.5 2.5
1 1
x (23) = [3.152 − 1.251(−30.8045)] = [41.6884] = 16.0032
2.605 2.605

302
Block 2 Miscellaneous Examples and Exercises
Iteration 4: Put k = 3 in Eqns. (4)
1 1
x 1( 4) = [6.2 − 5.2(16.0032)] = [−77.0167] = −30.8067
2.5 2.5
1 1
x (24 ) = [3.152 − 1.251(−30.8067)] = [41.6912] = 16.0043
2.605 2.605
***

Example 10: Solve the following system of equations by successive

relaxation over method.
2 x 1 − 3x 2 + x 3 = − 1

x 1 + 4 x 2 − 3x 3 = 0

x1 + x 2 + x 3 = 6

You may use x ( 0) = [1 1] and w = 1.25 .

T
1

Solution: We apply successive over relaxation method and obtain

1
x 1( k+1) = (1 − w ) ⋅ x 1( k ) + w ⋅ (−1 + 3x (2k ) − x (3k ) )
2
1
x (2k+1) = (1 − w ) ⋅ x (2k ) + w ⋅ (0 − x1( k +1) + 3x (3k ) ) (5)
4
1
x (3k+1) = (1 − w ) ⋅ x 3( k ) + w ⋅ (6 − x 1( k+1) − x (2k +1) )
1
where k = 0,1, 2....

We substitute w = 1.25 in Eqns. (5) and perform the following iterations

of SOR method.

Iteration 1: Put k = 0 and x (0) = [1 1] in Eqns. (5).

T
1
1
x 1(1) = (1 − 1.25) ⋅ 1 + 1.25 ⋅ [−1 − 3(1) − (1)] = 0.375
2
1
x (21) = (1 − 1.25) ⋅1 + 1.25 ⋅ [0 − (0.375) + 3(1)] = 0.5703
4
1
x (31) = (1 − 1.25) ⋅ 1 − 1.25 ⋅ [6 − (0.375) − (0.5703)] = 6.0684
1

Iteration 2: Put k = 1 in Eqns. (5).

1
x 1( 2) = (1 − 1.25) ⋅ 0.375 + 1.25 ⋅ [−1 + 3(0.5703) − (6.0684)] = − 3.4421
2
1
x (22 ) = (1 − 1.25) ⋅ 0.5703 + 1.25 ⋅ [0 − (−3.4421) + 3(6.0684)] = 6.6222
4
1
x (32) = (1 − 1.25) ⋅ 6.0684 + 1.25 ⋅ [6 − (−3.4421) − (6.6222)] = 2.0079
1
303
Block 2 Solution of Linear Algebraic Equations
Iteration 3: Put k = 2 in Eqns. (5).
1
x 1( 3) = (1 − 1.25) ⋅ (−3.4421) + 1.25 ⋅ [−1 + 3(6.6222) − (2.0079)] = 11.3972
2
1
x (23) = (1 − 1.25) ⋅ 6.6222 + 1.25 ⋅ [0 − (11.3972) + 3 (2.0079)] = − 3.3348
4
1
x (33) = (1 − 1.25) ⋅ −2.0079 + 1.25 ⋅ [6 − (11.3972) + 3 (−3.3348)] = − 3.08
1

Iteration 4: Put k = 3 in Eqns. (5).

1
x 1( 4) = (1 − 1.25) ⋅11.3972 + 1.25 ⋅ [−1 + 3(−3.3348) − (−3.08)] = − 7.8021
2
1
x (24) = (1 − 1.25) ⋅ (−3.3348) + 1.25 ⋅ [0 − (−7.8021) + 3(−3.08)] = 0.3844
4
1
x (34 ) = (1 − 1.25) ⋅ (−3.08) + 1.25 ⋅ [6 − (−7.8021) − (0.3844)] = 17.5421
1

We summarise all the iterations in Table 4.

Table 4

Number of Iteration x1(k +1) x(k2+1) x (k3 +1)

(k − 1)
1 0.375 0.5703 6.0684
2 − 3.4421 6.6222 2.0079
3 11.3972 − 3.3348 − 3.08
4 − 7.8021 0.3844 17 .5421

From Table 4, we observe that the solution vector is not converging.

Hence, the given system of equations is divergent.
***

Example 11: Use power method to find the dominant eigenvalue and
corresponding eigenvector. You may consider decimal upto two places.
2 3 1
 
0 −1 2
 
0 0 3

You may use v ( 0) = [1 1] .

T
1

2 3 1
 
Solution: A = 0 −1 2
 
0 0 3

We apply power method and get the following iterations:

304
Block 2 Miscellaneous Examples and Exercises

Iteration 1: y (0 ) = v ( 0) = [1 1] .
T
1
2 3 1 1 6
    
y (1) = Av (0) = 0 −1 2 1 = 1
    
0 0 5 1 3

{ }
Now m1 = max y1(1) , y (21) , y 3(1) = max{6, 1, 3} = 6

6   1 
1 (1) 1   
and v (1) = y = 1  = 0.17 
m1 6   
3  0.5 

2 3 1  1   3 
    
Iteration 2: y ( 2) = Av (1) = 0 −1 2 0.17 = 0.83
    
0 0 3  0.5   1.5 

m 2 = max{3, 0.83,1.5} = 3

 3   1 
1 ( 2) 1 
v ( 2)
= y = 0.83 = 0.28
m2 3
 1.5   0.5 

2 3 1  1  3.33
    
Iteration 3: y (3) = Av ( 2) = 0 −1 2 0.28 = 0.72
    
0 0 3  0.5   1.5 
   

m 3 = max { 3.33, 0.72, 1.5 } = 3.33

 0.33  1 
1 ( 3) 1    
v ( 3) = y = 0.72 = 0.22
m3 3.33    
 1.5  0.46

2 3 1  1   3.1 
    
Iteration 4: y ( 4) = Av (3) = 0 −1 2 0.22 = 0.68
    
0 0 3 0.45 1.35 
   

m 4 = max{3.1, 0.68,1.35} = 3.1

 3. 1   1 
1 ( 4) 1    
v ( 4) = y = 0.68 = 0.22
m4 3. 1    
1.35  0.44

305
Block 2 Solution of Linear Algebraic Equations

2 3 1   1   3.1 
    
Iteration 5: y (5) = Av ( 4) = 0 −1 2 0.22 = 0.65
    
0 0 3 0.44 1.31

m5 = 3.1

 3.1   1 
1 (5) 1    
v ( 5) = y = 0.65 =  0.21
m5 3. 1    
1.31 0.42

2 3 1  1  3.05
    
Iteration 6: y (6) = Av (5) = 0 −1 2  0.21 = 0.63
    
0 0 3 0.42 1.27 

m 6 = 3.05

3.05  1 
1 (6) 1    
v ( 6) = y = 0.63 = 0.21
m6 3.05    
1.27  0.41

2 3 1   1  3.04
    
Iteration 7: y (7 ) = Av (6) = 0 −1 2 0.21 = 0.62
    
0 0 3 0.41 1.24 

m 7 = 3.04

3.04   1 
1 ( 7) 1    
v (7) = y = 0.62 =  0.2 
m7 3.04    
1.24  0.41

2 3 1  1  3.02
    
Iteration 8: y (8) = Av ( 7) = 0 −1 2  0.2  =  0.61
    
0 0 3 0.41 1.23 

m 8 = 3.02

3.02  1 
1 (8) 1    
v (8) = y =  0.61 =  0.2 
m8 3.02    
1.23  0.41

306
Block 2 Miscellaneous Examples and Exercises

2 3 1  1  3.02
    
Iteration 9: y (9) = Av (8) = 0 −1 2  0.2  =  0.61
    
0 0 3 0.42 1.22 

m 9 = 3.02

3.02  1 
1 (9 ) 1    
v ( 9) = y =  0.61 = 0.2
m9 3.02    
1.22  0.4

( y (r9) )
After 9 iterations, the ratios for r = 1, 2, 3 are
( v (r8) )
3.02 0.61 1.22
, , = 3.02, 3.05, 3.05, respectively.
1 0.2 0.4

The maximum error in this case is 0.03 . Hence, the dominant eigenvalue
1
 
is 3.02 and the corresponding dominant eigenvector is 0.2 .
 
0.4

***

Example 12: Estimate the eigenvalues of the matrix

− 3 0 2 1
 
0 −2 0 2
A= 
0 1 −2 0
 
 0 −1 2 3
using Gershgorin bounds. Draw rough sketch of the region.
Solution: The eigen values of A lie in the following regions:

i) Maximum absolute row sums are:

First row absolute sum = 3 + 0 + 2 + 1 = 6
Second row absolute sum = 0 + 2 + 0 + 2 = 4
Third row absolute sum = 0 + 1 + 2 + 0 = 3
Fourth row absolute sum = 0 + 1 + 2 + 3 = 6

Hence, λ ≤ max {6, 4, 3, 6} = 6

ii) Maximum absolute column sums are:

First column absolute sum = 3 + 0 + 0 + 0 = 3
Second column absolute sum = 0 + 2 + 1 + 1 = 4
Third column absolute sum = 2 + 0 + 2 + 2 = 6
Fourth column absolute sum = 1 + 2 + 0 + 3 = 6

Hence, λ ≤ max {3, 4, 6, 6} = 6

307
Block 2 Solution of Linear Algebraic Equations
iii) Union of circles row wise are:
λ −3 ≤3

λ−2 ≤2

λ − 2 ≤1

λ −3 ≤ 3

Hence, union is λ − 3 ≤ 3 .

iv) Union of circles column wise are:

λ−3 ≤ 0

λ−2 ≤2

λ−2 ≤4

λ −3 ≤3

Hence, union is λ − 2 ≤ 4 .

Therefore, the Gershgorin bounds for the eigen values would be

the intersection of λ = 6, λ − 3 ≤ 3 and λ − 2 ≤ 4, which is λ − 3 ≤ 3 .
The shaded region in Fig. 1 is the region in which all the
eigenvalues lie.

Fig. 1
***

308
Block 2 Miscellaneous Examples and Exercises
Now you may try the following exercises.

E1) Solve the following system of equations

4x 1 + x 2 + x 3 = 4
x 1 + 4 x 2 − 2x 3 = 4

3x 1 + 2 x 2 − 4 x 3 = 6
using Crout LU decomposition method.

E2) Find the largest eigenvalue in magnitude of the matrix

2 1 1
 
A = 1 1 0
 
1 0 2

using the power method. Take the initial approximation to the

eigenvector as v ( 0) = [1 1 1] and perform 4 iterations.
T

E3) Solve the following system of equations

x 1 + 2 x 2 + 3x 3 = 5

2x 1 + 8x 2 + 22 x 3 = 6
3x1 + 22x 2 + 82 x 3 = −10
using Gauss-elimination method with pivoting.

E4) Find the interval which contains all the eigen value of the matrix
1 2 3
 
A = 2 −3 4
 
 3 4 2

using Gerchgorin bounds.

E5) Using four iterations of inverse power method find the least eigen
value and the corresponding eigen vector of the matrix.

2 −1 0
 
− 1 20 − 2
 
 0 −2 1 

Assume the initial approximation to the eigen vector as

v ( 0) = [1 1 1] .
T

E6) Determine the rate of convergence of the Gauss-Seidal method for

solving the system of equations

309
Block 2 Solution of Linear Algebraic Equations

4 0 2
 x 1   
4
   x  =  − 3 .
0 5 2
   2  
5  x 3   
4 10 2

E7) Find the inverse of the matrix

3 2 1
 
A = 2 3 2 .
 
1 2 2

Using Crout LU decomposition method.

E8) Jacobi iteration method is used to solve the system of equations

4 0 3  x 1  0.5
     
0 3 2  x 2  = 0.5
     
 3 2 5  x 3   0 

Determine the rate of convergence of the method.

E9) Estimate the eigenvalues of the matrix

1 2 − 1
 
1 1 1
 
1 3 − 1

using the Gerschgorin bound. Draw a rough sketch of the region

in whcih the eigenvalues lie.

E10) Using Gauss Seidel iteration method, solve the system of

equations:
20x + y − 2z = 17
3x + 20 y − z = −18
2x − 3y + 20z = 25
Perform four iterations with zero vector as an initial approximation.

E11) Perform three iterations of the inverse power method to obtain the
smallest eigenvalue in magnitude of the matrix
1 1
 .
 − 3 5

Take the initial approximation to the eigenvector as [0.9, 1.0] .

T

E12) The Gauss-Seidel method is used to solve the system of

equations:

310
Block 2 Miscellaneous Examples and Exercises

− 2 1 5   x 1   15 
    
 4 −8 1   x 2  = − 21
    
 4 −1 − 2  x 3   − 1 

Check whether the method converges for the given system of

equations or not.

E13) The Gauss-Jacobi method is used to solve the system of

equations
4 0 3  x 1  0.5
    
0 3 2  x 2  = 0.5
    
 3 2 5  x 3   0 

i) Determine the rate of convergence of the method.

ii) Find the number of iterations needed to make max ε (i k ) ≤ 10−2.
i

E14) Using Gauss-Jordan method, find the inverse of the matrix

 2 1 3
 − 1 − 1 2
 
 0 2 1 

E15) Perform 3 iterations of the power method to estimate the highest

eigenvalue (in magnitude) of the matrix:
 1 4 3
 4 3 1
 
1 3 4

Use v ( 0) = [1.2, 1.15, 1.15] as the initial approximation to the

T

eigenvector. Also, find the corresponding eigenvector.

E16) Consider the following system of equations:

28x + 4 y − z = 30
2x + 17 y + 4z = 27
− x + 3y + 10z = 22
Starting with x ( 0 ) = [0 0 0]T perform four iterations of (i) Gauss-
Seidel method (ii) Gauss-Jacobi method and (iii) SOR method with
optional value of the relaxation factor.

E17) Solve the system of linear equations

x 1 + x 2 + 2x 3 = 2
3x 1 + 2 x 2 + x 3 = − 1
x 1 + 4 x 2 + 3x 3 = 1
using the Doolittle LU decomposition method. 311
Block 2 Solution of Linear Algebraic Equations
E18) Solve the system of equations
x1 + x 2 + x 3 = 6
4x 1 + x 2 + 5x 3 = 23

8x 1 + 2x 2 + 3x 3 = 32
using Gauss elimination method with partial pivoting.

SOLUTIONS/ANSWERS
E1) We take A = LU,
 l 11 0 0 1 u12 u13 
   
where L = l 21 l 22 0  and U = 0 1 u 23 
   
l 31 l 32 l 33  0 0 1 

We multiply L and U and compare the elements of A with

corresponding elements of LU and get
4 0 0   
  1 1 1 
  4 
0 , U = 
15 4
L = 1 4  
 0 1 − 3
 5  5
3 4
− 4   
  0 0 1 


4 −1
Suppose Ly = b  y1 = 1, y 2 = , y3 =
5 2
1 1
and Ux = y  x 3 = − , x 2 = , x1 = 1
2 2

E2) We apply power method and get the following iterations:

Iteration 1: y (0) = v ( 0) = [1 1]
T
1
2 1 1  1  4
    
y (1) = A v ( 0) = 1 1 0 1 =  2
    
1 0 2 1  3
m1 = 4
 1 
1 (1)  
v (1) = y =  0.5 
m1  
0.75

3.25
 
Iteration 2: y ( 2) = A v (1) =  1.5 
 
 2.5 
312
Block 2 Miscellaneous Examples and Exercises

m 2 = 3.25
 1 
1 ( 2)  
v ( 2) = y = 0.4615
m2  
0.7692

3.2307 
 
Iteration 3: y (3) = A v ( 2) = 1.4615 
 
2.5384
m 3 = 3.2307
 1 
1 ( 3)  
v ( 3) = y =  0.4523
m3  
0.7857

3.2380 
 
Iteration 4: y ( 4) = A v (3) = 1.4523 
 
2.5714
m 4 = 3.2380
 1 
1 ( 4)  
v ( 4) = y = 0.4485
m4  
 0.7941

After four iterations, the largest eigenvalue in magnitude is 3.280.

1 2 3 5 
 
E3) [A | b ] =  2 8 22 6 
 
 3 22 82 − 10 R 1 ← R 3

3 22 82− 10
  2
≈ 2 8 22 6  R 2 ← R 2 − R1
  3
1 2 3 5  R ← R − R 1
3 3 1
3
3 22 82 − 10
 
0 − 20 − 98 38 
≈ 3 3 3 
 − 16 − 73 25  4
0 3 3 3  R3 ← R3 − R2
  5
313
Block 2 Solution of Linear Algebraic Equations

3 22 82 − 10
 
0 − 20 − 98 38 
≈ 3 3 3 
 
0 0 9 −9 
5

5 


Using back substitution method, we get x 3 = −1, x 2 = 3, x 1 = 2

E4) λ −1 ≤ 5  − 4 ≤ λ ≤ 6

λ + 3 ≤ 6  −9 ≤ λ ≤ 3

λ − 2 ≤ 7  −5 ≤ λ ≤ 9

The required interval is − 4 ≤ λ ≤ 3.

E5) The inverse power method is given by

1 (k+1)
Ay ( k +1) = v (k) , v ( k+1) = y
m k+1
( y k+1 ) r
m k+1 = max | ( y k +1 ) r |, λ =
r (vk )r

Iteration 1: v ( 0) = [1 1]
T
1

y (1) = [0.6129 1.4516]

T
0.2258
m1 = 1.4516

v (1) = [0.4222 1]
T
0.1555

Iteration 2: y ( 2) = [0.2874 1.3053]

T
0.1526
m 2 = 1.3053

v ( 2) = [0.2202 1]
T
0.1169

Iteration 3: y (3) = [0.1819 1.2873]

T
0.1436
m 3 = 1.2873

v ( 3) = [1.1413 1]
T
0.1116

Iteration 4: y ( 4) = [0.1410 1.2815]

T
0.1407
m 4 = 1.2815

v ( 4 ) = [0.1248 1.2793]
T
0.1396

The dominant eigenvalue is 1.2815.

314
Block 2 Miscellaneous Examples and Exercises
Thus, the smallest eigenvalue in magnitude for the matrix A is
1
= 0.7803 and the corresponding eigvenvector is
1.2815
[0.1248 0.1396 1.2793]T .
E6) The Gauss-Seidal iteration method in matrix form is given by
x (n +1) = −(D + L) −1 Ux ( n ) + (D + L) −1 b = Hx ( n ) + c

From the given system of equations, we have

4 0 0 0 0 2
   
D + L = 0 5 0  ; and U = 0 0 2
   
5 4 10 0 0 0

We obtain
 
 50 0 0 0 0 2 0 0 − 12 
1      2
H = −(D + L ) −1 U = −  0 40 0 0 0 2 = 0 0 − 
200      5
 − 25 − 16 20 0 0 0 0 0 41 
 100 


Now eigenvalues of H are λ = 0, 0, 0.41

Since, ρ(H ) = 0.41 < 1 , the method converges. The rate of

convergence is given by
v = − log10 (ρ(H )) = 0.3872

 
3 0 0
 5

E7) L = 2 0
3
 
1 4 3
 3 5 

 2 1
1 3 3
 
4
U = 0 1 5

 
0 0 1
 

 1 
 3 0 0
 
L−1 = − 25 3
5
0
 
 1 − 43 5
 3 3 
315
Block 2 Solution of Linear Algebraic Equations
 1 
1 − 23
5 
 
U −1 = 0 1 − 4
5
 
0 0 1 
 

2 −2 1 
1  
A −1 = U −1L−1 =  − 2 5 − 4
3  
 1 −4 5 

E8) Iteration matrix is

H = − D −1 (L + U )

1   3
4 0 0 0 0 3 0 0 4
     
= 0 1 0 0 0 2 = −  0 0 2
3 3
     
0 0 1 3 2 0  3 2
0
 5   5 5 

Eigenvalues of H are given by the characteristic equation

43
| H − λ I | = 0  λ = 0, ±
60
ρ(H ) = 0.8466 (Spectral radius)

v = − log10 (0.8466) = 0.0723 (Rate of convergence)

Thus, the rate of convergence of the method is 0.0723 .

E9) The eigenvalue lie in the regions

i) | λ | ≤ max{4, 3, 5} = 5

ii) | λ | ≤ max{3, 6, 3} = 6

iii) The union of circle

| λ − 1 | ≤ 3, | λ − 1 | ≤ 2, | λ + 1 | ≤ 4
i.e., union of | λ − 1 | ≤ 3, and | λ + 1 | ≤ 4

iv) The union of circle

| λ − 1 | ≤ 2, | λ − 1 | ≤ 5, | λ + 1 | ≤ 2
i.e., union of | λ − 1 | ≤ 5

Required region is the shaded region in Fig. 2.

316
Block 2 Miscellaneous Examples and Exercises

Fig. 2

E10) We apply Gauss Seidel method and get

1
x ( k +1) = [17 − y ( k ) + 2z ( k ) ]
20
1
y ( k+1) = [−18 − 3x ( k+1) + z ( k ) ] (6)
20
1
z ( k+1) = [25 − 2 x ( k +1) + 3y ( k+1) ]
20
where k = 0,1, 2,...

We perform the following iterations

Iteration 1: Starting with x ( 0 ) = [0 0 0]T and putting k = 0 in

Eqn. (6), we get

1
x (1) = [17 − y ( 0) + 2z (0) ] = 0.85
20

1
y (1) = [−18 − 3x (1) + z ( 0) ] = −1.0275
20

1
z (1) = [25 − 2x (1) + 3y (1) ] = 1.0109
20

Iteration 2: Put k = 1 in Eqn. (6)

1
x ( 2) = [17 − y (1) + 2z (1) ] = 1.0025
20

1
y ( 2) = [−18 − 3x ( 2 ) + z (1) ] = −0.9998
20

1
z ( 2) = [25 − 2x ( 2) + 3y ( 2) ] = 0.9998
20
317
Block 2 Solution of Linear Algebraic Equations
Iteration 3: Put k = 2 in Eqn. (6)
1
x ( 3) = [17 − y ( 2) − 2z ( 2) ] = 1.0000
20
1
y ( 3) = [−18 − 3x (3) + z ( 2) ] = −1.0000
20
1
z ( 3) = [25 − 2 x (3) + 3y (3) ] = 1.0000
20

Iteration 4: Put k = 3 in Eqn. (6)

x (4) = 1
y ( 4 ) = −1

z ( 4) = 1

1 1 0.9
E11) Here A =   , and v (0) =  
− 3 5  1 

0.44 0.95
Iteration 1: Ay (1) = v (0)  y (1) =  , m1 = 0.46, v (1) =  
0.46  1 

0.47  0.97
Iteration 2: Ay ( 2 ) = v (1)  y ( 2 ) =  ,
 2 m = 0.48, v ( 2)
=  
0.48   1 

0.47  0.95
Iteration 3: Ay ( 3) = v ( 2)  y (3) =  , m 3 = 0.49, v ( 3) =  
0.49  1 

 y ( 3) 
The ratios  (r2)  for r = 1, 2 are 0.48, 0.49, respectively. Therefore,
 vr 
the maximum error is 0.01 . Hence, the smallest eigenvalue in
1
magnitude of the matrix A is µ = ≈ 2.08 and the corresponding
λ
T
eigenvector is [0.95 1] .

E12) We apply Gauss-Seidel method and get

−1
− 2 0 0  0 1 5
   
H = −(D + L) −1 U = −  4 −8 0  0 0 1
   
 4 − 1 − 2 0 0 0
 1 5
0 2 2
 
1 11
= 0 4 8

 
0 7 69 
 8 16 
318
Block 2 Miscellaneous Examples and Exercises

 1 
5
− λ 2 
2
 
1 11
Now, [H − λI ] =  0 4
−λ 8

 
 0 7 69
− λ
 8 16 

 73 1
H − λI = λ λ2 − λ −  = 0
 16 8

 73 1
 λ  λ2 − λ −  = 0
 16 8
 λ = 0, 4.59, − 0.027
 ρ(H ) = 4.59 > 1 (Spectral radius)

Hence, the method is divergent.

1   3
4 0 0 0 0 3 0 0 4
     
E13) H = −D −1 (L + U) = −  0 1 0 0 0 2 = −  0 0 2
3 3
     
0 0 1 3 2 0 3 2
0
 5   5 5 

Eigenvalues of H are given by the characteristics equation

 43 
H − λI = 0  λ λ2 −  = 0
 60 

43
 λ = 0, ±
60

43
and ρ( H) = ≈ 0.6466 (Spectral radius)
60

ν = − log10 (0.8466) = 0.0723. (Rate of convergence)

2
Number of Iterations = ≈ 28
v

Thus the rate of convergence of the method is 0.0723 and the

number of iterations required to achieve the error less than or
equal to 10 −2 is 2.

2 1 3 1 0 0
 
E14) [A | I] = − 1 −1 2 0 1 0
  1
 0 2 1 0 0 1 R 1 ← R 1
2

319
Block 2 Solution of Linear Algebraic Equations
 1 3 1 
1 2 2 2
0 0
 
≈ − 1 −1 2 0 1 0
 
0 2 1 0 0 1 R 2 ← R 1 + R 2
 
 1 3 1 
1 2 2 2
0 0
 
≈ 0 −1 2 0 1 0
2
 
0 1 1 0 0 1  R 2 ← −2 R 2
 
 1 3 1 
1 2 2 2 0 0
  1
≈ 0 1 −7 − 1 − 2 0 R 1 ← R 1 − 2 R 2
 
0 2 1 0 0 1 R 3 → R 3 − 2R 2
 
1 0 5 1 1 0
 
≈ 0 1 −7 − 1 − 2 0
  1
0 0 15 2 4 1 R 3 ← R 3
15
1 0 5 1 1 0
 
 
≈ 0 1 −7 −1 − 2 0  R 1 ← R 1 − 5R 3
 2 4 1
0 0 1
15 15 15  R 2 ← R 2 + 7 R 3
 1 
1 0 0 3
− 13 − 1
3
 
1 2 7
≈ 0 1 0 − 15 − 15 15
 = [I | A −1 ]
 
0 0 1 2 4 1 
 15 15 1 

9.25
 
E15) Iteration 1: y (1) = Av ( 0) =  9.4 
 
9.25

0.984
 
m1 = 9.4, and v (1) = 1 
 
0.984

7.9362 
 
Iteration 2: y ( 2) = Av (1) = 7.9202 
 
7.9202 

320
Block 2 Miscellaneous Examples and Exercises

 1 
 
m 2 = 7.965, and v ( 2) = 0.998
 
0.998

7.9859
 
Iteration 3: y (3) = Av ( 2) =  7.992 
 
7.9859
0.9992
 
m3 = 7.992 and v (3) =  1  .
 
0.9992

E16) The system of equation can be written as with pivoting

28x + 4 y − z = 32
2x + 17 y + 4z = 35
x + 3y + 10z = 24

i) Gauss-Seidel Method
1
x ( k +1) = (30 − 4 y ( k ) + z ( k ) )
28
1
y ( k+1) = (27 − 2x ( k +1) − 4z ( k ) ) (7)
17
1
z ( k+1) = (22 + x ( k +1) − 3y ( k +1) )
10
where k = 0,1, 2,...

Iteration 1: Put k = 0 and x ( 0) = [0 0] in Eqn. (7)

T
0
x (1) = 1.0714
y (1) = 1.4622
z (1) = 1.8685

Iteration 2: Put k = 1 in Eqn. (7)

x ( 2) = 0.9293
y ( 2 ) = 1.0393
z ( 2) = 1.9811

Iteration 3: Put k = 2 in Eqn. (7)

x (3) = 0.9937
y (3) = 1.0052
z (3) = 1.9978 321
Block 2 Solution of Linear Algebraic Equations
Iteration 4: Put k = 3 in Eqn. (7)

x ( 4) = 0.9992

y ( 4 ) = 1.0006

z ( 4) = 1.9997

ii) Gauss-Jacobi Method gives

1
x ( k +1) = (30 − 4 y ( k ) + z ( k ) )
28
1
y ( k+1) = (27 − 2x ( k ) − 4z ( k ) )
17
1
z ( k +1) = (22 + x ( k ) − 3y ( k ) ) (8)
10
where k = 0,1, 2,...

Iteration 1: Put k = 0 and x ( 0) = [0 0] in Eqn. (8)

T
0

x (1) = 1.0714

y (1) = 1.5882

z (1) = 2.2

Iteration 2: Put k = 1 in Eqn. (8)

x ( 2) = 0.9231

y ( 2 ) = 0.0489

z ( 2) = 2.0089

Iteration 3: Put k = 2 in Eqn. (8)

x ( 3) = 1.0019

y (3) = 1.5564

z (3) = 1.8987

Iteration 4: Put k = 3 in Eqn. (8),

x ( 4) = 0.9933

y ( 4 ) = 0.9977

z ( 4) = 1.9855

iii) Before we use SOR method, let us find the optimal value of
w. The iteration matrix for Jacobi method is
322
Block 2 Miscellaneous Examples and Exercises
H = −D −1 (L + U)
 1 
− 28 0 0 

0 4 − 1

= 0 1
− 17 0  2 0 4

  
 0 1 
 − 1 3 0 
0 − 10
 
 1 
 0 − 17 28 
 
= − 1 0 − 10
3

 3 
− 1 − 17 0 
 2 4 

ρ(H ) = max {1 − 0.27191, 1 − 0.05231, 10.32431}

= 0.3243 = µ (let)

The optimal value of relaxation factor is w o = 1.1667 .

However using w = 1.16 and starting with [0 0 0]T in SOR

method we get the following iterations:
T
Iteration 1: x (1) = [1.2429 1.6727 2.114]

Iteration 2: x ( 2) = [0.8544 0.8811 2.0062]

T

Iteration 3: x (3) = [1.0433 1.0114 2]

T

Iteration 4: x ( 4) = [0.9912 0.9994 1.9992]

T

1 0 0 1 1 2 
   
E17) L = 3 1 0, U = 0 −1 −5
   
1 −3 1 0 0 − 14

Ly = b  y1 = 2, y 2 = −7, y 3 = −22
−2 −6 11
Ux = y  x1 = , x2 = , x3 =
7 7 7

1 1 1 6
 
E18) [A | b] = 4 1 5 23
 
8 2 3 32 R 1 ↔ R 3
8 2 3 32
  1
≈ 4 1 5 23 R 1 ← R 2 − R 1
  2
1 1 1 
6 R ←R −1R
3 3 1
8 323
Block 2 Solution of Linear Algebraic Equations

 
8 2 3 32
 7

≈ 0 0 2
7
 
0 3 5
2  R2 ↔ R3
 4 8 
 
8 2 3 32
 3 5

≈ 0 4 8
2
 
0 0 7
7
 2 

We now use back substitution method and get

x 3 = 2, x 2 = 1, x1 = 3

324