Contents

1 General informationon numerical analysis and scientific calculation 3

1.1 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Floating-point arithmetic and rounding errors . . . . . . . . . . . . . . 3
1.2.1 Representation of numbers in the machine . . . . . . . . . . . . 3
1.2.2 Roundoff error . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Matrix Analysis 7
2.1 Vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.1 Matrix operations . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 Invertible matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.3 The transpose of a matrix . . . . . . . . . . . . . . . . . . . . . 13
2.2.4 The trace of a square matrix . . . . . . . . . . . . . . . . . . . . 13
2.2.5 The determinant . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.6 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . 15

3 Direct Methods for Solving Linear System 19

3.1 Triangular Systems of Equations (Back Substitution) . . . . . . . . . . 21
3.2 Gauss Elimination method . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 The matrix form of Gaussian Elimination: The LU factorization . . . . 25

4 Iterative Methods for Solving Linear Systems 27

4.1 Generality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.2 Description of the Methods of Jacobi, Gauss-Seidel, and Relaxation . . 30
4.3 Jacobi’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.4 Gauss-Seidel method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.5 Relaxation methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.6 Notes on the implementation of iterative methods . . . . . . . . . . . . 35
4.7 Convergence of the Methods of Jacobi and Gauss-Seidel . . . . . . . . . 36

1
2 CONTENTS
Chapter 1

General informationon numerical

analysis and scientific calculation

1.1 Motivations

Numerical analysis is a branch of applied mathematics concerned with the develop-

ment of numerical tools and methods for the calculation of approximations of solutions
to mathematical problems that would be difficult, if not impossible, to obtain by ana-
lytical means.
Its objective is in particular to introduce detailed computational procedures that
can be implemented by computers (electronic, mechanical or human) and to analyse
their characteristics and performance.

1.2 Floating-point arithmetic and rounding errors

1.2.1 Representation of numbers in the machine

Reminder on scientific notation

1. 1. in basis 10
In base 10, the scientific notation of a real takes the following form:

s × 10e ,

with:

• e an integer, called exponent,

• s a real, such that 1 ≤ |s| < 10 or s = 0, called Significant.

3
4CHAPTER 1. GENERAL INFORMATIONON NUMERICAL ANALYSIS AND SCIENTIFIC CALCULA

The significant corresponds to the significant number (and the sign), while the
exponent corresponds to the order of magnitude.
Example 1.1.
369 = 3 × 102 + 6 × 101 + 9 × 100 .

2. In basis 2
In base 2, the scientific notation of a real takes the following form:

s × 2e

with:
• e an integer, called exponent,
• s a real, such that 1 ≤ |s| < 2 or s = 0, called Significant.
Example 1.2.

10012 = 1 × 23 + 0 × 22 + 0 × 21 + 1 × 20 .

Floating point numbers

Definition 1.1. A floating-point is a way of representing and performing arithmetic
operations on real numbers in computing. It’s a numerical data type that allows you to
represent a very large value (e.g., 1.23×108 8) or a very small value (e.g., 1.23×10−88 ).
It could also represent very large negative number (e.g., −1.23 × 108 8) and very small
negative number (e.g., −1.2310−88 ).
The term ”floating-point” refers to the fact that the decimal point can ”float” or be
positioned anywhere within the number, enabling the representation of both very large
and very small numbers.
It consists of representing a number by:

sign × mantissa × 2e ,

with
• the sign, positive or negative;
• mantissa (or significance), a positive real such that 1 ≤ mantisse < 2;
• exponent e, an integer.
Over the years, a variety of floating-point representations have been used in com-
puters. In 1985, the IEEE 754 Standard for Floating-Point Arithmetic was established,
and since the 1990s, the most commonly encountered representations are those defined
by the IEEE
IEEE 754 has 3 basic components (s, e, m) :
1.2. FLOATING-POINT ARITHMETIC AND ROUNDING ERRORS 5

s (sign) e (exponent) m (mantissa)

1bit w bits t bits

Types w t Total
Single precision 8 bits 23 bits 32 bits
Double precision 11 bits 52 bits 64 bits

Example 1.3.

1.2.2 Roundoff error

In computing, a roundoff error, also called rounding error is the difference between
the result produced by a given algorithm using exact arithmetic and the result produced
by the same algorithm using rounded arithmetic.
in short, there are two major facets of roundoff errors involved in numerical calcu-
lations:
1. Data measurement is not accurate (due to instruments);

2. The use of approximate values instead of exact values (e.g. 3.14 instead of π)
Definition 1.2. Suppose x∗ is an approximation to the number x. The absolute
error in this approximation noted E is given by

E = |x − x∗ |.

Example 1.4.
Value Representation Approximate value Absoluted Error
1 ¯
0,142857142857.. = 0,142857 0,142857 1
× 10−6
7 7
π 3,141592653589793.. 3,141592 0,653589793.. × 10−6
e 2,718281828459045.. 2,718281 0,828459045.. × 10−6
Example 1.5. If x = 100 and x∗ = 100.1 the absolute error is 0.1, whereas if x = 1
and x∗ = 1.1 the absolute error is still 0.1. Note that if x∗ is an approximation to x,
then x∗ is an equally good approximation to a with respect to the absolute error.
For some purposes the absolute error may be what we want, but in many cases it is
reasonable to say that the error is smaller in the first example above than in the second,
since the numbers involved are bigger. The relative error takes this into account
Definition 1.3. Suppose x∗ is an approximation to the number x. The relative
error in this approximation noted Er is given by
|x − x∗ | E
Er = = .
|x| |x|
6CHAPTER 1. GENERAL INFORMATIONON NUMERICAL ANALYSIS AND SCIENTIFIC CALCULA

We note that the relative error is obtained by scaling the absolute error with the
size of the number that is approximated. If we compute the relative errors in the
approximations above we find that it is 0.001 when x = 100 and x∗ = 100.1, while
when x = 1 and x∗ = 1.1 the relative error is 0.1. In contrast to the absolute error,
we see that the relative error tells us that the approximation in the first case is much
better than in the latter case.
Chapter 2

Matrix Analysis

2.1 Vector spaces

Definition 2.1. A vector space E over a field K or K-vector space consists of a set of
E (elements of E are called vectors), a field K (elements of K are scalars) and the two
operations:

• Vector addition of E × E in E :

E×E →E
(u, v) → u + v

• Scalar Multiplication of K × E in E :

K×E →E
(λ, u) 7→ λ · u

Where both the operations must satisfy the following condition

1. Commutativity: u + v = v + u, for all u, v ∈ E.

2. Associativity: u + (v + w) = (u + v) + w, for all u, v, w ∈ E.

3. Additive identity: There exists an element 0 ∈ E such that 0 + u = u for all

u ∈ E.

4. Additive inverse: For every u ∈ E, there exists an element u′ ∈ E such that

u + u′ = 0.

5. Multiplicative identity: 1 · u = u, for all u ∈ E.

7
8 CHAPTER 2. MATRIX ANALYSIS

6. Distributivity: λ · (u + v) = λ · u + λ · v and (λ + µ) · u = λ · u + µ · u, for all

u,v ∈ E and λ, µ ∈ K.

7. λ · (µ · u) = (λµ) · u, for all λ, µ ∈ K, u ∈ E.

Example 2.1. R2 is an R-vector space.

2.2 Matrices
Definition 2.2. A real n × m matrix A is a rectangular array

a11 a12 · · · a1m

a21 a22 · · · a2m
A= .. .. . . ..
. . . .
an1 an2 · · · anm
Or (ai,j ) 1≤i≤n
1≤j≤m

More generally, we may take the entries aij from an abstract field K, but for now
we suppose K = R, the real numbers.
The diagonal entries are the aij with i = j, i.e. the aii .
The set of real n × m matrices is denoted

Mnm (R) or Mn×m (R) or Rn×m .

In the case that m = n we also write Mn (R), and we call a matrix in Mn (R) a
square matrix.
Example 2.2. For example, M3 (R) is the set of all real 3 × 3 matrices, and M5,7 (2Z)
is the set of all 5 × 7 matrices with even integer entries.

2.2.1 Matrix operations

• Equality between matrix: Let A and B matrices of Mn,p (K), and i = 1 · · · n, j =
1 · · · p. A is equal to B if ∀i, ∀j, aij = bij .

• Addition:
Definition 2.3. Two or more matrices of the same order can be added by adding
the corresponding elements of the matrices. If A and B are two matrices with the
same dimension n × m, then the addition of matrices A and B is C = A + B of
dimension n × m defined by
cij = aij + bij .

In other words, we sum coefficients by coefficients.

2.2. MATRICES 9
Å ã Å ã Å ã
3 −2 0 5 3 3
Example 2.3. If A = and B = then A + B = .
1 7 2 −1 3 6
Å ã
′ −2
On the other hand, if B = then A + B ′ is not defined.
8
Proposition 2.1. Let A, B et C three matrices in Mn,p (K). Let α ∈ K and
β ∈ K two scalars.
1. Commutative Property of matrix addition : A + B = B + A.
2. Associative Property of matrix addition: A+(B+C) = (A+B)+C.
3. Additive Identity of matrix addition: A + 0 = A.
4. (α + β)A = αA + βA,
5. α(A + B) = αA + βB.

• Multiplication: The primary condition for the multiplication of two matrices

is the number of columns in the first matrix should be equal to the number of
rows in the second matrix, and hence the order of the matrix is important.
Definition 2.4. Let A = (aij ) a matrix n × p and B = (bij ) a matrix p × q.
Then, the multiplication of matrices A and B is the matrix C = AB n × q, where
it’s coefficients cij are defined by:
p
X
cij = aik bkj .
k=1
Ñ é
Å ã 0 1
1 2 3
Example 2.4. If A = and B = 2 1
4 5 0
3 4
Å ã Å ã
1×0+2×3+3×3 1×1+2×1+3×4 15 15
then A × B = = .
4×0+5×2+0×3 4×1+5×1+0×4 10 9
Remark 2.1.

1. The multiplication of matrices does not follow commutative property,

AB ̸= BA.
Example 2.5.
Å ãÅ ã Å ã Å ãÅ ã Å ã
5 1 2 0 14 3 2 0 5 1 10 2
= but =
3 −2 4 3 −2 −6 4 3 3 −2 29 −2

2. AB = 0 does not imply A = 0 or B = 0. It may happen that the product

of two non-zero matrices is zero. In other words, we can have A ̸= 0 and
B ̸= 0 but AB = 0.
10 CHAPTER 2. MATRIX ANALYSIS

Example 2.6.
Å ã Å ã Å ã
0 −1 2 −3 0 0
A= B= and AB =
0 5 0 0 0 0

3. AB = AC does not imply B = C. We can have AB = AC et B ̸= C.

Example 2.7.
Å ã Å ã Å ã Å ã
0 −1 4 −1 2 5 −5 −4
A= B= C= and AB = AC =
0 3 5 4 5 4 15 12
Proposition 2.2.

1. Associative Property of matrix addition: A(BC) = (AB)C.

2. Distributivity Property of matrix multiplication with respect to
addition: A(B + C) = AB + AC et (B + C)A = BA + CA.
3. A × 0 = 0 et 0 × A = 0.

2.2.2 Invertible matrix

Definition 2.5. Let A a square matrix of dimension n × n. If there exist a square
matrix B of dimension n × n such that
AB = I et BA = I,
A is said to be invertible matrix. We call B the inverse of A and we denote it A−1 .
Å ã
1 2
Example 2.8. Let A = . To study whether A is invertible is to study the
0 3
Å ã
a b
existence of a matrix B = such that AB = I and BA = I.
c d
We have
Å ãÅ ã Å ã Å ã Å ã
1 2 a b 1 0 a + 2c b + 2d 1 0
AB = I ⇐⇒ = ⇐⇒ =
0 3 c d 0 1 3c 3d 0 1
This equality is equivalent to the system:
 

 a + 2c = 1 
 a=1
b + 2d = 0 b = − 32
 
⇐⇒

 3c = 0 
 c=0
3d = 1 d = 13
 

1 − 32
Å ã
Then B = . To prove that it is suitable, it is necessary also show BA = I
0 13
1 − 32
Å ã
−1
equality. The matrix A is therefore invertible and A = .
0 13
2.2. MATRICES 11

Proposition 2.3. If A is invertible, then its inverse is unique.

Proposition 2.4. Let A be an invertible matrix. Then A−1 is also invertible and we
have:
(A−1 )−1 = A.
Proposition 2.5. Let A and B be two invertible matrices of the same dimension. Then
AB is invertible and
(AB)−1 = B −1 A−1 .

Matrix 2 × 2
Å ã
a b
To calculate the inverse of a 2 × 2 matrix , consider the matrix A = .
c d
Proposition 2.6. If ad − bc ̸= 0, then A is invertible and
Å ã
−1 1 d −b
A =
ad − bc −c a

Gauss-Jordn Matrix Inversion Method

We will demonstrate how to derive the inverse of a matrix using the Gauss-Jordan
(or reduced row) elimination method. Finding the inverse of a matrix implies several
prerequisites, including the matrix being invertible. To be invertible, a matrix must
be a square matrix and its determinant must be non-zero. A matrix with a zero
determinant is considered singular, which means it is not invertible.
Before we begin, it’s important to note that a matrix can represent a system of
linear equations solvable using row elementary operations. These operations preserve
the set of solutions represented by the matrix. There are three types of row elementary
operations:
1. Li ← λLi with λ ̸= 0: Multiplying each element of a row by a non-zero real.
2. Li ← Li + λLj with λ ∈ K(et j ̸= i): Replacing a row with the sum of itself and
a multiple of another row.
3. Li ← Lj : Swapping two rows of the matrix.
Ñ é
1 2 1
Example 2.9. Let’s calculate the inverse of A = 4 0 −1 .
−1 2 2
We start with: Ñ é
1 2 1 1 0 0
(A | I) = 4 0 −1 0 1 0
−1 2 2 0 0 1
12 CHAPTER 2. MATRIX ANALYSIS

To make 0s appear on the first column, on the second line, we will use the elementary
operation L2 ← L2 − 4L1 which leads to the augmented matrix:
Ñ é
1 2 1 1 0 0
0 −8 −5 −4 1 0 L2 ←L2 −4L1
−1 2 2 0 0 1

Then a 0 on the first column, on the third line with L3 ← L3 + L1 :

Ñ é
1 2 1 1 0 0
0 −8 −5 −4 1 0
0 4 3 1 0 1 L3 ←L3 +L1

Divide the second row L2 by −8, this give us

Ñ é
1 2 1 1 0 0
1
0 1 58 12 − 18 0 L2 ← − L2
8
0 4 3 1 0 1

We continue in order to make 0s appear everywhere below the diagonal:

Ñ é
1 2 1 1 0 0
0 1 58 12 − 81 0
0 0 12 −1 21 1 L3 ←L3 −4L2

1 0 − 14 0 1
Ñ é
4
0
5 1 1
0 1 8 2
−8 0
0 0 12 −1 12 1
Then
1 0 − 41 0 1
Ñ é
4
0
0 1 58 1
2
− 1
8
0
0 0 1 −2 1 2 L3 ←2L3

We will do the the elementary operation L1 ← L1 + 41 L3 and L2 ← L2 − 58 L3 :

1 0 0 − 12 21 1
Ñ é
2
0 1 0 4 − 4 − 54
7 3
.
0 0 1 −2 1 2

Thus the inverse of A is the matrix obtained on the right

Ñ 1 1 1
é
−2 2 2
A−1 = 7
4
− 34 − 54
−2 1 2
2.2. MATRICES 13

2.2.3 The transpose of a matrix

Definition 2.6. If A = (aij ) be an m × n matrix, then the matrix obtained by inter-
changing the rows and columns of A would be the transpose of A. It is denoted by A′
or AT . In other words, if
á ë
a11 a12 . . . a1p
a21 a22 . . . a2p
A= .. .. .. ,
. . .
an1 an2 . . . anp

then á ë
a11 a21 . . . an1
a12 a22 . . . an2
AT = .. .. .. .
. . .
a1p a2p . . . anp

Example 2.10.
Ñ éT Ñ é
1 2 3 1 4 −7
4 5 −6 = 2 5 8
−7 8 9 3 −6 9
Ñ éT
0 3 Å ã
0 1 −1
1 −5 =
3 −5 2
−1 2
Ñ é

T 1
1 −2 5 = −2
5

Theorem 2.1. 1. (A + B)T = AT + B T .

2. (αA)T = αAT .

3. (AB)T = B T AT .

4. If A is invertible, then AT is also invertible and we have (AT )−1 = (A−1 )T .

2.2.4 The trace of a square matrix

Definition 2.7. The trace of a square matrix A, denoted tr(A), is the sum of the
elements on its main diagonal (from the upper left to the lower right). It is only
defined for a square matrix (n × n). In other words,

tr A = a11 + a22 + · · · + ann .

14 CHAPTER 2. MATRIX ANALYSIS
Å ã
2 1
Example 2.11. • If A= , then tr A = 2 + 5 = 7.
0 5
Ñ é
1 1 2
• For B = 5 2 8 , tr B = 1 + 2 − 10 = −7.
11 0 −10

Theorem 2.2. Let A et B two matrices n × n. Then:

1. tr(A + B) = tr A + tr B.

2. tr(αA) = α tr A for all α ∈ K.

3. tr AT = tr A.

4. tr(AB) = tr(BA).

2.2.5 The determinant

Minor of a couple (i,j) of a determinant is the determinant obtained by deleting its

ith row and jth column in which element aij lies. Minor of an element (i,j) is denoted
by Mi,j .
Cofactor of a couple (i,j), denoted by Ci,j is defined by Ci,j = (–1)i+j Mi,j .

Theorem 2.3. Let A = (ai,j )1≤i,j≤n be a square matrix of order n, (Ci,j )1≤i,j≤n its
cofactors.

• By computing the expansion along the i − th row:

n
X
det(A) = ai,j Ci,j .
j=1

• By computing the expansion along the j − th column:

n
X
det(A) = ai,j Ci,j
i=1

Example 2.12. Let us compute (again) the determinant of the following matrix
Ñ é
1 −1 2
A= 6 3 1 .
4 5 3
2.2. MATRICES 15

Expanding cofactors along the first column, we find that

1 −1 2
3 1 −1 2 −1 2
6 3 1 = 1 × (−1)1+1 × + 6 × (−1)2+1 × + 4 × (−1)3+1 ×
5 3 5 3 3 1
4 5 3
= 4 + 6 × 13 + 4 × (−7)
= 54
Determinant properties: ∀A ∈ Mn (K)
1. det(AB) = det(BA) = det(A)det(B).
2. If A is invertible, det(A−1 ) = 1
det(A)
.

3. det(AT ) = det(A).
4. det(αA)n = αn det(A).
5. detI n = 1.
6. If two columns are identical on A then detA = 0.
7. If a column of A is a linear combination of the other columns then detA = 0.

2.2.6 Eigenvalues and Eigenvectors

Definition 2.8. Let A ∈ Mn (K).
• λ is said to be the eigenvalue of the matrix A if there is a non-zero vector X ∈ Kn
such that
AX = λX.

• The vector X is then called the eigenvector of A associated with the eigenvalue
λ.
Example 2.13. Let A ∈ M3 (R) be the matrix
Ñ é
1 3 3
A= −2 11 −2
8 −7 6
Ñ é
−1
• X1 = 0 is the eigenvector of A. Indeed,
1
Ñ éÑ é Ñ é Ñ é
1 3 3 −1 2 −1
AX1 = −2 11 −2 0 = 0 = −2 0 = −2X1 .
8 −7 6 1 −2 1
16 CHAPTER 2. MATRIX ANALYSIS

So X1 is an eigenvector of A associated with the eigenvalue λ = −2.

Theorem 2.4. Let A be a n × n matrix. Let λ1 ,λ2 , ...,λr be eigenvalues of A and let
X1 , X2 , ..., Xr be an eigenvector associated with λi (for 1 ≤ i ≤ r). If the eigenvalues
λi are distinct, then the eigenvectors Xi are linearly independent.
Proposition 2.7. Let A ∈ Mn (K) and λ ∈ K. Then:
λ is an Eigenvalue of A ⇐⇒ det (A − λIn ) = 0.
Definition 2.9. Let A ∈ Mn (K). The characteristic polynomial of A is
χA (X) = det (A − XIn ) .
The proposition 2.7 then becomes :
λ Eigenvalue of A ⇐⇒ χA (λ) = 0.

Procedure: Finding Eigenvalues and Eigenvectors

Let A be an n × n matrix.
1. First, find the eigenvalues λ of A by solving the equation χA (λ) = 0.
2. For each λ, find the basic eigenvectors X ̸= 0 by finding the basic solutions to
(A − λI)X = 0.
To verify your work, make sure that AX = λX for each λ and associated eigenvector
X.
Example 2.14. Ñ é
1 3 3
A= −2 11 −2
8 −7 6
Therefore
χA (X) = det (A − λI3 )
ÑÑ é Ñ éé
1 3 3 1 0 0
= det −2 11 −2 −λ 0 1 0
8 −7 6 0 0 1
1−λ 3 3
= −2 11 − λ −2
8 −7 6−λ
= ···
= −λ3 + 18λ2 − 51λ − 182
= −(λ + 2)(λ − 7)(λ − 13).
Then, the eigenvalues are: −2, 7 and 13.
2.2. MATRICES 17

Proposition 2.8. If a matrix A ∈ Mn (K) admits n eigenvalues then

• The sum of the eigenvalues is tr(A);

• The product of the eigenvalues is det(A).

Definition 2.10. Let A be an n × n matrix with eigenvalues λ1 , ..., λn . Then the

spectral radius ρ(A) of A is
ρ(A) = max |λi |.

Example 2.15. The spectral radius of matrix A of example 2.14 is:

ρ(A) = max |λi | = max {| − 2|,|7|,|13|} = 13.

18 CHAPTER 2. MATRIX ANALYSIS
Chapter 3

Direct Methods for Solving Linear

System

In this chapter, we call for direct methods because in the absence of rounding
errors they would finally give the exact solution after a finite number of elementary
operations. This methods are used only if the number of equations is not too high
(n ≤ 100).

Definition 3.1. Direct techniques are methods that theoretically give the exact so-
lution to the system in a finite number of steps.

So, we will consider this methods for solving a linear system of n equations in n
variables. Such a system has the form:


 a11 x1 + a12 x2 + · · · + a1n xn = b1
 a21 x1 + a22 x2 + · · · + a2n xn = b2

.. (1)


 .
 a x + a x + ··· + a x = b
n1 1 n2 2 nn n n

where we are given the constants aij , for each i, j = 1, 2, ..., n, and bi , for each
i = 1, 2, ..., n.
The linear system (1) can be rewritten as the following matrix-vector form:

AX = B, (2)

where coefficient matrix A = (aij ) ∈ Mn (R) and B the vector of components

(b1 , b2 , . . . , bn ) and X the unknown vector of components (x1 , x2 , . . . , xn ).

Theorem 3.1. A given square system, say, Ax = B gives a very unique result for
every column matrix B. This is possible if and only if the determinant of A is not
equal to zero, i.e., detA ̸= 0.

19
20 CHAPTER 3. DIRECT METHODS FOR SOLVING LINEAR SYSTEM

To solve this kind of system, the first idea that comes to mind is the use of Cramer’s
formulas given by:
det Ai
xi = , for i = 1, . . . ,n.
det A
Or Ai is the square matrix formed by replacing the k − th column of A by the column
vector B.
As a result, for the resolution of the system (2) by these method, a total of
TCramer = (n + 1)2 n! − 1 elementary operations, for example for n = 5, we have
TCramer = 4319 and for n = 10, we have TCramer ∽ 4 × 108 .
In this case, it becomes very slow in execution time (of calculations) as soon as n
exceeds 4.
Example 3.1. Solve the given system of equations using Cramer’s Rule.

 x + 2y + 3z = −5
3x + y − 3z = 4
−3x + 4y + 7z = −7


Solution: So, in order to solve the given equation, we will make four matrices.
These matrices will help in getting the values of x, y, and z .
Ñ é
1 2 3
A= 3 1 −3
−3 4 7
Then
Ñ é Ñ é Ñ é
−5 2 3 1 −5 3 1 2 −5
A1 4 1 −3 , A2 3 4 −3 , A3 3 1 4
−7 4 7 −3 −7 7 −3 4 −7
Then,
det A1 −40
x= = = −1,
det A 40
det A2 40
y= = = 1,
det A 40
det A3 −80
z= = = −2.
det A 40
Therefore,
(x,y,z) = (−1,1, − 2).
Another method consists in calculating the inverse matrix of A, that is to say:
A−1 AX = A−1 B then X = A−1 B.
To do this, we use a total of T Inverse = n! (n2 + n + 1) + 3n2 − n elementary op-
erations, for example for n = 5, we have TInverse = 3790 and for n = 10, we have
TInverse ∽ 4.108 .
This method is therefore no more advantageous than the first.
3.1. TRIANGULAR SYSTEMS OF EQUATIONS (BACK SUBSTITUTION) 21

3.1 Triangular Systems of Equations (Back Substi-

tution)

If the matrix A of the system (2) is triangular, the resolution is trivial.

1. If A is upper triangular, the system (2) can be written in the following form:

 a11 x1 + a12 x2 + . . . + ain xi + . . . a1n xn = b1



 a22 x2 + . . . + ai2 xi + . . . a2n xn = b2

 ... .. .. .. ..
. . . .

 aii xi + ... ain xn = bi
.. .. ..

 ..
.



 . . .

ann xn = bn

and since ni=1 aii = det(A) ̸= 0, the system is solved by first computing xn in
Q
the last equation and then xn−1 and so on, thus leading to
®
xn = Äbn /ann ä
xi = bi − nj=i+1 aij xj a1ii
P

2. If A is lower triangular, the system (2) can be written in the following form:

 a11 x1 = b1



 a21 x1 + a22 x2 = b2

 .. ... .. ..
. . .

 ai1 x1 + ai2 x2 + . . . + aii xi = bi
.. .. .. .. ..

 ..
.



 . . . . .

an1 x1 + an2 x2 + . . . + ani xi + . . . ann xn = bn

in this case the resolution algorithm becomes:

®
x1 = Äb1 /a11 ä
Pi−1
xi = bi − j=1 aij xj a1ii

Example 3.2. Use back substitution to solve the linear system



 4x1 −x2 +2x3 +3x4 = 20
2x2 +7x3 −4x4 = −7


 6x3 +5x4 = −4
3x4 = −6.

22 CHAPTER 3. DIRECT METHODS FOR SOLVING LINEAR SYSTEM

Solving for x4 in the last equation yields

6
= 2.
x4 =
3
Using x2 in the third equation, we obtain

6 − 5(2)
x3 = = −1.
6
Now x3 = −1 and x4 = 2 are used to find x2 in the second equation:

7 − 7(−1) + 4(2)
x2 = = −4.
2
Finally, x1 is obtained using the first equation:

20 + 1(−4) − 2(−1) − 3(2)

x1 = = 3.
4
Therefore,
(x1 ,x2 ,x3 ,x4 ) = (3, − 4, − 1,2).

3.2 Gauss Elimination method

This method is based on the transformation of the linear system Ax = B into an

equivalent system A′ x = B ′ , A′ is an upper triangular matrix.
This method can be summarized in the following two steps for solving a system of
linear equations:

1. Transforming the given system into a “triangular system” equivalent to the orig-
inal system, i.e,
transformation  (n) 
[A, B] −→ A ; B (n) , A(n) Upper triangular matrix,
 (1) (1) (1) (1)
  (n) (n) (n) (n)

a11 a12 . . . . . . .a1n b1 a11 a12 . . . . . . .a1n b1
 (1) (1) (1) (1) (n) (n) (n)
 a21 a22 . . . . . . .a2n b2  0 a22 . . . . . . .a2n b2
  
 −→ 
 
......... .........
 
   
(1) (1) (1) (1) (n) (n)
an1 an1 . . . . . . ..ann bn 0 0 . . . . . . ann bn

2. Solve the resulting trigonometric system A(n) x = B (n) by the “back substitution”
method of which x is the exact solution of the system Ax = B.

The steps: We pose

A = A(1) ; B = B (1) .
 3.2. GAUSS ELIMINATION METHOD 23

Step1: Converting the given system into a trigonometric system.

(1)
If the pivot element a11 ̸= 0, we do the following operation

 L(2)
1 = L1
(1)
(1)
ai1
 L(2) (1)
i = Li − (1) L1
(1)
a11

(1)
(If a11 = 0 it changed the order of the equations, that is, replace the first equation
(1)
with another equation so that it becomes a11 ̸= 0).
We then obtain:
 (2) (2) (2) (2)

a11 a12 . . . . . . .a1n b1
(2) (2) (2)
0 a22 . . . . . . .a2n b2
î ó  
(2) (2)
A ;B =
 
0 .........

 
(2) (2) (2)
0 an1 . . . . . . ann bn

And so on:

 L(k+1)
i
(k)
= Li , i = 1, . . . ,k (k)
At the k-th step aik
(k) ; akk ̸= 0
 L(k+1)
i
(k)
= Li −
(k)
(k) Lk , i = k + 1, . . . ,n
akk

Finally, we get a trigonometric system:


(n) (n) (n) (n)
 a11 x1 + a12 x2 + . . . . . . . . . + a1n xn = b1

(n) (n) (n)
0 + a22 xn + . . . . . . . . . + a2n xn = b2
 (n) (n)
0 + 0 + . . . . . . . . . . . . . . . . + ann xn = bn


Step2: : Solve the resulting trigonometric system by back-substitution method. So we

get the values of the variables in their inverse order.
Remark 3.1. Gaussian elimination requires 2/3n3 additions and multiplications and
n2 /2 divisions. Thus, if n = 10 Gaussian elimination requires 700 operations (to be
compared with Cramer formulas).
Example 3.3. Using the Gaussian Elimination method, find the solution to the fol-
lowing set of linear coefficients:

 x1 + 3x2 + 3x3 = 0
2x + 3x2 = 0
 1
3x1 + 2x2 + 6x3 = 11
We pose Ñ é Ñ é
1 3 3 0
A = A(1) = 2 3 0 ; B = B (1) = 0
3 2 6 11
 24 CHAPTER 3. DIRECT METHODS FOR SOLVING LINEAR SYSTEM

Step1: Converting the given system into a trigonometric system.

(1)
Since a11 ̸= 0, we do the following operation
 (2) (1)

 L1 = L1
 (1)
 (2) (1) a21 (1) (1) (1)
L2 = L2 − (1) L = L2 − 2L1
a11 1
 (1)
a31 (1)
 L(2) (1) (1) (1)

3 = L3 − (1) L1 = L3 − 3L1

a11

Then:  
î ó 1 3 3 ;0
A(2) ; B (2) =  0 −3 −6 ; 0 
0 −7 −3 ; 11
(2)
Since a22 ̸= 0 ,

(3) (2)
 L1(3) = L1(2)


L2 = L2
(2)
a32
 L(3) (2) (2) (2) (2)
L2 = L3 − 73 L1

3 = L3 −

(2)
a22

We obtain,  
î ó 1 3 3 ;0
A(3) ; B (3) =  0 −3 −6 ; 0 
0 0 11 ; 11

Therefore,  
î ó 1 3 3 ;0
A(3) ; B (3) =  0 −3 −6 ; 0 
0 0 11 ; 11

Step2: : Solve the resulting trigonometric

 system by back-substitution
 method. 
 x1 + 3x2 + 3x3 = 0  x1 = −3x2 − 3x3  x1 = 3
Resolution of A(3) x = B (3) ⇒ 0 − 3x2 − 6x3 = 0 ⇒ x2 = −2x3 ⇒ x = −2
 2
0 + 0 + 11x3 = 11 x3 = 1 x3 = 1
 
Ñ é
3
Then, the solution is X = −2 .
1
3.3. THE MATRIX FORM OF GAUSSIAN ELIMINATION: THE LU FACTORIZATION25

3.3 The matrix form of Gaussian Elimination: The

LU factorization

In linear algebra, LU decomposition (also called LU factorization) factorizes a ma-

trix as the product of a lower triangular matrix and an upper triangular matrix. The
LU decomposition can be viewed as the matrix form of Gaussian Elimination.
We assume that the system Ax = B admits an unique solution and that A is a
square matrix. An LU decomposition is a decomposition of the form A = LU , with

1. L Lower triangular matrix,

2. U Upper triangular matrix.

To be able to facilitate the resolution of Ax = B in two simple steps

ß
Ly = B
(3)
Ux = y

The matrix U is Ü method (A(n) x = B (n) ), we pose U = A(n) .

obtained by the Gaussê
1 0 0
α21 1 0 0 a
(k)
The matrix L = ; such that αik = ik (k)
1 akk
αn1 αnn−1 1
Remark 3.2. 1. The LU decomposition does not always exist on A even if it is
invertible.

Theorem 3.2. A necessary and sufficient condition for an invertible matrix A

to decompose in the form A = LU is that

det Ak ̸= 0, for all k = 1, . . . ,n − 1,

with Ak = (a)ki,j=1 , i.e, the minors are non-zero..

(k)
2. If αkk ̸= 0 the Gaussian method is applicable and therefore A is factored in the
form LU .

Example 3.4. Resolve the following system by LU decomposition

Ñ é Ñ é
1 3 3 0
A= 2 3 0 B= 0
3 2 6 11
26 CHAPTER 3. DIRECT METHODS FOR SOLVING LINEAR SYSTEM

Solution:

Determine the matrix U = A(n) by the Gauss method according to Example 3.3, we
get: Ñ é
1 3 3
U = A(3) = 0 −3 −6
0 0 11
The matrix L is defined by
Ñ é
1 0 0 (k)
aik
L= α21 1 0 αik = (k)
α31 α32 1 akk

Then Ñ é
1 0 0
L= 2 1 0
3 73 1
the resolution of Ax = B in two simple steps
Ñ éÑ é Ñ é  
1 0 0 y1 0  y1 = 0  y1 = 0
Ly = B ⇒ 2 1 0 y2 = 0 ⇒ 2y1 + y2 = 0 ⇒ y =0
 2
3 73 1 y3 11 7
3y1 + 3 y2 + y3 = 11 y3 = 11


Ñ éÑ é Ñ é  
1 3 3 x1 y1  x1 + 3x2 + 3x3 = 0  x1 = 3
Ux = y ⇒ 0 −3 −6 x2 = y2 ⇒ −3x2 − 6x3 = 0 ⇒ x = −2
 2
0 0 11 x3 y3 11x3 = 11 x3 = 1

Chapter 4

Iterative Methods for Solving

Linear Systems

In this chapter, we consider the linear system:

Ax = b, (4)
or A ∈ Mn (K) is invertible. When n is very large (n ≥ 100) the resolution of the system
(4) for the direct methods always become complicated. In this case, we will use another
class of methods for solving linear systems consists in approximating solutions using
iterative methods. These methods can be very efficient in terms of computational
run times and memory usage.
The basic idea is this: Given the linear system (4), find another matrix G and a
vector c, such that
1. The matrix I − G is invertible.
2. The unique solution x of the system (4) is identical to the unique solution x of
the system
x = Gx + c,
0 k
and then, starting from any vector x , compute the sequence x given by
k+1 k
x = G x +c, k ∈ N.
k
Under certain conditions (to be clarified soon), the sequence (x) converges to a
limit x which is the unique solution of x = Gx + c, and thus of the system (4).
In this chapter we will discuss three iterative methods for solving linear systems:
1. Jacobi’s method
2. Gauss-Seidel’s method
3. The relaxation method.

27
28 CHAPTER 4. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS

4.1 Generality
Definition 4.1. A square matrix A is said to be strictly diagonally dominant if
n
X
∀i ∈ [1, n], |aii | > |aj | .
j=1,j̸=i

Example 4.1. •
é
|a11 | > |a12 | + |a13 | |7| > |2| + |0|
Ñ
7 2 0 
3 5 −1 |a22 | > |a21 | + |a23 | |5| > |3| + | − 1|
0 5 −6

|a33 | > |a31 | + |a32 | | − 6| > |0| + |5|


It is therefore a diagonally dominant matrix.

•
é
|a11 | < |a12 | + |a13 | |20| < |12| + |25|
Ñ
20 12 25 
12 −18 2 |a22 | > |a21 | + |a23 | | − 18| > |12| + |2|
1 2 5

|a33 | > |a31 | + |a32 | |5| > |1| + |2|


This is not a diagonally dominant matrix.

Definition 4.2. A symmetric matrix A is a matrix that is symmetric along the diag-
onal, which means:
AT = A.

Example 4.2.
Ñ é Ñ é
20 12 5 20 12 5
A= 12 15 2 , AT = 12 15 2
5 2 25 5 2 25

Definition 4.3. Let A ∈ Mn (R). The leading Principal Minors of matrix A are the
determinants of the leading principal submatrices. For a 3×3 matrix, these submatrices
are obtained by progressively considering the top-left 1 × 1, 2 × 2, and 3 × 3 blocks of
the original matrix.

Example 4.3. Ñ é
2 0 1
A= 0 −1 1
1 0 −2
4.1. GENERALITY 29

The leading Principal Minors ∆i=1,2,3 are calculated as follows:

∆1 = 2
2 0
∆2 = = (2) × (−1) − (0) × (0) = −2
0 −1
2 0 1
−1 1 0 −1
∆3 = 0 −1 1 =2 +1 =4+1=5
0 −2 1 0
1 0 −2

Theorem 4.1. Let A ∈ Mn (R). A is positive definite if and only if all leading principal
minors are strictly positive, i.e., ∆i > 0 for i = 1,...,n.

Example 4.4. Ñ é
20 12 5
A= 15 15 2
5 2 25
A is positive definite.

Matrix Norms:

Definition 4.4. The norm of a square matrix A is a non-negative real number denoted
∥A∥. There are several different ways of defining a matrix norm, but they all share the
following properties:

1. ∥A∥ ≥ 0 for any square matrix A.

2. ∥A∥ = 0 if and only if the matrix A = 0.

3. ∥kA∥ = |k|∥A∥, for any scalar k.

4. ∥A + B∥ ≤ ∥A∥ + ∥B∥.

5. ∥AB∥ ≤ ∥A∥∥B∥.

The 1−norm: It is the maximum absolute column sum. Put simply, we sum the
absolute values down each column and then take the biggest answer, i.e.,
n
X
∥A∥1 = max |aij | .
j=1...n
i=1

Example 4.5. Calculate the 1-norm of

Ñ é
20 12 5
A= 12 15 2
5 2 25
30 CHAPTER 4. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS

n
X
∥A∥1 = max |aij |
j=1...n
i=1
= max {|20| + |12| + |5|,|12| + |15| + |2|,|5| + |2| + |25|}
= max {37,29,32} = 37.

The infinity-norm: It is the maximum absolute row sum. Put simply, we sum
the absolute values along each row and then take the biggest answer, i.e.,
n
X
∥A∥∞ = max |aij | .
i=1...n
i=1

Example 4.6. Calculate the infinity-norm of

Ñ é
2 0 1
B= 0 −1 1
1 0 −2

n
X
∥B∥∞ = max |aij |
i=1...n
j=1

= max {|2| + |0| + |1|,|0| + | − 1| + |1|,|2| + |0| + | − 2|}

= max {3,2,3} = 3.

4.2 Description of the Methods of Jacobi, Gauss-

Seidel, and Relaxation
The methods described in this chapter are instances of the following scheme: Given
the linear system (4), suppose we can write A in the form

A = M − N,

with M invertible, and “easy to invert,” which means that M is close to being a
diagonal or a triangular matrix.
Then, the system (4) is equivalent to

M x = N x + b.

that is,
x = M −1 N x + M −1 b.
4.2. DESCRIPTION OF THE METHODS OF JACOBI, GAUSS-SEIDEL, AND RELAXATION31

Therefore, we are in the situation described in the previous sections with

G = M −1 N and c = M −1 b.

In fact, since A = M − N , we have

G = M −1 N = M −1 (M − A) = I − M −1 A,

which shows that I − G = M −1 A is invertible.

The iterative method associated with the matrix G = M −1 N is given by

k+1 k
x = M −1 N x +M −1 b, k ≥ 0,

0
starting from any arbitrary vector x.
To describe the various choices of M and N , it is convenient to write A in terms
of three submatrices D, E, F ,as

A = D − E − F,

where

• The only nonzero entries in D are the diagonal entries in A:

Ö è
a11 0 ···
D= .. .
.
··· 0 ann

• The only nonzero entries in −E are entries in A below the diagonal:

á ë
0
a21 0 0
−E = .. .. .
. .
an1 · · · an,n−1 0

• The only nonzero entries in −F are entries in A above the diagonal:

á ë
0 a12 · · · a1n
.. ..
−F = . . .
0 an−1,n
0
32 CHAPTER 4. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS

4.3 Jacobi’s method

In Jacobi’s method, we pick
M = D and N = E + F.
So that
G = M −1 N = D−1 (E + F ) = I − D−1 .
As a matter of notation, we let
GJ = I − D−1 = D−1 (E + F ),
which is called Jacobi’s matrix .
k
The corresponding method, Jacobi’s iterative method, computes the sequence (x)
using the recurrence
k+1 k
x = D−1 (E + F ) x +D−1 b.
That is,
n
k+1 X aij k bi
xi = − xj + pour i = 1, . . . , n
j=1
aii aii
j̸=i
k
The Jacobi’s method consists in constructing the sequence (x)k∈N of Mn,1 (R). As-
suming 0
x∈ Mn,1 (R)
Å ã
k+1 1
Pn aij k
xi = aii bi − j=1 aii xj
j̸=i

Example 4.7.
Ñ é Ñ é Ñ é
2 −1 0 1 0
0
A= −1 4 −1 b= 1 with x= 0
0 −1 2 1 0
We use the Jacobi’s method
Ü ê
n
k+1 1 X k
xi = bi − aij xj ; i = 1, . . . , n.
aii j=1
i̸=j

The system is written

  k+1  k

k+1 k k 1
1 x1 = 2 1+ x2
 x1 = a11 (b1 − (a12 x2 +a13 x3 ))

 

 
  
k+1 k k k+1 k k
x2 = a122 (b2 − (a21 x1 +a23 x3 )) ⇒ x2 = 41 1+ x1 + x3
 
 k+1 k k
 
 k+1 k
 
x3 = a133 (b3 − (a31 x1 +a32 x2 )) 1
x3 = 2 1+ x2
 
4.4. GAUSS-SEIDEL METHOD 33
Ñ é
0
0
From x= 0 we get:
0

k 0 1 2 3 4
x1 0 0,5 0,625 0,75 0,78125
x2 0 0,25 0,5 0,5625 0,625
x3 0 0,5 0,625 0,75 0,78125

4.4 Gauss-Seidel method

In Gauss-Seidel’s method, we pick M = D − E et N = F .
So that
GGS = M −1 N = (D − E)−1 F.
GGS is called Gauss-Seidel’s matrix .
The corresponding method, Gauss-Seidel’s iterative method, computes the sequence
k
(x) using the recurrence

k+1 k
x = (D − E)−1 F x +(D − E)−1 b.

That is,
i−1 n
k+1 1 X k+1 1 X k bi
xi = − aij xj − aij xj + .
aii j=1 aii j=i+1 aii

k
This method consists in constructing the sequence (x)k∈N of Mn,1 (R). Assuming
(0
x∈ Mn,1 (R)
k+1 k+1 k
x i = − a1ii i−1
P 1
Pn bi
j=1 aij xj − aii j=i+1 aij xj + aii

Example 4.8.
Ñ é Ñ é Ñ é
2 −1 0 1 0
0
A= −1 4 −1 b= 1 avec x= 0
0 −1 2 1 0

We use the Gauss-Seidel’s method,

n n
!
k 1 X k+1 X k
xi = bi − aij xj − aij xj ; i = 1, . . . , n
aii j<i j>i
34 CHAPTER 4. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS

The system is written

  k+1  k

k+1 k k 1
1 x1 = 2 1+ x2
 x1 = a11 (b1 − (a12 x2 +a13 x3 ))

 

 
  
k+1 k+1 k k+1 k+1 k
x2 = 1
(b
a22 2
− (a21 x1 +a23 x3 )) ⇒ x2 = 41 1+ x1 + x3
 k+1 k+1 k+1
  
 k+1 k+1
 1

 x3 = (b − (a31 x1 +a32 x2 )) 1
x3 = 2 1+ x2
 
a33 3

Ñ é
0
0
From x= 0 we get:
0

k 0 1 2 3 4
x1 0 0,5 0,6875 0,796875 0,82421875
x2 0 0,375 0,59375 0,6484375 0,66210938
x3 0 0,6875 0,796875 0,82421875 0,83105469

Remark 4.1.

• One of the advantages of the method of Gauss-Seidel is that is requires only half
of the memory used by Jacobi’s method, since we only need
k+1
uk+1 k k
1 , . . . , ui−1 , ui+1 , . . . , un .

to compute uk+1
i .

• We also show that in certain important cases (for example, if A is a triangular

matrix), the method of Gauss-Seidel converges faster than Jacobi’s method (in
this case, they both converge or diverge simultaneously).

4.5 Relaxation methods

For this method, we take the following decomposition
Å ã
D
M= −E ,
ω
1−ω
Å ã
N= D + F with ω ∈ (0,2).
ω

The number ω is called the Parameter of Relaxation.

So, the Relaxation’s matrix is defined by
ã−1 Å
1−ω
Å ã
−1 D
GR = M N= −E D+F .
ω ω
4.6. NOTES ON THE IMPLEMENTATION OF ITERATIVE METHODS 35

The corresponding method, Relaxation’s iterative method, computes the sequence

k
(x) using the recurrence
ã−1 Å ã−1
1−ω
Å ã Å
k+1 D k D
x= −E D+F x+ −E b.
ω ω ω
k k+1
When the vector x is known, these methods are determined xi by
k+1 k k+1∗
xi = (1 − ω) xi +ω xi , (5)
k+1∗
where xi is determined by a Jacobi or Gauss-Seidel method as follows:
• With the Jacobi method, we obtain
Ü ê
n
k+1 ω X k k
xi = bi − aij xj + (1 − ω) xi .
aii j=1
j̸=i

This method its called the Over-Relaxation method (or JOR).

• With the Gauss-Seidel method, we obtain

i−1 n
!
k+1 ω X k+1 X k k
xi = − aij xj + aij xj −bi + (1 − ω) xi .
aii j=1 j=i+1

This method its called the Successive Over-Relaxation method (or SOR).
In particular,
1. If 0 < ω < 1, the method is called Under-Relaxation.

2. If 1 < ω < 2, the method is called Over-Relaxation.

3. If ω = 1, we find the Gauss-Seidel or Jacobi method.

4.6 Notes on the implementation of iterative meth-

ods
n
The basic idea of iterative methods is to construct a sequence of vectors x that
enjoy the property of convergence
n
x = lim x .
n→∞
36 CHAPTER 4. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS

where x is the solution to (4). In practice, the iterative process is stopped at the
minimum value of n such that
n
∥ x −x∥ < ε,
where ε is a fixed tolerance and ∥.∥ is any convenient vector norm.
However, since x must verify Ax = b, so ∥Ax − b∥ = 0, we can also consider the
stopping test as:
n
1. ∥A x −b∥ < ε.
n n−1
∥x− x ∥
2. k < ε.
∥x∥

n n−1
3. ∥ x − x ∥ < ε.
In practice, we add also a stop condition on the number of iterations for avoid too
many calculations.

4.7 Convergence of the Methods of Jacobi and Gauss-

Seidel
All the methods seen in the previous section define a sequence of iterates of the form
n+1 n
x = G x +c
in which G is a certain iteration matrix. If the above iteration converges, its limit
x satisfies
x = Gx + c.
In the case where the above iteration arises from the splitting A = M − N , it is
easy to see that the solution x to the above system is identical to that of the original
system Ax = b.
Theorem 4.2 (Necessary and sufficient condition).
The following statements are equivalent:
1. The Iterative method converges;
2. The spectral radius of G is less than one: ρ(G) < 1;
3. For at least one subordinate norm matrix standard, we have: ∥G∥ < 1.
Theorem 4.3. If A is a strictly diagonally dominant matrix, then the associated Jacobi
and Gauss-Seidel iterations converge for any x0 .
Proposition 4.1 (Sufficient condition).
If A is a symmetric and positive definite. Then, the Iterative method converge.
4.7. CONVERGENCE OF THE METHODS OF JACOBI AND GAUSS-SEIDEL 37

Example 4.9. Study of the convergence of the Jacobi and Gauss-Seidel methods in the
case where the matrix of the linear system is:

1. Ñ é
−4 1 0
A1 = 1 −4 1 ;
0 1 −4
the two methods converge.

2. Ñ é
1 2 −2
A2 = 1 1 1 .
2 2 1
the Jacobi method converges and the Gauss-Seidel method diverges.