Matrices

Rza Mustafayev

Khazar University, Baku, Azerbaijan

Rza Mustafayev Khazar University 1 / 53

 Matrices were for the first time introduced in the XIXth , but were previously used as arrays for the
resolution of linear systems.

Matrices are widely employed in a large variety of domains:

In physics, such as for the study of electrical circuits, quantum mechanics, optics ...

In computer science, such as to code and encrypt messages, manage databases ...

In chemistry, for example to balance chemical reactions ...

In economics for optimization of problems such capital, manufacturing ...

Here we will introduce the basic principles of matrices ...

Rza Mustafayev Khazar University 2 / 53

 Introduction to Matrices

Rza Mustafayev Khazar University 3 / 53

Definitions
A matrix is a two-dimensional array of numbers or expressions arranged in a set of rows and columns.
Let m, n ∈ N. A matrix A is said to be of size m × n if the array contains m rows and n columns, and we write A
is a m × n matrix. It should be noted that rows come always first!
A m × n matrix A is represented as follows
 a11 ··· a1n   a11 ··· a1n 
   
 . .. .  . .. .
   
A =  . . , A =  . . , A = (aij )1≤i ≤m,1≤j ≤n , or simply A = (aij )
 
 . . .   . . . 
am1 ··· amn am1 ··· amn

where aij are entries or coefficients of the matrix A. Each entry has two subscripts. The first one stands for the
row number and the second one stands for the column number.
If m = 1, then the matrix is defined to be a row matrix or a row vector, i.e.,
A = (a1 , a2 , . . . , an ).

If n = 1, then the matrix is defined to be a column matrix or a column vector, i.e.,

 a1 
 
 a2 
A =  .  .
 
 . 
 . 
am

If m = n, then the matrix A will be called as a square matrix. In this case, a11 , a22 , . . . , ann are called as the
main diagonal entries of the matrix A. We denote
diag(A) = (a11 , a22 , . . . , ann ).

Rza Mustafayev Khazar University 4 / 53

Examples:

!
3 (−2)
1) A = is a 2 × 2 matrix.
1 7
!
3 2 (−2)
2) A = is a 2 × 3 matrix.
1 0 7
 
3
3) A = 1 is a 3 × 1 matrix or a column vector.
 
4
 
4) A = 3 1 4 is a 1 × 3 matrix or a row vector.

Rza Mustafayev Khazar University 5 / 53

Definitions
A square matrix of size n × n with entries (aij ), where

1, if i = j

,

aij = 
0,

if i , j
is called the identity matrix of size n × n and denoted by In . That is,

1 0 ··· 0

 
0 1 ··· 0

.  .

In =  . . ..

 . . . 
 . . . . 
0 0 ··· 1

We denote by Omn the so-called null matrix (or zero matrix) of size m × n containing only zero entries:

0 ··· 0

 
 . . 
..

Omn =  . . .  .
 . . 
0 ··· 0

Let K be any of C, R, Q, or Z. Then, we denote by Mmn (K) the set of all matrices of size m × n with entries in K
(By Mn (K), we denote the set of all square n × n matrices with entries in K).

Rza Mustafayev Khazar University 6 / 53

Examples:

!
3 2
1) A = ∈ M22 (N).
1 7
!
3 2 (−2)
2) A = ∈ M23 (Z).
1 0 7

(1/2)
 
3) A = (3/2) ∈ M31 (Q).
 
2
 
4) A = π (1/3) e ∈ M13 (R).

Rza Mustafayev Khazar University 7 / 53

 Elementary Matrix Arithmetic

Rza Mustafayev Khazar University 8 / 53

Definition (Equality of Matrices): Two matrices A = (aij ) and B = (bij ) are said to be equal (and written as A = B)
if and only if they have same size and the corresponding entries are equal, i.e.,
aij = bij for all i , j .

So,
 a11 ... a1n  b ... b1n 
   
 .   11
.. .   . .. .

 . . = . . ⇐⇒ aij = bij for 1 ≤ i ≤ m, 1 ≤ j ≤ n.

 . . .   . . . 

am1 ... amn bm1 ... bmn
Matrices which are not equal are called as different.

Thus, matrices of different sizes are always different.

Rza Mustafayev Khazar University 9 / 53

Definition (Addition of Matrices): The sum of two matrices A = (aij ) and B = (bij ) of the same sizes is written as
A + B and is the matrix A + B = (aij + bij ), i.e., the matrix obtained by adding the corresponding entries of A and B.

So,
 a11 ··· a1n  b ··· b1n  a + b11 ··· a1n + b1n 
     
 .   11   11
.. .   . .. . . .. .

 . .  +  .. .  =  . .  .
  
 . . .   . .   . . . 
am1 ··· amn bm1 ··· bmn am1 + bm1 ··· amn + bmn

Note: Matrices of different sizes cannot be added.

Matrix subtraction is similarly defined: The difference of two matrices A = (aij ) and B = (bij ) of the same sizes is
written as A − B and is the matrix A − B = (aij − bij ).

Hence,
 a11 ··· a1n  b ··· b1n  a − b11 ··· a1n − b1n 
     
 .   11   11
.. .   . .. . . .. .

 . .  −  .. .  =  . .  .
  
 . . .   . .   . . . 

am1 ··· amn bm1 ··· bmn am1 − bm1 ··· amn − bmn

! !
3 (−2) 0 5
Example: Let A = ,B= , compute A + B and A − B.
1 7 2 (−1)

Rza Mustafayev Khazar University 10 / 53

Proposition: Let A and B be two matrices of the same size. Then
A + B = B + A,
that is, matrix addition is commutative.

Proposition: Let A, B, and C be three matrices of the same size. Then

A + (B + C) = (A + B) + C,
that is, matrix addition is associative.

Proposition: Let A be any matrix. Then for the null matrix of the size same as of A
A + O = A.

Rza Mustafayev Khazar University 11 / 53

Definition (Scalar Multiplication): The product of any m × n matrix A = (aij ) by any scalar α is written as αA and is
the m × n matrix αA = (αaij ), i.e., the matrix whose entries obtained by multiplying each entry of A by α.

Thus,
 a11 ... a1n  αa ... α a1n 
   
 .   11
.. .   . .. . 

α  . . .  =  .. . .  .
 . .   . 
am1 ... amn α am1 ... α amn

! !
3 2 1 (−1) 1 (−2)
Exercise: Let A = ,B= , α = −2, and β = 3. Compute αA + βB.
1 0 1 (−2) 1 (−3)

Proposition: Let A be any matrix. Then

1 A = A.

Proposition: Let α, β ∈ K and A, B ∈ Mmn (K). Then

(α + β)A = αA + βA
α(A + B) = αA + αB
α(βA) = (αβ)A.

Rza Mustafayev Khazar University 12 / 53

Definition (Negative of a Matrix): The negation of any m × n matrix A = (aij ) is formed by negating each entry of
the matrix, that is,
−A = −1 A.

So,
 a11 ... a1n  −a ... −a1n 
   
 .   11
.. .   . .. . 

−  . . .  =  . . .  .
 . .   . . 
am1 ... amn −am1 ... −amn

Proposition: Let A be any matrix. Then

A + (−A) = O.
Here O is the null matrix of the size same as of A.

Rza Mustafayev Khazar University 13 / 53

Definition (Matrix Multiplication): Let A = (aij ) ∈ Mmn (K) and B = (bjk ) ∈ Mnp (K), where m, n, p ∈ N. The
product C = AB is defined to be the m × p matrix (cik ) with entries
n
X
cik = ai1 b1k + ai2 b2k + · · · + aij bjk + · · · + ain bnk = aij bjk for 1 ≤ i ≤ m, 1 ≤ k ≤ p .
j =1

That is, the entry cik of the product is obtained by multiplying term-by-term the entries of the ith row of A and the k th
column of B, and summing the n products. In other words, cik is the dot product of the ith row of A and the k th
column of B

Hence,

 a11 a12 ··· a1n  b11 b12 ··· b1p 

   
 a21 a22 ··· a2n  b b22 ··· b2p 
 
  21
 . . .. .  ·  . . .. .
 
 .. . .   . . .

 . . .   . . . . 

am1 am2 ··· amn bn1 bn2 ··· bnp
 a11 b11 + · · · + a1n bn1 a11 b12 + · · · + a1n bn2 ··· a11 b1p + · · · + a1n bnp 
 
 a21 b11 + · · · + a2n bn1 a21 b12 + · · · + a2n bn2 ··· a21 b1p + · · · + a2n bnp 

. . .  .
 
= 
. . .

 . . ··· . 

am1 b11 + · · · + amn bn1 am1 b12 + · · · + amn bn2 ··· am1 b1p + · · · + amn bnp

The product AB is defined if and only if the number of columns in A equals the number of rows in B.

Rza Mustafayev Khazar University 14 / 53

Rza Mustafayev Khazar University 15 / 53
Proposition: Let α ∈ K, A ∈ Mmn (K), B ∈ Mnp (K), where m, n, p ∈ N. Then
(αA) B = α (A B) = A(α B).

Proposition: Let A ∈ Mmn (K), B, C ∈ Mnp (K), where m, n, p ∈ N. Then

A (B + C) = A B + A C.

Proposition: Let A, B ∈ Mmn (K), C ∈ Mnp (K), where m, n, p ∈ N. Then

(A + B) C = A C + B C.

Remark: From previous two propositions we see that matrix multiplication is distributive with respect to matrix
addition.

Remark: Matrix multiplication is not commutative, in general, i.e.,

A B , B A.

Rza Mustafayev Khazar University 16 / 53

Proposition: Let A ∈ Mmn (K), B ∈ Mnp (K), and C ∈ Mpq (K), where m, n, p , q ∈ N. Then
A (B C) = (A B) C.
In other words, matrix multiplication is associative.

Proof: Let A = (aij ), B = (bjk ), and C = (ckl ). Since the entry at (i , k ) of the matrix AB is
n
X
aij bjk ,
j =1

then the entry at (i , l ) of the matrix (AB)C is

p
X n
X !
aij bjk ckl .
k =1 j =1

The entry at (j , l ) of the matrix BC is

p
X
bjk ckl ,
k =1

then the entry at (i , l ) of the matrix A(BC) is

n
X p
X !
aij bjk ckl .
j =1 k =1

Interchanging the sums, we arrive at

n
X p
X ! X p
n X p X
X n p
X n
X !
aij bjk ckl = aij bjk ckl = aij bjk ckl = aij bjk ckl .
j =1 k =1 j =1 k =1 k =1 j =1 k =1 j =1

Thus, A (B C) = (A B) C. □

Rza Mustafayev Khazar University 17 / 53

Exercise:  
! 2 1 0 
0 2 (−2) 
1) Let A = , B = 0 1 0 . Compute A B.
6 (−4) 0
(−2) (−3)
 
2
! !
5 1 2 0
2) Let A = ,B= . Compute A B and B A.
3 (−2) 4 3

Proposition: Let A be any matrix. Then

OA = O and A O = O.

Remark: In contrary to the scalar case, A B = O does not imply that either A = O or B = O and A B = A C does not
necessarily imply that B = C.

! ! !
0 (−10 4 (−1) 2 5
Exercise: Let A = ,B= , and C = . Compute A B and A C.
0 3 5 4 5 4

Proposition: Let A be any square matrix of size n × n. Then

A In = A and In A = A.

Rza Mustafayev Khazar University 18 / 53

Definition (Power of a Matrix): Let A = (aij ) be a square matrix of n × n size. We defined the successive powers of A
as follows:
A0 = In and Ap +1 = Ap A for every p ∈ N.

Proposition: Let n ∈ N. Then

(In )p = In for p = 0, 1, 2, . . .
(On )p = On for p = 1, 2, . . .

Proposition: Let A, B ∈ Mn (N). Then

AB = BA =⇒ An B = B An for all n ≥ 2.

Proof: We are going to prove by induction. Let A, B ∈ Mn (N) such that A B = B A. Then

A2 B = A A B = A B A = B A A = B A 2 .
Assume that An B = B An for some n ∈ N such that n ≥ 3. Let us show that An+1 B = B An+1 . Indeed:
An+1 B = An A B = An B A = B An A = B An+1 .
The proof is completed. □

The previous statement can be generalized as follows:

Proposition: Let A, B ∈ Mn (N). Then

AB = BA =⇒ An Bk = Bk An for all n, k ∈ N.

Rza Mustafayev Khazar University 19 / 53

  
1 0 1
(−1) 0. Compute Ap for p ∈ N.

Example: Let A = 0
 
0 0 2

Solution: Note that

3 1 0 22 − 1
      
1 0 11 0 1 1 0
A2 = 0 (−1) (−1) (−1)2 0  ,
     
00 0 = 0 1 0 = 0
22
       
0 0 2 0 0 2 0 0 4 0 0
7 1 0 23 − 1
      
1 0 3 1 0 1 1 0
A3 = 0 (−1) (−1) (−1)3 0  ,
     
1 0 0
 0 = 0 0 = 0
23
      
0 0 4 0 0 2 0 0 8 0 0
15 1 0 24 − 1
      
1 0 71 0 1 1 0
A4 = 0 (−1) (−1) (−1)4 0  .
     
00
 0 = 0 1 0  = 0
24
      
0 0 8 0 0 2 0 0 16 0 0

Let us show by induction principle that

2p − 1
 
1 0
A p = 0 (−1)p

0 
2p
 
0 0
for each p ∈ N.
It is true when p = 1, 2, 3, 4. Suppose at p, and show it at p + 1:

2p − 1 1 1 0 2p +1 − 1

    
1 0 1 0
A p +1 = A p A = 0 (−1)p (−1) (−1)p +1  .
   
0  0
 0 = 0 0
2p 2 p +1
   
0 0 0 0 2 0 0
The proof is completed.

Rza Mustafayev Khazar University 20 / 53

Theorem (Binomial formula): Let A, B ∈ Mn (N) such that A B = B A. Then, for every p ∈ N, we have
p !
X p
(A + B)p = Ap −k Bk . (1)
k =0
k

Proof: We prove (1) by induction. It is true when p = 1. Assuming that (1) holds for p = m, we prove it for p = m + 1.
(A + B)m+1 = (A + B)m (A + B)
m ! ! m ! ! m ! !
X m m−k k X m m−k k X m m−k k
= A B (A + B) = A B A+ A B B
k =0
k k =0
k k =0
k
m ! m ! m ! m +1 !
X m X m X m X m
= Am−k +1 Bk + Am−k Bk +1 = Am−k +1 Bk + Am−k +1 Bk
k =0
k k =0
k k =0
k k =1
k −1
m ! m !
X m X m
= A m +1 + A m − k +1 B k + Am−k +1 Bk + Bm+1
k =1
k k =1
k −1
m " ! !#
X m m
= A m +1 + + Am+1−k Bk + Bm+1
k =1
k k −1
m ! m +1 !
X m+1 X m + 1 m+1−k k
= A m +1 + Am+1−k Bk + Bm+1 = A B .
k =1
k k =0
k

The proof is completed. □

Remark: Note that if A B , B A, then

(A + B)2 = A2 + A B + B A + B2 .
Rza Mustafayev Khazar University 21 / 53
    
2 0 0
 0 0 0

Example: Let A = 0 2 0 and B = 2 0 0. Compute
   
0 0 2 3 1 0
1 Ap and Bp for p ≥ 0.
2 Show that A B = B A.
3 Compute (A + B)p for p ≥ 0.

p
Solution: 1) Observe that A = 2I3 . So, Ap = (2I3 )p = 2p I3 = 2p I3 . Note that
    
0 0 0 0 0 0 0 0 0
B2 = 2 0 ,
   
0 0 2 0 0 = 0 0
     
3 1 0 3 1 0 2 0 0
    
0 0 0 0 0 0 0 0 0
B3 = 0 0 = O3 .
   
0 0 2 0 0 = 0 0
     
2 0 0 3 1 0 0 0 0

So, Bp = O3 for p ≥ 3.
2) Since A B = (2I3 ) B = 2B and B A = B (2I3 ) = 2B, we see that A B = B A.
3) We can use the binomial formula. Hence,
p ! ! ! !
X p p p p p −1 p p −2 2
(A + B)p = Ap −k Bk = A I3 + A B+ A B
k =0
k 0 1 2

p (p − 1) p −2
= 2p I3 + p 2p −1 I3 B + 2 I3 B 2
2
p (p − 1) p −2 2
= 2p I3 + p 2p −1 B + 2 B
2

Rza Mustafayev Khazar University 22 / 53

Definition (Transposition of a Matrix): The transpose of an m × n matrix A = (aij ) is the n × m matrix AT (read as A
transpose), which has the first row of A as its first column, second row of A as its second column, and so on, that is
AT = (aji ). Thus,

 a11 a12 ··· a1n  a11 a21 ··· am1 

   
 a21 a22 ··· a2n  a12 a22 ··· am2 
 
 .
 
the transpose of A =  . . .. . is A =  .
T
. .. .
 
 . . .
  . . .
 . . . . 
  . . . . 

am1 am2 ··· amn a1n a2n ··· amn
In particular, transposition converts row vectors to column vectors and vice versa.

Obviously, (AT )T = A for any matrix A. Observe that (αA)T = αAT for any scalar α and matrix A.

Exercise: Find the transpose matrix of

   
 1 0 0   1 3  
0  , 4 , 3 .
 
A =  1 4 B =  0 C= 1 2
(−2) (−2) (−3) (−3)
   
0

Proposition: If A and B are two matrices of the same size, then

(A + B)T = AT + BT .

Rza Mustafayev Khazar University 23 / 53

Proposition: Let A ∈ Mmn (K), B ∈ Mnp (K), where m, n, p ∈ N. Then

(A B)T = BT AT .

Proof: Let A = (ai j ) and B = (bjk ). Recall that the entry at (i , k ) of the matrix AB is
n
X
aij bjk .
j =1

So, the entry at (i , k ) of the matrix (A B)T is

n
X
akj bji .
j =1

On the other hand, the entry at (i , k ) of the matrix BT AT is the dot product of the ith row of BT with the k th column of
AT , which are the ith column of B
b1i 
 
b2i 
 
 . 
 
 .. 
 
bni
with the k th row of A  
ak 1 ak 2 ··· akn

, respectively. Thus, the entry at (i , k ) of the matrix BT AT is

n
X
bji akj .
j =1

Thus, (A B)T = BT AT . □

Rza Mustafayev Khazar University 24 / 53

Transposition gives a rise to useful classes of matrices as follows:

Definition (Symmetric and Skew-symmetric Matrices): Let A be a square matrix.

A is said to be Symmetric matrix if AT = A.
A is said to be skew-symmetric matrix if AT = −A.

Remark: Obviously, the main diagonal entries of a skew-symmetric matrix are zeros.

(−2)
  
(−5)

3 2   0 1 
(−5) is symmetric while B =  5 (−3) is skew-symmetric.
 
Example: A =  3 8 0
(−5) (−1)
   
2 1 3 0

Remark: Any real square matrix A may be written as the sum of a symmetric matrix and a skew-symmetric matrix.
More precisely, we write
1  1 
A= A + AT + A − AT .
2 2
| {z } | {z }
symmetric skew-symmetric

    
(−1) 0

3 3 2  3 4 2  0
(−5) = 4 (−5) + 1 0 , where the first and the second
    
Example: Note that 5 4 4 0
(−5) (−5)
     
2 4 2 4 0 0 0
matrices on the right-hand side are symmetric and skew-symmetric, respectively.

Rza Mustafayev Khazar University 25 / 53

Definition (Orthogonal Matrices): Let A be a square matrix. A is said to be orthogonal matrix is A AT = I.

Example: Show that the following matrices are orthogonal:

!
1 1
1 A= √1
2 1 (−1)

√ !
1 3 1
√
2 B= 2
1 (− 3)


(−2)

2 1 
1 1 
3 C= 3  2 2 
(−2)

2 1

Rza Mustafayev Khazar University 26 / 53

Definition (Upper and Lower Triangular Matrices): Let A = (aij ) be a square matrix.
If aij = 0 for i > j, then A is called as upper triangular matrix.
If aij = 0 for i < j, then A is called as lower triangular matrix.

Figure: Upper and lower triangular matrices, respectively. All the entries in the red triangles are zero.

Rza Mustafayev Khazar University 27 / 53

Definition (Trace of a Matrix): Let A = (aij ) be a square matrix of n × n size. We call the trace of A the sum of its
main diagonal entries:
tr(A) := a11 + a22 + · · · + ann .

 
 1 0 0 

Exercise: The trace of A =  1 4 0  is tr(A) = 1 + 4 − 3 = 2.
−2 −2 −3
 

Proposition: Let A, B ∈ Mn (K) and α ∈ K. Then

tr(A + B) = tr(A) + tr(B)
tr(αA) = αtr(A)
tr(AT ) = tr(A)
tr(AB) = tr(BA)

Proof: Let us prove the last property. Assume that A = (aij ) and B = (bij ). Since
n
X n
X n
X ! n
X n
X n
X !
diag(AB) = a1k bk 1 , a2k bk 2 , . . . , ank bkn and diag(BA) = b1i ai1 , b2i ai2 , . . . , bni ain ,
k =1 k =1 k =1 i =1 i =1 i =1

then, interchanging the sums, we have

n X
X n n X
X n
tr(AB) = aik bki = bki aik = tr(BA).
i =1 k =1 k =1 i =1

The proof of the last property is completed. □

Rza Mustafayev Khazar University 28 / 53

 Inverse of a Matrix

Rza Mustafayev Khazar University 29 / 53

Definition (Inverse of a Matrix): Let A be a square n × n matrix. If there exists a square matrix B of size n × n such
that
A B = In ,
then we say that B is a right inverse of A .

Similarly, if there exists a square matrix B of size n × n such that

B A = In ,
then we say that B is a left inverse of A.

Not all matrices have inverses in this way, but it is an interesting fact that the left and right inverses of a square matrix
A, if they exist, are the same.

Proposition: Let A be a square n × n matrix. Suppose that B is either a right or a left inverse of A. Then B is a
two-sided inverse, i.e.
A B = B A = I,
and this inverse is unique, so if C satisfies either C A = In or A C = In , then C = B.

Notation: We denote this unique inverse of a square matrix A when it exists by A−1 . If A−1 exists, we say A is
invertible. The subset of invertible matrices of Mn (K) is denoted by GLn (K).

The inverse of a square matrix does not always exist. If it is the case, we say that the matrix is singular.

Rza Mustafayev Khazar University 30 / 53

Proposition: If a matrix A is invertible, then A−1 is also invertible and
(A−1 )−1 = A.

Proposition: If A and B are invertible square matrices of the same size, then A B is also invertible and
(A B)−1 = B−1 A−1 .

Proof: Using the associativity law,

(A B) (B−1 A−1 ) = A (B (B−1 A−1 )) = A ((B B−1 ) A−1 ) = A (I A−1 ) = A A−1 = I.
Hence B−1 A−1 is a right inverse of A B, and therefore is the unique inverse of A B. □

Proposition (Simplification by an invertible Matrix): Let A, B ∈ Mn (K) and C ∈ GLn (K). Then
AC = BC =⇒ A = B.

Proposition: If A is invertible, then so is AT , and

−1
(AT ) = (A−1 )T .

Proof: Since
(A−1 )T (AT ) = (A A−1 )T ) = IT = I,
−1
then (A−1 )T is a left inverse of (AT ). So, (A−1 )T is the inverse of (AT ), that is, (A−1 )T = (AT ) . □

Rza Mustafayev Khazar University 31 / 53

Finding the inverse of a matrix (Direct method): A direct method to find the inverse of a n × n matrix A is to
determine the entries of the n × n matrix B that verifies
A B = B A = In .

Exercise:

1) Study the invertibility of the following matrices :

! !
1 2 3 0
A= , B= .
0 3 5 0

2) Is the square unity matrix In invertible ? If yes, determine its inverse ?

3) Is the square null matrix On invertible ?

Remark: The direct method is mainly feasible for n ≤ 3.

Rza Mustafayev Khazar University 32 / 53

Finding the inverse of a matrix (Determinant method for 2 × 2 matrices):
!
a b
Proposition: Let A = a 2 × 2 matrix. If ad − bc , 0, then A is invertible and
c d
!
1 d (−b )
A−1 = ,
ad − bc (−c ) a

where ad − bc is called the determinant of A.

Exercise:

If possible, compute the inverse of the following matrices:

! ! ! !
3 1 2 (−3) 0 2 a+1 1
A= ,B= ,C= ,D= with a ∈ R.
7 2 (−5) 4 3 0 2 a

Rza Mustafayev Khazar University 33 / 53

 Elementary Matrices

Rza Mustafayev Khazar University 34 / 53

Definition: The elementary operations on the lines Li of A are:
Multiply any line by a nonzero number: Li ← λLi with λ , 0
Replace any line with itself added to a multiple of another line: Li ← Li + λLj with λ , 0 and i , j
Interchange the lines: Li ↔ Lj (Swap operation).

Thus, if
a
 11 a12 ··· a1n 
 . . .. .

 . . .

 . . . .


 ai1 ai2 ··· ain 

 . . . 
..

A =  . . . .  ,
 . . . 
 a
 j1 aj2 ··· ajn 
 . . . 
..

 . . . 
 . . . . 
am1 am2 ··· amn
then
a
 11 a12 ··· a1n 
 . . .. .

 . . .

 . . . .


λai1 λai2 ··· λain 

 . . .. . 

ALi ←λLi =  . . . .  .
 . . . 
 a
 j1 aj2 ··· ajn 
 . . . 
..

 . . . 
 . . . . 
am1 am2 ··· amn

Rza Mustafayev Khazar University 35 / 53

Similarly,
 a
 11 a12 ··· a1n

 . . .. . 
. . . .
. . .
 
 
ai1 + λaj1 ai2 + λaj2 ··· ain + λajn 
 
. . .. .
 
ALi ←Li +λLj =  . . .  ,

 . . . . 
 a
 j1 aj2 ··· ajn 
. . .. .
 
. . .

. . . .
 
 
am1 am2 ··· amn
and
a
 11 a12 ··· a1n 
 . . .

 . . .. .

 . . . .


 aj1
 aj2 ··· ajn 
 . . .. . 

ALi ↔Lj =  . . . .  .
 . . . 
 a
 i1 ai2 ··· ain 
 . . . 
..

 . . . 
 . . . . 
am1 am2 ··· amn

Rza Mustafayev Khazar University 36 / 53

Definition: By applying one of the elementary operations to the identity matrix In , we obtain the so-called elementary
matrices:

ELi ←λLi , ELi ←Li +λLj or ELi ↔Lj ,

respectively.

Example: For n = 3,
     
1 0 0 1 0 0 1 0 0
0, 0, 1.
  
EL2 ←5L2 = 0 5 EL2 ←L2 +2L1 = 2 1 EL2 ↔L3 = 0 0
     
0 0 1 0 0 1 0 1 0

Proposition: Application of any elementary operation to a matrix A is equivalent to multiplication of the matrix A from
the left by the corresponding elementary matrix:
applying the elementary operation Li ↔ Lj to A one gets ELi ↔Lj × A

applying the elementary operation Li ← λLi to A one gets ELi ←λLi × A

applying the elementary operation Li ← Li + λLj to A one gets ELi ←Li +λLj × A

Exercise: Check your understanding by calculating EL1 ↔L2 B, EL2 ←λL2 B, and EL3 ←L3 +2L2 B directly for
 
 1 2 3 4
(−1) 0.

B =  2 1
(−2)
 
3 1 1

Rza Mustafayev Khazar University 37 / 53

Proposition: The applied elementary operations are reversible, therefore the elementary matrices are invertible.
Namely,
If E is obtained by multiplying row i by scalar λ, then E−1 is obtained by multiplying row i by scalar 1/λ:
 −1
ELi ←λLi = ELi ←(1/λ)Li .

If E is obtained by adding λ times row j to row i, then E−1 is obtained by subtracting λ times row j from row i:
 −1
ELi ←Li +λLj = ELi ←Li −λLj .

−1
If E is obtained by switching rows i and j, then E is also obtained by switching rows i and j:
 −1
ELi ↔Lj = ELi ↔Lj .

Exercise:
!
1 0
Let A = . Find A−1 .
0 2
 
1 0 0
0. Find B−1 .

Let B = 2 1
 
0 0 1

1 0 0 0
 
0 1 0 0

Let C = 
0 . Find C−1 .
0 0 1
0 0 1 0

Rza Mustafayev Khazar University 38 / 53

 Row Echelon Form

Rza Mustafayev Khazar University 39 / 53

Clearing the first column: Our first basic technique using the row operations converts any matrix A to another matrix
B of the form
b11 b12 ··· b1k 
 
 0 b · · · b

22 2k 
 ,

 . . .. .
 
 .. .
. . .
. 
 
0 bn2 ··· bnk
so all but possibly the first entry in the first column is zero. We can furthermore arrange that b11 is either 1 or 0.

1 If all entries in the first column are already zero there is nothing to do.
2 Else use the swap operation to arrange that the top left entry a11 in the matrix is nonzero.
3 Now use operation Lj ← Lj + µL1 for each j ≥ 2 and for suitable values of µ to ensure the rest of the first column
is zero.

Exercise:

Apply this method to convert the following matrices:

     
0 1 1  1 0 1 1 0 4 7 10
0, B = (−1)
  
A = 1 1 0 2 1, C = 2 5 8 11.
(−1)
     
1 0 1 1 5 9 3 6 9 12

Rza Mustafayev Khazar University 40 / 53

Definition: We say that two matrices A and B are row equivalent if one can be changed to the other by a sequence
of elementary operations, and we write A ∼ B.

   
1 2 3
 1 0 1

Exercise: Are A = 1 1 2 and B = 0 1 1 equivalent?
   
0 1 1 0 0 0

Definition: A matrix is in row echelon form if it contains no adjacent rows of the form
!
0 0 ··· 0 x1 x2 ···
0 0 ··· 0 y1 y2 ···
with y1 , 0 (irrespective of what x1 is). In other words, we say that a matrix A is in row echelon form if the number of
zeroes at the beginning of each row is strictly increasing row by row.

Definition: If a matrix is in row echelon form, then the first nonzero entry of each row is called a pivot, and the
columns in which pivots appear are called pivot columns.

Exercise: Which matrix is in row echelon form?

1 2 (−1) 3

  
(−1)
 
(−1)

0 1 2 0 2
0 2 4
  
A = 
0 , B = 0 1 2 , C = 0 1 2 .
0 0 1   
0 0 0 0 0 0
0 0 0 0

Rza Mustafayev Khazar University 41 / 53

Any matrix can be converted to row echelon form by row operations. This procedure, called forward elimination, is as
follows:
1 Apply "clearing the first column" (ignoring any initial columns of zeros) until the matrix is of the form

0 ··· 0 x a1 ··· ak 

 
0 ··· 0 0 b11 ··· b1k 


 .. . . . .. .  .
 
. . . . .
 . . . . . 

0 ··· 0 0 bm1 ··· bmk

2 Now put the matrix

 b11 ··· b1k 
 
 . .. .

 . .

 . . . 

bm1 ··· bmk
into row echelon form by "clearing the first column" of this matrix (again, ignoring any initial column of zeros).
Note that row operations on this matrix correspond to row operations on the original. Continue in this way until
the whole matrix is in row echelon form.

Rza Mustafayev Khazar University 42 / 53

Here we give Gauss forward elimination algorithm:

Algorithm (Forward Elimination): The input is a matrix A = (aij ). The output is a row echelon form of A.

Step 1: Find the first column with a nonzero entry. Let j1 denote this column.

a Arrange so that a1 j1 , 0. That is, if necessary, apply swap operation so that a nonzero entry appears in the first
row in column j1 .
b Use a1 j1 as a pivot to obtain 0’s below a1 j1 . Specifically, for i > 1, apply the operation Li ← Li − (ai j1 /a1 j1 )L1 .

Step 2: Repeat Step 1 with the submatrix formed by all rows excluding the first row. Here we let j2 denote the first
column in the submatrix with a nonzero entry. Hence, at the end of Step 2, we have a2 j2 , 0.

Step 3 to r: Continue the above process until a submatrix has only zero rows.

We emphasize that at the end of the algorithm, the pivots will be

a1 j1 , a2 j2 , . . . , ar jr
where r denotes the number of nonzero rows in the final row echelon matrix.

Rza Mustafayev Khazar University 43 / 53

Example: Transform the following matrices to their row echelon forms ?
   
 1 2 3 4
  1 2 3 

A =  0 2 4 6, B =  1 4 0 
(−1) (−2) (−2) (−3)
   
0 1 0

Exercise: Use row operations to put the following into row echelon form:
     
(−1)
 
0 1 1
 1 1 1 1
 1 2 3 5

0 1 2 3 

A = 0 1 0, B = 2 3 4 5, C = 1 2 3 4, D =  0 1 2 3 4 .
(−1) (−2) (−3) (−4) (−5)
       
1 1 0 3 4 5 6 0 0 1 1

Definition: If A can be converted to the row echelon form matrix B using row operations, and B has exactly n nonzero
rows, then the rank of A, denoted by rank(A), is k .

Exercise: Say what the ranks of the matrices in the previous exercise are?

Rza Mustafayev Khazar University 44 / 53

 Reduced Row Echelon Form

Rza Mustafayev Khazar University 45 / 53

Definition: A matrix A is said to be in reduced row echelon form if it is an echelon matrix satisfying the following two
properties:
Each pivot (leading nonzero entry) is equal to 1.
Each pivot is the only nonzero entry in its column.

Remark: The major difference between a row echelon matrix and a reduced row echelon matrix is that in a row
echelon matrix there must be zeros below pivots, but in a reduced row echelon matrix, each pivot must also equal to 1
and there must also be zeros above the pivots.

The zero matrix O and the identity matrix I of any size are important special examples of reduced row echelon
matrices.

Example: Which matrix is in reduced row echelon form ?

     
1 0 2 0 1 1 2 0 1 0 2 0
(−1) (−1) (−1)
  
A = 0 1 0, B = 0 1 0, C = 0 1 0.
     
0 0 0 1 0 0 0 1 0 0 0 2

Rza Mustafayev Khazar University 46 / 53

Any matrix in row echelon form can be converted to reduced row echelon form by row operations. This procedure,
called Gauss backward elimination, is as follows:

Algorithm (Backward Elimination): The input is a matrix A = (aij ) in row echelon form with pivot entries
a1 j1 , a2 j2 , . . . , ar jr .
The output is the reduced row echelon form of A.

Step 1:

a Apply Lr ← (1/ar jr )Lr (Use row scaling so that the last pivot equals 1).
b Apply Li ← Li + (−ai jr )Lr for i = r − 1, r − 2, . . . , 2, 1 (Use ar jr = 1 to obtain 0’s above the pivot).

Step 2 to r-1: Repeat Step 1 for rows Lr −1 , Lr −2 , . . . , L2 .

Step r: Apply L1 ← (1/a1 j1 )L1 (Use row scaling so that the first pivot equals 1).

 
 1 2 3 4

Example: Transform A =  0 2 4 6 to its reduced row echelon form.
−1
 
0 1 0

Theorem: Let A ∈ Mnp (R), there exists a unique matrix U in reduced row echelon form obtained from A by means of
elementary operations on lines.

Rza Mustafayev Khazar University 47 / 53

 Converting to the Identity Matrix

Rza Mustafayev Khazar University 48 / 53

If a matrix A is n × k where n ≤ k , then the leftmost n × n block of A can sometimes be converted to the identity matrix
using row operations. In fact, this can be always done if A has echelon form
a
 11 ··· a1n a1n+1 ··· a1k 

 0 a22 ··· a2n a2n+1 ··· a2k 

 0
 0 a33 ··· a3n a3n+1 ··· a3k 
 . . . .

 . .. . . .

 . . . . .


0 ··· 0 ann ann+1 ··· ank
with each of the diagonals ajj , 0, or in other words, if A has rank n.

We know that, in this case, applying Gauss backward elimination algorithm, the latter can be transformed to the
following reduced row echelon form:
1
 0 0 ··· 0 b1n+1 ··· b1k 

0
 1 0 ··· 0 b2n+1 ··· b2k 

0 0 1 ··· 0 b3n+1 ··· b3k 
 .. . . .

..

. . .

 .
 . . . .


0 0 ··· 0 1 bnn+1 ··· bnk

Rza Mustafayev Khazar University 49 / 53

Definition: If A is an n × m matrix and B is an n × k matrix, the augmented matrix (A | B) is the n × (m + k ) matrix
you get by writing down the entries of A and B next to one another in the obvious way.

Thus, if
a11 ··· a1m  b11 ··· b1k 
   
 . .  . .
.. ..
 
A =  . . and B =  . .  ,
 
 . . . 
  . . . 
an1 ··· anm bn1 ··· bnk
then
a11 ··· a1m b11 ··· b1k 
 
 . .. . . .. .
 
(A | B) =  . . . . . . 
 . . . . 

an1 ··· anm bn1 ··· bnk

Rza Mustafayev Khazar University 50 / 53

To compute the inverse of an n × n square matrix A, apply row operations to the augmented matrix (A | I), where I is
the n × n identity matrix, to get (I | B) for some matrix B. This is not always possible, but as we have already seen it will
be possible if A can be converted to some echelon form with n nonzero rows, that is, if rank(A) = n. Then A−1 = B.

To see why the method works, recall that each elementary row operation corresponds to multiplying on the left by an
elementary row operation matrix. So applying several row operations to (A | I) corresponds to multiplying on the left by
a product
E = E1 E2 · · · Ek
of row operation matrices. If the result of these operations is (I | B) then by associativity of matrix multiplication
E(A | I) = (I | B),
that is,
EA = I and EI = B.
Consequently,
A−1 = E = B.

Recall that a square n × n matrix has rank n if and only if its reduced row echelon form is In . Thus, we have proven the
sufficient part of the following statement.

Theorem: A matrix A ∈ Mn (R) is invertible if and only if its reduced row echelon form is In .

 
 1 2 3 

Example: Show that A =  1 4 0  is invertible.
(−2) (−2) (−3)
 

Rza Mustafayev Khazar University 51 / 53

Finding the inverse of a matrix (Gauss method):

Gauss method allows to find the inverse of a matrix A by means of the following steps :

1) transform the matrix A to the identity matrix using elementary operations,

2) apply the same previous elementary operations to the identity matrix to find A−1 .

 1 0 1 0
   
 1 2 1   0 2 (−2) 0

(−1) and B = 

Exercise: Compute the inverse of A =  4 0  using Gauss method.
(−1) 2 0 1
(−1)
 
2 2
0 2 1 3

Rza Mustafayev Khazar University 52 / 53

 Thank You For Your Attention

Rza Mustafayev Khazar University 53 / 53