Linear Algebra

Osama Sohrab
National University Of Computer & Emergining Sciences, Peshawar

Spring 2025

Osama Sohrab Linear Algebra 1 / 186

Chapter 1: Systems of Linear Equations and Matrices

Osama Sohrab Linear Algebra 2 / 186

Problem
A man is ordered by his doctor to take 5 units of vitamin A, 13 units of vitamin B, and 23
units of vitamin C each day. Three brands of vitamin pills are available, and the number
of units of each vitamin per pill are shown in the accompanying table.

Vitamin

Brand A B C

1 1 2 4

2 1 1 3

3 0 1 1

Osama Sohrab Linear Algebra 3 / 186

(a) Find all combinations of pills that provide exactly the required amount of vitamins
(no partial pills allowed).
(b) If brands 1, 2, and 3 cost $3, $2, and $5 per pill, respectively, find the least expensive
treatment.

Osama Sohrab Linear Algebra 4 / 186

Solution
Required:

Vit A: 05 units
Vit B: 13 units
Vit C: 23 units

Let x be the number of pills from brand 1, y be the number of pills from brand 2 and z
be the number of pills from brand 3 which fulfill our requirement. Then we have

Vit A Eq : x + y + 0z = 5
Vit B Eq : 2x + y + z = 13
Vit C Eq : 4x + 3y + z = 23.

Osama Sohrab Linear Algebra 5 / 186

So we have a linear system, in which we have three equations and three unknowns and
we want its solution. The augmented matrix is given as:
 
1 1 0 5
 2 1 1 13 
 

4 3 1 23
Performing the row operations:

R2 − 2R1 → R2 , R3 − 4R1 → R3 ,

we get:  
1 1 0 5
 0 −1 1 3 
 

0 −1 1 3
Osama Sohrab Linear Algebra 6 / 186
Performing the row operation:
R3 − R2 → R3
We get:  
1 1 0 5
 0 −1 1 3 
 

0 0 0 0
From R2 :
−y + z = 3 =⇒ z − 3 = y or y = z − 3 z is free
From R1 :
x + y = 5 =⇒ x = 5 − y
Substituting y = z − 3:
x = 5 − (z − 3) = 5 − z + 3 = 8 − z ⇒ x = 8 − z
Osama Sohrab Linear Algebra 7 / 186
Thus
x = 8 − z, y = z − 3, z is free.

   
x 8−z
y  = z − 3 , z ∈ {3, 4, 5, 6, 7, 8}
   

z z
(b) For z = 3, we have:
   
x 5
y  = 0
   

z 3
So, the least expensive treatment is:

$5(3) + $0(2) + $3(5) = $30

Osama Sohrab Linear Algebra 8 / 186
Practice Problem
The accompanying figure shows a network of one-way streets with traffic flowing in the
directions indicated. The flow rates along the streets are measured as the average number
of vehicles per hour. Is it possible to close the road from A to B for construction and
keep traffic flowing on the other streets? Explain.

Osama Sohrab Linear Algebra 9 / 186

1.3: Matrices and Matrix Operations

Definition
A matrix is a rectangular array of numbers. The numbers in the array are called
the entries of the matrix.

Some examples of matrices are

   
1 2 3 " # 4
h i 4 5 6
4 5 6 , π e 3 , , 1
   
1 2 3
7 8 9 6

A matrix with m rows and n columns is referred to as an m × n matrix or as having size

m × n.
Osama Sohrab Linear Algebra 10 / 186
A general m × n matrix is
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
 
A=
 .. .. ... ..  (1)
 . . . 


am1 am2 · · · amn

When a compact notation is desired, matrix (1) can be written as:

A = [aij ]m×n or A = [aij ]

The entry in row i and column j of a matrix A is also commonly denoted by the symbol
(A)ij . Thus, for matrix (1) above, we have:

(A)ij = aij
Osama Sohrab Linear Algebra 11 / 186
and for the matrix: " #
2 −3
A=
7 0

We have:
(A)11 = 2, (A)12 = −3, (A)21 = 7, and (A)22 = 0.

Osama Sohrab Linear Algebra 12 / 186

Row Vector and Column Vector
A matrix with only one row is called a row vector (or a row matrix), and a matrix with
only one column is called a column vector (or a column matrix).

Row and column vectors are of special importance, and it is common practice to denote
them by boldface lowercase letters rather than capital letters. For such matrices, double
subscripting of the entries is unnecessary. Thus a general 1×n row vector a and a general
m × 1 column vector b would be written as:
 
b1
 b2 
h i  
a = a1 a2 · · · an and b = 
 .. 

 . 
bm

Osama Sohrab Linear Algebra 13 / 186

Square Matrix and Main Diagonal
A matrix A with n rows and n columns is called a square matrix of order n, and the
shaded entries a11 , a22 , . . . , ann in (2) are said to be on the main diagonal of A.
 
a11 a12 · · · a1n
a21 a22 · · · a2n 
 

A= .. .. . . .  (2)

 . . . .. 
an1 an2 · · · ann

Osama Sohrab Linear Algebra 14 / 186

Equal Matrices
Two matrices are defined to be equal if they have the same size and their corre-
sponding entries are equal.

Osama Sohrab Linear Algebra 15 / 186

Problem " # " # " #
a b 1 2 −1 1 0
Given A = ,B= , and C = , discuss the possibility that
c d 3 0 1 −1 2

A = B, B = C, A = C.

Solution
A = B is impossible because A and B are of different sizes: A is 2 × 2 whereas B is
2 × 3. Similarly, B = C is impossible. But A = C is possible provided that corresponding
entries are equal:
" # " #
a b 1 0
=
c d −1 2
means a = 1, b = 0, c = −1, and d = 2.
Osama Sohrab Linear Algebra 16 / 186
 Matrices Addition & Subtraction
If A and B are matrices of the same size, then the sum A + B is the matrix
obtained by adding the entries of B to the corresponding entries of A, and the
difference A − B is the matrix obtained by subtracting the entries of B from
the corresponding entries of A. Matrices of different sizes cannot be added or
subtracted.

In matrix notation, if A = [aij ] and B = [bij ] have the same size, then:

(A + B)ij = (A)ij + (B)ij = aij + bij and (A − B)ij = (A)ij − (B)ij = aij − bij .

Osama Sohrab Linear Algebra 17 / 186

Consider the matrices
   
2 1 0 3 −4 3 5 1 " #
1 1
A = −1 0 2 4 , B= 2 2 0 −1 , C= .
   
2 2
4 −2 7 0 3 2 −4 5

Then
   
−2 4 5 4 6 −2 −5 2
A + B =  1 2 2 3 , A − B = −3 −2 2 5 .
   

7 0 3 5 1 −4 11 −5

The expressions A + C, B + C, A − C, and B − C are undefined.

Osama Sohrab Linear Algebra 18 / 186
 Scalar Multiples of a Matrix
If A is any matrix and c is any scalar, then the product cA is the matrix obtained
by multiplying each entry of the matrix A by c. The matrix cA is said to be a
scalar multiple of A.

In matrix notation, if A = [aij ], then:

(cA)ij = c(A)ij = caij

Osama Sohrab Linear Algebra 19 / 186

Given the matrices:
" # " # " #
2 3 4 0 2 7 9 −6 3
A= , B= , C=
1 3 1 −1 3 −5 3 0 12

We have:
" # " # " #
4 6 8 0 −2 −7 1 3 −2 1
2A = , (−1)B = , C=
2 6 2 1 −3 5 3 1 0 4

It is common practice to denote (−1)B by −B.

Osama Sohrab Linear Algebra 20 / 186

 Product of Two Matrices
If A is an m × r matrix and B is an r × n matrix, then the product AB is the
m × n matrix whose entries are determined as follows: To find the entry in row i
and column j of AB, single out row i from the matrix A and column j from the
matrix B. Multiply the corresponding entries from the row and column together,
and then add the resulting products.

Consider the matrices:

 
" # 4 1 4 3
1 2 4
A= , B = 0 −1 3 1
 
2 6 0
2 7 5 2
Since A is a 2 × 3 matrix and B is a 3 × 4 matrix, the product AB is a 2 × 4 matrix.
Osama Sohrab Linear Algebra 21 / 186
To determine, for example, the entry in row 2 and column 3 of AB, we single out row
2 from A and column 3 from B. Then, as illustrated below, we multiply corresponding
entries together and add up these products.

 
" # 4 1 4 3 " #
1 2 4  □ □ □ □
0 −1 3 1 =

2 6 0 □ □ 26 □
2 7 5 2

(2 · 4) + (6 · 3) + (0 · 5) = 26

Osama Sohrab Linear Algebra 22 / 186

The entry in row 1 and column 4 of AB is computed as follows:
 
" # 4 1 4 3 " #
1 2 4  □ □ □ 13
0 −1 3 1 =

2 6 0 □ □ □ □
2 7 5 2
(1 · 3) + (2 · 1) + (4 · 2) = 13
The computations for the remaining entries are
(1 · 4) + (2 · 0) + (4 · 2) = 12
(1 · 1) − (2 · 1) + (4 · 7) = 27
(1 · 4) + (2 · 3) + (4 · 5) = 30
(2 · 4) + (6 · 0) + (0 · 2) = 8
(2 · 1) − (6 · 1) + (0 · 7) = −4
(2 · 3) + (6 · 1) + (0 · 2) = 12
Osama Sohrab Linear Algebra 23 / 186
Thus " #
12 27 30 13
AB =
8 −4 26 12

Osama Sohrab Linear Algebra 24 / 186

Partitioned Matrices
A matrix can be subdivided or partitioned into smaller matrices by inserting horizontal
and vertical rules between selected rows and columns. For example, the following are
three possible partitions of a general 3 × 4 matrix A—the first is a partition of A into four
submatrices A11 , A12 , A21 , and A22 ; the second is a partition of A into its row vectors
r1 , r2 , and r3 ; and the third is a partition of A into its column vectors c1 , c2 , c3 , and c4 :
 
a11 a12 a13 a14 " #
A11 A12
A =  a21 a22 a23 a24  =
 
A21 A22
a31 a32 a33 a34

Osama Sohrab Linear Algebra 25 / 186

    
a11 a12 a13 a14 r1
A = a21 a22 a23 a24  = r2 
   

a31 a32 a33 a34 r3

 
a11 a12 a13 a14 h i
A =  a21 a22 a23 a24  = c1 c2 c3 c4
 

a31 a32 a33 a34

Osama Sohrab Linear Algebra 26 / 186

Matrix Multiplication by Columns and by Rows
Partitioning has many uses, one of which is for finding particular rows or columns of
a matrix product AB without computing the entire product. Specifically, the following
formulas, show how individual column vectors of AB can be obtained by partitioning B
into column vectors and how individual row vectors of AB can be obtained by partitioning
A into row vectors.
h i h i
AB = A b1 b2 · · · bn = Ab1 Ab2 · · · Abn

(AB computed column by column)

Osama Sohrab Linear Algebra 27 / 186

    
a1 a1 B
 a2   a2 B 
   
AB = 
 .. B =  .. 
  
 .  . 
am am B

(AB computed row by row)

In words, these formulas state that

jth column vector of AB = A[jth column vector of B]

ith row vector of AB = [ith row vector of A]B

Osama Sohrab Linear Algebra 28 / 186

Problem
Compute AB if    
2 3 5 8 9
A = 1 4 7 , B = 7 2 .
   

0 1 8 6 1
Solution The columns of B are
   
8 9
b1 = 7 and b2 = 2 ,
   

6 1
so     
2 3 5 8 67
Ab1 = 1 4 7 7 = 78 ,
    

0 1 8 6 55
Osama Sohrab Linear Algebra 29 / 186
and
    
2 3 5 9 29
Ab2 = 1 4 7 2 = 24 .
    

0 1 8 1 10
Hence,
 
67 29
AB = [Ab1 Ab2 ] = 78 24 .
 

55 10

Osama Sohrab Linear Algebra 30 / 186

Powers of a Matrix
If A is a square matrix, then we define the nonnegative integer powers of A to be

A0 = I and An = AA · · · A [n factors]

and if A is invertible, then we define the negative integer powers of A to be

A−n = (A−1 )n = A−1 A−1 · · · A−1 [n factors]

Because these definitions parallel those for real numbers, the usual laws of nonnegative
exponents hold; for example,

Ar As = Ar+s and (Ar )s = Ars .

Osama Sohrab Linear Algebra 31 / 186

Transpose of a Matrix

If A is any m × n matrix, then the transpose of A, denoted by AT , is defined

to be the n × m matrix that results by interchanging the rows and columns of A;
that is, the first column of AT is the first row of A, the second column of AT is
the second row of A, and so forth.

Osama Sohrab Linear Algebra 32 / 186

The following are some examples of matrices and their transposes.
   
a11 a12 a13 a14 2 3
A = a21 a22 a23 a24  , B = 1 4 , C = [1 3 5], D = [4].
   

a31 a32 a33 a34 5 6

 
a11 a21 a31  
  " # 1
a12 a22 a32  2 1 5
AT =   , BT = , T
C = 3 , DT = [4].
 
a a23 a33  3 4 6
 13
5

a14 a24 a34
Observe that not only are the columns of AT the rows of A, but the rows of AT are the
columns of A. Thus the entry in row i and column j of AT is the entry in row j and
column i of A; that is,
(AT )ij = (A)ji
Osama Sohrab Linear Algebra 33 / 186
Theorem
If the sizes of the matrices are such that the stated operations can be performed,
then:
(AT )T = A
(A + B)T = AT + B T
(A − B)T = AT − B T
(kA)T = kAT
(AB)T = B T AT

Osama Sohrab Linear Algebra 34 / 186

Theorem (Properties of Matrix Arithmetic)

Assuming that the sizes of the matrices are such that the indicated operations can
be performed, the following rules of matrix arithmetic are valid:
(a) A + B = B + A [Commutative law for matrix addition]
(b) A + (B + C) = (A + B) + C [Associative law for matrix addition]
(c) A(BC) = (AB)C [Associative law for matrix multiplication]
(d) A(B + C) = AB + AC [Left distributive law]
(e) (B + C)A = BA + CA [Right distributive law]
(f ) A(B − C) = AB − AC
(g) (B − C)A = BA − CA

Osama Sohrab Linear Algebra 35 / 186

(h) a(B + C) = aB + aC
(i) a(B − C) = aB − aC
(j) (a + b)C = aC + bC
(k) (a − b)C = aC − bC
(l) a(bC) = (ab)C
(m) a(BC) = (aB)C = B(aC)

Osama Sohrab Linear Algebra 36 / 186

WARNINGS:
❶ In general, AB ̸= BA.
❷ The cancellation laws do not hold for matrix multiplication. That is, if
AB = AC, then it is not true in general that B = C.
❸ If a product AB is the zero matrix, you cannot conclude in general that either
A = O or B = O.

Osama Sohrab Linear Algebra 37 / 186

Example on Point 1
Let " # " #
5 1 2 0
A= , B= .
3 −2 4 3
Then: " #" # " #
5 1 2 0 14 3
AB = = ,
3 −2 4 3 −2 −6
" #" # " #
2 0 5 1 10 2
BA = = .
4 3 3 −2 29 −2

∴ AB ̸= BA

Osama Sohrab Linear Algebra 38 / 186

Example on Point 2
Consider the matrices
" # " # " #
0 1 1 1 2 5
A= , B= , C=
0 2 3 4 3 4

Here " #
3 4
AB = AC =
6 8
But
B ̸= C

Osama Sohrab Linear Algebra 39 / 186

Example on Point 3
Here are two matrices for which AB = O, but A ̸= O and B ̸= O:
" # " #
0 1 3 7
A= , B=
0 2 0 0

Osama Sohrab Linear Algebra 40 / 186

 Definition of Trace
If A is a square matrix, then the trace of A, denoted by tr(A), is defined to be the
sum of the entries on the main diagonal of A. The trace of A is undefined if A is
not a square matrix.

The following are examples of matrices and their traces.

 
  −1 2 7 0
a11 a12 a13  
3 5 −8 4 
A = a21 a22 a23  , B = 
  
1 2 7 −3
a31 a32 a33
 
4 −2 1 0

tr(A) = a11 + a22 + a33 tr(B) = −1 + 5 + 7 + 0 = 11

Osama Sohrab Linear Algebra 41 / 186
Properties of trace

tr(A + B) = tr(A) + tr(B)

tr(AB) = tr(BA)
tr(cA) = c tr(A)
tr(At ) = tr(A)

Osama Sohrab Linear Algebra 42 / 186

 Zero Matrices
A matrix whose entries are all zero is called a zero matrix.

Some examples are

   
0 0 0 0 " #
0 0 0 h i
0 , 0 0 0 , , 0 0 0 0
   
0 0 0
0 0 0 0

We will denote a zero matrix by O unless it is important to specify its size, in which case
we will denote the m × n zero matrix by Om×n .

Osama Sohrab Linear Algebra 43 / 186

Properties of Zero Matrices
If c is a scalar, and if the sizes of the matrices are such that the operations can be
perfomed, then:
(a) A + O = O + A = A
(b) A − O = A
(c) A − A = A + (−A) = O
(d) 0A = O
(e) If cA = O, then c = 0 or A = O.

Osama Sohrab Linear Algebra 44 / 186

 Identity Matrices
A square matrix with 1’s on the main diagonal and zeros elsewhere is called an
identity matrix.

Some examples are

 
  1 0 0 0
" # 1 0 0  
h i 1 0 0 1 0 0
1 , , 0 1 0 ,
   
0 1 0 0 1 0
0 0 1
 
0 0 0 1

An identity matrix is denoted by the letter I. If it is important to emphasize the size, we

will write In for the n × n identity matrix.
Osama Sohrab Linear Algebra 45 / 186
 Diagonal Matrices
A square matrix in which all the entries off the main diagonal are zero is called a
diagonal matrix.

Here are some examples:

 
  6 0 0 0
" # 1 0 0   " #
2 0 0 −4 0 0 0 0
, 0 1 0 , 0 0 0 0 ,
   
0 5 0 0
0 0 1
 
0 0 0 8

Osama Sohrab Linear Algebra 46 / 186

A general n × n diagonal matrix D can be written as
 
d1 0 · · · 0
 0 d2 · · · 0 
 
D=  .. .. . . . .. 

. . .
0 0 · · · dn

Osama Sohrab Linear Algebra 47 / 186

Triangular Matrices
A square matrix in which all the entries above the main diagonal are zero is
called lower triangular, and a square matrix in which all the entries below the
main diagonal are zero is called upper triangular. A matrix that is either upper
triangular or lower triangular is called triangular.

 
a11 a12 a13 a14
 
 0 a22 a23 a24 
 
 0 0 a a 
 33 34 

0 0 0 a44

A general 4 × 4 upper triangular matrix

Osama Sohrab Linear Algebra 48 / 186
 Symmetric Matrices

A square matrix A is said to be symmetric if AT = A.

The following matrices are symmetric since each is equal to its own transpose.
 
  6 0 0 0
" # 1 4 5  
7 −3 0 −4 0 0
, 4 −3 0 , 
   
−3 5 0 0 0 0
5 0 7
 
0 0 0 8

Osama Sohrab Linear Algebra 49 / 186

 Equivalent Matrices

Matrices A and B are said to be row equivalent if either (hence each) can be
obtained from the other by a sequence of elementary row operations.

Listed below are two elementary matrices and the operations that produce them.
 
" # 1 0 3
1 0
 0 1 0 
 
0 −3
0 0 1
Multiply the second row of I2 by −3.
Add 3 times the third row of I3 to the first row.

Osama Sohrab Linear Algebra 50 / 186

 Elementary Matrices
A matrix E is called an elementary matrix if it can be obtained from an identity
matrix by performing a single elementary row operation.

Listed below are two elementary matrices and the operations that produce them.
 
1 0 0 0
" #  
1 0 0 0
 0 1
0 −3 0 0
 1 0
0 1 0 0
Multiply the second row of I2 by −3. Interchange the second and fourth rows of I4 .

Osama Sohrab Linear Algebra 51 / 186

Elementary Row Operations

Symbol Meaning
Ri ↔ Rj Interchange rows i and j
kRi Multiply the ith row by the nonzero constant k
Ri + kRj Multiply the jth row by k and add to the ith row

Osama Sohrab Linear Algebra 52 / 186

Determinant
The determinant of a 1 × 1 matrix [a11 ] is

det[a11 ] = a11

For 2 × 2 matrices the formula is

" #
a b a b
det = = ad − bc
c d c d

Let’s talk about minor and cofactor of an element of a matrix with the help of which we
will calculate determinants of higher order matrices.

Osama Sohrab Linear Algebra 53 / 186

Minor & Cofactor of an element (In a Square Matrix)

If A is a square matrix, then the minor of entry Aij is denoted by Mij and is
defined to be the determinant of the submatrix that remains after the ith row and
jth column are deleted from A. The number (−1)i+j Mij is denoted by Cij and is
called the cofactor of entry aij .

Osama Sohrab Linear Algebra 54 / 186

The minor of entry a11 is
3 1 -4
5 6
M11 = 2 5 6 = = 16
4 8
1 4 8
The cofactor of a11 is
C11 = (−1)1+1 M11 = M11 = 16
Similarly, the minor of entry a32 is
3 1 −4
3 −4
M32 = 2 5 6 = = 26
2 6
1 4 8
The cofactor of a32 is
C32 = (−1)3+2 M32 = −M32 = −26
Osama Sohrab Linear Algebra 55 / 186
Definition of a General Determinant
If A is an n × n matrix, then the number obtained by multiplying the entries in
any row or column of A by the corresponding cofactors and adding the resulting
products is called the determinant of A, and the sums themselves are called
cofactor expansions of A. That is,

det(A) = a1j C1j + a2j C2j + · · · + anj Cnj

[cofactor expansion along the jth column]

and
det(A) = ai1 Ci1 + ai2 Ci2 + · · · + ain Cin
[cofactor expansion along the ith row]

Osama Sohrab Linear Algebra 56 / 186

Problem
Find the determinant of the matrix
 
3 1 0
A = −2 −4 3
 

5 4 −2

Solution
By cofactor expansion along the first row:
3 1 0
−4 3 −2 3 −2 −4
det(A) = −2 −4 3 =3 −1 +0
4 −2 5 −2 5 4
5 4 −2

= 3(−4) − 1(−11) + 0 = −1
Osama Sohrab Linear Algebra 57 / 186
By cofactor expansion along the first column

3 1 0
−4 3 1 0 1 0
det(A) = −2 −4 3 =3 − (−2) +5
4 −2 4 −2 −4 3
5 4 −2

= 3(−4) − (−2)(−2) + 5(3) = −1

The best strategy for cofactor expansion is to expand along a row or column with
the most zeros.

Osama Sohrab Linear Algebra 58 / 186

A Useful Technique for Evaluating 2 × 2 and 3 × 3 Determinants

In the 2 × 2 case, the determinant can be computed by forming the product of the entries
on the rightward arrow and subtracting the product of the entries on the leftward arrow.
In the 3 × 3 case we first recopy the first and second columns as shown in the figure,
after which we can compute the determinant by summing the products of the entries on
the rightward arrows and subtracting the products on the leftward arrows.

Osama Sohrab Linear Algebra 59 / 186

For example,

Osama Sohrab Linear Algebra 60 / 186

Properties of the Determinant

Theorem Zero Row or Column

Let A be a square matrix. If A has a row of zeros or a column of zeros, then
det(A) = 0.

For example
1 0 3
0 0
= 0, 4 0 5 =0
1 2
6 0 7

Osama Sohrab Linear Algebra 61 / 186

 Theorem
Let A be a square matrix. Then det(AT ) = det(A).

For example, for the matrix

" # " #
1 2 1 3
A= , AT =
3 4 2 4

Observe that
" #
1 2 1 3
det(A) = = −2, det(AT ) = = −2
3 4 2 4

Osama Sohrab Linear Algebra 62 / 186

Theorem
Let A be an n × n matrix.
(a) If B is the matrix that results when a single row or single column of A is
multiplied by a scalar k, then det(B) = k det(A).
(b) If B is the matrix that results when two rows or two columns of A are
interchanged, then det(B) = − det(A).
(c) If B is the matrix that results when a multiple of one row of A is added
to another or when a multiple of one column is added to another, then
det(B) = det(A).

Osama Sohrab Linear Algebra 63 / 186

 Theorem
If A is a square matrix with two proportional rows or two proportional columns,
then det(A) = 0.

Each of the following matrices has two proportional rows or columns; thus, each has a
determinant of zero.
 
  3 −1−5 4
" # 1 −2 7  
−1 4  6 −2 5 2 
, −4 8 5 ,
   
−2 8 5 8 1 4 
2 −4 3
 
−9 3 −12 15

Osama Sohrab Linear Algebra 64 / 186

 Theorem
If A and B are square matrices of the same size, then

det(AB) = det(A) det(B)

For example, Consider the matrices

" # " # " #
3 1 −1 3 2 17
A= , B= , AB =
2 1 5 8 3 14
Now
det(A) = 1, det(B) = −23, and det(AB) = −23
Clearly,
det(AB) = det(A) det(B), as guaranteed by the Theorem.
Osama Sohrab Linear Algebra 65 / 186
If A is a square matrix of order n × n, then

det(kA) = k n det(A)

Osama Sohrab Linear Algebra 66 / 186

Theorem (Determinant of a Triangular Matrix)

Suppose A is an n × n triangular matrix (upper or lower). Then

det A = a11 a22 · · · ann ,

where a11 , a22 , . . . , ann are the entries on the main diagonal of A.

3 0 0 0
−3 0 0
2 6 0 0
= 3(6)(−4)(−2) = 144, 0 6 0 = (−3)(6)(4) = −72.
5 9 −4 0
0 0 4
7 2 4 −2

Osama Sohrab Linear Algebra 67 / 186

Theorem
If A is invertible, then
1
det(A−1 ) =
det(A)

Osama Sohrab Linear Algebra 68 / 186

 Theorem
Let A, B, and C be n × n matrices that differ only in a single row, say the rth,
and assume that the rth row of C can be obtained by adding corresponding entries
in the rth rows of A and B. Then

det(C) = det(A) + det(B)

The same result holds for columns.

For example,
     
1 7 5 1 7 5 1 7 5
det  2 0 3  = det 2 0 3 + det 2 0 3  .
     

1 + 0 4 + 1 7 + (−1) 1 4 7 0 1 −1
Osama Sohrab Linear Algebra 69 / 186
In general,
det(A + B) ̸= det(A) + det(B)

Consider
" # " # " #
1 2 3 1 4 3
A= , B= , A+B = .
2 5 1 3 3 8
We have det(A) = 1, det(B) = 8, and det(A + B) = 23; thus

det(A + B) ̸= det(A) + det(B).

Osama Sohrab Linear Algebra 70 / 186

Evaluating Determinants by Row Reduction
Evaluating the determinant of an n × n matrix by the method of cofactor expansion re-
quires a Herculean effort when the order of the matrix is large. To expand the determinant
of, say, a 5 × 5 matrix with nonzero entries requires evaluating five cofactors that are
determinants of 4 × 4 submatrices; each of these in turn requires four additional cofactors
that are determinants of 3 × 3 submatrices, and so on.

The idea of the method is to reduce the given matrix to upper triangular form by
elementary row operations, then compute the determinant of the upper triangular matrix
(an easy computation), and then relate that determinant to that of the original matrix.

Osama Sohrab Linear Algebra 71 / 186

Problem
Evaluate det(A) where
 
0 1 5
A = 3 −6 9
 

2 6 1
Solution
0 1 5
det(A) = 3 −6 9
2 6 1
3 −6 9
= − 0 1 5 R1 ↔ R2
2 6 1

Osama Sohrab Linear Algebra 72 / 186

 1 −2 3
= −3 0 1 5 3 was common in R1
2 6 1
1 −2 3
= −3 0 1 5 R3 − 2R1
0 10 −5
1 −2 3
= −3 0 1 5 R3 − 10R2
0 0 −55
= (−3)(1)(1)(−55) Triangular Matrix
= 165

Osama Sohrab Linear Algebra 73 / 186

Nonzero row or column in a matrix means a row or column that contains
at least one nonzero entry.
A leading entry of a row refers to the leftmost nonzero entry (in a nonzero
row).

Osama Sohrab Linear Algebra 74 / 186

Echelon Forms
A matrix is said to be in Echelon form if it has the following properties:
❶ Every nonzero row precedes every zero row.
❷ In each successive nonzero row, the number of zeros before the leading entry of a
row increases row by row.

Reduced Echelon Forms

If a matrix in echelon form satisfies the following additional conditions, then it is in
reduced echelon form (or reduced row echelon form).
❶ The leading entry in each nonzero row is 1.
❷ Each leading 1 is the only nonzero entry in its column.

Osama Sohrab Linear Algebra 75 / 186

The matrices
   
2 −3 2 1 1 0 0 29
0 1 −4 8  and 0 1 0 16
   

0 0 0 25 0 0 1 3

are in echelon form. In fact, the second matrix is in reduced echelon form.

Osama Sohrab Linear Algebra 76 / 186

Thorem
The reduced echelon form of a matrix is unique

Osama Sohrab Linear Algebra 77 / 186

Adjoint Matrix
If A is any n × n matrix and Cij is the cofactor of aij , then the matrix
 
C11 C12 · · · C1n
 C21 C22 · · · C2n 
 
 . .. .. 
 . ...
 . . . 


Cn1 Cn2 · · · Cnn

is called the matrix of cofactors from A. The transpose of this matrix is called
the adjoint of A and is denoted by adj(A).

Osama Sohrab Linear Algebra 78 / 186

For example, if  
3 −12
A = 1 6 3
 

2 −4 0
then
C11 = 12 C12 = 6 C13 = −16
C21 = 4 C22 = 2 C23 = 16
C31 = 12 C32 = −10 C33 = 16

so the matrix of cofactors is  

12 −16
6
4 2 16 
 

12 −10 16

Osama Sohrab Linear Algebra 79 / 186

and the adjoint of A is  
12 4 12
adj(A) =  6 −10 2
 

−16 16 16

Osama Sohrab Linear Algebra 80 / 186

Inverse of a Matrix
Let A be an n × n matrix. If there exists an n × n matrix B such that

AB = BA = I,

where I is the n × n identity, then the matrix A is said to be nonsingular or

invertible. The matrix B is said to be the inverse of A.

Osama Sohrab Linear Algebra 81 / 186

For example, the matrix " #
2 1
A=
1 1
is nonsingular or invertible since the matrix
" #
1 −1
B=
−1 2
is its inverse. To verify this, observe that
" #" # " #
2 1 1 −1 1 0
AB = = =I
1 1 −1 2 0 1
and " #" # " #
1 −1 2 1 1 0
BA = = = I.
−1 2 1 1 0 1
Osama Sohrab Linear Algebra 82 / 186
Theorem
If A and B are invertible matrices then:
(i) (AB)−1 = B −1 A−1
−1
(ii) (A−1 ) = A

−1 T
(iii) AT = (A−1 )

Osama Sohrab Linear Algebra 83 / 186

Theorem
If A is invertible and n is a nonnegative integer, then:
n
(i) (An )−1 = A−n = (A−1 )
(ii) For k ̸= 0, (kA)−1 = k −1 A−1

Osama Sohrab Linear Algebra 84 / 186

Theorem
A square matrix A is invertible if and only if det(A) ̸= 0.

Osama Sohrab Linear Algebra 85 / 186

Theorem (Inverse of a Matrix Using Its Adjoint)

If A is invertible, then
1
A−1 = adj(A)
det(A)

Osama Sohrab Linear Algebra 86 / 186

Theorem ( Inverse of a 2 × 2 Matrix )

The matrix " #

a b
A=
c d
is invertible if and only if ad − bc ̸= 0, in which case the inverse is given by the
formula " #
1 d −b
A−1 =
ad − bc −c a

Osama Sohrab Linear Algebra 87 / 186

Problem
Determine whether the matrix is invertible. If so, find its inverse. Let
" #
6 1
A= .
5 2

Solution:
The determinant of A is

det(A) = (6)(2) − (1)(5) = 7,

which is nonzero. Thus, A is invertible, and its inverse is

" # " #
2 1
1 2 −1 −
A−1 = = 75 67 .
7 −5 6 −7 7

Osama Sohrab Linear Algebra 88 / 186

Problem
Find the inverse of the matrix
 
3 −12
A = 1 6 3 .
 

2 −4 0
Solution

C11 = 12, C12 = 6, C13 = −16, C21 = 4, C22 = 2, C23 = 16,

C31 = 12, C32 = −10, C33 = 16
The adjugate matrix is
 
12 4 12
adj(A) =  6 −10 ,
2 det(A) = 64.
 

−16 16 16
Osama Sohrab Linear Algebra 89 / 186
The inverse of A is
   12 4 12

12 4 12 64 64 64
1 1 
A−1 =
 6 2 10 
adj(A) =  6 2 −10 = − 64 .

 64 64
det(A) 64
−16 16 16 − 16
64
16
64
16
64

Another method for finding the inverse of a matrix is illustrated in the coming slides.

Osama Sohrab Linear Algebra 90 / 186

 Theorem ( Finding the Inverse )

If an n × n matrix A can be transformed into the n × n identity I by a sequence of

elementary row operations, then A is nonsingular. The same sequence of operations
that transforms A into the identity I will also transform I into A−1 .

It is convenient to carry out these row operations on A and I simultaneously by means

of an n × 2n matrix obtained by augmenting A with the identity I as shown here:
 
a11 a12 · · · a1n 1 0 · · · 0
 a21 a22 · · · a2n 0 1 · · · 0 
 
[A | I] =  .
 .. . . .. .. .. . . .. 
 .. . . . . . . . 
an1 an2 · · · ann 0 0 · · · 1

Osama Sohrab Linear Algebra 91 / 186

Osama Sohrab Linear Algebra 92 / 186
Problem
Find the inverse of  
1 2 3
A = 2 5 3
 

1 0 8
Solution
 
1 2 3 1 0 0
 2 5 3 0 1 0 
 

1 0 8 0 0 1
 
1 2 3 1 0 0
 0 1 −3 −2 1 0  R2 − 2R1 & R3 − R1
 

0 −2 5 −1 0 1
Osama Sohrab Linear Algebra 93 / 186
  
1 2 3 1 0 0
 0 1 −3 −2 1 0  R3 + 2R2
 

0 0 −1 −5 2 1
 
1 2 3 1 0 0
 0 1 −3 −2 1 0  −R3
 

0 0 1 5 −2 −1
 
1 2 0 −14 6 3
 0 1 0 13 −5 −3  R1 − 3R3 & R2 + 3R3
 

0 0 1 5 −2 −1

Osama Sohrab Linear Algebra 94 / 186

  
1 0 0 −40 16 9
 0 1 0 13 −5 −3  R1 − 2R2
 

0 0 1 5 −2 −1
Thus  
−40 16 9
A−1 =  13 −5 −3
 

5 −2 −1

Osama Sohrab Linear Algebra 95 / 186

Often it will not be known in advance if a given n × n matrix A is invertible.
However, if it is not, then it will be impossible to reduce A to In by elementary
row operations. This will be signaled by a row of zeros appearing on the left side
of the partition at some stage of the inversion algorithm. If this occurs, then you
can stop the computations and conclude that A is not invertible.

Osama Sohrab Linear Algebra 96 / 186

Showing that a Matrix is not Invertible
Consider the matrix  
1 6 4
A= 2 4 −1 .
 

−1 2 5
The computations are as follows:
 
1 6 4 1 0 0
 2 4 −1 0 1 0 
 

−1 2 5 0 0 1
 
1 6 4 1 0 0
 0 −8 −9 −2 1 0  R2 − 2R1 & R3 + R1
 

0 8 9 1 0 1
Osama Sohrab Linear Algebra 97 / 186
  
1 6 4 1 0 0
 0 −8 −9 −2 1 0  R3 + R2
 

0 0 0 −1 1 1

Since we have obtained a row of zeros on the left side, A is not invertible.

Osama Sohrab Linear Algebra 98 / 186

Practice Problem
Find the inverse of the following matrix, if it exists.
 
0 1 2
A = 1 0 3
 

4 −3 8

Osama Sohrab Linear Algebra 99 / 186

Inverse of a Diagonal Matrix
A diagonal matrix is invertible if and only if all of its diagonal entries are nonzero; in this
case, the inverse of D is
 
1/d1 0 ··· 0
 0 1/d2 · · · 0
 
−1

D =
 .. .. ... .. 
 . . .


0 0 · · · 1/dn

Osama Sohrab Linear Algebra 100 / 186

Power of a Diagonal Matrix
Powers of diagonal matrices are easy to compute; we leave it for you to verify that if D
is the diagonal matrix above and k is a positive integer, then
 
dk1 0 · · · 0
 0 dk2 · · · 0 
 
k
D =  .. .. . . . .. 

. . .
0 0 · · · dkn

Osama Sohrab Linear Algebra 101 / 186

For example, if  
1 0 0
A = 0 −3 0
 

0 0 2
then
     
1 0 0 1 0 0 1 0 0
A−1 = 0 − 13 0  , 5
A = 0 −243 0 , A−5 1
= 0 − 243 0.
     

0 0 12 0 0 32 0 0 1
32

Osama Sohrab Linear Algebra 102 / 186

Practice Problem
If det(A) = 2 and det(B) = 5, calculate det(A3 B −1 AT B 2 ).

Osama Sohrab Linear Algebra 103 / 186

Rank
The number of nonzero rows in the echelon form of a matrix is called the rank of
the matrix.

Osama Sohrab Linear Algebra 104 / 186

Theorem
An n × n square matrix A has rank n if and only if

det A ̸= 0

Osama Sohrab Linear Algebra 105 / 186

Problem
For what value of a does the system has non-trivial solutions.

x + 2y + z = 0
x + 3y + 6z = 0
2x + 3y + az = 0

Osama Sohrab Linear Algebra 106 / 186

Practice Problem
Compute the rank of  
1 1 −1 4
A = 2 1 3 0 .
 

0 1 −5 8

Osama Sohrab Linear Algebra 107 / 186

Linear Equation
A linear equation in n variables x1 , x2 , . . . , xn is an equation that can be expressed in
the form
a1 x 1 + a2 x 2 + . . . + an x n = b

where a′ s are not all zero. If b = 0 in the above equation then we have

a1 x1 + a2 x2 + . . . + an xn = 0

which is called a homogeneous linear equation in the variables x1 , x2 , . . . , xn .

Osama Sohrab Linear Algebra 108 / 186

 A system of linear equations (or more briefly, a linear system) is a collection of
one or more linear equations involving the same variables.

For example,

2x + 3y = 4
5x + 6y = 7

or,

5p + 2q − 3r = 4
5p + 6q + 7r = 8
9p − 4q + 2r = 3
Osama Sohrab Linear Algebra 109 / 186
A general linear system of m equations in the n unknowns x1 , x2 , . . . , xn can be written
as
a11 x1 + a12 x2 + · · · + a1n xn = b1
a21 x1 + a22 x2 + · · · + a2n xn = b2
.. .. .. .. (⋆)
. . . .
am1 x1 + am2 x2 + · · · + amn xn = bm

Osama Sohrab Linear Algebra 110 / 186

 A solution of a linear system in n unknowns x1 , x2 , . . . , xn is a sequence of n
numbers s1 , s2 , . . . , sn for which the substitution

x1 = s 1 , x2 = s2 , . . . , xn = sn

makes each equation a true statement.

For example, x = 1, y = −2 is a solution of

5x + y = 3
2x − y = 4

Osama Sohrab Linear Algebra 111 / 186

 A linear system is said to be consistent if it has at least one solution, and
inconsistent if it has no solutions. If a linear system is consistent, it has ei-
ther
a unique solution (that is, precisely one solution), or
infinitely many solutions.

For example, consider the linear system

a1 x + b 1 y = c 1
a1 x + b 1 y = c 2

in which the graphs of the equations are lines in the xy-plane. There are three possibilities.
Osama Sohrab Linear Algebra 112 / 186
Osama Sohrab Linear Algebra 113 / 186
For a linear system with three equations and three variables, each equation in the system
represents a plane in 3D-space.

Figure 1: Unique Solution

Osama Sohrab Linear Algebra 114 / 186

 (a) Infinite Solution (b) No Solution

Osama Sohrab Linear Algebra 115 / 186

In matrix form, we can write the linear system (⋆) as
    
a11 a12 · · · a1n x1 b1
 a21 a22 · · · a2n   x2   b2 
    
 .  =  . 
 . .. .. 
...
   
 .
 . . .   ..   .. 
am1 am2 · · · amn xn bm

Let    
a11 a12 · · · a1n a11 a12 · · · a1n b1
 a21 a22 · · · a2n   a21 a22 · · · a2n b2
   

A=
 .. .. .. ,
..  Ã = [A | b] = 
 .. .. .. .. .. 
 . . . .   . . . . .


am1 am2 · · · amn am1 am2 · · · amn bm

A is called coefficient matrix and à = [A | b] is called an augmented matrix.

Osama Sohrab Linear Algebra 116 / 186
Homogeneous Linear System
A linear system is said to be homogeneous if the constant terms on the right hand side
of (⋆) are all zero; that is, the system has the form

a11 x1 + a12 x2 + · · · + a1n xn = 0

a21 x1 + a22 x2 + · · · + a2n xn = 0
.. .. .. ..
. . . .
am1 x1 + am2 x2 + · · · + amn xn = 0

Every homogeneous linear system is consistent because all such systems have x1 =
0, x2 = 0, · · · , xn = 0 as a solution. This solution is called the trivial solution; if
thereare other solutions, they are called nontrivial solutions.

Osama Sohrab Linear Algebra 117 / 186

Because a homogeneous linear system always has the trivial solution, there are only two
possibilities for its solutions:
The system has only the trivial solution.
The system has infinitely many solutions in addition to the trivial solution.

Osama Sohrab Linear Algebra 118 / 186

The set of all possible solutions is called the solution set of the linear system.
Two linear systems are called equivalent if they have the same solution set. That
is, each solution of the first system is a solution of the second system, and each
solution of the second system is a solution of the first.

Osama Sohrab Linear Algebra 119 / 186

Fundamental Theorem for Linear Systems

(a) Existence: A linear system of m equations in n unknowns x1 , x2 , · · · , xn

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
.. .. .. .. (⋆)
. . . .
am1 x1 + am2 x2 + · · · + amn xn = bm

is consistent, that is, has solution(s), if and only if the coefficient matrix A and the
augmented matrix à = [A|b] have the same rank.
(b) Uniqueness: The system (⋆) has precisely one solution if and only if this common
rank r of A and equals n.

Osama Sohrab Linear Algebra 120 / 186

(c) Infinitely many solutions: If this common rank r is less than n, the system (⋆)
has infinitely many solutions. All of these solutions are obtained by determining r
suitable unknowns in terms of the remaining n − r unknowns, to which arbitrary
values can be assigned.
(d) Gauss Elimination: If solutions exist, they can all be obtained by the Gauss
elimination.

Osama Sohrab Linear Algebra 121 / 186

 r1 = Rank of Coefficient Matrix
r2 = Rank of Augmented Matrix
n = Number of Unknowns
Linear System

r1 = r2 r1 ̸= r2

Consistent Inconsistent

r1 = r2 = n r1 = r2 < n

Unique Solution Infinite Solution

Osama Sohrab Linear Algebra 122 / 186

Practice Problem
Use fundamental theorem of linear system to check the consistency of the following linear
system. If the system is consistent, then find the solution set.

8x + 5y + 11z = 30
−x − 4y + 2z = 3
2x − y + 5z = 12

Osama Sohrab Linear Algebra 123 / 186

Practice Problem
Find a, b and c so that the linear system

x + ay + cz = 0
bx + cy − 3z = 1
ax + 2y + bz = 5

has the solution x = 3, y = −1, z = 2.

Osama Sohrab Linear Algebra 124 / 186

Methods for Solving a Linear System

Gaussian Elimination with Back-substitution

❶ Write the augmented matrix of the linear system.
❷ Use elementary row operations to rewrite the matrix in row-echelon form.
❸ Write the linear system corresponding to the matrix in row-echelon form, and
use back-substitution to find the solution.

Osama Sohrab Linear Algebra 125 / 186

Problem
Solve the following linear system.
x2 + x3 − 2x4 = −3
x1 + 2x2 − x3 = 2
2x1 + 4x2 + x3 − 3x4 = −2
x1 − 4x2 − 7x3 − x4 = −19
Solution
The augmented matrix for this system is
 
0 1 1 −2 −3
 
 1 2 −1 0 2 
 
 2 4
 1 −3 −2 

1 −4 −7 −1 −19
Osama Sohrab Linear Algebra 126 / 186
  
1 2 −1 0 2
 
 0 1 1 −2 −3 
  R1 ↔ R2
 2 4
 1 −3 −2 

1 −4 −7 −1 −19

 
1 2 −1 0 2
 
 0 1
 1 −2 −3 
 R3 − 2R1 & R4 − R1
 0 0
 3 −3 −6 

0 −6 −6 −1 −21

Osama Sohrab Linear Algebra 127 / 186

  
1 2 −1 0 2
 
 0 1 1 −2 −3 
  R4 + 6R1
 0 0
 3 −3 −6  
0 0 0 −13 −39

 
1 2 −1 0 2
 
 0 1 1 −2 −3  1 1
  R3 & − R4
 0 0
 1 −1 −2 
 3 13
0 0 0 1 3

Osama Sohrab Linear Algebra 128 / 186

From R4 we have x4 = 3 . From R3 , we have

x3 − x4 = −2
⇒ x3 = −2 + x4
⇒ x3 = −2 + 3
⇒ x3 = 1

From R2 we have

x2 + x3 − 2x4 = −3
⇒ x2 = −3 − x3 + 2x4
⇒ x2 = −3 − 1 + 6
⇒ x2 = 2
Osama Sohrab Linear Algebra 129 / 186
Similarly from R1 we have

x1 + 2x2 − x3 = 2
⇒ x1 = 2 − 2x2 + x3
⇒ x1 = 2 − 4 + 1
⇒ x1 = −1

Thus    
x1 −1
   
x2   2 
 = 
x   1 
 3  
x4 3

Osama Sohrab Linear Algebra 130 / 186

Gauss-Jordan Elimination
In Gaussian elimination, we apply elementary row operations to a matrix to obtain a
row-echelon form. A second method of elimination, called Gauss-Jordan elimination
after Carl Friedrich Gauss and Wilhelm Jordan (1842–1899), continues the reduction
process until a reduced row-echelon form is obtained.

Osama Sohrab Linear Algebra 131 / 186

Problem
Use Gauss-Jordan elimination to solve the system.

x − 2y + 3z = 9
−x + 3y = −4
2x − 5y + 5z = 17

Solution
The augmented matrix for this system is
 
1 −2 3 9
 −1 3 0 −4 
 

2 −5 5 17

Osama Sohrab Linear Algebra 132 / 186

  
1 −2 3 9
 0 1 3 5  R2 + R1 & R3 − 2R1
 

0 −1 −1 −1

 
1 −2 3 9
 0 1 3 5  R3 + R2
 

0 0 2 4

 
1 −2 3 9
1
 0 1 3 5  R3
 
2
0 0 1 2

Osama Sohrab Linear Algebra 133 / 186

  
1 0 9 19
 0 1 3 5  R1 + 2R2
 

0 0 1 2
 
1 0 0 1
 0 1 0 −1  R1 − 9R3 & R2 − 3R3
 

0 0 1 2

From R3 we have x3 = 2 . From R2 , we have x2 = −1 and from R1 we have

x1 = 1 . Thus
   
x1 1
x2  = −1
   

x3 2
Osama Sohrab Linear Algebra 134 / 186
Problem
Use Gauss-Jordan elimination to solve the system.

x1 + 3x2 − 2x3 + 2x5 = 0

2x1 + 6x2 − 5x3 − 2x4 + 4x5 − 3x6 = −1
5x3 + 10x4 + 15x6 = 5
2x1 + 6x2 + 8x4 + 4x5 + 18x6 = 6

Solution
The RREF of the augmented matrix is

Osama Sohrab Linear Algebra 135 / 186

  
1 3 0 4 2 0 0
 
 0 0 1 2 0 0 0 
 
 0 0 0 0 0 1 1 
 3 
0 0 0 0 0 0 0
1
From R3 , we have x6 = . From R2 , we have x3 + 2x4 = 0 ⇒ x3 = −2x4 . From R1 ,
3
we have x1 + 3x2 + 4x4 + 2x5 = 0 ⇒ x1 = −3x2 − 4x4 − 2x5 . Thus
           
x1 −3x2 − 4x4 − 2x5 −3 −4 −2 0
           
x2   x2  1 0  0  0
           
x  
 3  −2x4 
0
 
−2
   0  0
 =  = x2   + x 4   + x5   +  
    
x4   x4  0 1  0  0
           
x   x5 0 0
      1  0
 5      
1 1
x6 3
0 0 0 3
Osama Sohrab Linear Algebra 136 / 186
 Linear System

m=n m ̸= n

det(A) ̸= 0 det(A) = 0 Gaussian Elimination Gauss-Jordan Elimination

Unique Solution Gaussian Elimination Gauss-Jordan Elimination

Matrix Inversion Gaussian Elimination Cramer’s Rule

Gauss-Jordan Elimination

m is number of equations and n is number of unknowns.

Osama Sohrab Linear Algebra 137 / 186

Theorem ( Crammer’s Rule )

If a system of n linear equations in n unknowns has a coefficient matrix A with a

nonzero determinant |A|, then the solution of the system is

det(A1 ) det(A2 ) det(An )

x1 = , x2 = , ..., xn =
det(A) det(A) det(A)

where the ith column of Ai is the column of constants in the system of equations.

Osama Sohrab Linear Algebra 138 / 186

Problem
Use Cramer’s Rule to solve the system of linear equations for x.

−x + 2y − 3z = 1
2x +z =0
3x − 4y + 4z = 2

Solution
The determinant of the coefficient matrix is
−1 2 −3
|A| = 2 0 1 = 10.
3 −4 4

The determinant is nonzero, so the solution is unique.

Osama Sohrab Linear Algebra 139 / 186
Apply Cramer’s Rule to solve for x, as shown below.

1 2 −3
0 0 1
2 −4 4 (1)(−1)5|2 − 4| (1)(−1)(−8) 4
x= = = =
10 10 10 5

By Applying Cramer’s Rule to solve for y and z. We will get

3 8
y=− z=−
2 5
Thus    4 
x 5
   3
y  = − 2 
z − 85
Osama Sohrab Linear Algebra 140 / 186
 Theorem
If A is an invertible n × n matrix, then for every n × 1 matrix b, the system of
equations Ax = b has exactly one solution, namely, x = A−1 b.

For example, consider the linear system

x1 + 2x2 + 3x3 = 5
2x1 + 5x2 + 3x3 = 3
x1 + 8x3 = 17

Osama Sohrab Linear Algebra 141 / 186

In matrix form this system can be written as Ax = b, where
     
1 2 3 x1 5
A = 2 5 3 , x = x2  , b =  3 
     

1 0 8 x3 17
 
−40 16 9
A−1 =  13 −5 −3
 

5−2 −1
    
−40 16 9 5 1
−1
x = A b =  13 −5 −3  3  = −1
    

5 −2 −1 17 2
or x1 = 1, x2 = −1, x3 = 2.
Osama Sohrab Linear Algebra 142 / 186
A network is a set of branches through which something “flows.” For example, the
branches might be electrical wires through which electricity flows, pipes through which
water or oil flows, traffic lanes through which vehicular traffic flows, or economic linkages
through which money flows. A point where two or more branches meet is called a node
or a junction.

A
A & B are junctions

Osama Sohrab Linear Algebra 143 / 186

 30

D
35 55
B
15

60
Figure 3: A, B, C, and D are junctions.

Osama Sohrab Linear Algebra 144 / 186

Junction Rule
At each of the junction in the network, the total flow into that junction must equal
the total flow out.

I2

I1
I3

Figure 4: I1 = I2 + I3

Osama Sohrab Linear Algebra 145 / 186

Problem
A network of one-way streets is shown in the accompanying diagram. The rate of flow of
cars into junction A is 500 cars per hour, and 400 and 100 cars per hour emerge from B
and C, respectively. Find the possible flows along each street.

Osama Sohrab Linear Algebra 146 / 186

Solution
Suppose the flows along the streets are f1 , f2 , f3 , f4 , f5 and f6 cars per hour in the
directions as shown in the figure below.

Osama Sohrab Linear Algebra 147 / 186

By applying junction rule at each node, we have

At A 500 = f1 + f2 + f3
At B f1 + f4 + f6 = 400
At C f3 + f5 = f6 + 100
At D f2 = f4 + f5

So we have a linear system of four equations in six unknowns.

f1 + f2 + f3 = 500
f1 + f4 + f6 = 400
f3 + f5 − f6 = 100
f2 − f4 − f5 = 0

Osama Sohrab Linear Algebra 148 / 186

Augmented Matrix is  
1 1 1 0 | 500
0 0
 
1
 0 0 1 0 1 | 400

0
 0 1 0 1 −1 | 100

0 1 0 −1 −1 0 | 0

RREF of the augmented matrix is

 
1 0 0 1 | 400
0 1
 
0 1
 0 −1 −1 0 | 0  
0 0
 1 0 1 −1 | 100

0 0 0 0 0 0 | 0

The system is consistent and number of free variables = 6 − 3 = 3.

Osama Sohrab Linear Algebra 149 / 186
From R1 , R2 and R3 we have the following:

f1 = 400 − f4 − f6
f2 = f4 + f5
f3 = 100 − f5 + f6

Thus    
f1 400 − f4 − f6
   
f2   f4 + f5 
   
f  100 − f + f 
 3  5 6
 = 
f4   f4 
   
f   f 
 5  5 
f6 f6
Osama Sohrab Linear Algebra 150 / 186
Of course, not all these solutions may be acceptable in the real situation. For example,
the flows f1 , f2 , · · · , f6 are all positive in the present context. This imposes constraints
on the flows: f1 ≥ 0 and f3 ≥ 0 which becomes

f4 + f6 ≤ 400
f5 − f6 ≤ 100

Osama Sohrab Linear Algebra 151 / 186

Practice Problems
Exercise Set 1.1: 1, 2, 5,6
Exercise Set 1.2: 1,2, 15-22
Exercise Set 1.3: 1, 3, 13
Exercise Set 1.4: 10
Exercise Set 1.5: 11, 12
Exercise Set 1.10: 1-4

Osama Sohrab Linear Algebra 152 / 186

Balancing Chemical Equations
Chemical compounds are represented by chemical formulas that describe the atomic
makeup of their molecules. For example, water is composed of two hydrogen atoms and
one oxygen atom, so its chemical formula is H2 O; and stable oxygen is composed of two
oxygen atoms, so its chemical formula is O2 .
A chemical equation is said to be balanced if for each type of atom in the reaction, the
same number of atoms appears on each side of the arrow. For example

CH4 + 2O2 −→ CO2 + 2H2 O

by which we mean that one methane molecule combines with two stable oxygen molecules
to produce one carbon dioxide molecule and two water molecules.

Osama Sohrab Linear Algebra 153 / 186

If a chemical equation is not balanced, for example

CH4 + O2 −→ CO2 + H2 O

The standard convention is to use the smallest positive integers that will balance the
equation. To balance this equation we must find positive integers x1 , x2 , x3 and x4 such
that
x1 (CH4 ) + x2 (O2 ) −→ x3 (CO2 ) + x4 (H2 O)

For each of the atoms in the equation, the number of atoms on the left must be equal
to the number of atoms on the right. Expressing this in tabular form we have

Osama Sohrab Linear Algebra 154 / 186

 Left Side Right Side
Carbon x1 = x3
Hydrogen 4x1 = 2x4
Oxygen 2x2 = 2x3 + x4

from which we obtain the homogeneous linear system

x1 − x3 = 0
4x1 − 2x4 = 0
2x2 − 2x3 − x4 = 0

Osama Sohrab Linear Algebra 155 / 186

The augmented matrix for this system is
 
1 0 −1 0 0
 4 0 0 −2 0 
 

0 2 −2 −1 0

The reduced row echelon form of this matrix is

 
1 0 0 − 12 0
 0 1 0 −1 0 
 

0 0 1 − 12 0
from which we conclude that the general solution of the system is
1 1
x1 = x4 , x2 = x4 , x3 = x4
2 2
Osama Sohrab Linear Algebra 156 / 186
where x4 is arbitrary. The smallest positive integer values for the unknowns occur when
we let x4 = 2, so the equation can be balanced by letting x1 = 1, x2 = 2, x3 = 1,
x4 = 2.

Osama Sohrab Linear Algebra 157 / 186

Practice Problem
Write a balanced equation for the given chemical reaction.
(a)
C3 H8 + O2 → CO2 + H2 O [propane combustion]

(b)
C6 H12 O6 → CO2 + C2 H5 OH [fermentation of sugar]

Osama Sohrab Linear Algebra 158 / 186

Polynomial Interpolation
An important problem in various applications is to find a polynomial whose graph passes
through a specified set of points in the plane; this is called an interpolating polynomial
for the points. The simplest example of such a problem is to find a linear polynomial

y = ax + b (3)

whose graph passes through two known distinct points, (x1 , y1 ) and (x2 , y2), in the xy-
plane. You have probably encountered various methods in analytic geometry for finding
the equation of a line through two points, but here we will give a method based on linear
systems that can be adapted to general polynomial interpolation.

Osama Sohrab Linear Algebra 159 / 186

Osama Sohrab Linear Algebra 160 / 186
The graph of (3) is the line y = ax + b, and for this line to pass through the points
(x1 , y1 ), (x2 , y2 ), we must have

y1 = ax1 + b and y2 = ax2 + b

so we have the following linear system

ax1 + b = y1
ax2 + b = y2

Solving the linear system we can obtain a and b and then by putting these value in (3)
we can get the equation of line.

Osama Sohrab Linear Algebra 161 / 186

Theorem ( Polynomial Interpolation )

Given any n points in the xy-plane that have distinct x-coordinates, there is a
unique polynomial of degree n − 1 or less whose graph passes through those points.

Osama Sohrab Linear Algebra 162 / 186

Problem
Find a polynomial that interpolates the points (−2, 3), (−1, 5), (0, 1), (1, 4) and (2, 10).
Solution
Since we have five points, we will choose a fourth-degree polynomial
p(x) = a0 + a1 x + a2 x2 + a3 x3 + a4 x4 .
Substituting the given points into p(x) produces the linear system
a0 − 2a1 + 4a2 − 8a3 + 16a4 = 3
a0 − a1 + a2 − a3 + a4 = 5
a0 = 1
a0 + a1 + a2 + a3 + a4 = 4
a0 + 2a1 + 4a2 + 8a3 + 16a4 = 10
Osama Sohrab Linear Algebra 163 / 186
Solving the linear system, we obtain

30 101 18 17
a0 = 1, a1 = −
, a2 = , a3 = , a4 = −
24 24 24 24
which means the polynomial function is

30 101 2 18 3 17 4
p(x) = 1 − x+ x + x − x
24 24 24 24

1
= (24 − 30x + 101x2 + 18x3 − 17x4 ).
24

Osama Sohrab Linear Algebra 164 / 186

Practice Problem
A forester wants to estimate the age (in years) of a tree by measuring the diameter of
the trunk (in cm). He obtains the following data:

Tree 1 Tree 2 Tree 3

Trunk Diameter 5 10 15
Age 3 5 6

Estimate the age of a tree with a trunk diameter of 12 cm.

Osama Sohrab Linear Algebra 165 / 186

Practice Problem
Three Nissans, two Fords, and four Chevrolets can be rented for $106 per day. At the
same rates, two Nissans, four Fords, and three Chevrolets cost $107 per day, whereas
four Nissans, three Fords, and two Chevrolets cost $102 per day. Find the rental rates
for all three kinds of cars.

Osama Sohrab Linear Algebra 166 / 186

Practice Problem
Solve the following system of equations for x and y.

x2 + xy − y 2 = 1
2x2 − xy + 3y 2 = 13
x2 + 3xy + 2y 2 = 0

[Hint: These equations are linear in the new variables x1 = x2 , x2 = xy, and x3 = y 2 .]

Osama Sohrab Linear Algebra 167 / 186

Graph Theory
Directed Graphs
A directed graph is a finite set of elements, {P1 , P2 , · · · , Pn } together with a finite
collection of ordered pairs (Pi , Pj ) of distinct elements of this set, with no ordered pair
being repeated. The elements of the set are called vertices, and the ordered pairs are
called directed edges, of the directed graph.

Figure 5
Osama Sohrab Linear Algebra 168 / 186
We use the notation Pi → Pj (which is read ”Pi is connected to Pj ”) to indicate that
the directed edge (Pi , Pj ) belongs to the directed graph. Geometrically, we can visualize
a directed graph by representing the vertices as points in the plane and representing the
directed edge Pi → Pj by drawing a line or arc from vertex Pi to vertex Pj , with an arrow
pointing from Pi to Pj . If both Pi → Pj and Pj → Pi hold (denoted Pi ↔ Pj ), we draw
a single line between Pi and Pj with two oppositely pointing arrows (as with P2 and P3
in the Figure 5).
A directed graph may have separate “components” of vertices that are connected only
among themselves; and some vertices, such as P5 , may not be connected with any other
vertex. Also, because Pi → Pi is not permitted in a directed graph, a vertex cannot be
connected with itself by a single arc that does not pass through any other vertex.

Osama Sohrab Linear Algebra 169 / 186

Figure 6 shows diagrams representing three more examples of directed graphs. With a
directed graph having n vertices, we may associate an n × n matrix M = [mij ], called
the vertex matrix of the directed graph. Its elements are defined by
(
1 if Pi → Pj
mij =
0 otherwise
for i, j = 1, 2, · · · , n.

(a) (b) (c)

Figure 6
Osama Sohrab Linear Algebra 170 / 186
For the three directed graphs in Figure 6 the corresponding vertex matrices are
 
  0 1 0 0 1  
0 1 0 0 0 1 0 0
0 0 1 1 0
 
   
0 0 1 0   1 0 1 0
M = 0 1 0 1 ,
 M = 0 0 0 1 0 , M = 1 0 0 1

0 1 0 0 1
     
0 0 0 0 1 0 0 0
0 1 1 0 0

By their definition, vertex matrices have the following two properties:

All entries are either 0 or 1.
All diagonal entries are 0.
Conversely, any matrix with these two properties determines a unique directed
graph having the given matrix as its vertex matrix.
Osama Sohrab Linear Algebra 171 / 186
For example, the matrix  
0 1 1 0
 
0 0 1 0
M =
1

 0 0 1

0 0 0 0

determines the directed graph in Figure 7.

Figure 7
Osama Sohrab Linear Algebra 172 / 186
Influences Within a Family
A certain family consists of a mother, father, daughter, and two sons. The family members
have influence, or power, over each other in the following ways: the mother can influence
the daughter and the oldest son; the father can influence the two sons; the daughter can
influence the father; the oldest son can influence the youngest son; and the youngest son
can influence the mother. We may model this family influence pattern with a directed
graph whose vertices are the five family members. If family member A influences family
member B , we write A → B. Figure 174 is the resulting directed graph, where we have
used obvious letter designations for the five family members. The vertex matrix of this
directed graph is

Osama Sohrab Linear Algebra 173 / 186

 Figure 8

The father cannot directly influence the mother; that is, F → M is not true. But he
can influence the youngest son, who can then influence the mother. We write this as
F → Y S → M and call it a 2-step connection from F to M . Analogously, we call
M → D a 1-step connection, F → OS → Y S → M a 3-step connection, and so
forth.
Osama Sohrab Linear Algebra 174 / 186
Theorem
(r)
Let M be the vertex matrix of a directed graph and let mij be the (i, j)-th element
(r)
of M r . Then mij is equal to the number of r-step connections from Pi to Pj .

Osama Sohrab Linear Algebra 175 / 186

Figure 9 is the route map of a small airline that services the four cities P1 , P2 , P3 , P4 .

Figure 9

Osama Sohrab Linear Algebra 176 / 186

As a directed graph, its vertex matrix is
 
0 1 1 0
 
1 0 1 0
M =
1

 0 0 1

0 1 1 0

Also,
   
2 0 1 1 1 3 3 1
   
1 1 1 1 2 2 3 1
M2 = 
0
, and M 3 =  
 2 2 0

4
 0 2 2

2 0 1 1 1 3 3 1

Osama Sohrab Linear Algebra 177 / 186

If we are interested in connections from city P4 to city P3 , we may use the previous
theorem to find their number. Because m43 = 1, there is one 1-step connection; because
(2) (3)
m43 = 1, there is one 2-step connection; and because m43 = 3, there are three 3-step
connections. To verify this, from Figure (9) we find

1-step connections from P4 to P3 : P4 → P 3

2-step connections from P4 to P3 : P4 → P 2 → P3

3-step connections from P4 to P3 : P4 → P 3 → P4 → P3

P4 → P2 → P1 → P3
P4 → P3 → P1 → P3

Osama Sohrab Linear Algebra 178 / 186

Practice Problems
Exercise Set 10.5: 1, 2, 3

Osama Sohrab Linear Algebra 179 / 186

The set of all ordered n-tuples of real numbers is denoted by Rn . When convenient,
ordered n-tuples can be denoted in matrix notation as column vectors. For example, the
matrix  
s1
 s2 
 
.
.
.
sn
can be used as an alternative to (s1 , s2 , · · · , sn )

Osama Sohrab Linear Algebra 180 / 186

It may have occurred to you that an ordered pair (v1 , v2 ) can represent either a
vector with components v1 and v2 or a point with coordinates v1 and v2 (and
similarly for ordered triples). Both are valid geometric interpretation. The term
“vector” is used in various ways in mathematics, physics, engineering, and other
applications.

Osama Sohrab Linear Algebra 181 / 186

 CH # 04 General Vector Spaces

Osama Sohrab Linear Algebra 182 / 186

Although the notions of vectors and scalars that we will study in this text have their
origins in physics and engineering, we will be more concerned with using them to build
mathematical structures and then applying those structures to such diverse fields as
genetics, computer science and economics.

Osama Sohrab Linear Algebra 183 / 186

4.1: Real Vector Spaces
Definition Let V be an arbitrary nonempty set of objects on which two operations are
defined: addition, and multiplication by numbers called scalars. By addition we mean
a rule for associating with each pair of objects u and v in V an object u + v, called
the sum of u and v; by scalar multiplication we mean a rule for associating with each
scalar k and each object u in V an object ku, called the scalar multiple of u by k. If
the following axioms are satisfied by all objects u, v, w in V and all scalars k and m,
then we call V a vector space and we call the objects in V vectors.

Osama Sohrab Linear Algebra 184 / 186

❶ If u and v are objects in V , then u + v is in V .
❷ u + v = v + u.
❸ u + (v + w) = (u + v) + w.
❹ There is an object 0 in V , called a zero vector for V , such that
0 + u = u + 0 = u for all u in V .
❺ For each u in V , there is an object −u in V , called a negative of u, such that
u + (−u) = (−u) + u = 0.
❻ If k is any scalar and u is any object in V , then ku is in V .
❼ k(u + v) = ku + kv.
❽ (k + m)u = ku + mu.
❾ k(mu) = (km)(u).
❿ 1u = u.
Osama Sohrab Linear Algebra 185 / 186
Examples
Rn with the Standard Operations Is a Vector Space

Let V = Rn , and define the vector space operations on V to be the usual operations of
addition and scalar multiplication of n-tuples; that is,

u + v = (u1 , u2 , . . . , un ) + (v1 , v2 , . . . , vn ) = (u1 + v1 , u2 + v2 , . . . , un + vn )

ku = (ku1 , ku2 , . . . , kun )

The set V = Rn is closed under addition and scalar multiplication because the foregoing
operations produce n-tuples as their end result, and these operations satisfy Axioms 2, 3,
4, 5, 7, 8, 9, and 10.

Osama Sohrab Linear Algebra 186 / 186