Lecture 9

Lecture 9

Matrix Algebra

9.1 Sums of Matrices

We begin with the definition of matrix addition.

Definition 9.1: Given matrices

   
a11 a12 · · · a1n b11 b12 · · · b1n
 a21 a22 · · · a2n   b21 b22 · · · b2n 
   
A =  .. .. .. ..  , B =  .. .. .. ..  ,
 . . . .   . . . . 
am1 am2 · · · amn bm1 bm2 · · · bmn

both of the same dimension m × n, the sum A + B is defined as

 
a11 + b11 a12 + b12 ··· a1n + b1n
 a21 + b21 a22 + b22 ··· a2n + b2n 
 
A+B= .. .. .. .. .
 . . . . 
am1 + bm1 am2 + bm2 · · · amn + bmn

Next is the definition of scalar-matrix multiplication.

Definition 9.2: For a scalar α we define αA by

   
a11 a12 · · · a1n αa11 αa12 · · · αa1n
 a21 a22 · · · a2n   αa21 αa22 · · · αa2n 
   
αA = α  .. .. .. ..  =  .. .. .. ..  .
 . . . .   . . . . 
am1 am2 · · · amn αam1 αam2 · · · αamn

75
Matrix Algebra

Example 9.3. Given A and B below, find 3A − 2B.

   
1 −2 5 5 0 −11
A = 0 −3
 9 , B =  3 −5 1 
4 −6 7 −1 −9 0
Solution. We compute:
   
3 −6 15 10 0 −22
3A − 2B =  0 −9 27 −  6 −10 2 
12 −18 21 −2 −18 0
 
−7 −6 37
= −6 1 25
14 0 21

Below are some basic algebraic properties of matrix addition/scalar multiplication.

Theorem 9.4: Let A, B, C be matrices of the same size and let α, β be scalars. Then

(a) A + B = B + A (d) α(A + B) = αA + αB

(b) (A + B) + C = A + (B + C) (e) (α + β)A = αA + βA
(c) A + 0 = A (f) α(βA) = (αβ)A

9.2 Matrix Multiplication

Let TB : Rp → Rn and let TA : Rn → Rm be linear mappings. If x ∈ Rp then TB (x) ∈ Rn
and thus we can apply TA to TB (x). The resulting vector TA (TB (x)) is in Rm . Hence, each
x ∈ Rp can be mapped to a point in Rm , and because TB and TA are linear mappings the
resulting mapping is also linear. This resulting mapping is called the composition of TA
and TB , and is usually denoted by TA ◦ TB : Rp → Rm (see Figure 9.1). Hence,

(TA ◦ TB )(x) = TA (TB (x)).

Because (TA ◦ TB ) : Rp → Rm is a linear mapping it has an associated standard matrix,

which we denote for now by C. From Lecture 8, to compute the standard matrix of any
linear mapping, we must compute the images of the standard unit vectors e1 , e2 , . . . , ep under
the linear mapping. Now, for any x ∈ Rp ,

TA (TB (x)) = TA (Bx) = A(Bx).

Applying this to x = ei for all i = 1, 2, . . . , p, we obtain the standard matrix of TA ◦ TB :

 
C = A(Be1 ) A(Be2 ) · · · A(Bep ) .

76
 Lecture 9

Rm
Rp TB Rn TA
b b

x b TB (x) TA (TB(x))

(TA ◦ TB )(x)

Figure 9.1: Illustration of the composition of two mappings.

Now Be1 is
 
Be1 = b1 b2 · · · bp e1 = b1 .

And similarly Bei = bi for all i = 1, 2, . . . , p. Therefore,

 
C = Ab1 Ab2 · · · Abp

is the standard matrix of TA ◦ TB . This computation motivates the following definition.

 
Definition 9.5: For A ∈ Rm×n and B ∈ Rn×p , with B = b1 b2 · · · bp , we define the
product AB by the formula
 
AB = Ab1 Ab2 · · · Abp .

The product AB is defined only when the number of columns of A equals the number of
rows of B. The following diagram is useful for remembering this:

(m × n) · (n × p) → m × p

From our definition of AB, the standard matrix of the composite mapping TA ◦ TB is

C = AB.

In other words, composition of linear mappings corresponds to matrix multiplication.

Example 9.6. For A and B below compute AB and BA.

 
  −4 2 4 −4
1 2 −2
A= , B =  −1 −5 −3 3 
1 1 −3
−4 −4 −3 −1

77
Matrix Algebra

Solution. First AB = [Ab1 Ab2 Ab3 Ab4 ]:

 
  −4 2 4 −4
1 2 −2
AB =  −1 −5 −3 3 
1 1 −3
−4 −4 −3 −1

2
=
7

2 0
=
7 9

2 0 4
=
7 9 10
 
2 0 4 4
=
7 9 10 2

On the other hand, BA is not defined! B has 4 columns and A has 2 rows.

Example 9.7. For A and B below compute AB and BA.

   
−4 4 3 −1 −1 0
A =  3 −3 −1  , B =  −3 0 −2 
−2 −1 1 −2 1 −2

Solution. First AB = [Ab1 Ab2 Ab3 ]:

  
−4 4 3 −1 −1 0
AB =  3 −3 −1   −3 0 −2 
−2 −1 1 −2 1 −2

−14
=  8
3

−14 7
=  8 −4
3 3
 
−14 7 −14
= 8 −4 8 
3 3 0

78
 Lecture 9

Next BA = [Ba1 Ba2 Ba3 ]:

  
−1 −1 0 −4 4 3
BA =  −3 0 −2   3 −3 −1 
−2 1 −2 −2 −1 1

1
= 16

15

1 −1
=  16 −10
15 −9
 
1 −1 −2
=  16 −10 −11 
15 −9 −9

On the other hand:  

−14 7 −14
AB =  8 −4 8 
3 3 0
Therefore, in general AB 6= BA, i.e., matrix multiplication is not commutative.

An important matrix that arises frequently is the identity matrix In ∈ Rn×n of size
n:  
1 0 0 ··· 0
0 1 0 · · · 0
 
In =  .. .. .. .. 
. . . · · · .
0 0 0 ··· 1
You should verify that for any A ∈ Rn×n it holds that AIn = In A = A. Below are some
basic algebraic properties of matrix multiplication.

Theorem 9.8: Let A, B, C be matrices, of appropriate dimensions, and let α be a scalar.

Then
(1) A(BC) = (AB)C
(2) A(B + C) = AB + AC
(3) (B + C)A = BA + CA
(4) α(AB) = (αA)B = A(αB)
(5) In A = AIn = A

If A ∈ Rn×n is a square matrix, the kth power of A is

Ak = |AAA
{z· · · A}
k times

79
Matrix Algebra

Example 9.9. Compute A3 if  

−2 3
A= .
1 0
Solution. Compute A2 :
    
2 −2 3 −2 3 7 −6
A = =
1 0 1 0 −2 3
And then A3 :
  
3 2 7 −6 −2 3
A =A A=
−2 3 1 0
 
−20 21
=
7 −6
We could also do:
    
3 2 −2 3 7 −6 −20 21
A = AA = = .
1 0 −2 3 7 −6

9.3 Matrix Transpose

We begin with the definition of the transpose of a matrix.

Definition 9.10: Given a matrix A ∈ Rm×n , the transpose of A is the matrix AT whose
ith column is the ith row of A.

If A is m × n then AT is n × m. For example, if

 
0 −1 8 −7 −4
 −4 6 −10 −9 6 
A=  9

5 −2 −3 5 
−8 8 4 7 7
then  
0 −4 9 −8

 −1 6 5 8 

AT = 
 8 −10 −2 4 
.
 −7 −9 −3 7 
−4 6 5 7

Example 9.11. Compute (AB)T and BT AT if

 
  −2 1 2
−2 1 0
A= , B =  −1 −2 0 .
3 −1 −3
0 0 −1

80
 Lecture 9

Solution. Compute AB:

 
  −2 1 2
−2 1 0
AB =  −1 −2 0 
3 −1 −3
0 0 −1
 
3 −4 −4
=
−5 5 9

Next compute BT AT :
  
−2 −1 0 −2 3
BT AT =  1 −2 0   1 −1 
2 0 −1 0 −3
 
3 −5
= −4
 5  = (AB)T
−4 9

The following theorem summarizes properties of the transpose.

Theorem 9.12: Let A and B be matrices of appropriate sizes. The following hold:
(1) (AT )T = A
(2) (A + B)T = AT + BT
(3) (αA)T = αAT
(4) (AB)T = BT AT

A consequence of property (4) is that

(A1 A2 . . . Ak )T = ATk ATk−1 · · · AT2 AT1

and as a special case

(Ak )T = (AT )k .

Example 9.13. Let T : R2 → R2 be the linear mapping that first contracts vectors by a
factor of k = 3 and then rotates by an angle θ. What is the standard matrix A of T?
Solution. Let e1 = (1, 0) and e2 = (0, 1) denote the standard
 unit vectors in R2 . From
Lecture 8, the standard matrix of T is A = T(e1 ) T(e2 ) . Recall that the standard matrix
of a rotation by θ is  
cos(θ) − sin(θ)
sin(θ) cos(θ)
Contracting e1 by a factor of k = 3 results in ( 13 , 0) and then rotation by θ results in
1 
cos(θ)
3
1 = T(e1 ).
3
sin(θ)

81
Matrix Algebra

Contracting e2 by a factor of k = 3 results in (0, 13 ) and then rotation by θ results in

 1 
− 3 sin(θ)
1 = T(e2 ).
3
cos(θ)

Therefore, "1 #
  3
cos(θ) − 13 sin(θ)
A = T(e1 ) T(e2 ) =
1 1
3
sin(θ) 3
cos(θ)
1
On the other hand, the standard matrix corresponding to a contraction by a factor k = 3
is
"1 #
3
0
1
0 3

Therefore, " # "1 # "1 #

cos(θ) − sin(θ) 3
0 3
cos(θ) − 13 sin(θ)
= =A
sin(θ) cos(θ) 0 1 1
sin(θ) 1
cos(θ)
| {z } | {z 3 } 3 3
rotation contraction

After this lecture you should know the following:

• know how to add and multiply matrices
• that matrix multiplication corresponds to composition of linear mappings
• the algebraic properties of matrix multiplication (Theorem 9.8)
• how to compute the transpose of a matrix
• the properties of matrix transposition (Theorem 9.12)

82
 Lecture 10

Lecture 10

Invertible Matrices

10.1 Inverse of a Matrix

The inverse of a square matrix A ∈ Rn×n generalizes the notion of the reciprocal of a non-
zero number a ∈ R. Formally speaking, the inverse of a non-zero number a ∈ R is the unique
number c ∈ R such that ac = ca = 1. The inverse of a 6= 0, usually denoted by a−1 = a1 , can
be used to solve the equation ax = b:

ax = b ⇒ a−1 ax = a−1 b ⇒ x = a−1 b.

This motivates the following definition.

Definition 10.1: A matrix A ∈ Rn×n is called invertible if there exists a matrix C ∈

Rn×n such that AC = In and CA = In .

If A is invertible then can it have more than one inverse? Suppose that there exists C1 , C2
such that ACi = Ci A = In . Then

C2 = C2 (AC1 ) = (C2 A)C1 = In C1 = C1 .

Thus, if A is invertible, it can have only one inverse. This motivates the following definition.

Definition 10.2: If A is invertible then we denote the inverse of A by A−1 . Thus,

AA−1 = A−1 A = In .

Example 10.3. Given A and C below, show that C is the inverse of A.

   
1 −3 0 −14 −3 −6
A =  −1 2 −2  , C =  −5 −1 −2 
−2 6 1 2 0 1

83
Invertible Matrices

Solution. Compute AC:

    
1 −3 0 −14 −3 −6 1 0 0
AC =  −1 2 −2   −5 −1 −2  = 0 1 0
−2 6 1 2 0 1 0 0 1

Compute CA:
    
−14 −3 −6 1 −3 0 1 0 0
CA =  −5 −1 −2   −1 2 −2  = 0 1 0
2 0 1 −2 6 1 0 0 1

Therefore, by definition C = A−1 .

Theorem 10.4: Let A ∈ Rn×n and suppose that A is invertible. Then for any b ∈ Rn
the matrix equation Ax = b has a unique solution given by A−1 b.

Proof: Let b ∈ Rn be arbitrary. Then multiplying the equation Ax = b by A−1 from the
left we obtain that

A−1 Ax = A−1 b
⇒ In x = A−1 b
⇒ x = A−1 b.

Therefore, with x = A−1 b we have that

Ax = A(A−1 b) = AA−1 b = In b = b

and thus x = A−1 b is a solution. If x̃ is another solution of the equation, that is, Ax̃ = b,
then multiplying both sides by A−1 we obtain that x̃ = A−1 b. Thus, x = x̃. 
Example 10.5. Use the result of Example 10.3. to solve the linear system Ax = b if
   
1 −3 0 1
A = −1
 2 −2 , b = −3 .
 
−2 6 1 −1

Solution. We showed in Example 10.3 that

 
−14 −3 −6
A−1 =  −5 −1 −2  .
2 0 1

Therefore, the unique solution to the linear system Ax = b is

    
−14 −3 −6 1 1
A−1 b =  −5 −1 −2  −3 = 0
2 0 1 −1 1

84
 Lecture 10

Verify:     
1 −3 0 1 1
 −1 2 −2  0 = −3
−2 6 1 1 −1

The following theorem summarizes the relationship between the matrix inverse and ma-
trix multiplication and matrix transpose.

Theorem 10.6: Let A and B be invertible matrices. Then:

(1) The matrix A−1 is invertible and its inverse is A:

(A−1 )−1 = A.

(2) The matrix AB is invertible and its inverse is B−1 A−1 :

(AB)−1 = B−1 A−1 .

(3) The matrix AT is invertible and its inverse is (A−1 )T :

(AT )−1 = (A−1)T .

Proof: To prove (2) we compute

(AB)(B−1A−1 ) = ABB−1A−1 = AIn A−1 = AA−1 = In .

To prove (3) we compute

AT (A−1)T = (A−1 A)T = ITn = In .

10.2 Computing the Inverse of a Matrix

 
If A ∈ Mn×n is invertible, how do we find A−1 ? Let A−1 = c1 c2  · · · cn and we will
find expressions for ci . First note that AA−1 = Ac1 Ac2 · · · Acn . On the other hand,
we also have AA−1 = In = e1 e2 · · · en . Therefore, we want to find c1 , c2 , . . . , cn such
that    
Ac1 Ac2 · · · Acn = e1 e2 · · · en .
| {z } | {z }
AA−1 In

To find ci we therefore need to solve the linear system

 Ax
 = ei . Here the image vector “b”
is ei . To find c1 we form the augmented matrix A e1 and find its RREF:
   
A e1 ∼ In c1 .

85
Invertible Matrices

We will need to do this for each

 c2 , . . . , cn so we might as well form the combined augmented
matrix A e1 e2 · · · en and find the RREF all at once:
   
A e1 e2 · · · en ∼ In c1 c2 · · · cn .

In summary, to determine if A−1 exists and to simultaneously compute it, we compute the
RREF of the augmented matrix  
A In ,
that is, A augmented with the n × n identity matrix. If the RREF of A is In , that is
   
A I n ∼ I n c1 c2 · · · cn

then  
A−1 = c1 c2 · · · cn .
If the RREF of A is not In then A is not invertible.
 
1 3
Example 10.7. Find the inverse of A = if it exists.
−1 −2
 
Solution. Form the augmented matrix A I2 and row reduce:
 
  1 3 1 0
A I2 =
−1 −2 0 1

Add rows R1 and R2 :    

1 3 1 0 R1 +R2 1 3 1 0
−−−−→
−1 −2 0 1 0 1 1 1
−3R +R
Perform the operation −−−2−−→
1
:
   
1 3 1 0 −3R2 +R1 1 0 −2 −3
−−−−−→
0 1 1 1 0 1 1 1

Thus, rref(A) = I2 , and therefore A is invertible. The inverse is

 
−1 −2 −3
A =
1 1

Verify:     
−1 1 3 −2 −3 1 0
AA = = .
−1 −2 1 1 0 1

 
1 0 3
Example 10.8. Find the inverse of A =  1 1 0  if it exists.
−2 0 −7

86
 Lecture 10

 
Solution. Form the augmented matrix A I3 and row reduce:
   
1 0 3 1 0 0 1 0 3 1 0 0
−R1 +R2 , 2R1 +R2
1 1 0 0 1 0 −−−−−−−−−−→ 0 1 −3 −1 1 0
−2 0 −7 0 0 1 0 0 −1 2 0 1
−R3 :    
1 0 3 1 0 0 1 0 3 1 0 0
−R3
0 1 −3 −1 1 0 − −→ 0 1 −3 −1 1 0 
0 0 −1 2 0 1 0 0 1 −2 0 −1
3R3 + R2 and −3R3 + R1 :
   
1 0 3 1 0 0 1 0 0 7 0 3
3R3 +R2 , −3R3 +R1
0 1 −3 −1 1 0  − −−−−−−−−−−→ 0 1 0 −7 1 −3
0 0 1 −2 0 −1 0 0 1 −2 0 −1
Therefore, rref(A) = I3 , and therefore A is invertible. The inverse is
 
7 0 3
A−1 = −7 1 −3
−2 0 −1
Verify:     
1 0 3 7 0 3 1 0 0
AA−1 =  1 1 0  −7 1 −3 = 0 1 0
−2 0 −7 −2 0 −1 0 0 1

 
1 0 1
Example 10.9. Find the inverse of A =  1 1 −2  if it exists.
−2 0 −2
 
Solution. Form the augmented matrix A I3 and row reduce:
   
1 0 1 1 0 0 1 0 1 1 0 0
−R1 +R2 , 2R1 +R2
 1 1 −2 0 1 0 − −−−−−−−−−→ 0 1 −3 −1 1 0
−2 0 −2 0 0 1 0 0 0 2 0 1
We need not go further since the rref(A) is not I3 (rank(A) = 2 ). Therefore, A is not
invertible.

10.3 Invertible Linear Mappings

Let TA : Rn → Rn be a matrix mapping with standard matrix A and suppose that A is
invertible. Let TA−1 : Rn → Rn be the matrix mapping with standard matrix A−1 . Then
the standard matrix of the composite mapping TA−1 ◦ TA : Rn → Rn is

A−1 A = In .

87
Invertible Matrices

Therefore, (TA−1 ◦ TA )(x) = In x = x. Let’s unravel (TA−1 ◦ TA )(x) to see this:

(TA−1 ◦ TA )(x) = TA−1 (TA (x)) = TA−1 (Ax) = A−1 Ax = x.

Similarly, the standard matrix of (TA ◦ TA−1 ) is also In . Intuitively, the linear mapping TA−1
undoes what TA does, and conversely. Moreover, since Ax = b always has a solution, TA is
onto. And, because the solution to Ax = b is unique, TA is one-to-one.

The following theorem summarizes equivalent conditions for matrix invertibility.

Theorem 10.10: Let A ∈ Rn×n . The following statements are equivalent:

(a) A is invertible.
(b) A is row equivalent to In , that is, rref(A) = In .
(c) The equation Ax = 0 has only the trivial solution.
(d) The linear transformation TA (x) = Ax is one-to-one.
(e) The linear transformation TA (x) = Ax is onto.
(f) The matrix equation Ax = b is always solvable.
(g) The columns of A span Rn .
(h) The columns of A are linearly independent.
(i) AT is invertible.

Proof: This is a summary of all the statements we have proved about matrices and matrix
mappings specialized to the case of square matrices A ∈ Rn×n . Note that for non-square
matrices, one-to-one does not imply ontoness, and conversely.

Example 10.11. Without doing any arithmetic, write down the inverse of the dilation
matrix " #
3 0
A= .
0 5
Example 10.12. Without doing any arithmetic, write down the inverse of the rotation
matrix " #
cos(θ) − sin(θ)
A= .
sin(θ) cos(θ)

After this lecture you should know the following:

• how to compute the inverse of a matrix
• properties of matrix inversion and matrix multiplication
• relate invertibility of a matrix with properties of the associated linear mapping (1-1,
onto)
• the characterizations of invertible matrices Theorem 10.10

88
 Lecture 11

Lecture 11

Determinants

11.1 Determinants of 2 × 2 and 3 × 3 Matrices

Consider a general 2 × 2 linear system

a11 x1 + a12 x2 = b1
a21 x1 + a22 x2 = b2 .

Using elementary row operations, it can be shown that the solution is

b1 a22 − b2 a12 b2 a11 − b1 a21
x1 = , x2 = ,
a11 a22 − a12 a21 a11 a22 − a12 a21
provided that a11 a22 − a12 a21 6= 0. Notice the denominator is the same in both expressions.
The number a11 a22 − a12 a21 then completely characterizes when a 2 × 2 linear system has a
unique solution. This motivates the following definition.

Definition 11.1: Given a 2 × 2 matrix

 
a a
A = 11 12
a21 a22

we define the determinant of A as

 
a11 a12
det A = det = a11 a22 − a12 a21 .
a21 a22

An alternative notation for det A is using vertical bars:

   
a11 a12 a11 a12 
det =  .
a21 a22 a21 a22 

89
Determinants

Example 11.2. Compute the determinant of A.

 
3 −1
(i) A =
8 2
 
3 1
(ii) A =
−6 −2
 
−110 0
(iii) A =
568 0
Solution. For (i):  
3 −1
det(A) = 
  = (3)(2) − (8)(−1) = 14
8 2
For (ii):  
3 1 
det(A) = 
 = (3)(−2) − (−6)(1) = 0
−6 −2
For (iii):  
−110 0
det(A) =   = (−110)(0) − (568)(0) = 0
568 0

As in the 2 × 2 case, the solution of a 3 × 3 linear system Ax = b can be shown to be

Numerator1 Numerator2 Numerator3
x1 = , x2 = , x3 =
D D D
where
D = a11 (a22 a33 − a23 a32 ) − a12 (a21 a33 − a23 a31 ) + a13 (a21 a32 − a22 a31 ).
Notice that the terms of D in the parenthesis are determinants of 2 × 2 submatrices of A:
D = a11 (a22 a33 − a23 a32 ) − a12 (a21 a33 − a23 a31 ) + a13 (a21 a32 − a22 a31 ).
|  {z  } |  {z  } |  {z  }
a22 a23  a21 a23  a21 a22 
     
     
a32 a33  a31 a33  a31 a32 
     

Let      
a22 a23 a21 a23 a21 a22
A11 = , A12 = , and A13 = .
a32 a33 a31 a33 a31 a32
Then we can write
D = a11 det(A11 ) − a12 det(A12 ) + a13 det(A13 ).
 
a22 a23
The matrix A11 = is obtained from A by deleting the 1st row and the 1st column:
a32 a33
 
a11 a12 a13  
a 22 a23
A = a21 a22 a23  −→ A11 =
 .
a32 a33
a31 a32 a33

90
 Lecture 11

 
a21 a23
Similarly, the matrix A12 = is obtained from A by deleting the 1st row and the
a31 a33
2nd column:  
a11 a12 a13  
a21 a 23
A = a21 a22 a23  −→ A12 = .
a31 a33
a31 a32 a33
 
a21 a22
Finally, the matrix A13 = is obtained from A by deleting the 1st row and the 3rd
a31 a32
column:  
a11 a12 a13  
a21 a22
A = a21 a22 a23  −→ .
a31 a32
a31 a32 a33
Notice also that the sign in front of the coefficients a11 , a12 , and a13 , alternate. This motivates
the following definition.

Definition 11.3: Let A be a 3 × 3 matrix. Let Ajk be the 2 × 2 matrix obtained from
A by deleting the jth row and kth column. Define the cofactor of ajk to be the number
Cjk = (−1)j+k det Ajk . Define the determinant of A to be

det A = a11 C11 + a12 C12 + a13 C13 .

This definition of the determinant is called the expansion of the determinant along the
first row. In the cofactor Cjk = (−1)j+k det Ajk , the expression (−1)j+k will evaluate to
either 1 or −1, depending on whether j + k is even or odd. For example, the cofactor of a12
is
C12 = (−1)1+2 det A12 = − det A12
and the cofactor of a13 is
C13 = (−1)1+3 det A13 = det A13 .
We can also compute the cofactor of the other entries of A in the obvious way. For example,
the cofactor of a23 is
C23 = (−1)2+3 det A23 = − det A23 .
A helpful way to remember the sign (−1)j+k of a cofactor is to use the matrix
 
+ − +
− + − .
+ − +
This works not just for 3 × 3 matrices but for any square n × n matrix.

Example 11.4. Compute the determinant of the matrix

 
4 −2 3
A = 2 3 5
1 0 6

91
Determinants

Solution. From the definition of the determinant

det A = a11 C11 + a12 C12 + a13 C13

= (4) det A11 − (−2) det A12 + (3) det A13

     
3 5 2 5 2 3
= 4  1 6 + 3 1 0
+2   
0 6

= 4(3 · 6 − 5 · 0) + 2(2 · 6 − 1 · 5) + 3(2 · 0 − 1 · 3)

= 72 + 14 − 9

= 77

We can compute the determinant of a matrix A by expanding along any row or column.
For example, the expansion of the determinant for the matrix
 
a11 a12 a13
A = a21 a22 a23 
a31 a32 a33

along the 3rd row is

     
a12 a13  a11 a13  a11 a12 
det A = a31   − a32 
a21 a23  + a33 a21 a22  .
  
a22 a23 

And along the 2nd column:

     
a21 a23  a11 a13  a11 a13 
det A = −a12 
  + a22 
  − a32 
 .
a31 a33  a31 a33  a21 a23 

The punchline is that any way you choose to expand (row or column) you will get the same
answer. If a particular row or column contains zeros, say entry ajk , then the computation of
the determinant is simplified if you expand along either row j or column k because ajk Cjk = 0
and we need not compute Cjk .

Example 11.5. Compute the determinant of the matrix

 
4 −2 3
A= 2  3 5
1 0 6

Solution. In Example 11.4, we computed det(A) = 77 by expanding along the 1st row.

92
 Lecture 11

Notice that a32 = 0. Expanding along the 3rd row:

det A = (1) det A31 − (0) det A32 + (6) det A33
   
−2 3 4 −2
=   +6 
3 5 2 3 

= 1(−2 · 5 − 3 · 3) + 6(4 · 3 − (−2) · 2)

= −19 + 96

= 77

11.2 Determinants of n × n Matrices

Using the 3 × 3 case as a guide, we define the determinant of a general n × n matrix as
follows.

Definition 11.6: Let A be a n × n matrix. Let Ajk be the (n − 1) × (n − 1) matrix

obtained from A by deleting the jth row and kth column, and let Cjk = (−1)j+k det Ajk
be the (j, k)-cofactor of A. The determinant of A is defined to be

det A = a11 C11 + a12 C12 + · · · + a1n C1n .

The next theorem tells us that we can compute the determinant by expanding along any
row or column.

Theorem 11.7: Let A be a n × n matrix. Then det A may be obtained by a cofactor

expansion along any row or any column of A:

det A = aj1 Cj1 + aj2 Cj2 + · · · + ajn Cjn .

We obtain two immediate corollaries.

Corollary 11.8: If A has a row or column containing all zeros then det A = 0.

Proof. If the jth row contains all zeros then aj1 = aj2 = · · · = ajn = 0:

det A = aj1 Cj1 + aj2 Cj2 + · · · + ajn Cjn = 0.

93
Determinants

Corollary 11.9: For any square matrix A it holds that det A = det AT .

Sketch of the proof. Expanding along the jth row of A is equivalent to expanding along
the jth column of AT .

Example 11.10. Compute the determinant of

 
1 3 0 −2
 1 2 −2 −1 
A=
 0

0 2 1 
−1 −3 1 0

Solution. The third row contains two zeros, so expand along this row:

det A = 0 det A31 − 0 det A32 + 2 det A33 − det A34

   
 1 3 −2   1 3 0 

= 2  1 2 −1  −  1 2 −2 
 −1 −3 0   −1 −3 1 
      
 2 −1  1 −1 1 2 
= 2 1  − 3 − 2 
−3 0  −1 0  −1 −3
    
 2 −2  1 −2
− 1  − 3 
−3 1  −1 1 

= 2((0 − 3) − 3(0 − 1) − 2(−3 + 2)) − ((2 − 6) − 3(1 − 2))

=5

Example 11.11. Compute the determinant of

 
1 3 0 −2
 1 2 −2 −1 
A=
 0

0 2 1 
−1 −3 1 0

94
 Lecture 11

Solution. Expanding along the second row:

det A = − det A21 + 2 det A22 − (−2) det A23 − 1 det A24
   
 3 0 −2   1 0 −2 
   
= −  0 2 1  + 2  0 2 1 
 −3 1 0   −1 1 0 
   
 1 3 −2   1 3 0 
   
+ 2  0 0 1  −  0 0 2 
 −1 −3 0   −1 −3 1 

= −1(−3 − 12) + 2(−1 − 4) + 2(0) − (0)

=5

11.3 Triangular Matrices

Below we introduce a class of matrices for which the determinant computation is trivial.

Definition 11.12: A square matrix A ∈ Rn×n is called upper triangular if ajk = 0

whenever j > k. In other words, all the entries of A below the diagonal entries aii are
zero. It is called lower triangular if ajk = 0 whenever j < k.

For example, a 4 × 4 upper triangular matrix takes the form

 
a11 a12 a13 a14
 0 a22 a23 a24 
A= 0

0 a33 a34 
0 0 0 a44
Expanding along the first column, we compute
 
a22 a23 a24    
 a33 a34 
det A = a11  0 a33 a34  = a11 a22   = a11 a22 a33 a44 .
0 0 a44

0 a44 
The general n × n case is similar and is summarized in the following theorem.

Theorem 11.13: The determinant of a triangular matrix is the product of its diagonal
entries.

After this lecture you should know the following:

• how to compute the determinant of any sized matrix
• that the determinant of A is equal to the determinant of AT
• the determinant of a triangular matrix is the product of its diagonal entries

95
Determinants

96
 Lecture 12

Lecture 12

Properties of the Determinant

12.1 ERO and Determinants

Recall that for a matrix A ∈ Rn×n we defined
det A = aj1 Cj1 + aj2 Cj2 + · · · + ajn Cjn
where the number Cjk = (−1)j+k det Ajk is called the (j, k)-cofactor of A and
 
aj = aj1 aj2 · · · ajn
denotes the jth row of A. Notice that
 
Cj1
 Cj2 

 
det A = aj1 aj2 · · · ajn  ..  .
 . 
Cjn
 
If we let cj = Cj1 Cj2 · · · Cjn then
det A = aj · cTj .
In this lecture, we will establish properties of the determinant under elementary row opera-
tions and some consequences. The following theorem describes how the determinant behaves
under elementary row operations of Type 1.

Theorem 12.1: Suppose that A ∈ Rn×n and let B be the matrix obtained by interchang-
ing two rows of A. Then det B = − det A.
   
a11 a12 a21 a22
Proof. Consider the 2 × 2 case. Let A = and let B = . Then
a21 a22 a11 a12
det B = a12 a21 − a11 a22 = −(a11 a22 − a12 a21 ) = − det A.
The general case is proved by induction.

This theorem leads to the following corollary.

97
Properties of the Determinant

Corollary 12.2: If A ∈ Rn×n has two rows (or two columns) that are equal then
det(A) = 0.

Proof. Suppose that A has rows j and k that are equal. Let B be the matrix obtained by
interchanging rows j and k. Then by the previous theorem det B = − det A. But clearly
B = A, and therefore det B = det A. Therefore, det(A) = − det(A) and thus det A = 0.


Now we consider how the determinant behaves under elementary row operations of Type
2.

Theorem 12.3: Let A ∈ Rn×n and let B be the matrix obtained by multiplying a row of
A by β. Then det B = β det A.

Proof. Suppose that B is obtained from A by multiplying the jth row by β. The rows of A
and B different from j are equal, and therefore

Bjk = Ajk , for k = 1, 2, . . . , n.

In particular, the (j, k) cofactors of A and B are equal. The jth row of B is βaj . Then,
expanding det B along the jth row:

det B = (βaj ) · cTj

= β(aj · cTj )

= β det A.

Lastly we consider Type 3 elementary row operations.

Theorem 12.4: Let A ∈ Rn×n and let B be the matrix obtained from A by adding β
times the kth row to the jth row. Then det B = det A.

Proof. For any matrix A and any row vector r = [r1 r2 · · · rn ] the expression

r · cTj = r1 Cj1 + r2 Cj2 + · · · + rn Cjn

is the determinant of the matrix obtained from A by replacing the jth row with the row r.
Therefore, if k 6= j then
ak · cTj = 0

98
 Lecture 12

since then rows k and j are equal. The jth row of B is bj = aj + βak . Therefore, expanding
det B along the jth row:

det B = (aj + βak ) · cTj

= aj · cTj + β ak · cTj

= det A.

Example 12.5. Suppose that A is a 4 × 4 matrix and suppose that det A = 11. If B is
obtained from A by interchanging rows 2 and 4, what is det B?

Solution. Interchanging (or swapping) rows changes the sign of the determinant. Therefore,

det B = −11.

Example 12.6. Suppose that A is a 4 × 4 matrix and suppose that det A = 11. Let
a1 , a2 , a3 , a4 denote the rows of A. If B is obtained from A by replacing row a3 by 3a1 + a3 ,
what is det B?

Solution. This is a Type 3 elementary row operation, which preserves the value of the de-
terminant. Therefore,
det B = 11.

Example 12.7. Suppose that A is a 4 × 4 matrix and suppose that det A = 11. Let
a1 , a2 , a3 , a4 denote the rows of A. If B is obtained from A by replacing row a3 by 3a1 + 7a3 ,
what is det B?

Solution. This is not quite a Type 3 elementary row operation because a3 is multiplied by
7. The third row of B is b3 = 3a1 + 7a3 . Therefore, expanding det B along the third row

det B = (3a1 + 7a3 ) · cT3

= 3a1 · cT3 + 7a3 · cT3

= 7(a3 · cT3 )

= 7 det A

= 77

99
Properties of the Determinant

Example 12.8. Suppose that A is a 4 × 4 matrix and suppose that det A = 11. Let
a1 , a2 , a3 , a4 denote the rows of A. If B is obtained from A by replacing row a3 by 4a1 + 5a2 ,
what is det B?

Solution. Again, this is not a Type 3 elementary row operation. The third row of B is
b3 = 4a1 + 5a2 . Therefore, expanding det B along the third row

det B = (4a1 + 5a2 ) · cT3

= 4a1 · cT3 + 5a2 · cT3

=0+0

=0

12.2 Determinants and Invertibility of Matrices

The following theorem characterizes invertibility of matrices with the determinant.

Theorem 12.9: A square matrix A is invertible if and only if det A 6= 0.

Proof. Beginning with the matrix A, perform elementary row operations and generate a
sequence of matrices A1 , A2 , . . . , Ap such that Ap is in row echelon form and thus triangular:

A ∼ A1 ∼ A2 ∼ · · · ∼ Ap .

Thus, matrix Ai is obtained from Ai−1 by performing one of the elementary row operations.
From Theorems 12.1, 12.3, 12.4, if det Ai−1 6= 0 then det Ai 6= 0. In particular, det A = 0 if
and only if det Ap = 0. Now, Ap is triangular and therefore its determinant is the product
of its diagonal entries. If all the diagonal entries are non-zero then det A = det Ap 6= 0. In
this case, A is invertible because there are r = n leading entries in Ap . If a diagonal entry
of Ap is zero then det A = det Ap = 0. In this case, A is not invertible because there are
r < n leading entries in Ap . Therefore, A is invertible if and only if det A 6= 0.

12.3 Properties of the Determinant

The following theorem characterizes how the determinant behaves under scalar multiplication
of matrices.

Theorem 12.10: Let A ∈ Rn×n and let B = βA, that is, B is obtained by multiplying
every entry of A by β. Then det B = β n det A.

100
 Lecture 12

Proof. Consider the 2 × 2 case:

 
βa11 βa12 
det(βA) =  
βa12 βa22 

= βa11 · βa22 − βa12 · βa21

= β 2 (a11 a22 − a12 a21 )

= β 2 det A.

Thus, the statement holds for 2 × 2 matrices. Consider a 3 × 3 matrix A. Then

det(βA) = βa11 |βA11| − βa12 |βA12 | + βa13 |βA13 |

= βa11 β 2 |A11 | − βa12 β 2 |A12 | + βa13 β 2 |A13 |

= β 3 (a11 |A11 | − a12 |A12 | + a13 |A13 |)

= β 3 det A.

The general case can be treated using mathematical induction on n.

Example 12.11. Suppose that A is a 4 × 4 matrix and suppose that det A = 11. What is
det(3A)?
Solution. We have

det(3A) = 34 det A

= 81 · 11

= 891

The following theorem characterizes how the determinant behaves under matrix multi-
plication.

Theorem 12.12: Let A and B be n × n matrices. Then

det(AB) = det(A) det(B).

Corollary 12.13: For any square matrix det(Ak ) = (det A)k .

101
Properties of the Determinant

Corollary 12.14: If A is invertible then

1
det(A−1 ) = .
det A

Proof. From AA−1 = In we have that det(AA−1) = 1. But also

det(AA−1 ) = det(A) det(A−1 ).

Therefore
det(A) det(A−1 ) = 1
or equivalently
1
det A−1 = .
det A

Example 12.15. Let A, B, C be n × n matrices. Suppose that det A = 3, det B = 0, and

det C = 7.
(i) Is AC invertible?
(ii) Is AB invertible?
(iii) Is ACB invertible?
Solution. (i): We have det(AC) = det A det C = 3 · 7 = 21. Thus, AC is invertible.
(ii): We have det(AB) = det A det B = 3 · 0 = 0. Thus, AB is not invertible.
(iii): We have det(ACB) = det A det C det B = 3·7·0 = 0. Thus, ACB is not invertible.
After this lecture you should know the following:
• how the determinant behaves under elementary row operations
• that A is invertible if and only if det A 6= 0
• that det(AB) = det(A) det(B)

102
 Lecture 13

Lecture 13

Applications of the Determinant

13.1 The Cofactor Method

Recall that for A ∈ Rn×n we defined

det A = aj1 Cj1 + aj2 Cj2 + · · · + ajn Cjn

where Cjk = (−1)j+k det Ajk is called the (j, k)-Cofactor of A and
 
aj = aj1 aj2 · · · ajn
 
is the jth row of A. If cj = Cj1 Cj2 · · · Cjn then
 
Cj1
  C
  j2 
det A = aj1 aj2 · · · ajn  ..  = aj · cTj .

 . 
Cjn

Suppose that B is the matrix obtained from A by replacing row aj with a distinct row ak .
To compute det B expand along its jth row bj = ak :

det B = ak · cTj = 0.

The Cofactor Method is an alternative method to find the inverse of an invertible matrix.
Recall that for any matrix A ∈ Rn×n , if we expand along the jth row then

det A = aj · cTj .

On the other hand, if j 6= k then

aj · cTk = 0.
In summary, (
det A, if j = k
aj · cTk =
0, if j 6= k.

103
Applications of the Determinant

Form the Cofactor matrix

   
C11 C12 · · · C1n c1
 C21 C22 · · · C2n   c2 
 
 
Cof(A) =  .. .. ..  =  ..  .
 . . ··· .   .
Cn1 Cn2 · · · Cnn cn

Then,

a1
 a2   
T  
A(Cof(A)) =  ..  cT1 cT2 · · · cTn
.
an
 
a1 cT1 a1 cT2 · · · a1 cTn
 
 a cT a cT · · · a cT 
 2 1 2 2 2 n
 
= 
 .. .. .. .. 
 . . . . 
 
T T T
an c1 an c2 · · · an cn
 
det A 0 ··· 0
 
 
 0 det A · · · 0 
 
= 
 .. .. .. .. 
 . . . . 
 
0 0 · · · det A

This can be written succinctly as

A(Cof(A))T = det(A)In .

Now if det A 6= 0 then we can divide by det A to obtain

 
1
A (Cof(A))T = In .
det A

This leads to the following formula for the inverse:

1
A−1 = (Cof(A))T
det A

Although this is an explicit and elegant formula for A−1, it is computationally intensive,
even for 3 × 3 matrices. However, for the 2 × 2 case it provides a useful formula to compute

104
 Lecture 13

   
a b d −c
the matrix inverse. Indeed, if A = we have Cof(A) = and therefore
c d −b a
 
−1 1 d −b
A = .
ad − bc −c a

When does an integer matrix have an integer inverse? We can answer this question
using the Cofactor Method. Let us first be clear about what we mean by an integer matrix.

Definition 13.1: A matrix A ∈ Rm×n is called an integer matrix if every entry of A is

an integer.

Suppose that A ∈ Rn×n is an invertible integer matrix. Then det(A) is a non-zero integer
and (Cof(A))T is an integer matrix. If A−1 is also an integer matrix then det(A−1 ) is also
an integer. Now det(A) det(A−1) = 1 thus it must be the case that det(A) = ±1. Suppose
on the other hand that det(A) = ±1. Then by the Cofactor method

1
A−1 = (Cof(A))T = ±(Cof(A))T
det(A)

and therefore A−1 is also an integer matrix. We have proved the following.

Theorem 13.2: An invertible integer matrix A ∈ Rn×n has an integer inverse A−1 if and
only if det A = ±1.

We can use the previous theorem to generate integer matrices with an integer inverse
as follows. Begin with an upper triangular matrix M0 having integer entries and whose
diagonal entries are either 1 or −1. By construction, det(M0 ) = ±1. Perform any sequence
of elementary row operations of Type 1 and Type 3. This generates a sequence of matrices
M1 , . . . , Mp whose entries are integers. Moreover,

M0 ∼ M1 ∼ · · · ∼ Mp .

Therefore,
±1 = det(M) = det(M1 ) = · · · = det(Mp ).

105
Applications of the Determinant

13.2 Cramer’s Rule

The Cofactor method can be used to give an explicit formula for the solution of a linear
system where the coefficient matrix is invertible. The formula is known as Cramer’s Rule.
To derive this formula, recall that if A is invertible then the solution to Ax = b is x = A−1 b.
Using the Cofactor method, A−1 = det1 A (Cof(A))T , and therefore
  
C11 C21 · · · Cn1 b1
1  12 C22
 C · · · Cn2   b2 
 
x=  . .. .. ..   ..  .
det A  .. . . .  . 
C1n C2n · · · Cnn bn

Consider the first component x1 of x:

1
x1 = (b1 C11 + b2 C21 + · · · + bn Cn1 ).
det A
The expression b1 C11 + b2 C21 + · · · + bn Cn1 is the expansion of the determinant along the
first column of the matrix obtained from A by replacing the first column with b:
 
b1 a12 · · · a1n
 b2 a22 · · · a2n 
 
det  .. .. .. .  = b1 C11 + b2 C21 + · · · + bn Cn1
. . . .. 
bn an2 · · · ann

Similarly,
1
x2 = (b1 C12 + b2 C22 + · · · + bn Cn2 )
det A
and (b1 C12 + b2 C22 + · · · + bn Cn2 ) is the expansion of the determinant along the second
column of the matrix obtained from A by replacing the second column with b. In summary:

Theorem 13.3: (Cramer’s Rule) Let A ∈ Rn×n be an invertible matrix. Let b ∈ Rn

and let Ai be the matrix obtained from A by replacing the ith column with b. Then the
solution to Ax = b is  
det A1
1   det A2 

x=  ..  .
det A  . 
det An

Although this is an explicit and elegant formula for x, it is computationally intensive, and
used mainly for theoretical purposes.

106