Lecture Notes in Linear Algebra

Dr. Abdullah Al-Azemi

Mathematics Department
Kuwait University

January 7, 2019
Contents

1 Linear Equations and Matrices 1

1.1 Systems of Linear Equations and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2.1 Solving System of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.4 Inverses; Algebraic Properties of Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.5 A Method for Finding A−1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

1.6 More on Linear Systems and Invertible Matrices . . . . . . . . . . . . . . . . . . . . . . 36

1.7 Diagonal, Triangular, and Symmetric Matrices . . . . . . . . . . . . . . . . . . . . . . . 45

2 Determinants 53

2.1 Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2.2 Evaluating Determinants by Row Reduction . . . . . . . . . . . . . . . . . . . . . . . . 61

3 Euclidean Vector Spaces 69

3.1 Vectors in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.1.1 Vectors Operations: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.2 Norm and Dot Product in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

4 General Vector Spaces 81

4.1 Real Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

4.2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

i
ii CONTENTS

4.3 Linear Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4.4 Basis and Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.5 Row Space, Column Space, and Null Space . . . . . . . . . . . . . . . . . . . . . . . . . 112

5 Eigenvalues and Eigenvectors 121

5.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.2 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.5 The Cross Product in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

3.3 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

3.4 Lines and Planes in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

 Chapter

1 Linear Equations and Matrices

1.1 Systems of Linear Equations and Matrices

A general linear system of m equations in the n unknowns x1 , x2 , · · · , xn :

a11 x1 + a12 x2 + a13 x3 + · · · + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + · · · + a2n xn = b2
.. (1.1.1)
.
am1 x1 + am2 x2 + am3 x3 + · · · + amn xn = bm
A system in the Form (1.1.1) is called a homogeneous system if b1 = b2 = · · · = bm = 0.
Otherwise it is called a non-homogeneous system.
A solution, if any, of a linear system in n unknowns x1 , x2 , · · · , xn is a sequence of n numbers
s1 , s2 , · · · , sn . That is, x1 = s1 , x2 = s2 , · · · , xn = sn is a solution satisfying all the above
equations.
Any system might have a unique solution, no solutions, or infinite solutions. In general, we
say that the system is consistent if it has at least one solution, and inconsistent if it has no
solutions.

Example 1.1.1

x + y = 1
Solve the following linear system:
2x − y = 5

Solution:

Clearly adding the two (non-homogeneous) equations, we get 3x = 6. Thus, x = 2 and hence
y = −1. Therefore, the system has a unique soltion: x = 2 and y = −1.

1
2 Chapter 1. Linear Equations and Matrices

Example 1.1.2

x + y = 1
Solve the following linear system:
2x + 2y = 2

Solution:

We can eliminate x from the second equation by adding −2 times the first equation to the
second. Thus, we get
x + y = 1
0 = 0
Thus, we simply can omit the second equation. Therefore, the solutions are of the form x = 1−y.
Setting a parameter t ∈ R for y, we get infinite solutions of the form x = 1 − t and y = t.
Therefore, the system has infinite solutions.

Example 1.1.3

x1 + x 2 − x3 = 1
Solve the following linear system: x2 − 2x3 = 0
2x1 + 2x2 − 2x3 = 5

Solution:

Adding the third equation to −2 times the first equation, we get 0 = 3 which is impossible.
Therefore, this system has no solutions.

Definition 1.1.1

An m × n matrix A is a rectangular array of m · n real numbers arranged in m horizontal rows

and n vertical columns. That is,
 
 a11 a12 ... a1j ... a1n 
a21 a22 ... a2j ... a2n
 
.. .. .. .. .. ..
 
 

. . . . . . 
A=
 
 
 ai1 ai2 ... aij ... ain 
.. .. .. .. .. ..
 
. . . . . .
 
 
 
am1 am2 ... amj ... amn
1.1. Systems of Linear Equations and Matrices 3

In the matrix A above, we have:

• a11 , a12 , . . . , amn are called elements (or entries) of the matrix.
• The entry aij lies in the intersection of row i and column j.
• The size (or order) of A is m by n, written as m × n.
• The matrix is called a square matrix if m = n.
• We write Mm×n for the class of all real matrices of size m × n.
If a matrix A is of size 1 × n, then we say that A is a row vector. In addition, if A is of size n × 1,
we say that A is a column vector . In Chapter 3, we speak of n-vectors to be elements of Rn .

Example 1.1.4
 
 3 0 1
 
2 3 −2
Matrix A =  ∈ M2×3 and matrix B =  4 2 0 ∈ M3×3 .
 
4 1 −1  
−1 4 1

Matrix Form of Linear System of Equations:

Given a linear system of m equations on n unknowns as in (1.1.1), we transform this system
into the matrix form as in (1.1.2)
 
 a11 a12 · · · a1n b1 
· · · a2n
 
a21 a22 b2 
.. .. .. .. ..  (1.1.2)


. . . . . 
 

 
am1 am2 · · · amn bm

We say that the form of (1.1.2) is the augmented matrix form . Note that each row in the
augmented matrix form correspond to an equation in the associated system.

Example 1.1.5

Here is an example of transforming a system of equations into its augmented matrix form:
 
x1 + x2 − x3 = 1  1 1 −1 1 
− 2x3 = 0  0 1 −2 0 
 
x2 →  
 
2x1 + 2x2 − 2x3 = 5 2 2 −2 5
4 Chapter 1. Linear Equations and Matrices

The basic method for solving a linear system is to perform algebraic operations on the system that
do not change the solution set so that it produces a simpler version of the same system.

In matrix form, these algebraic operations are called elementary row operations:

? Given a matrix A ∈ Mm×n , we define the following elementary row operations:

1. interchanging a row by another row,

2. multiplying a row by a non-zero scalar,
3. adding a multiple of a row to another row. That is,
replace ai1 , ai2 , · · · , ain by ai1 + cak1 , ai2 + cak2 , · · · , ain + cakn , for 1 ≤ i, k ≤ n.

Example 1.1.6

Use elementary row operations to solve the non-homogeneous system:

x1 + 2x2 + 3x3 = 9
2x1 − x2 + x3 = 8
3x1 − x3 = 3

Solution:

We first transform the system into its augmented matrix form. Then we use the elementary row
operations
 to simplify the form and finally we get the solution
 in thesimpler system.

1 2 3 9 1 2 3 9  − 1 r2 →r2  1 2 3 9
 r2 −2r1 →r2   r1 −2r2 →r1
   
A =  2 −1 1 8 r −3r →r 0 −5 −5 −10 1
 − − −−− −→ 

 −− 5
− − −→

 0 1 1 2 −−−−−−→
 3 1 3 − 2 r3 →r3   r3 −3r2 →r3
3 0 −1 3 0 −6 −10 −24 0 3 5 12
     
1 0 1 5 1 1 0 1 5 1 0 0 2
 2 r3 →r3   r1 −r3 →r1 
0 1 1 2 −−−−→ 0 1 1 2 −−−−−−→ 0 1 0 −1
 
    r2 −r3 →r2  
0 0 2 6 0 0 1 3 0 0 1 3
Therefore, the solution is x1 = 2, x2 = −1, and x3 = 3. That is the system is consistent and has
a unique solution.

Definition 1.1.2

An m × n matrix A is called row equivalent to an m × n matrix B if B can be obtained by

applying a finite sequence of elementary row operation to A. In this case, we write A ≈ B.
1.1. Systems of Linear Equations and Matrices 5

? Properties of matrix equivalence: For any m × n matrices A, B, and C we have

1. A ≈ A,
2. A ≈ B ⇒ B ≈ A,
3. A ≈ B and B ≈ C ⇒ A ≈ C.
6 Chapter 1. Linear Equations and Matrices

1.2 Gaussian Elimination

Definition 1.2.1

A matrix A ∈ Mm×n is said to be in the reduced row echelon form (r.r.e.f. for short) if it
satisfies the following conditions:

1. the row of zeros (if any) should be at the bottom of A,

2. the leading (first) entry of a non-zero row should be 1,
3. the leading entry of row i + 1 should be on the right of the leading entry of row i,
4. the columns that contain a leading entry 1, all its other entries are zeros.

The matrix A is said to be in the row echelon form (r.e.f.) if the last condition is not satisfied.

Example 1.2.1

Here aresome examples of some matrices to clarify the previous conditions:

1 0 0 3 0
(1) A = 
 0 0 1 −3 0 is r.r.e.f.


0 0 0 0 0
1 3 0 4
(2) B = 
0 0 0 0 is not r.r.e.f. since it has a row of zeros in the second row.

 
0 0 1 −2
 
1 0 4 5
(3) C = 0 2 −2 3 is not r.r.e.f. since the leading entry in the second row is not 1.
 
 
0 0 1 2
 
1 0 0 2
(4) D = 0 0 1 3 is not r.r.e.f. since the leading entry of the third row is on the left of
 
 
0 1 0 0
the leading entry of the second row. Switch 2nd and 3rd rows to get the r.r.e.f. form.

Theorem 1.2.1

Every non-zero m × n matrix is row equivalent to a unique matrix in the r.r.e.f.

1.2. Gaussian Elimination 7

1.2.1 Solving System of Linear Equations

We write O to denote the zero matrix whose entries are all 0. We write In (or simply) to denote the
identity matrix with 1’s on the main diagonal and zero elsewhere.

In the augmented matrix form, we add a vertical bar | to recognize the scalars in the last column
of the matrix. That is a system of linear equations might be transformed into augmented matrix form
h i
A | B . Moreover, the system itself is recognized as AX = B, where A is the coefficients, X is the
vector (one column) of unknowns, and B are the scalars.

Solving AX = B

AX = B where B 6= O (non-homogenous) AX = O (homogenous)

unique sol no sol infinite sol trivial sol non-trivial sol

Remark 1.2.1

? Guass-Jordan method for solving AX = B:

h i
1. Find the r.r.e.f. of augmented matrix A | B ,
2. Solve the reduced system.

Note that, the solution of the reduced system is the solution of the original one.

Theorem 1.2.2

Let AX = B and CX = D be two linear systems of equations. If [ A | B ] ≈ [ C | D ], then the

two system have the same solutions.
8 Chapter 1. Linear Equations and Matrices

i. Solving Non-Homogenous System AX = B with B 6= O

Example 1.2.2

Solve the following system using the Guass-Jordan method.

x + 2y + 3z = 9
2x − y + z = 8
3x − z = 3

Solution:
     
1 2 3 9 1 2 3 9 1 1 2 3 9
 r2 −2r1 →r2   − 5 r2 →r2   r1 −2r2 →r1
2 −1 1 8 −−−−−−→ 0 −5 −5 −10 −−−−−→ 0 1 1 2 −−−−−−→

  r3 −3r1 →r3   − 1 r3 →r3   r3 −3r2 →r3
3 0 −1 3 0 −6 −10 −24 2
0 3 5 12
     
1 0 1 5 1 1 0 1 5 1 0 0 2
 2 r3 →r3   r1 −r3 →r1 
0 1 1 2 −−−−→ 0 1 1 2 −−−−−−→ 0 1 0 −1
 
    r2 −r3 →r2  
0 0 2 6 0 0 1 3 0 0 1 3
 
 2 
−1 is a unique
Therefore, the reduced system is: x = 2, y = −1, and z = 3. Thus, X =  
 
3
solution to the system.

Remark 1.2.2

The system AX = B has a unique solution if and only if A ≈ I.

Example 1.2.3

Solve the following system using the Guass-Jordan method.

x1 + x2 − x3 + 4x4 = 1
+ x2 − 3x3 + 4x4 = 0
2x1 + 2x2 − 2x3 + 8x4 = 2

Solution:
1.2. Gaussian Elimination 9

     
1 1 −1 4 1 1 1 −1 4 1 1 0 2 0 1
 r3 −2r1 →r3   r1 −r2 →r1 
0 1 −3 4 0 −− −− − −→ 0 1 −3 4 0 −
− − − − −
→ 0 1 −3 4 0
 
 
   
2 2 −2 8 2 0 0 0 0 0 0 0 0 0 0
Therefor, the reduced system is:

x1 + 2x3 = 1 x1 = 1 − 2x3
⇒
x2 − 3x3 + 4x4 = 0 x2 = 3x3 − 4x4

We now fix x3 = r and x4 = t for r, t ∈ R to get the following infinite many solutions:
 
 1 − 2r 
= 1 − 2r 3r − 4t
 
x1
⇒ X=   for all r, t ∈ R.
x2 = 3r − 4t r
 
 
 
t

Remark 1.2.3

The system AX = B has infinity many solutions if the number of unknowns (columns of matrix
A) is more than the number of equations (rows of matrix A).

Example 1.2.4

Solve the following system using the Guass-Jordan method.

x1 + x2 − x3 + 4x4 = 1
+ x2 − 3x3 + 4x4 = 0
2x1 + 2x2 − 2x3 + 8x4 = 3

Solution:
   
1 1 −1 4 1 1 1 −1 4 1
 r3 −2r1 →r3 
0 1 −3 4 0 −−−−−−→ 0 1 −3 4 0
 
   
2 2 −2 8 2 0 0 0 0 1
At this point, it can be seen that the third row suggests that 0 = 1 which is not possible.
Therefore, this system has no solution.
10 Chapter 1. Linear Equations and Matrices

Remark 1.2.4

The system AX = B has no solution whenever A ≈ a matrix with a row of zeros while [ A | B ] ≈
a matrix with no rows of zeros.

ii. Solving Homogenous System AX = O

Example 1.2.5

Solve the following system using the Guass-Jordan method.

x + 2y + 3z = 0
x + 3y + 2z = 0
2x + y − 2z = 0

Solution:
     
1 2 3 0 1 2 3 0 1 0 5 0 −1
r3 →r3
r −r →r r −2r →r
1 3 2 0 0 1 −1 0 0 1 0
2
−− −−1−−→
2 1
−− −−2−−→
1
−1 −11
−−−−→
     
  r3 −2r1 →r3   r3 +3r2 →r3  
2 1 −2 0 0 −3 −8 0 0 0 −11 0
   
1 0 5 0 1 0 0 0
 r1 −5r3 →r1 
0 1 −1 0 −−−−−−→ 0 1 0 0
 
  r2 +r3 →r2  
0 0 1 0 0 0 1 0
 
0
Therefore, the reduced system is: x = 0, y = 0, and z = 0. Thus, X = 0 is a trivial solution.
 
 
0

Remark 1.2.5

The system AX = O has a trivial solution whenever A ≈ I.

1.2. Gaussian Elimination 11

Example 1.2.6

Solve the following system using the Guass-Jordan method.

x1 + x2 − x3 + 4x4 = 0
+ x2 − 3x3 + 4x4 = 0
2x1 + 2x2 − 2x3 + 8x4 = 0

Solution:
     
1 1 −1 4 0 1 1 −1 4 0 1 0 2 0 0
 r3 −2r1 →r3   r1 −r2 →r1 
0 1 −3 4 0 −−−−−−→ 0 1 −3 4 0 −−−−−−→ 0 1 −3 4 0
 
     
2 2 −2 8 0 0 0 0 0 0 0 0 0 0 0
Therefor, the reduced system is:

x1 + 2x3 = 0 x1 = −2x3
⇒
x2 − 3x3 + 4x4 = 0 x2 = 3x3 − 4x4

We now fix x3 = r and x4 = t for r, t ∈ R to get the following non-trivial solution:

 
 −2r 
= −2r 3r − 4t
 
x1
⇒ X=   for all r, t ∈ R.
x2 = 3r − 4t r
 
 
 
t

Remark 1.2.6

The system AX = O has a non-trivial solution if the number of unknowns (columns of matrix
A) is greater than the number of equations (rows of matrix A) in the reduced system.
12 Chapter 1. Linear Equations and Matrices

Exercise 1.2.1

1. Find the reduced row echelon form (r.r.e.f.) of the following matrix:
 
2 4 6 0
1 2 4 0
 
 
1 3 3 1

x + y + z = 0
x + 2y + 3z = 0
2. Solve the systems:
x + 3y + 4z = 0
x + 4y + 5z = 0.
x1 + 2x2 − 3x3 = 6
3. Solve the systems 2x1 − x2 + 4x3 = 1
x1 − x2 + x3 = 3.
1.3. Matrix Operations 13

1.3 Matrix Operations

Let A be a given m × n matrix  

 a11 a12 ··· a1n 
 
a
 21 a22 ··· a2n 
A=  . .. .. 

.
 .. . . 
 
 
am1 am2 ··· amn
Then we might write A = [aij ] where aij correspond to the entry at row i and column j. We also might
write (A)ij to denote the same entry aij .

We also use the terms: row vector, rowi (A), and column vector, colj , to denote the ith row and
j th column of the matrix A. That is
 
 a1j 
 
 a2j 
 
rowi (A) = ai1 ai2 ··· ain and colj (A) =  . 
 .  .
1×n  . 
 
amj
m×1

An n × n matrix is called a square matrix. Moreover, the entries a11 , a22 , · · · , ann are said to
be on the main diagonal of A.
The trace of A, denoted by tr(A), is the sum of the entries on the main diagonal. If A is not a
square matrix, then the trace of A is undefined.

Remark 1.3.1

Two m × n matrices A and B are said to be equal if (A)ij = (B)ij for all 1 ≤ i ≤ m and
1 ≤ j ≤ n.

Example 1.3.1

Find the values of a, b, c, and d if

   
a + b c + d  1
=
2
.
c−d a−b −2 1

Solution:
14 Chapter 1. Linear Equations and Matrices

Since the two matrices are equal. Then


a+b=1 
+ =⇒ 2a = 2 =⇒ a = 1 and hence b = 0.
a−b=1 

c+d=2 
+ =⇒ 2c = 0 =⇒ c = 0 and hence d = 2.
c − d = −2 

• Matrix transpose

Definition 1.3.1

If A = [aij ] is an m × n matrix, then the n × m matrix AT = [aji ], where 1 ≤ j ≤ n and

1 ≤ i ≤ m, is called the transpose of A. Observe that AT is resulted by interchanging rows
and columns of A.

Example 1.3.2

Here are some examples of matrices and their transpose:

 
a b c  
1 4 0
 
A = d e f , B= , and C= 2 5 7
 
  −1 2 3 1×3
g h i 2×3
3×3
     
a d g 1 −1 2
A = b e
T
h , B = 4 2 
T
, and C =
T
5
     
     
c f i 0 3 7
3×3 3×2 3×1
1.3. Matrix Operations 15

• Matrix addition and substraction

To add two m × n matrices A = [aij ] and B = [bij ], we must have the size(A) = size(B). Then,
A ± B = C, where C = [cij ] with cij = aij ± bij , for all 1 ≤ i ≤ m and 1 ≤ j ≤ n. That is,

(A ± B)ij = (A)ij ± (B)ij = aij ± bij .

Example 1.3.3

If possible, find A + B and AT − B, where

 
2 1  

1 2 4
A= 3 
0

, and B =  .
5 1 1

 
−1 1 2×3
3×2

Solution:

Clearly, A+B is not possible as they have different sizes. On the other hand, size(AT ) = size(B),
and      
2 3 −1 1 2 4 1 1 −5
A −B =
T
− = .
1 0 1 5 1 1 −4 −1 0
2×3 2×3 2×3

• Matrix multiplication

If A = [aij ] is any m × n matrix and c is any scalar, then c A = [c aij ] for all 1 ≤ i ≤ m and 1 ≤ j ≤ n.
That is, (cA)ij = c(A)ij = c aij . The matrix c A is called a scalar multiple of A.

Definition 1.3.2

An n-vector X = (x1 x2 · · · xn ) can be written as X = (x1 , x2 , · · · , xn ). The dot (inner)

product of the n-vectors X = (x1 , x2 , · · · , xn ) and Y = (y1 , y2 , · · · , yn ) is defined by
n
X · Y = x1 y1 + x2 y2 + · · · + xn yn = (1.3.1)
X
xi yi .
i=1
16 Chapter 1. Linear Equations and Matrices

Example 1.3.4

If X = (1, 0, 2, −1) and Y = (3, 5, −1, 4), then X · Y = 3 + 0 + (−2) + (−4) = −3.

Definition 1.3.3

Let A = [aij ] be an m × p matrix and B = [bij ] be a p × n matrix. Then, the product of A and
B is the m × n matrix AB = [(AB)ij ], where (AB)ij = rowi (A) · colj (B). That is,
p
(AB)ij = ai1 b1j + ai2 b2j + · · · + aip bpj = aik bkj , for 1 ≤ i ≤ m and 1 ≤ j ≤ n.
X

k=1

The product is undefined if the number of columns of A not equals the number of rows of B.

Example 1.3.5
 
 2 1
     
1 3 −1 2+0+1 1 + 6 − 2 3 5
If A =  , and B =  0 2 , then AB =  = .
 
2 0 1   4+0−1 2+0+2 3 4
2×3 −1 2 2×2
3×2

Example 1.3.6

Find AB and AT B if possible, where

   
1 0 2 4 1 −1
A= , and B =  .
1 −2 0 2 5 1

Solution:

• AB is not defined # columns in A is not the same as # rows of B.

     
1 1 4 + 2 1 + 5 −1 + 1  6 6 0
 
 4 1 −1
• A B=
T
−2 0

=
 −4 −10 −2  = −4 −10 −2
  
 2 5
 1    
2 0 8 2 −2 8 2 −2
1.3. Matrix Operations 17

Remark 1.3.2
   
1 0 0 0
In general, AB 6= BA. For instance, consider A =   and B =  . Check it yourself!!
1 0 1 1


It is always true that AB = BA if A = In or that A = B.

Example 1.3.7
 
Let A = 1 −2 0 . Find all values of c so that c A · AT = 15.

Solution:
 
   1 
Note that cA · AT = c 1 −2 0 ·

−2

= 5c. Therefore, 5c = 15 and hence c = 3.
0
 

Example 1.3.8
   
1
−2 3   1 4
Let A = 42 1 , and B =  3 −1 . Compute the (3, 2)-entry, (AB)32 , of AB.
   
   
0 1 −2 −2 2

Solution:

(AB)32 = row3 (A) · col2 (B) = [0 1 − 2] · [4 − 1 2]T = −1 − 4 = −5.

Theorem 1.3.1

If A is an m × p matrix and B is a p × n matrix, then

(a) for each 1 ≤ j ≤ n, colj (AB) = A colj (B).

 
(b) for each 1 ≤ i ≤ m, rowi (AB) = rowi (A) B.

TRUE or FALSE:
? If a matrix B has a column of zeros, then the product AB has a column of zeros as well. (TRUE).
reason: Assume that column j of B is a column of zero. Then, colj (AB) = A colj (B) = 0.
18 Chapter 1. Linear Equations and Matrices

Example 1.3.9
 
1 2  

−2 3 4
Let A = 3 4 and B =  .
 
3 2 1

 
−1 5 2×3
3×2

1. Find the first column of AB.

2. Find the second row of AB.

Solution:

We simply use Theorem 1.3.1 as follows:

   
1 2 
4

 −2

1. col1 (AB) = A col1 (B) = 3 4 =6
  
 3

  
−1 5 17
 
−2 3 4
   
2. row2 (AB) = row2 (A) B = 3 4 = 6 17 16
3 2 1

Definition 1.3.4

Let A be an m × n matrix and X ∈ Rn (be an n-vector), then

    
 a11 a12 ··· a1n   x1   a11 x1 + a12 x2 + · · · + a1n xn 
···  a21 x1 + a22 x2 + · · · + a2n xn 
    
 a21 a22 a2n   x2 
AX =  .
 . .. .. ..   ..  = 
    .. 
 . . . .   .   .


am1 x1 + am2 x2 + · · · + amn xn
    
am1 am2 ··· amn xn
     
 a11   a12   a1n 
     
 a21   a22   a2n 
= x1  ..  + x2  ..  + · · · + xn  .. 
     = x1 col1 (A) + x2 col2 (A) + · · · + xn coln (A).
 .   .   . 

     
am1 am2 amn

That is, AX is a linear combination of columns of A and the entries of X are the coefficients.
1.3. Matrix Operations 19

Example 1.3.10

Write the following product as a linear combination of the columns of the first matrix.
     
1 3   1 3
 3 
2

1 = 3 2 + (−2) 
 
1 .

  −2    
4 2 4 2

Example 1.3.11
 
 1 2  
−2 3 4
Let A = 
 3 4

and B =  . Find the second column of AB as a linear combination
  3 2 1
−1 5
of columns of A.

Solution:
     
1 2  
 1  2
 3

col2 (AB) = A col2 (B) = 3 4 = 3  3  + 2 4
    
 2

    
−1 5 −1 5

Example 1.3.12

If A and B are n × n matrices, then

1. tr(cA) = c tr(A), where c is any real number.

2. tr(AB) = tr(BA).

Solution:

1. tr(cA) = c (A)11 + c (A)22 + · · · + c (A)nn = c[(A)11 + (A)22 + · · · + (A)nn ] = c tr(A).

2.
n n X
n n X
n n
tr(AB) = (AB)ii = (A)ij (B)ji = (B)ji (A)ij = (BA)jj = tr(BA).
X X X X

i=1 i=1 j=1 j=1 i=1 j=1

20 Chapter 1. Linear Equations and Matrices

Exercise 1.3.1
 
1 2
1. Let A =  . Compute A2 + I2 .
0 −1
 
1 2  
0 1 −3
2. Let A = 4 5 and B =  . Find the third row of AB.
 
  −3 1 4
3 6
 
1 1  
4 1 −1
3. Let A = 0 2 and B =  . Find AB and express the second column of AB
 
  2 5 1
2 0
as a linear
 combination
 of the columns of A.
1 2 7
4. Let A = 1 −5 7

. Find tr(A).
 
0 −1 10
 
1 1
5. Let A =  . Find A1977 , and find all matrices B such that AB = BA.
0 1
 
1 1
6. Let A = 2
1
2 .
1
Find A100 .
2 2
 
1 2
7. Find a 2 × 2 matrix B 6= O and B 6= I2 so that AB = BA if A = 
0 1
8. Show that if A and B are n × n matrices, then tr(A + B) = tr(A) + tr(B).
9. Show that there are no 2 × 2 matrices A and B so that AB − BA = I2 .
1.4. Inverses; Algebraic Properties of Matrices 21

1.4 Inverses; Algebraic Properties of Matrices

Theorem 1.4.1: Properties of Matrix Arithmetic

Let A, B, and C be matrices of appropriate sizes, and let r, s ∈ R. Then:

1. A + B = B + A (Commutative law for matrix addition)

2. A + (B + C) = (A + B) + C (Associative law for matrix addition)
3. A(BC) = (AB)C (Associative law for matrix multiplication)
4. A(B + C) = AB + AC (Left distributive law)
5. (A + B)C = AC + BC (right distributive law)
6. A(B − C) = AB − AC
7. (A − B)C = AC − BC
8. r(A + B) = rA + rB
9. r(A − B) = rA − rB
10. (r + s)A = rA + sA
11. (r − s)A = rA − sA
12. r(sA) = (rs)A
13. r(AB) = (rA)B = A(rB)

Proof:

We only proof (1) and (10). Let A = [aij ] and B = [bij ] for 1 ≤ i ≤ m and 1 ≤ j ≤ n. Then

1. A + B = [aij + bij ] = [bij + aij ] = B + A.

10. (r + s)A = (r + s)[aij ] = [(r + s) aij ] = [r aij + s aij ] = [r aij ] + [s aij ] = rA + sA.

Theorem 1.4.2: Properties of Zero Matrices

Let c be a scalar, and let A be a matrix of an appropriate size. Then:

1. A + O = O + A = A
2. A − O = A
3. A − A = A + (−A) = O
22 Chapter 1. Linear Equations and Matrices

4. OA = O
5. If cA = O, then c = 0 or A = O.

Proof:

We only proof (5). Let A = [aij ] for 1 ≤ i ≤ m and 1 ≤ j ≤ n. If cA = O, then for each i and
j, we have (cA)ij = c(A)ij = c aij = 0. Therefore, either c = 0 or we have aij = 0 for all i and j.
Therefore, either c = 0 or A = O.

         
0 1 1 3 2 5 1 2 1 2
Let A =  ,B= ,C= , and D =  . Then, AB = AC =  , but
0 2 1 2 1 2 0 0 2 4
B 6= C. That is, the cancellation law does not hold here.
For any a, b ∈ R, we have ab = 0 implies that a = 0 or b = 0. However, in matrices we have
AD = O but A 6= O and D 6= O.
Moreover, the n × n identity matrix I commutes with any other matrix. That is, AI = IA = A
for any n × n matrix A.

Definition 1.4.1

A matrix A ∈ Mn×n is called nonsingular or invertable if there exists a matrix B ∈ Mn×n

such that
A B = B A = In .

In particular, B is called the inverse of A and is denoted by A−1 . If there is no such B, we

say that A is singular which means that A has no inverse.

Example 1.4.1

Here is an example of a 2 × 2 matrix along with its inverse.

       
1 1  3 −1  3 −1 1
=
1 1
=
0
= I2 .
2 3 −2 1 −2 1 2 3 0 1
1.4. Inverses; Algebraic Properties of Matrices 23

Theorem 1.4.3

If a matrix has an inverse, then its inverse is unique.

Proof:

Assume that A ∈ Mn×n is a matrix with two inverses B and C, then

B = B In = B (A C) = (B A) C = In C = C.

Therefore, A has a unique inverse.

Theorem 1.4.4
 
a b
The matrix A =  is nonsingular iff ad − bc 6= 0, in which case the inverse of A is
c d
 
1  d −b
A −1
= .
ad − bc −c a

Proof:

We simply show that AA−1 = A−1 A = I. Here, we only do the following:

     
a b 1  d −b 1 ad − bc −ab + ab
A A−1 = · =
c d ad − bc −c a ad − bc cd − cd −bc + ad
 
1 ad − bc 0
=  = I.
ad − bc 0 −bc + ad

We note that the quantity ad − bc above is called the determinant of the 2 × 2 matrix A and is
denoted by det (A) = ad −bc. The

determinant will be discussed in general sizes in Chapter 2.
1 2
As an example, if A =  , then det (A) = 3 − 2 = 1 and hence the inverse of A is
1 3
   
3 −2  3 −2
A−1 = 1  = .
3−2
−1 1 −1 1
24 Chapter 1. Linear Equations and Matrices

Theorem 1.4.5

If A and B are nonsingular n × n matrices, then A B is nonsingular and (A B)−1 = B −1 A−1 .

Proof:

Since both A and B are nonsingular, then both A−1 and B −1 exist. Thus

A B (B −1 A−1 ) = A (B B −1 ) A−1 = A I A−1 = A A−1 = I.

A B (B −1 A−1 ) = A (B B −1 ) A−1 = A I A−1 = A A−1 = I.

Thus, A B is nonsingular and (A B)−1 = B −1 A−1 .

Remark 1.4.1

Let m and n be nonnegative integers and let A and B be two matrices of appropriate sizes, then

1. A0 = I and An = A A · · · A (n-times),
2. Am An = Am+n ,
3. (Am )n = Amn ,
4. (AB)n = (AB)(AB) · · · (AB), (n-times), and in general (AB)n 6= An B n .
5. In general, (A + B)2 = (A + B)(A + B) = A2 + AB + BA + B 2 6= A2 + 2AB + B 2 .

Theorem 1.4.6

If A is a nonsingular matrix and n is a nonnegative integer, then:

−1
1. A−1 is nonsingular and (A−1 ) = A.
2. An is nonsingular and (An )−1 = A−n = (A−1 )n .
3. k A is nonsingular for any nonzero scalar k, and (k A)−1 = k −1 A−1 .

Proof:

We use the idea of the definition. A matrix B is the inverse of matrix C if BC = CB = I.

1. Clearly, A−1 A = A A−1 = I and hence A is the inverse of A−1 .

2. An A−n = An−n = A0 = I. That is A−n is the inverse of An .
3. (kA)(k −1 A−1 ) = (kk −1 )(AA−1 ) = I. Also, (k −1 A−1 )(kA) = I. Thus, (kA)−1 = k −1 A−1 .
1.4. Inverses; Algebraic Properties of Matrices 25

Theorem 1.4.7

Let A and B be two matrices of appropriate size and let c be a scalar. Then:

1. (AT )T = A,

2. (A ± B)T = AT ± B T ,

3. (c A)T = c AT , and

4. (AB)T = B T AT . This result can be extended to three or more factors.

Exercise 1.4.1

If A is a square matrix and n is a positive integer, is it true that (An )T = (AT )n ? Justify your
answer.

Theorem 1.4.8

If A is nonsingular matrix, then AT is also nonsingular and (AT )−1 = (A−1 )T .

Proof:

We simply show that the product AT (A−1 )T = (A−1 )T AT = I. Clearly, AT (A−1 )T = (A−1 A)T =
I T = I. Similarly, (A−1 )T AT = I.

Example 1.4.2
   
2 0 1 4
Let A =  and B =  . Find AB, and B T AT .
1 −1 3 5

Solution:
     
2+0 8 + 0  2 8 2 −2
Clearly, AB =  = . Moreover, B T AT = (AB)T = 
1−3 4−5 −2 −1 8 −1
26 Chapter 1. Linear Equations and Matrices

Definition 1.4.2

Let A be an n × n (square) matrix and p(x) = a0 + a1 x + · · · + am xm be any polynomial. Then

we define the matrix polynomial in A as the n × n matrix p(A) where

p(A) = a0 In + a1 A + · · · + am Am .

Example 1.4.3
 
−1 2
Compute p(A) where p(x) = x2 − 2x − 3 and A =  .
0 3

Solution:

 2    
−1 2 −1 2 1 0
p(A) = A2 − 2A − 3I2 =  − 2 − 3
0 3 0 3 0 1
       
1 4 −2 4 3 0 0 0
= − − = =O
0 9 0 6 0 3 0 0

Example 1.4.4

Show that if A2 + 5A − 2I = O, then A−1 = 12 (A + 5I).

Solution:

A2 + 5A − 2I = O implies that 2I = A2 + 5A = A(A + 5I). Then I = 21 A(A + 5I). Therefore,

 
I=A 1
2
(A + 5I) , which shows that A−1 = 12 (A + 5I).

Example 1.4.5

Show that if A is a square matrix such that Ak = O for some positive integer k, then the matrix
A is nonsingular and (I − A)−1 = I + A + A2 + · · · + Ak−1 .

Solution:
1.4. Inverses; Algebraic Properties of Matrices 27

h i h i
(I − A)(I + A + · · · + Ak−1 ) = I + @
A +Z
AZ2 + · · · + Ak−1 A +Z
HH − @ AZ2 + Z
AZ3 + · · · + Ak−1
HH + Ak
H H

= I − Ak = I − O = I.
28 Chapter 1. Linear Equations and Matrices

Exercise 1.4.2
 
 1
 3
 
1. Let A =  . Find all constants c ∈ R such that (cA)T · (cA) = 5.
−1
 

3
 

2. Let A be an n × n matrix such that A3 = O. With justification, prove or disprove that

(In − A)−1 = A2 + A + In .

3. A square matrix A is said to be idempotent if A2 = A.

(a) Show that if A is idempotent, then so is I − A.

(b) Show that if A is idempotent, then 2A − I is nonsingular and is its own inverse.

4. Let p1 (x) = x2 − 9, p2 (x) = x + 3, and p3 (x) = x − 3. Show that p1 (A) = p2 (A)p3 (A) for
any square matrix A.
5. Let A be a square matrix. Then

(a) Show that (I − A)−1 = I + A + A2 + A3 if A4 = O.

(b) Show that (I − A)−1 = I + A + A2 + · · · + An if An+1 = O.

6. Let Jn be the n × n matrix each of whose entries is 1. Show that if n > 1, then

1
(I − Jn )−1 = I − Jn .
n−1

7. Let A, B, and C be n × n matrices such that D = AB + AC is non-singular.

(a) Find A−1 if possible.

(b) Find (B + C)−1 if possible.
   
1 2 1 −1 2
8. Let A =  , and B =  . Find C if (B T + C)A−1 = B T .
1 3 3 2 0
   
1 2 0 1 1 0
9. Let A−1 = 
0 1 −1 and B = 1
 
0 0

. Find C if A C = B T .
   
2 0 1 0 1 1
h
Hint: Consider multiplying both sides from the left with A−1 .]
1.5. A Method for Finding A−1 29

1.5 A Method for Finding A−1

Unless otherwise specified, all matrices in this section are considered to be square matrices.

Theorem 1.5.1

If A is an n × n matrix, then the following statements are equivalent:

1. A is nonsingular.
2. AX = O has only the trivial solution.
3. A is row equivalent to In .

Remark 1.5.1

? How to find A−1 for a given A ∈ Mn×n :

1. form the augmented matrix [ A | In ],

2. find the r.r.e.f. of [ A | In ], say [ C | B ]:

(a) if C = In , then A is non-singular and A−1 = B,

(b) if C 6= In , then C has a row of zeros and A is singular matrix with no inverse.

Example 1.5.1
 
1 1
Find, if possible, A−1 for A = 
2 3

Solution:
     
1 1 1 0 r2 −2r1 →r2 1 1
−−−−−−→
1 0 r1 −r2 →r1 1 0
−−−−−−→
3 −1
.
2 3 0 1 0 1 −2 1 0 1 −2 1
 
3 −1
Therefore, A is non-singular and A−1 = .
−2 1
30 Chapter 1. Linear Equations and Matrices

Example 1.5.2
 
1 0 1
Find, if possible, A−1 for A = 1 1 2
 
 
2 0 1

Solution:
     
1 0 1 1 0 0 1 0 1 1 0 0 1 0 1 1 0 0
 r2 −r1 →r2   −r3 
1 1 2 0 1 0 −−−−−−→ 0 1 1 −1 1 0 −−→ 0 1 1 −1 1 0
 
  r3 −2r1 →r3    
2 0 1 0 0 1 0 0 −1 −2 0 1 0 0 1 2 0 −1
   
1 0 0 −1 0 1 −1 0 1
r1 −r3 →r1
−−−−−−→ 0 1 0 −3 1 1 . Therefore, A is non-singular and A = −3
−1
1 1.
   
r2 −r3 →r2    
0 0 1 2 0 −1 2 0 −1

Example 1.5.3
 
1 2 −3
Find, if possible, A−1 for A = 1 −2 1
 
 
5 −2 −3

Solution:
   
1 2 −3 1 0 0 1 2 −3 1 0 0
 r2 −r1 →r2   r3 −3r2 →r3
1 −2 1 0 1 0 r −5r →r 0 −4
−− −− − −→ 4 −1 1 0 −−−−−−→

 
 3 1 3 
5 −2 −3 0 0 1 0 −12 12 −5 0 1
 
 1 2 −3 1 0 0

 0 −4 4 −1 1 0

.
 
0 0 0 −2 −3 1
At this point, we can conclude that this matrix is singular with no inverse because of the fact
that the third row in the first part is a zero row. In particular, A−1 does not exist.

Remark 1.5.2

Let A be an n × n matrix:

1. if A is row equivalent to In , then it is non-singular,

1.5. A Method for Finding A−1 31

Example 1.5.4
 
−3 0
A= is row equivalent to I2 .
0 5

2. if A is row equivalent to a matrix with a row of zeros, then it is singular.

Example 1.5.5
 
−1 1
A= is row equivalent to a matrix with a row of zeros.
0 0

TRUE or FALSE:

? If A and B are

singular

matrices,

then

so is A + B.  
(FALSE).
1 0 0 0 are two singular matrices, while A + B = 1 0
reason: A =   and B =  is a non-
0 0 0 1 0 1
singular.

? If A and B are

non-singular

matrices,

then

so is A + B. 
(FALSE).

1 0 −1 0 0 0
reason: A =   and B =   are two non-singular matrices, while A + B =   is a
0 1 0 −1 0 0
singular.
32 Chapter 1. Linear Equations and Matrices

Exercise 1.5.1
 
1 0 1
1. Let A = T
1

1 0

. Find (2A)−1 .
 
2 0 1
h
Hint: If c is a nonzero constant, then what is (cA)−1 ?]
 
1 2 3    
2. Let A−1 = 
0 1 2

. Find all x, y, z ∈ R such that x y z A= 1 2 3 .

0 0 1
   
0 1 −1 1 2 −1
3. Let A = 1 1 0 and B = 1 3 0.
   
   
0 1 2 1 1 −1
(a) Find B −1 .

(b) Find C if A = B C.
1.6. More on Linear Systems and Invertible Matrices 33

1.6 More on Linear Systems and Invertible Matrices

Theorem 1.6.1

A (nonhomogeneous) system of linear equations has zero, one, or infinitely many solutions.

Proof:

If AX = B, B 6= O has no solutions or one solution, then we are done. So, we only need to show
that if the system has more than one solution, then it has infinitely many solutions. Assume
that Y and Z are two solutions for the system AX = B. Then, for r, s ∈ R, define U = r Y + s Z
to get:

AU = A(r Y + s Z) = A(r Y ) + A(s Z) = r (AY ) + s (AZ) = r B + s B = (r + s)B.

If r + s = 1, then U is a solution to the system. Since there are infinitely many choices for r and
s in R, the system has infinitely many solutions.

TRUE or FALSE:

? If X1 and X2 are two solutions for AX = B, then 23 X1 − 2X2 is also a solution. (FALSE).
reason: Since 3
2
−2 = −1
2
6= 1.

Theorem 1.6.2

If A is a nonsingular n×n matrix, then for each n×1 matrix B, the system of equations AX = B
has a unique solution, namely X = A−1 B.

Proof:

Given AX = B, we multiply both sides (from left) by A−1 to get X = A−1 B. That is A−1 B is
a solution to the system.
To show that it is unique, assume that Y is any solution for AX = B. Hence AY = B and again
Y = A−1 B.
34 Chapter 1. Linear Equations and Matrices

Example 1.6.1

Solve the following system ”A X = B” (given in its matrix form)

    
1 0 1 x  0 
1 1 2 y  =  1 
    
    
2 0 1 z −1

Solution:

We solve the system by using A−1 , we can find the inverse of A as in Example 1.5.2, to get
 
−1 0 1
A−1 =
−3 1 1

 
2 0 −1

Then, using Theorem 1.6.2, we get the following unique solution:

      
x −1 0 1   0  −1
X = y  = A B = −3
−1
1 1  1  =  0 .
      
      
z 2 0 −1 −1 1

Example 1.6.2

Solve the following system ”A X = O”

    
1 0 1 x 0
1 1 2 y  = 0
    
    
2 0 1 z 0

Solution:

We can find the inverse of A as in Example 1.5.2, to get

 
−1 0 1
A−1 = −3 1 1
 
 
2 0 −1
Then, using Remark 1.6.2, we get the trivial solution:
      
x −1 0 1  0 0
X = y  = A O = −3
−1
1 1 0 = 0 .
      
      
z 2 0 −1 0 0
1.6. More on Linear Systems and Invertible Matrices 35

• Solving a sequence of systems with a common coefficient matrix

Given the systems AX = B1 ; AX = B2 ; · · · ; AX = Bk , an efficient way to solve these systems

at once is to form the augmented matrix

[ A | B1 | B2 | · · · | Bk ],

and use the Guass-Jordan elimination to solve all of the system at the same time.

Example 1.6.3

Solve the following linear systems:

x + z = b1 i. b1 = 0, b2 = 1, and b3 = 1;
x + y + 2z = b2 for
ii. b1 = 1, b2 = 2, and b3 = 3.
2x + 3z = b3

Solution:
     
1 0 1 0 1 1 0 1 0 1 1 0 0 −1 0
 r2 −r1 →r2   r1 −r2 →r1 
1 1 2 1 2 r −2r →r 
−− −− − −→ 0 1 1 1 1 r −r →r 
−
− −− − −
→ 0 1 0 0 0
 
 
 3 1 3  2 3 2  
2 0 3 1 3 0 0 1 1 1 0 0 1 1 1
Therefore, the solution of the system (i) is x = −1, y = 0, z =1 and the solution of system (ii)
is x = 0, y = 0, z = 1.

Theorem 1.6.3

Let A be a square matrix. Then

1. If B is a square matrix satisfying BA = I, then B = A−1 .

2. If B is a square matrix satisfying AB = I, then B = A−1 .

Proof:

1. Assuming BA = I, we show that A is nonsingular by showing that the system AX = O

has only the trivial solution. Multiplying both sides (from left) by B, we get B (AX) =
B O which implies (BA)X = IX = X = O. Thus, the system AX = O has only the
36 Chapter 1. Linear Equations and Matrices

trivial solution. Theorem 1.5.1 implies that A is nonsingular. Therefore BA = I implies

(BA)A−1 = IA−1 . Hence B = A−1 .
2. Using part (a), AB = I implies that A = B −1 . Taking the inverse for both sides, we get
A−1 = B as desired.

Theorem 1.6.4: Extended Version of Theorem 1.5.1

If A is an n × n matrix, then the following statements are equivalent:

1. A is nonsingular.
2. AX = O has only the trivial solution.
3. A is row equivalent to In .
4. AX = B is consistent for every n × 1 matrix B.
5. AX = B has exactly one solution for every n × 1 matrix B.

Proof:

We only show that 5 implies 1: Assume that the system AX = B has one solution for every
n × 1 matrix B. Let X1 , X2 , · · · , Xn be the solutions (respectively) to the following systems:
     
1 0 0
0 1 0
     

AX = 
0 , AX = 
0 , AX = 0 .
     
  ··· ,
 ..   ..   .. 
 
. . .
     
0 0 1

If C is the n × n matrix formed by the column vectors X1 , X2 , · · · , Xn , we get

AC = A [X1 | X2 | · · · | Xn ] = [AX1 | AX2 | · · · | AXn ] = In

By Theorem 1.6.3, we get C = A−1 . Thus A is nonsingular.

Theorem 1.6.5

Let A and B be two square matrices of the same size. If AB is nonsingular, then A and B must
be also nonsingular.
1.6. More on Linear Systems and Invertible Matrices 37

In what follows, we discuss when a given system is consistent based on the remarks in Section
1.2

Example 1.6.4

Consider the system:

x + y + z = 2
x + 2y − z = 2
x + 2y + (a − 5)z = a
2

Find all values of a so that the system has:

(a) a unique solution (consistent).

(b) infinite many solutions (consistent).
(c) no solution (inconsistent).

Solution:

In this kind of questions, it is not necessary to get the r.r.e.f. So, we will try to focus on the last
row
 which contains the term  ”a” as follows:   
 1 1 1 2 1 1 1 2 1 1 1 2
 r2 −r1 →r2   r3 −r2 →r3 
 
 1 2 −1 2 −−−−−−→ 0 1 −2 0 −−−−−−→ 0 1 −2 0
 

  r3 −r1 →r3    
1 2 a −5 a
2
0 1 a −6 a−2
2
0 0 a −4 a−2
2

At this point, we have:

(a) a unique solution if a2 − 4 6= 0 ⇐⇒ a 6= ±2 ⇐⇒ a ∈ R\{−2, +2}.

n o n o
(b) infinite solutions if a2 − 4 = 0 and a − 2 = 0 ⇐⇒ a = ±2 and a = +2 ⇐⇒ a = +2.
n o n o
(c) no solution if a2 − 4 = 0 and a − 2 6= 0 ⇐⇒ a = ±2 and a 6= +2 ⇐⇒ a = −2.

Example 1.6.5

Discuss the consistency of the following non-homogenous system:

x + 3y + kz = 4
2x + ky + 12z = 6

Solution:
38 Chapter 1. Linear Equations and Matrices

   
1 3 k 4 −
r2 −2r1 →r2
−−−−−→ 
1 3 k 4
.
2 k 12 6 0 k − 6 12 − 2k −2
The system is consistent only if k 6= 6:

(a) can not have a unique solution (not equivalent to I because it is not square matrix!!),
(b) has infinite solutions when k 6= 6,
(c) has no solutions when k = 6.
1.6. More on Linear Systems and Invertible Matrices 39

Exercise 1.6.1
   
1 0 3 3   1 
0 1 1 −1   0 
   
1. Let A = 
 and B = 
  . Find all value(s) of a such that the system
1 −2 3 1   0 
 

0 2 0 a2 + 1 a+2
   

AX = B is consistent.
2. Find all value(s) of a for which the system

x − y + (a + 3)z = a3 − a − 7
−x + ay − az = a
2(a − 1)y + (a2 + 2)z = 8a − 14.

has (a) no solution (inconsistent), (b) unique solution (consistent), and (c) infinite many
solutions (consistent).
3. Discuss the consistency of the following homogenous system:

x + y − z = 0
x − y + 3z = 0
x + y + (a − 5)z = 0
2

4. Show that if C1 and C2 are solutions of the system AX = B, then 4C1 − 3C2 is also a
solution of this system.
5. Let U and V be two solutions of the homogenous system AX = O. Show that rU + sV
(for r, s ∈ R) is a solution to the same system.
6. Let U and V be two solutions of the non-homogenous system AX = B. Show that U − V
is a solution to the homogenous system AX = O.
7. If A is nonsingular matrix, then AAT and AT A are both nonsingular matrices.
8. Show that if A, B, and A + B are invertible matrices with the same size, then

A (A−1 + B −1 ) B (A + B)−1 = I.

9. Let A be an m × n matrix, and B be an m × 1 column vector. Show that the system

AX = B has a solution if and only if B is a linear combination of columns of A.
[Hint: Recall Definition 1.3.4].
40 Chapter 1. Linear Equations and Matrices

1.7 Diagonal, Triangular, and Symmetric Matrices

The ”diagonal matrix”, D = [dij ], is an n × n matrix so that dij = 0 for all i 6= j.

   
d11 0 ··· 0  d1 0 ··· 0
 0 0  0 0
   
d22 ··· d2 ···
D=  . .. .. ..  it is also can be written as D = . .. .. .. 
   
 .. . . .   .. . . .

  
   
0 0 ··· dnn 0 0 ··· dn

Moreover, we sometime write D = diag(d1 , d2 , · · · , dn ). If all scalars in the diagonal matrix are
equal, say equal c, then D is said to be a scalar matrix. In particular, the identity matrix In is
a scalar matrix with c = 1. That is In = diag(1, 1, · · · , 1).

1. D is nonsingular if and only if all of its diagonal entries are nonzero; in this case we have
 
1/d1 0 ··· 0 
 0 1/d2 0 
 
···
D−1 =  .

.. ... .. 

 .. . . 
 
 
0 0 ··· 1/dn

2. If k is a positive integer, then Dk is computed as

 
k
d1 0 ··· 0
0 0
 
dk2 ···
D =k
.

.. ... .. 

 .. . .
 
 
0 0 ··· dkn

3. If A is an n × m matrix and B is an m × n matrix, then

 
 d1 row1 (A) 
 d row (A) 
   
 2 2
DA = .. and B D = d1 col1 (B) d2 col2 (B) ··· dn coln (B)

.
 
 
 
 
dn rown (A)
1.7. Diagonal, Triangular, and Symmetric Matrices 41

Example 1.7.1
 
1 0 −1
   
1 0 0
1 1 1 −1 0
1 0
 
Let D = 0 3 0 , A=  , and B = 
2 3 1 2 . Compute D−3 , AD,
   
2 1 4
     
0 0 2 4 2 1 3
5 1
 
−1
and DB.

Solution:
   
3
Note that D = diag(1, 3, 2). Hence D−1 = diag 1, 13 , 21 and thus D−3 = (D−1 ) = diag 1, 27
1 1
,8 .
   
1 0 −1 1 0 −2
 
 1 0 0 
1 1 0 1 3 0
    
AD =  0 3 0 =  = col (A) 3 col2 (A) 2 col3 (A) ,
 
1
2 1 4 2 3 8
    
 0 0 2
5 1 5 3
  
−1 −2
and       
1 0 0 1 1 −1 0 1 1 −1 0  row1 (B) 
DB = 
0 3 0

2 3 1 2

=
6 9 3 6

=
3 row2 (B) .

      
0 0 2 4 2 1 3 8 4 2 6 2 row3 (B)

Example 1.7.2

Find all 2 × 2 diagonal matrices A that satisfy the equation A2 − 3A + 2I = O.

Solution:
 
a 0
Assume that A =  be a 2 × 2 diagonal matrix. Then,
0 b
       
a2 0  3a 0  2 0 0 0
A2 − 3A + 2I =  − + = .
0 b2 0 3b 0 2 0 0

Hence, a2 − 3a + 2 = 0 and b2 − 3b + 2 = 0. That is (a − 1)(a − 2) = 0 and (b − 1)(b − 2) = 0

which implies that a = 1 or 2 and b = 1 or 2. Therefore,
       
1 0 1 0 2 0 2 0

A∈  0
, , , .
1 0 2 0 1 0 2 
42 Chapter 1. Linear Equations and Matrices

The ”lower triangular matrix”, L = [lij ], is an n × n matrix so that lij = 0 for all i < j. The
”upper triangular matrix”, U = [uij ], is an n × n matrix so that uij = 0 for all i > j.
   
 l11 u11 u12 ··· u1n 


 l21 l22
0 





 u22 ···

u2n 
L= , and U =
   

... ..
 
.. .. ..
 
0
 
. . . .
   
   
   
 

ln1 ln2 ··· lnn

unn

Theorem 1.7.1

• The transpose of a lower triangular matrix is upper triangular ma-

 
trix, and the transpose of an upper triangular matrix is lower trian-  
 i<j 
gular matrix. 



 
i>j
• The product of lower triangular matrices is lower triangular, and
 
 

the product of upper triangular matrices is upper triangular.

• A triangular matrix is nonsingular if and only if its diagonal entries
are all nonzero.
• The inverse of a nonsingular lower triangular matrix is lower triangular, and the inverse of a
nonsingular upper triangular matrix is upper triangular.

Example 1.7.3
 
8 0
Find a lower triangular matrix that satisfies A3 =  .
9 −1

Solution:
 
a 0
Assume that A =  be a 2 × 2 lower triangular matrix. Then,
b c
   
a 3
0  8 0
A3 =  2 = .
a b + c(ab + bc) c3 9 −1
Hence, a = 2 and c = −1 and thus 4b − (2b − b) = 9 implies that b = 3.
1.7. Diagonal, Triangular, and Symmetric Matrices 43

Definition 1.7.1

A square matrix is called symmetric if AT = A. It is called skew-symmetric if AT = −A.

A square matrix A = [aij ] is symmetric if aij = aji and it is skew-symmetric if aij = −aji .

   
1 2 3  0 2 −3
A = 2 4 5 is a symmetric; where B = −2 0 1 is a skew-symmetric.
   
   
3 5 0 3 −1 0

Example 1.7.4

Fill in the missing entries (marked with ×) to produce symmetric (or skew-symmetric) matrices.
 
× 2 ×
× × 0 .
 
 
−2 × ×

Theorem 1.7.2

If A and B are symmetric matrices with the same size, and k is any scalar, then:

1. AT is symmetric.
2. A + B and A − B are symmetric.
3. kA is symmetric.
4. AB is symmetric iff AB = BA.

Proof:

Note that A and B are symmetric matrices and hence AT = A and B T = B. Then,

1. (AT )T = A = AT . Then AT is symmetric.

2. (A ± B)T = AT ± B T = A ± B. Then A + B and A − B are symmetric.
3. (kA)T = kAT = kA. Then, kA is symmetric.
4. AB is symmetric iff (AB)T = AB iff B T AT = AB iff BA = AB.
44 Chapter 1. Linear Equations and Matrices

Theorem 1.7.3

If A is a symmetric nonsingular matrix, then A−1 is symmetric.

Proof:

Assume that A is symmetric nonsingular matrix. Then

−1
(A−1 )T = AT = A−1 .

Therefore, A−1 is symmetric.

Example 1.7.5

Show that AT A and AAT are both symmetric matrices.

Solution:

It is clear that (AT A)T = AT (AT )T = AT A which shows that AT A is symmetric. In addition,
(AAT )T = (AT )T AT = AAT shows that AAT is also symmetric.

Example 1.7.6

1. If A is nonsingular skew-symmetric matrix, then A−1 is skew-symmetric.

2. If A and B are skew-symmetric matrices, then so are AT , A + B, A − B and kA for any
scalar k.
3. Every square matrix A can be expressed as the sum of a symmetric matrix and a skew-
symmetric matrix.

Solution:
−1
1. Assume that AT = −A. Then (A−1 )T = (AT ) = (−A)−1 = −A−1 . That is A−1 is
skew-symmetric.
2. We only show that A + B is skew-symmetric: (A + B)T = AT + B T = −A + (−B) =
−(A + B).
   
3. If A is any square matrix, then A = 1
2
A + AT + 12 A − AT . Then we only need to show
   
that 1
2
A + AT and 1
2
A − AT are symmetric and skew-symmetric matrices, respectively.
1.7. Diagonal, Triangular, and Symmetric Matrices 45

Exercise 1.7.1

1. Show that if A and B are symmetric matrices, then AB − BA is a skew-symmetric matrix.

2. Let A be 2 × 2 skew-symmetric matrix. If A2 = A, then A = 0.
3. If A and B are lower triangular matrices, show that A + B is lower triangular as well.
4. If A and B are skew-symmetric matrices and AB = BA, then AB is symmetric.
5. Let A ∈ Mn×n . Show that

(a) AT + A is symmetirc.

(b) A − AT is skew-symmetric.

6. Let A ∈ Mn×n . Then A can be written as A = S + K, where S is symmetric matrix and

K is skew-symmetric
 matrix.

2 −1 3
7. Let A = 0 4 1. Find a symmetric matrix S and a skew symmetric matrix K
 
 
1 −2 −3
such that A = S + K.
46 Chapter 1. Linear Equations and Matrices
 Chapter

2 Determinants

2.1 Determinants

Definition 2.1.1

Let S = {1, 2, · · · , n} be a set of integers from 1 to n. A rearrangement of j1 , j2 , · · · , jn of

elements of S is called permutation of S.

Example 2.1.1

For S = {1, 2}, we have 2 permutations:

1 2, and 2 1.

While, for S = {1, 2, 3}, we have the following 6 permutations:

1 2 3, 1 3 2, 2 1 3, 2 3 1, 3 1 2, and 3 2 1.

In general, for S = {1, 2, · · · , n}, we have n! = n(n − 1)(n − 2) · · · 3 · 2 · 1 permutations.

Definition 2.1.2

A permutation j1 j2 · · · jn of the set S = {1, 2, · · · , n} is said to have an inversion if a larger

integer jr preceedes a smaller one js for r, s ∈ S. A permutation is called even with a positive
”+” sign or odd with a negative ”−” sign according to whether the total number of inversions
in it is even or odd, respectively.

47
48 Chapter 2. Determinants

Example 2.1.2

If S = {1, 2, 3}, then

permutation #inversions even-odd sign inversions

123 0 even + 12 13 23
132 1 odd − 13 12 3 2
213 1 odd − 2 1 23 13
231 2 even + 23 2 1 3 1
312 2 even + 3 1 3 2 12
321 3 odd − 3 2 3 1 2 1

TRUE or FALSE:

? The permutation 5 2 1 3 4 has a positive sign. (FALSE).

reason: The number of inversions is 5 which are 5 2, 5 1, 5 3, 5 4, and 2 1. So, this permutation has an
odd number of inversions and a negative sign.

Definition 2.1.3

Let A = (aij ) ∈ Mn×n . Then, the determinant of A, denoted by det(A) or A , is

det(A) = (±)a1j1 a2j2 a3j3 · · · anjn

X

where the summation ranges over all permutations j1 j2 · · · jn of the set S = {1, 2, · · · , n}. The
sign is taken as + or − according to the sign of the permutation.

Example 2.1.3
 
a a12 
Compute the determinant of A =  11 .
a21 a22

Solution:

Using the definition, we have det(A) = (±)a1j1 a2j2 , where j1 j2 is a permutation of S = {1, 2}.
P

Thus, j1 j2 ∈ {1 2, 2 1} and
det(A) = + a11 a22 − a12 a21 .
2.1. Determinants 49

Example 2.1.4
 
a11 a12 a13 
Compute the determinant of A = a21 a22 a23 .
 
 
a31 a32 a33

Solution:

Using the definition, we have det(A) = (±)a1j1 a2j2 a3j3 , where j1 j2 j3 is a permutation of S =
P

{1, 2, 3}. Thus, j1 j2 j3 ∈ {1 2 3, 1 3 2, 2 1 3, 2 3 1, 3 1 2, 3 2 1} and

det(A) = + a11 a22 a33 − a11 a23 a32 − a12 a21 a33 + a12 a23 a31 + a13 a21 a32 − a13 a22 a31 .

Moreover, this formula can be found by taking the sum of the positive product of the diagonal
entries and the negative product of the anti-diagonal entries in the following matrix:

+ + +− − −
a11 a12 a13 a11 a12

a21 a22 a23 a21 a22

a31 a32 a33 a31 a32

Determinants of 2 × 2 and 3 × 3 matrices can be evaluated using

+ + +− − −
+ − a11 a12 a13 a11 a12
a11 a12
A1 = a21 a22 A2 = a21 a22 a23 a21 a22
a31 a32 a33 a31 a32

For the 2 × 2 matrix A1 , we get det (A1 ) = a11 a22 − a12 a21 . It is simply the result of ”blue stripe”
product minus ”red stripe” product.
While for the 3 × 3 matrix A2 , we first recopy the first two columns and then we add up the
product of the blue stripes and subtract the product of the red stripes.
   
det (A2 ) = a11 a22 a33 + a12 a23 a31 + a13 a21 a32 − a13 a22 a31 + a11 a23 a32 + a12 a21 a33 .

3 0 1
3 2
For example, = −2, and 0 1 2 = 2.
4 2
1 0 1
50 Chapter 2. Determinants

Definition 2.1.4

If A is a square matrix, then the minor of entry aij is denoted by Mij and is defined to be the
determinant of the submatrix that remains after the ith row and j th column are deleted from A.
The number (−1)i+j Mij is denoted by Aij and is called the cofactor of entry aij .
Moreover, the cofactor matrix denoted by coef(A) = [Aij ] where 1 ≤ i, j ≤ n.

Example 2.1.5
 
1 2 1
Compute the minors, the cofactors, and the cofactor matrix of A, where A = 0 1 −1 .
 
 
3 1 −2

Solution:

1 −1 0 −1 0 1
M11 = = −1, M12 = = 3, M13 = = −3,
1 −2 3 −2 3 1

2 1 1 1 1 2
M21 = = −5, M22 = = −5, M23 = = −5,
1 −2 3 −2 3 1

2 1 1 1 1 2
M31 = = −3, M32 = = −1, M33 = = 1.
1 −1 0 −1 0 1

Thus, the cofactors are:

A11 = (−1)2 M11 = −1, A12 = (−1)3 M11 = −3, A13 = (−1)4 M11 = −3,

A21 = (−1)3 M11 = 5, A22 = (−1)4 M11 = −5, A23 = (−1)5 M11 = 5,

A31 = (−1)4 M11 = −3, A32 = (−1)5 M11 = 1, A33 = (−1)6 M11 = 1.
 
−1 −3 −3
Therefore, coef(A) =  5 −5 5.
 
 
−3 1 1
2.1. Determinants 51

We note that the minors Mij and the cofactors Aij are either the same or the negative of each
other. This results from the (−1)i+j entry in the cofactor value.

   
a a12  + −
If A =  11 , then, its sign matrix is  , and its minros and cofactors are:
a21 a22 − +

A11 = M11 = a22 ; A12 = −M12 = −a21 ; A21 = −M21 = −a12 ; and A22 = M22 = a11 .

Definition 2.1.5

If A is an n × n matrix, then the number resulted by the sum of multiplying the entries in any
row by the corresponding cofactors is called the determinant of A, and the sums themselves
are called cofactor expansion of A.
That is, the determinant of A using the cofactor expansion along the ith row is:

det (A) = ai1 Ai1 + ai2 Ai2 + · · · + ain Ain .

While the determinant of A using the cofactor expansion along the j th column is:

det (A) = a1j A1j + a2j A2j + · · · + anj Anj .

Theorem 2.1.1

Let A be an n × n matrix, then for each 1 ≤ i ≤ n we have


det (A) if i = k,



ai1 Ak1 + ai2 Ak2 + · · · + ain Akn =
0 if i 6= k.



and for each 1 ≤ j ≤ n


det (A) if j = k,



a1j A1k + a2j A2k + · · · + anj Ank =
0 if j 6= k.


52 Chapter 2. Determinants

Example 2.1.6

Compute the determinant of A by using the cofactor expansion method, where A is the matrix
of Example 2.1.5:  
1 2 1
A= 0 1 −1
 
 
3 1 −2

Solution:

Here we can choose any row or column to compute the determinant. Using the cofactor expansion
along the 1st row, we get

det (A) = a11 A11 + a12 A12 + a13 A13 = (1)(−1) + (2)(−3) + (1)(−3) = −10.

Choosing the first row of A and the cofactors of (for instance) the second row of A, we get

a11 A21 + a12 A22 + a13 A23 = (1)(5) + (2)(−5) + (1)(5) = 0.

Example 2.1.7
 
0 3 0 1
2 1 1 2
 
Compute det (A) using the cofactor expansion method, where A =  .
0 0 1 2
 

0 1 0 1
 

Solution:

We use the cofactor expansion along the 1st -column since it has the most zeros:

−0 3 0 1
3 0 1
2 1 1 2 = (0) A11 + (−1)2+1 (2) + (0) A31 + (0) A41
0 1 2
0 0 1 2
1 0 1
0 1 0 1

3+ 0 1
= (−2) = (−2) 3 1 = (−2) [3 − 1] = −4.
0 1 2
1 1
1 0 1
2.1. Determinants 53

Theorem 2.1.2

If A is an n × n triangular matrix (upper triangular, lower triangular, or diagonal), then

det (A) = a11 a22 · · · ann .

Example 2.1.8

We use Theorem 2.1.2 to compute:

3 0 0
2 5 0 = 30.
100 1987 2

Can you check the answer using another method?

54 Chapter 2. Determinants

Exercise 2.1.1
 
0 4 4 0
1 1 2 0
 
1. Let D =  . Evaluate D .
1 3 5 3
 

0 1 2 6
 

Final answer:

|D| = −12.
    
a+e b + f a b e f
2. Let A =  ,B= , and C =  . Show that A = B + C .
c d c d c d
 
1 4 2
3. Let A = 5 −3 6 . Compute the cofactors A11 , A12 , and A13 , and show that 5A11 −
 
 
2 3 2
3A12 + 6A13 = 0.
2.2. Evaluating Determinants by Row Reduction 55

2.2 Evaluating Determinants by Row Reduction

In this section, we introduce some basic properties and theorems to compute determinants.

Theorem 2.2.1

Let A be an n × n matrix.

1. If A is a square matrix with a row of zeros (or a column of zeros), then det (A) = 0.
2. If A is a square matrix, then det (A) = det (AT ).

a11 a12 a13 a11 a21 a31

a21 a22 a23 = a12 a22 a32
a31 a32 a33 a13 a23 a33

3. If B is obtained from A by multiplying a single row (or a single column) by a scalar k,

then det (B) = k det (A). This result can be generalized as det (k A) = k n det (A).

k a11 k a12 k a13 a11 a12 a13

a21 a22 a23 = k a21 a22 a23
a31 a32 a33 a31 a32 a33

4. If B is obtained from A by interchaning two rows (or columns), then det (B) = − det (A).

a21 a22 a23 a11 a12 a13

a11 a12 a13 = − a21 a22 a23
a31 a32 a33 a31 a32 a33

5. If B is obtained from A by adding a multiple of one row (one column) to another row
(column, respectively), then det (B) = det (A).

a11 + k a21 a12 + k a22 a13 + k a23 a11 a12 a13

a21 a22 a23 = a21 a22 a23
a31 a32 a33 a31 a32 a33

6. If A has two proportional rows (or columns), then det (A) = 0.

a11 a12 a13

k a11 k a12 k a13 = 0
a31 a32 a33

7. If A and B are two square matrices, then det (AB) = det (A) det (B).
56 Chapter 2. Determinants

Example 2.2.1

Evaluate the determinant of the matrix A, where

 
2 −2 5 3
6 −6 15 9
 
A=  .
0 3 1 9
 

1 4 2
 
−1

Solution:

Note that the second row of A is 3 times the first one. Then, det (A) = 0.

2 −2 5 3 2 −2 5 3
6 −6 15 9 r2 −3r1 →r2
== 0 0 0 0 = 0.
0 3 1 9 0 3 1 9
1 4 2 −1 1 4 2 −1

Example 2.2.2

Evaluate the determinant of the matrix A, where

 
2 −2 5
A = 0 3 1 .
 
 
1 4 2

Solution:

Note that the second row of A is 3 times the first one. Then, det (A) = 0.

2 −2 5 r1 ⇔r3
1 4 2 r3 −2r1 →r3
1 4 2
0 3 1 == (−1) 0 3 1 == (−1) 0 3 1
1 4 2 2 −2 5 0 −10 1

c2 ⇔c3
1 2 4 r3 −r2 →r3
1 2 4
== (−1)(−1) 0 1 3 == 0 1 3
0 1 −10 0 0 −13
= (1)(1)(−13) = −13
2.2. Evaluating Determinants by Row Reduction 57

Example 2.2.3

Evaluate the determinant of the matrix A, where

 
2 −2 5 1
3 3 1 3
 
A=  .
1 1 2 1
 

4 3 5 6
 

Solution:

Note that the second row of A is 3 times the first one. Then, det (A) = 0.

2 −2 5 1 0 −4 1 −1
−4 1 −1
3 3 1 3 0 0 −5 0 +
= + = 0 −5 0
1 1 2 1 1 1 2 1
−1 −3 2
4 3 5 6 0 −1 −3 2

= (−5) −4 −1 = (−5)[−8 − 1] = 45.

−1 2

Example 2.2.4

a b c 3g 3h 3i
Let d e f = −6. Compute 2a + d 2b + e 2c + f .
g h i d e f

Solution:

We use the properties stated in Theorem 2.2.1:

3g 3h 3i 2a + d 2b + e 2c + f
r1 ⇔r2 r1 −r2 →r1
2a + d 2b + e 2c + f −−−→ (−1)(−1)
r2 ⇔r3
d e f −− −1−−−→
r
3 3
d e f 3g 3h 3i

2a 2b 2c 1 a b c
r
2 1
(3) d e f −−→ (3)(2) d e f = (6)(−6) = −36.
g h i g h i
58 Chapter 2. Determinants

Theorem 2.2.2

1
If A is an n × n non-singular matrix, then A 6= 0 and A−1 = . This statement suggests
|A|
that if A = 0, then A is singular matrix.

Proof:

Since A is non-singular, A−1 exists, then

AA−1 = In (take the determinant for both sides)

A A−1 = In
A A −1
= 1.

1
Thus, A 6= 0 and A−1 = .
|A|

Example 2.2.5

Show that if A ∈ Mn×n is skew-symmetric matrix and n is odd, then A = 0.

Solution:

Since A is skew-symmetric, then AT = −A and taking the determinant for both sides

AT = −A
A T
= (−1)n A , where n is odd and (−1)n = −1.
A = AT = − A .

Therefore, A = − A which means that A = 0.

TRUE or FALSE:

? If A, B ∈ Mn×n with A = B , then A = B. (FALSE).

reason: I2 6= −I2 while I2 = 1 and − I2 = (−1)2 I2 = 1.
2.2. Evaluating Determinants by Row Reduction 59

Example 2.2.6

Show that if A−1 = AT , then A = 1 or A = −1.

Solution:

Since A−1 = AT , we have

1 2
|A−1 | = |AT | ⇐⇒ = A ⇐⇒ A = 1 ⇐⇒ A = ±1.
|A|

Example 2.2.7

Let A, B ∈ M3×3 with A = 2 and B = −2. Find 2 A−1 (B 2 )T .

Solution:

1
2 A−1 (B 2 )T = 23 A−1 B2 = 8 B B
|A|
1
= 8 ( ) (−2)(−2) = 16.
2
60 Chapter 2. Determinants

Exercise 2.2.1
 
1 0 0 3
2 7 0 6
 
1. Compute det (A) where A =  .
0 6 3 0
 

2 3 1 5
 

[Hint: Simply reduce A to a lower triangular matrix! Find a relation between the first and
the fourth columns.]
a b c −a −b −c
2. Let d e f = −6. Compute 2d 2e 2f .
g h i 3g 3h 3i
Final answer: 36.
3. Solve for x:
0 1 0
x 1
= x 3 −2 .
1 x−1
1 5 −1
Final answer: x = 1.
a1 b 1 c 1 b1 b2 b1 − 3b3
4. Given that a2 b2 c2 = 7, evaluate a1 a2 a1 − 3a3 .
a3 b 3 c 3 c1 c2 c1 − 3c3
Final answer:
 21.   
a1 a2 a3   2a3 2a2 2a1 
5. Let A =  b1 b2 b3 , and B =  b3 − a3 b 2 − a2 b 1 − a1 . If A = −4, find B .
   
   
c1 c2 c3 c3 + 3b3 c2 + 3b2 c1 + 3b1
Final answer: 8.
6. Let A, B ∈ Mn×n . Show that, if A B = In , then B A = In .
7. If |A B| = 0, then either A = 0 or B = 0.
8. If A B = In , then A 6= 0 and B 6= 0.
9. Show that if A is non-singular and A2 = A, then A = 1.
10. Show that for any A, B, C ∈ Mn×n , if A B = A C and A 6= 0, then B = C.
 
1 2 4
11. Find all values of α for which the matrix 1 3 9 is singular.
 
 
1 α α2
12. Let A and B be two n × n matrices such that A is invertible and B is singular. Prove that
A−1 B is singular.
2.2. Evaluating Determinants by Row Reduction 61

13. If A and B are 2 × 2 matrices with det (A) = 2 and det (B) = 5, compute 3 A2 (AB −1 )T .
 
7 1 0 3
2 0 0 0
 
14. Let A be a matrix with A−1 = . Find det (A).
1 3 5 4
 

6 2 0 5
 
62 Chapter 2. Determinants
 Chapter

3 Euclidean Vector Spaces

3.1 Vectors in Rn

In this chapter, we deal with vectors in Rn , or sometimes we simply call them n-vectors. For instance,
 
 y1 
 
 y2 
 
(row-vector) X = x1 x2 ··· xn , and (column-vector) Y =  . 
 .  are vectors in Rn .
 . 
 
yn

In the notion of Rn , A vector is simply written as X = (x1 , x2 , · · · , xn ). While A points is written as

X(x1 , x2 , · · · , xn ).

Vecors in R3 can be manipulated by arrows starting at initial

point and pointing at terminal point. In the Figure, the Q
X
P terminal point
vector X has its initial point at P and has its terminal point
initial point
at Q.
For instance, if P (p1 , p2 , p3 ) and Q(q1 , q2 , q3 ) are points in R3 , then the vector X is written as
−→
X = P Q = (p1 − q1 , p2 − q2 , p3 − q3 ). This vector is different from its opposite vector −X which
−→
is given by −X = QP .

3.1.1 Vectors Operations:

Let X = (x1 , x2 , · · · , xn ), Y = (y1 , y2 , · · · , yn ) ∈ Rn and c ∈ R. Then:

1. Adding or substracting two vectors, they must have the same number of components:

X ± Y = (x1 ± y1 , x2 ± y2 , · · · , xn ± yn ) ∈ Rn .

2. Multiplying a vector by a scalar:

c X = (c x1 , c x2 , · · · , c xn ) ∈ Rn .

63
64 Chapter 3. Euclidean Vector Spaces

3. The vector O = (0, 0, · · · , 0) ∈ Rn is called the zero vector.

4. Two vectors X and Y are said to be equal if x1 = y1 , x2 = y2 , · · · , xn = yn .

Example 3.1.1

If X = (2, 3, −1) and Y = (1, 0, 1) are two vectors of R3 , then −X = (−2, −3, 1) and 3Y − X =
(3, 0, 3) − (2, 3, −1) = (1, −3, 4).

Try to prove the statements in the following two Theorems.

Theorem 3.1.1: The Closure of Rn under Vector Addition and Scalar Multiplication

Let X, Y, Z ∈ Rn and let c, d ∈ R. Then

1. Rn is closed under vector addition (i.e. X + Y ∈ Rn ):

(a) X + Y = Y + X,
(b) X + (Y + Z) = (X + Y ) + Z,
(c) ∃ a vector O ∈ Rn such that X + O = O + X = X. ”additive identity”
(d) for any X ∈ Rn , ∃(−X) such that X + (−X) = (−X) + X = O. ”additive inverse”

2. Rn is closed under scalar multiplication (i.e. c X ∈ Rn ):

(a) c (X + Y ) = c X + c Y ,
(b) (c + d)X = c X + d X,
(c) c (d X) = (c d) X,
(d) 1 X = X, where 1 ∈ R.

Theorem 3.1.2

If X is a vector in Rn and c is any scalar, then

1. 0 X = O.
2. c O = O.
3. (−1) X = −X.
3.1. Vectors in Rn 65

Definition 3.1.1

If X ∈ Rn , then we say that X is a linear combination of the vectors V1 , V2 , · · · , Vn ∈ Rn , if

it can be expressed as
X = c1 V1 + c2 V2 + · · · + cn Vn ,

where c1 , c2 , · · · , cn are scalars in R. These scalars are called the coefficients of the linear
combination.
66 Chapter 3. Euclidean Vector Spaces

Exercise 3.1.1

Find all scalars c1 , c2 , c3 ∈ R such that

c1 (1, 2, 0) + c2 (2, 1, 1) + c3 (0, 0, 1) = (0, 0, 0).

Final answer: c1 = c2 = c3 = 0. ”Try to create a system of linear equations.”

3.2. Norm and Dot Product in Rn 67

3.2 Norm and Dot Product in Rn

Definition 3.2.1

Let X = (x1 , x2 , · · · , xn ) and Y = (y1 , y2 , · · · , yn ) in Rn . Then

1. The dot product of X and Y is defined as

n
X ·Y = xi yi = x1 y1 + x2 y2 + · · · + xn yn .
X

i=1

2. The norm (or length) of the vector X is defined as

q
kX k = x21 + x22 + · · · + x2n .

Remark 3.2.1

We note that, k X k2 = x21 + x22 + · · · + x2n = X · X. Therefore, k X k2 = X · X.

Theorem 3.2.1

If X ∈ Rn and c ∈ R, then:

1. k X k ≥ 0
2. k X k = 0 if and only if X = O.
3. k c X k = | c | k X k.

Proof:

The proof of the first two parts is easy. So we only prove the third statement of the Teorem.
Let X = (x1 , x2 , · · · , xn ). Then c X = (c x1 , c x2 , · · · , c xn ). Thus,
q q
kcX k = (c x1 )2 + (c x2 )2 + · · · + (cxn )2 = c2 (x21 + x22 + · · · + x2n )
q
= | c | x21 + x22 + · · · + x2n = | c | k X k.
68 Chapter 3. Euclidean Vector Spaces

Definition 3.2.2

A vector of norm 1 is called a unit vector in Rn . That is, U is a unit if k U k = 1.

Remark 3.2.2: Normalizing a Vector

1
If nonzero vector X ∈ Rn , then U = X is the unit vector in the same direction as X.
kXk

1 1 1
Clearly, kU k = X = kX k = k X k = 1.
kX k kX k kX k

The vectors i = (1, 0) and j = (0, 1) are called the standard units in R2 .
The vectors i = (1, 0, 0), j = (0, 1, 0) and k = (0, 0, 1) are called the standard units in R3 .
In general, the vectors E1 = (1, 0, · · · , 0), E2 = (0, 1, 0, · · · , 0), · · · , En = (0, · · · , 0, 1) are called
the standard unit vectors in Rn . Note that any vector X = (x1 , x2 , · · · , xn ) ∈ Rn is a linear
combination of these vectors:

X = x1 E1 + x2 E2 + · · · + xn En .

Remark 3.2.3: Another definition of dot product

If X and Y are nonzero vectors in Rn and if θ is the angle between X and

Y , then Y
X ·Y
cos θ = where 0 ≤ θ ≤ π.
kX kkY k
That is X · Y = k X k k Y k cos θ.
θ
Since, −1 ≤ cos θ ≤ 1, we get X
X ·Y
−1 ≤ ≤ 1.
kX kkY k

Theorem 3.2.2

If X, Y, Z ∈ Rn and c ∈ R. Then:

1. O · X = X · O = 0.
3.2. Norm and Dot Product in Rn 69

2. X · Y = Y · X.
3. X · (Y + Z) = X · Y + X · Z and X · (Y − Z) = X · Y − X · Z.
4. (X + Y ) · Z = X · Z + Y · Z and (X − Y ) · Z = X · Z − Y · Z
5. (c X) · Y = X · (c Y ) = c (X · Y ).
6. X · X ≥ 0 and X · X = 0 if and only if X = 0.

Proof:

All parts are easy to prove.

Example 3.2.1

Find the angle between X = (0, 1, 1, 0) and Y = (1, 1, 0, 0).

Solution:

X ·Y
We have cos θ = where X · Y = 0 + 1 + 0 + 0 = 1 and
kX kkY k
√ √
k X k = 02 + 12 + 12 + 02 = 2 = k Y k.

1 π
Therefore, cos θ = which implies that θ = .
2 3

Theorem 3.2.3: Cauchy-Schwarz Inequality

If X and Y are vectors in Rn , then | X · Y | ≤ k X k k Y k.

Theorem 3.2.4: Triangle Inequality

If X, Y, Z ∈ Rn , then: k X + Y k ≤ k X k + k Y k.

Proof:

By Remark 3.2.1, we have

k X + Y k2 = (X + Y ) · (X + Y ) = X · X + X · Y + Y · X + Y · Y

= k X k2 + 2 X · Y + k Y k2 absolute value.
70 Chapter 3. Euclidean Vector Spaces

= k X k2 + 2 | X · Y | + k Y k2 Cauchy-Schwarz Inequality.

≤ k X k2 + 2 k X k k Y k + k Y k2 = (k X k + k Y k)2 .

Example 3.2.2
 
If X and Y are vectors in Rn , then k X + Y k2 + k X − Y k2 = 2 k X k2 + k Y k2 .

Solution:

k X + Y k2 + k X − Y k2 = (X + Y ) · (X + Y ) + (X − Y ) · (X − Y )

= 2(X · X) + 2X · Y − 2X · Y + 2(Y · Y )
 
= 2 k X k2 + k Y k2 .

Remark 3.2.4

If X and Y are in Rn , then k X − Y k ≥ k X k − k Y k .

Proof:

Recall that for real values x and a, we have | x | ≤ a iff −a ≤ x ≤ a. That is a ≥ x and a ≥ −x.
Therefore, we simply show that k X − Y k ≥ k X k − k Y k and k X − Y k ≥ k Y k − k X k. First

k X k = k (X − Y ) + Y k ≤ k X − Y k + k Y k → k X k − k Y k ≤ k X − Y k.

For the second inequality, we use the first one (interchinging X and Y ) in the following way:

kX − Y k = kY − X k ≥ kY k − kX k by the first inequality.

Example 3.2.3

If k X k = 2 and k Y k = 3, what are the largest and smallest values possible for k X − Y k?

Solution:
3.2. Norm and Dot Product in Rn 71

By Triangle inequality, we have k X − Y k ≤ k X k + k Y k = 5 which is the largest values of

k X − Y k. For the smallest value, we use Remark 3.2.4. That is,

k X − Y k ≥ k X k − k Y k = | 2 − 3 | = 1.
72 Chapter 3. Euclidean Vector Spaces

Exercise 3.2.1

1. Use Cauchy-SchwarzâĂŹs inequality to show that

(ab − cd + xy)2 ≤ (a2 + d2 + y 2 )(b2 + c2 + x2 )

for all real numbers a, b, c, d, x, and y.

Hint: Find two suitable vectors in R3 .
2. Let U, V ∈ Rn be unit vectors. Prove that (U + 2V ) · (2U − V ) ≤ 3.
3. Let θ be the angle between the vectors U = (4, −2, 1, 2) and V = (4, 2, 5, 2). Find cos θ.
Final answer: 21
35
.
4. For any vectors X and Y in Rn , show that k X k ≤ k X − 2Y k + 2k Y k.
Hint: Triangle inequality.
5. Let X and Y be two vectors in Rn . Prove that kX − Y k ≤ kXk + kY k.
Hint: Triangle inequality.
6. Find a vector X, of length 6, in the opposite direction of Y = (1, 2, −2).
Hint: What is −6 k Y1 k Y ?
7. Let X and Y be vectors in Rn such that kXk = kY k. Show that (X + Y ) · (X − Y ) = 0.
8. Let U and V be two vectors in R3 such that k U k = 2 and k V k = 3.

(a) Find the maximum possible value for k 2U + 3V k.

(b) If U · V = 0, find k 2U + 3V k.

9. Let X, Y ∈ Rn . Find X · Y given that k X + Y k = 1 and k X − Y k = 5.

Final answer: −6.
10. Answer each of the following as True or False:

(a) If U and V are two unit vectors in Rn , then k U − 6V k ≥ 5.

(b) There exist X, Y ∈ R4 such that k X k = k Y k = 2 and X · Y = 6.

11. Find all values of a for which X · Y = 0, where X = (a2 − a, −3, −1) and Y = (2, a − 1, 2a).
Final answer: a = 1
2
or 3.
1 1
12. If X and Y are vectors in Rn , then X · Y = k X + Y k2 − k X − Y k2 .
4 4
13. Show that if X · Y = 0 for all Y ∈ Rn , then X = O. Use the standard unit vectors of Rn
for Y .
3.2. Norm and Dot Product in Rn 73

14. Show that if X · Z = Y · Z for all Z ∈ Rn , then X = Y . Use the standard unit vectors of
Rn for Z.
74 Chapter 3. Euclidean Vector Spaces
 Chapter

4 General Vector Spaces

4.1 Real Vector Spaces

Definition 4.1.1

A real vector space V is a set of elements with two operations ⊕ and satisfying the following
conditions. For short, we write (V, ⊕, ) is a vector space if

(α) if X, Y ∈ V, then X ⊕ Y ∈ V, that is ”V is closed under ⊕”: for all X, Y, Z ∈ V

(a) X ⊕ Y = Y ⊕ X,

(b) X ⊕ (Y ⊕ Z) = (X ⊕ Y ) ⊕ Z,

(c) there exists O ∈ V such that X ⊕ O = O ⊕ X = X,

(d) for each X ∈ V, there exists e ∈ V such that e ⊕ X = X ⊕ e = O.

(β) if X ∈ V and c ∈ R, then c X ∈ V, that is ”V is closed under ”: for all X, Y ∈ V and

for all c, d ∈ R

(a) c (X ⊕ Y ) = c X ⊕c Y,

(b) (c + d) X=c X ⊕d X,

(c) c (d X) = (c d) X,

(d) 1 X=X 1 = X.

Remark 4.1.1

(Rn , +, ·) is a vector space. That is, Rn with vector addition and scalar multiplication is a vector
space.

75
76 Chapter 4. General Vector Spaces

Example 4.1.1

Consider V = {(x, y, z) : x, y, z ∈ R} with

(x1 , y1 , z1 ) ⊕ (x2 , y2 , z2 ) = (x1 + x2 , y1 + y2 , z1 + z2 ),

and
c (x, y, z) = (cx, cy, 0).

Is (V, ⊕, ) a vector space? Explain.

Solution:

Clearly, the α conditions are satisfied because this is the usual vector addition and hence V is
closed under ⊕. Thus, we only check on the β conditions. Let X = (x1 , y1 , z1 ), Y = (x2 , y2 , z2 )
be any two vectors in V, then

1. c (X + Y ) = c (x1 + x2 , y1 + y2 , z1 + z2 ) = (cx1 + cx2 , cy1 + cy2 , 0) = (cx1 , cy1 , 0) +

(cx2 , cy2 , 0) = c X +c Y . This condition is satisfied.

2. (c + d) X = ((c + d)x1 , (c + d)y1 , 0) = (cx1 , cy1 , 0) + (dx1 , dy1 , 0) = c X +d X. This

condition is satisfied.

3. c (d X) = c (dx1 , dy1 , 0) = (cdx1 , cdy1 , 0) = (cd) X. This condition is satisfied.

4. 1 X = (x1 , y1 , 0) 6= (x1 , y1 , z1 ). This condition is NOT satisfied.

Therefore, (V, ⊕, ) is not a vector space.

Example 4.1.2

Let V = {(x, y, z) : x, y, z ∈ R and z > 0} associated with the operations:

(x1 , y1 , z1 ) ⊕ (x2 , y2 , z2 ) = (x1 + x2 , y1 + y2 , z1 + z2 ), and c (x, y, z) = (cx, cy, cz).

Is (V, ⊕, ) a vector space? Explain.

Solution:

No. If c ∈ R with c < 0, then c (x, y, z) = (cx, cy, cz) 6∈ V since cz < 0.
4.1. Real Vector Spaces 77

Example 4.1.3

Is the set of real numbers under the substraction and scalar multiplication a vector space?
Explain.

Solution:

NO. Clearly, for x, y ∈ R, x ⊕ y = x − y 6= y − x = y ⊕ x.

Theorem 4.1.1

Let V be a vector space, X is a vector in V, and c is a scalar, then:

1. The zero vector is unique in V.

2. 0 X = O.
3. c O = O.
4. (−1)X = −X.
5. If c X = O, then c = 0 or X = O.

Proof:

1. Assume that O1 and O2 are two zero vectors in V. Then

O1 = O1 + O2 = O2 ⇒ O1 = O2 .

2. 0X + 0X = (0 + 0)X = 0X. The negative inverse of 0X, namely −0X, is in V. Hence

0X + 0X + (−0X) = 0X + (−0X) ⇒ 0X = O.

3. O + O = O. Then c(O + O) = cO implies that cO + cO = cO. Adding the negative of cO,

namely −cO, to both sides, we get: cO + cO − cO = cO − cO and hence cO = O.
4. To show that (−1)X = −x, we simply show that X + (−1)X = O. Clearly, X + (−1)X =
(1 + (−1))X = 0X = O by part (2).
5. If c = 0, then we are done. Otherwise, assume c 6= 0. Then 1
c
cX = 1
c
O = O (by Part 3).
Hence 1 X = X = O.
78 Chapter 4. General Vector Spaces

Exercise 4.1.1

1. Let V = {(x, y, 0) | x, y ∈ R}, be associated with the operations:

(x1 , y1 , 0) ⊕ (x2 , y2 , 0) = (x1 + x2 , y1 + y2 , 0), and c (x, y, 0) = (cx, cy, 0).

Is (V, ⊕, ) form a vector space? Explain. (This is a Vector Space!!).

2. Let V = {(x, y) | x, y ∈ R}. Define addition and scalar multiplication on V as follows: for
each (x, y), (x0 , y 0 ) ∈ V and a ∈ R,

(x, y) ⊕ (x0 , y 0 ) = (x + x0 , y + y 0 ) and a (x, y) = (ay, ax).

Determine whether V with the given operations is a vector space. Justify your answer.
3. Consider R2 with the operations ⊕ and where (x, y) ⊕ (x0 , y 0 ) = (2x − x0 , 2y − y 0 ) and
c (x, y) = c(x, y). Does the property (c + d) X=c X ⊕d X hold for all c, d ∈ R
and all X ∈ R2 ? Explain.
4. Consider the set V = {(x, y, z) : x, y, z ∈ R} with the following operations

(x1 , y1 , z1 ) ⊕ (x2 , y2 , z2 ) = (x1 + x2 , y1 + y2 , z1 + z2 ) and c (x, y, z) = (z, y, x).

Is V a vector space? Explain.

5. Let V = {(x, y, z) : x, y, z ∈ R} and define

(x, y, z) ⊕ (a, b, c) = (x + a, y + b, z + c) and k (x, y, z) = (kx, ky, 0).

Show that V is not a vector space.

6. Consider the set V = {(x, y) | x, y ∈ R} with the following operations

(x, y) ⊕ (x0 , y 0 ) = (x − x0 , y − y 0 ) and k (x, y) = (kx, ky).

Determine whether (V, ⊕, ) is a vector space (justify your answer).

7. Let V be a real vector space. Show that 0X = O for any X ∈ V .
8. Let V be a vector space with a zero vector O. Show that the zero vector O of V is unique.
9. Prove that the negative of a vector X in a vector space V is unique.
10. Determine whether V = R is a vector space with respect to the following operations:
X ⊕ Y = 2X − Y and c X = cX, for all X, Y ∈ V and for all c ∈ R.
4.1. Real Vector Spaces 79

11. Determine whether V = {(x, y, z) : x, y, z ∈ R} is a vector space with respect to the

following operations: (x, y, z) ⊕ (x0 , y 0 , z 0 ) = (xx0 , yy 0 , zz 0 ) and c (x, y, z) = (cx, cy, cz),
for all (x, y, z), (x0 , y 0 , z 0 ) ∈ V and for all c ∈ R.
12. Answer each of the following as True or False:

(a) (T) V = {(x, y) ∈ R2 : y < 0} is closed under the operation c (x, y) = (cx, y).

(b) (F) V = {(x, y) ∈ R2 : y < 0} is closed under the operation c (x, y) = (cy, x).
80 Chapter 4. General Vector Spaces

4.2 Subspaces

Definition 4.2.1

Let (V, ⊕, ) be a vector space and let W ⊆ V be non-empty. If (W, ⊕, ) is a vector space,
then W is a subspace of V.

Remark 4.2.1

If V is a vector space, then V and {0} are subspaces of V. They are called trivial subspaces
of V.

Theorem 4.2.1

Let (V, ⊕, ) be a vector space and let W be a subset of V. Then, W is a subspace of V if and
only if the following conditions hold:

1. W 6= φ,
2. for all x, y ∈ W, x ⊕ y ∈ W,
3. for all x ∈ W and c ∈ R, c x ∈ W.

Example 4.2.1

Is W = {(x, y, 0, z 2 ) : x, y ∈ R and z ∈ Z} a subspace of R4 ? Explain.

Solution:

1. (0, 0, 0, 0) ∈ W and hence W is non-empty.

2. let (x1 , y1 , 0, z12 ), (x2 , y2 , 0, z22 ) ∈ W, then

(x1 , y1 , 0, z12 ) + (x2 , y2 , 0, z22 ) = (x1 + x2 , y1 + y2 , 0, z12 + z22 ) 6∈ W since z12 + z22 6= (z1 + z2 )2 .

For example, (0, 0, 0, 4), (0, 0, 0, 9) ∈ W while the sum of them (0, 0, 0, 15) 6∈ W. Therefore, W is
not a subspace of R4 .
4.2. Subspaces 81

Example 4.2.2

Is W = {(x, y, z) : x + y + z = 1, where x, y, z ∈ R} a subspace of R3 ? Explain.

Solution:

Clearly, (0, 0, 0) 6∈ W and hence W is not a vector space and it is not a subspace of R3 .

Example 4.2.3

Let W = {(a, b, c, d) : d = 2a − b and c = a}. Is (W, +, ·) a subspace of R4 ? Explain.

Solution:

1. (0, 0, 0, 0) ∈ W, then W 6= φ,
2. let X = (a1 , b1 , a1 , 2a1 − b1 ) and Y = (a2 , b2 , a2 , 2a2 − b2 ). Then,

X +Y = (a1 , b1 , a1 , 2a1 − b1 ) + (a2 , b2 , a2 , 2a2 − b2 )

= (a1 + a2 , b1 + b2 , a1 + a2 , 2(a1 + a2 ) − (b1 + b2 )) ∈ W.

3. for X = (a, b, a, 2a−b) ∈ W and k ∈ R, we have k(a, b, a, 2a−b) = (ka, kb, ka, 2(ka), −kb) ∈
W.

Therefore, W is a subspace of R4 .

Theorem 4.2.2

If W1 , W2 , · · · , Wn are subspaces of a vector space V, the the intersection of these subspaces is

also a subspace of V.

Proof:

Let W be the intersection of these subspaces. Then W is not empty since Wi contains the zero
vector for all 1 ≤ i ≤ n. Moreover, if X, Y ∈ W, then X, Y ∈ Wi for all i and hence X + Y ∈ Wi
which implies that X + Y ∈ W. Finaly, if c is a scalar and X ∈ W, then X ∈ Wi for all i and
hence c X ∈ Wi which implies that c X ∈ W. Therefore, W is a subspace of V.
82 Chapter 4. General Vector Spaces

Definition 4.2.2

Let AX = O be a homogenous system for A ∈ Mm×n and X ∈ Rn . We define the null space
(or the solution space of AX = O) of A by

W = {X : AX = O} ⊆ Rn .

Theorem 4.2.3

The solution space of the homogeneous system AX = O (null space of A), where A is m × n
matrix and X ∈ Rn is a subspace of Rn .

Proof:

Let W = {X : AX = O} ⊆ Rn be the null space of A. Then

1. Clearly, W 6= φ since AX = O always has a solution (either trivial or non-trivial),

2. If X, Y ∈ W, then AX = AY = O. But, A(X + Y ) = AX + AY = O + O = O. Thus,
X + Y ∈ W,
3. For any c ∈ R and X ∈ W, we have A(cX) = cAX = cO = O, thus cX ∈ W.

Therefore, W is a subsapce of Rn .

If AX = B with B 6= O and A ∈ Mm×n and X ∈ Rn . Then W = {X : AX = B} ⊆ Rn is not

necessarily a subspace of Rn . Simply, the system AX = B might have no solutions and then W
is empty.

Definition 4.2.3

Let X1 , X2 , · · · , Xn be vectors in a vector space V, a vector X ∈ V is called a linear combina-

tion of the vectors X1 , X2 , · · · , Xn if and only if X = c1 X1 + c2 X2 + · · · + cn Xn for some real
numbers c1 , c2 , · · · , cn .
4.2. Subspaces 83

Example 4.2.4

Determine whether the vector X = (2, 1, 5) is a linear combination of the set of vectors
{X1 , X2 , X3 } where X1 = (1, 2, 1), X2 = (1, 0, 2), and X3 = (1, 1, 0).

Solution:

X is a linear combination of X1 , X2 , X3 if we find numbers c1 , c2 , c3 so that X = c1 X1 + c2 X2 +

c3 X3 . Consider c1 (1, 2, 1) + c2 (1, 0, 2) + c3 (1, 1, 0) = (2, 1, 5), this is a system in three unknowns:

c1 + c2 + c3 = 2
2c1 + 0 + c3 = 1
c1 + 2c2 + 0 = 5

So,
 we solve the
 system:    
1 1 1 2 1 1 1 2 1 1 1 2
 r2 −2r1 →r2   r2 ⇔r3   r1 −r2 →r1
2 0 1 1 −−−−−−→ 0 −2 −1 −3 −−−→ 0 1 −1 3 −−−−−−→

  r3 −r1 →r3     r3 +2r2 →r3
1 2 0 5 0 1 −1 3 0 −2 −1 −3
     
1 0 2 −1 1 1 0 2 −1 1 0 0 1
 − 3 r3 →r3   r1 −2r3 →r1 
0 1 −1 3 −−−−−→ 0 1 −1 3 −−−−−−→ 0 1 0 2
 
    r2 +r3 →r2  
0 0 −3 3 0 0 1 −1 0 0 1 −1
   
c1   1
Therefore, c2  =  2 is a solution. Thus, X = X1 + 2X2 − X3 (check!).
   
   
c3 −1
Note that we can solve the problem as follows: The matrix of coefficient above has determinant
equals to 3 and hence the system has a unique solution. Therefore there are c1 , c2 , and c3 satis-
fying the linear combination equation. This show that X is a linear combination of X1 , X2 , X3
and we are done without solving the system.

Definition 4.2.4

Let S = {X1 , X2 , · · · , Xk } be a subset of a vector space V. Then, the set of all vectors in V that
are linear combination of the vectors in S is denoted by span S or span {X1 , X2 , · · · , Xk }.
Moreover, if W = span S then W is a subspace of V and we say that S spans W or that W is
spanned by S.
84 Chapter 4. General Vector Spaces

Theorem 4.2.4

If S = {X1 , X2 , · · · , Xk } is a nonempty set of vectors in a vector space V, then span S is a

subspace of V.

Proof:

Let W = span S = {Z : Z = c1 X1 + c2 X2 + · · · + ck Xk } ⊆ V. Then,

1. W 6= φ since Z = X1 + X2 + · · · + Xk ∈ W for ci = 1 for all i = 1, 2, · · · , k,

2. let Z1 = c1 X1 + c2 X2 + · · · + ck Xk , Z2 = d1 X1 + d2 X2 + · · · + dk Xk ∈ W, then

Z1 + Z2 = (c1 + d1 )X1 + (c2 + d2 )X2 + · · · + (ck + dk )Xk ∈ W,

3. for c ∈ R and Z = c1 X1 + c2 X2 + · · · + ck Xk ∈ W, we have

Z = c c1 X1 + c c2 X2 + · · · + c ck Xk ∈ W.

Therefore, W = span S is a subspace of V.

Theorem 4.2.5

If S is a nonempty set of vectors in a vector space V, then span S is the smallest subspace of
V that contains all of the vectors in S. That is, any other subspace that contains S contains
span S.

Example 4.2.5

Let S = {(1, 1, 0, 1), (1, −1, 0, 1), (0, 1, 2, 1)}. Determine whether X and Y belong to span S,
where X = (2, 3, 2, 3), and Y = (0, 1, 2, 3).

Solution:

For X, consider the system X = (2, 3, 2, 3) = c1 (1, 1, 0, 1) + c2 (1, −1, 0, 1) + c3 (0, 1, 2, 1). This
system has the unique solution: c1 = 2, x2 = 0, c3 = 1. Thus, X belongs to span S.
For Y we consider the system Y = (0, 1, 2, 3) = c1 (1, 1, 0, 1) + c2 (1, −1, 0, 1) + c3 (0, 1, 2, 1). This
system is inconsistent and has no solutions. Thus, Y does not belong to span S.
4.2. Subspaces 85

Recall that any vector in Rn can be written as a linear combination of the standard unit vectors
{E1 , E2 , · · · , En }. Thus, {E1 , E2 , · · · , En } spans Rn .

We now go back to show how can we solve Example 4.2.3 by three different method. One method
was already shown in that example.

Example 4.2.6

Let W = {(a, b, c, d) : d = 2a − b and c = a}. Is (W, +, ·) a subspace of R4 ? Explain.

Solution: Definition

We simply show it by the meaning of the definition of subspaces. Look at the Example 4.2.3.

Solution: Null Space

W = {(a, b, c, d) : 2a − b − d = 0 and a − c = 0}.

 
2 −1 0 −1
That is W is the solution space of AX = O where A =  and X = (a, b, c, d).
1 0 −1 0
Therefore, W is a subspace of R .4

Solution: The Span

W = {(a, b, a, 2a − b) : a, b ∈ R} = {a (1, 0, 1, 2) + b (0, 1, 0, −1) : a, b ∈ R}.

Therefore W = span {(1, 0, 1, 2), (0, 1, 0, −1)}. That is W is a subspace of R4 .

Example 4.2.7

Let S = {X1 , X2 } where X1 = (1, 1, 0), and X2 = (1, 1, 1). Does S spans R3 ? Explain.

Solution:

Let X = (a, b, c) be any vector in R3 . Consider

c1 (1, 1, 0) + c2 (1, 1, 1) = (a, b, c).

Note that we can not use the determinant argument here since we have no square matrix. Thus,
86 Chapter 4. General Vector Spaces

solving the system, we get

   
1 1 a 1 1 a
 r2 −r1 →r2 
1 1 b −
− − − − −
→ 0 0 b − a .
 

  
0 1 c 0 1 c

This system has no solution if b − a 6= 0. Therefore, X is not a linear combination of S and S

does not span R3 .

Example 4.2.8

Let S = {X1 , X2 } where X1 = (1, 1, 0), and X2 = (1, 1, 1). Does S spans R3 ? Explain.

Solution:

Let X = (a, b, c) be any vector in R3 . Consider

c1 (1, 1, 0) + c2 (1, 1, 1) = (a, b, c).

Note that we can not use the determinant argument here since we have no square matrix. Thus,
solving the system, we get
   
1 1 a 1 1 a
 r2 −r1 →r2 
1 1 b −−−−−−→ 0 0 b − a .
 
   
0 1 c 0 1 c

This system has no solution if b − a 6= 0. Therefore, X is not a linear combination of S and S

does not span R3 .
4.2. Subspaces 87

Exercise 4.2.1

1. Let X, Y ∈ Rn and W = {Z : Z = aX + bY, for a, b ∈ R}. Is W a subspace of Rn ?

Explain.
2. Let X1 = (1, 0, 2), X2 = (2, 0, 1) and X3 = (1, 0, 3) be vectors in R3 . Determine whether
the vector X = (1, 2, 3) can be written as a linear combination of X1 , X2 , and X3 .
n √  o
3. Determine whether the subset W = a, a + 2, 3a : a ∈ R of R3 is a subspace of R3 .
4. Determine whether the vectors X1 = (3, 0, 0, 0), X2 = (0, −1, 2, 1), X3 = (6, 2, −6, 0), and
X4 = (3, −2, 3, 3) spans the vector space R4 .
5. Show that the solution set of a homogeneous linear system AX = O in n unknowns is a
subspace of Rn .
6. Let W = {(a, 2a, b, a − b) : a, b ∈ R} be a subset of R4 . Show that W is a subspace of R4 .
7. Determine whether W1 and W2 are subspaces of R4 .

(a) W1 = {(a, b, c, d) : a2 + b2 + c2 + d2 > 0}.

(b) W2 = {(a, b, c, d) : a + 3b − 2c + 4d = 0 and a − 5b + 4c + 7d = 0}.

8. Determine whether W1 and W2 are subspaces of R3 .

(a) W1 = {(a, b, c) : a − c = b}.

(b) W2 = {(a, b, c) : ab ≥ 0}.
88 Chapter 4. General Vector Spaces

4.3 Linear Independence

Definition 4.3.1

The set of vectors S = {X1 , X2 , · · · , Xn } in a vector space V is said to be linearly dependent

if there exist constants c1 , c2 , · · · , cn not all zeros, such that

c1 X1 + c2 X2 + · · · + cn Xn = 0.

Otherwise, S is said to be linearly independent. That is, X1 , X2 , · · · , Xn are linearly inde-

pendent if whenever, c1 X1 + c2 X2 + · · · + cn Xn = 0, we must have c1 = c2 = · · · = cn = 0.

Note that the standard unit vectors are linearly independent in Rn since the homogeneous system

c1 (1, 0, · · · , 0) + c2 (0, 1, · · · , 0) + · · · + cn (0, 0, · · · , 1) = (0, 0, · · · , 0)

clearly has only the trivial solution c1 = c2 = · · · = cn = 0.

Example 4.3.1

Determine whether X1 = (1, 0, 1, 2), X2 = (0, 1, 1, 2), and X3 = (1, 1, 1, 3) in R4 are linearly
independent or linearly dependent? Explain.

Solution:

We solve the homogenous system: c1 X1 + c2 X2 + c3 X3 = 0. That is,

   
1 0 1 0 1 0 0 0
0 1 1 0
 → · · · → 0 1 0 0
   

 
1 1 1 0 0 0 1 0
   

2 2 3 0 0 0 0 0
   

Therefore, c1 = c2 = c3 = 0 and thus X1 , X2 , X3 are linearly independent.

4.3. Linear Independence 89

Example 4.3.2

Determine whether the vectors X1 = (1, 2, −1), X2 = (1, −2, 1), X3 = (−3, 2, −1), and X4 =
(2, 0, 0) in R3 is linearly independent or linearly dependent? Explain.

Solution:

Consider the homogenous system: c1 X1 + c2 X2 + c3 X3 + c4 X4 = 0. This system has a non-trivial

solutions because the number of unknowns (4) is greater than the number of equations (3).
Therefore, X1 , X2 , X3 , X4 are linearly dependent.
In addition, we can show that this set is linearly dependent by the mean of r.r.e.f. as follows:
   
1 1 −3 2 0 1 0 −1 1 0
2 −2 2 0 0 → · · · → 0 1 −2 1 0 .
   
 
  
−1 1 −1 0 0 0 0 0 0 0

From the reduced system above, we see that (from the third column) X3 = −X1 − 2X2 and that
(from the fourth column) X4 = X1 + X2 .

Theorem 4.3.1

A set S with two or more vectors is

1. Linearly dependent iff at least one of the vectors in S is a linear combination of the other
vectors in S.
2. Linearly independent iff no vectors in S is a linear combination of the other vectors in S.

Remark 4.3.1

Let S = {X1 , X2 , · · · , Xn } be a set of vectors in Rn and let A be an n × n matrix whose columns

are the n-vectors of S. Then,

1. if A is singular, then S is linearly dependent,

2. if A is non-singular, then S is linearly independent.
90 Chapter 4. General Vector Spaces

Theorem 4.3.2

1. A set that contains O ”the zero vector” is linearly dependent.

2. A set with exactly one vector is linearly independent iff that vector is not O.
3. A set with exactly two nonzero vectors is linearly independent iff neither vector is a scalar
multiple of the other.

Example 4.3.3

For what values of α are the vectors (−1, 0, −1), (2, 1, 2), (1, 1, α) in R3 linearly dependent?
Explain.

Solution:

We want the vectors to be linearly dependent, so consider the system c1 (−1, 0, −1) + c2 (2, 1, 2) +
c3 (1, 1, α) = (0, 0, 0). This system has non-trivial solutions only if A = 0, where A is the
matrix whose columns are [−1, 0, −1]T , [2, 1, 2]T , and [1, 1, α]T . That is,

-1 2 1
A = 0 1 1 = 0 ⇐⇒ −(α − 2) − (2 − 1) = 0 ⇐⇒ 2 − α − 1 = 0 ⇐⇒ α = 1.
-1 2 α

Therefore, if α = 1 the vectors are linearly dependent. Otherwise if α ∈ R\{1}, the vectors are
linearly independent.

Theorem 4.3.3

Let S = {X1 , X2 , · · · , Xm } be a set of vectors in Rn . If m > n, then S is linearly dependent.

Example 4.3.4

Suppose that S = {X1 , X2 , X3 } is a linearly independent set of vectors in a vector space V. Show
that T = {Y1 , Y2 , Y3 } is also linearly independent set, where Y1 = X1 + X2 + X3 , Y2 = X2 + X3 ,
and Y3 = X3

Solution:
4.3. Linear Independence 91

Consider the system c1 Y1 + c2 Y2 + c3 Y3 = 0. Therefore,

c1 Y 1 + c2 Y 2 + c3 Y 3 = 0
c1 (X1 + X2 + X3 ) + c2 (X2 + X3 ) + c3 (X3 ) = 0
c1 X 1 + c1 X 2 + c1 X 3 + c2 X 2 + c2 X 3 + c3 X 3 = 0
(c1 )X1 + (c1 + c2 )X2 + (c1 + c2 + c3 )X3 = 0
But X1 , X2 , X3 are linearly independent, thus c1 = c1 + c2 = c1 + c2 + c3 = 0. Therefore,
c1 = c2 = c3 = 0 and hence T is linealry independent.

Example 4.3.5

Suppose that S = {X1 , X2 , X3 } is a linearly dependent set of vectors in a vector space V.

Show that T = {Y1 , Y2 , Y3 } is also linearly dependent set, where Y1 = X1 , Y2 = X1 + X2 , and
Y3 = X 1 + X2 + X3

Solution:

Consider the system c1 Y1 + c2 Y2 + c3 Y3 = 0. Therefore,

c1 Y 1 + c2 Y 2 + c3 Y 3 = 0
c1 (X1 ) + c2 (X1 + X2 ) + c3 (X1 + X2 + X3 ) = 0
(c1 + c2 + c3 )X1 + (c2 + c3 )X2 + (c3 )X3 = 0
But X1 , X2 , X3 are linearly dependent, thus at least one of c1 +c2 +c3 , c2 +c3 , and c3 is non-zero.
Therefore, one of c1 , c2 , c3 is non-zero and hence T is linealry dependent.

Remark 4.3.2

The set of linearly independent vectors should be non-zeros distinct vectors.

Remark 4.3.3

Let S1 , S2 be two subsets of a vector space V with S1 ⊆ S2 . Then,

1. if S1 is linearly dependent, then S2 is linearly dependent,

2. if S2 is linearly independent, then S1 is linearly independent.
92 Chapter 4. General Vector Spaces

Example 4.3.6

Show that if S = {X1 , X2 , · · · , Xn } is a linearly independent set of vectors, then so is any subset
of S.

Solution:

Let T = {X1 , · · · , Xk } with k ≤ n. As S is linearly independent set, c1 X1 +· · · cn Xn = O implies

that c1 = · · · = cn = 0. To show that T is linearly independent, we show that c1 = · · · = ck = 0.
Consider the system c1 X1 + · · · + ck Xk = O. Then

c1 X1 + c2 X2 + · · · + ck Xk + (0)Xk+1 + · · · + (0)Xn = O.

This system has only the trivial solution and hence c1 = · · · = ck = 0. Therefore, T is linealy
independent.

Note that if S = {X1 , X2 , X3 } is a linear independent set, then as we have seen in the previ-
ous example the sets {X1 , X2 }, {X1 , X3 }, {X2 , X3 }, {X1 }, {X2 }, and {X3 } are also linearly
independent.

Example 4.3.7

Show that if S = {X1 , X2 , X3 } is a linearly dependent set of vectors in a vector space V, and X4
is any vector in V that is not in S, then {X1 , X2 , X3 , X4 } is also linearly dependent.

Solution:

Since S is linealy dependent set, the homogeneous system c1 X1 + c2 X2 + c3 X3 = O has a non

trivial solution (c1 , c2 , c3 ). Then the homogeneous system c1 X1 + c2 X2 + c3 X3 + c4 X4 = O also
has a nontrivial solution (c1 , c2 , c3 , 0).
4.3. Linear Independence 93

Exercise 4.3.1

1. Show that if {X1 , X2 } is a linearly dependent set, then one of the vector is a scalar multiple
of the other.
2. Show that any subset of a vector space V contains the zero vector is a linearly dependent
set.
3. Show that if {X1 , X2 , · · · , Xn } is a linearly dependent set, then we can express one of the
vectors in terms of the others.
4. Let X, Y, Z ∈ Rn be three nonzero vectors where the dot product of any (distinct) two
vectors is 0. Show that the set {X, Y, Z} is linearly independent.
94 Chapter 4. General Vector Spaces

4.4 Basis and Dimension

Definition 4.4.1

A set S = {X1 , X2 , · · · , Xn } of distinct nonzero vectors in a vector space V is called a basis iff

1. S spans V (V = span S),

2. S is linearly independent set.

The dimension of V is the number of vectors in its basis and is denoted by dim(V).

Example 4.4.1

Show that the set S = {X1 = (1, 0, 1), X2 = (0, 1, −1), X3 = (0, 2, 2)} is a basis for R3 .

Solution:

To show that S is a basis for R3 , we show that S is a linearly independent set that spans R3 .

1. S is linealy independent? Consider the homogenous system

c1 (1, 0, 1) + c2 (0, 1, −1) + c3 (0, 2, 2) = (0, 0, 0).

This system has a trivial solution if A 6= 0, where A is the matrix of coefficients. That
is,
1 0 0
1 2
|A| = 0 1 2 = (1) = 2 − (−2) = 4 6= 0.
−1 2
1 −1 2
Thus, the system has only the trivial solution and hence S is linearly independent.

2. S spans R3 ? For any X = (a, b, c) ∈ R3 , consider the nonhomogenous system:

c1 (1, 0, 1) + c2 (0, 1, −1) + c3 (0, 2, 2) = (a, b, c).

Since the A 6= 0 where A is the matrix of coefficients, the system has a unique solution
and thus S spans R3 .

Therefore, S is a basis for R3 .

4.4. Basis and Dimension 95

Remark 4.4.1

The set of standard unit vectors {E1 , E2 , · · · , En } ∈ Rn forms the standard basis for Rn and
hence dim(Rn ) = n.

Theorem 4.4.1

Let S = {X1 , X2 , · · · , Xn } be a basis for a vector space V. Then, every vector in V can be
written in exactly one way as a linear combination of the vectors in S.

Proof:

Let X ∈ V. Since S is a basis of V, then S spans V. That is, we can write

X = c1 X1 + c2 X2 + · · · + cn Xn , and (4.4.1)
X = d1 X1 + d2 X2 + · · · + dn Xn . (4.4.2)

By substracting Equation 4.4.2 out of Equation 4.4.1, we get

0 = (c1 − d1 )X1 + (c2 − d2 )X2 + · · · + (cn − dn )Xn .

But S is linearly independent set (it is a basis). Thus, c1 − d1 = 0, · · · , cn − dn = 0. Therefore,

c1 = d1 , c2 = d2 , · · · , cn = dn , and hence X can be written in one and only one way as a linear
combination of vectors in S.

Theorem 4.4.2

Let V be a finite-dimensional vector space, and let {X1 , X2 , · · · , Xn } be any basis:

1. If a set has more than n vectors, then it is linearly dependent.

2. If a set has fewer than n vectors, then it does not span V.

Theorem 4.4.3

All bases for a finite-dimensional vector space have the same number of vectors.
96 Chapter 4. General Vector Spaces

Theorem 4.4.4: The Plus/Minus Theorem

Let S be a nonempty set of vectors in a vector space V. Then

1. If S is linearly independent and X is a vector in V not in span S, then the set S ∪ {X}
is linearly independent.
2. If X is a vector in S that is a linear combination of the other vectors in S, then S − {X}
span the same space. That is, span S = span (S − {X}).

Example 4.4.2

Find a basis for and the dimension of the subspace of all vectors of the form (a, b, −a − b, a − b),
for a, b ∈ R.

Solution:

Let W = {(a, b, −a − b, a − b) | a, b ∈ R} ⊆ R4 . Let X be any vector of W, then

X = (a, b, −a − b, a − b) = a(1, 0, −1, 1) + b(0, 1, −1, −1) ∈ W.

Therefore, S = {(1, 0, −1, 1), (0, 1, −1, −1)} spans W.

Clearly,
c1 (1, 0, −1, 1) + c2 (0, 1, −1, −1) = (0, 0, 0, 0)

holds only if c1 = c2 = 0 which shows that S is linearly independent. That is, S is a basis for W
and dim(W) = 2.

Example 4.4.3

Find a basis for and the dimension of the solution space of the homogeneous system

x1 + x2 + 2x4 = 0
x2 − x3 + x4 = 0
x1 + x2 + 2x4 = 0

Solution:
4.4. Basis and Dimension 97

We first solve the system using Gauss-Jordan method:

   
1 1 0 2 0
r.r.e.f.
 1 0 1 1 0
0 1 −1 1 0 ≈ · · · · · · ≈  0 1 −1 1 0
   

  
1 1 0 2 0 0 0 0 0 0

Thus the solutions are: x1 = −x3 − x4 ; x2 = x3 − x4 and x3 = t and x4 = r for t, r ∈ R. That

is the solution space of the homogeneous system is W = {(−t − r, t − r, t, r) : t, r ∈ R}.
Therefore, any vector X in W is of the form: X = t(−1, 1, 1, 0) + r(−1, −1, 0, 1) which means
that S = {(−1, 1, 1, 0), (−1, −1, 0, 1)} spans W.
As S is a linearly independent set (none of the vectors is a scalar multiple of the other), S forms
a basis for W, and hence the solution space has dimension 2.

Theorem 4.4.5

Let V be an n -dimensional vector space, and let S = {X1 , X2 , · · · , X n } be a set in V. Then,

S is a basis for V iff S spans V or S is linearly independent.

The set S = {X1 = (1, 5), X2 = (1, 4)} is linear independent in the 2-dimensional vector space
R2 . Hence, S forms a basis for R2 .
Moreover, considering S = {X1 = (1, 0, 5), X2 = (1, 0, 4), X3 = (1, 1, 1)}, we see that X1 and X2
form a linear independent set in the xz-plane. The vector X3 is outside of the xz-plane, so the
set S is linearly independent set in R3 . Hence, S forms a basis for R3 .

Example 4.4.4

Find all values of a for which S = {(a2 , 0, 1), (0, a, 2), (1, 0, 1)} is a basis for R3 .

Solution:

Since dim(R3 ) = 3 = size of S, it is enough to show that S is linearly independent (or it spans
R3 ) to show that it is a basis for R3 . Consider c1 (a2 , 0, 1) + c2 (0, a, 2) + c3 (1, 0, 1) = (0, 0, 0).
98 Chapter 4. General Vector Spaces

Clearly, S is linearly independent if A 6= 0, where A is the coefficient matrix. That is,

a2 0 1
0 = a(a − 1) 6= 0 =⇒ a 6= 0 and a 6= ±1.
2
0 a
1 2 1

Therefore, S is a basis for R3 if a ∈ R\{−1, 0, 1}.

Theorem 4.4.6: Reduction and Extension Theorem

Let S be a finite set of vectors in a finite-dimensional vector space V.

1. If S spans V but is not a basis, then S can be reduced to a basis for V by removing
appropriate vectors from S.
2. If S is a linearly independent set that is not a basis for V, then S can be extended to a
basis for V by adding appropriate vectors to S.

Remark 4.4.2: How to construct a basis?

Let V be a vector space and S = {X1 , X2 , · · · , Xn } is a subset of V. The procedure to find a

subset of S that is a basis for W = span S is:

1. form the linear combination c1 X1 + c2 X2 + · · · + cn Xn = 0,

2. form the augmented matrix of the homogenous system in step (1),
3. find the r.r.e.f. of the augmented matrix,
4. Vectors in S corresponding to leading columns form a basis for W = span S.

Example 4.4.5

Let S = {X1 = (1, 0, 1), X2 = (1, 1, 1), X3 = (0, −1, 0), X4 = (2, 1, 2)} be a set of vectors in R3 .
Find a subset of S that is a basis for W = span S, and find the dimension of W.

Solution:

We form the homogenous system: c1 X1 + c2 X2 + c3 X3 + c4 X4 = O to find a linearly independent

4.4. Basis and Dimension 99

subset of S:    
1 1 0 2 0
r.r.e.f.
 1 0 1 1 0
0 1 −1 1 0 ≈ ······ ≈  0 1 −1 1 0
   
   
1 1 0 2 0 0 0 0 0 0
The leading entries are pointing (appear) on the first two columns, namely columns 1 and 2.
Therefore, {X1 , X2 } is linearly independent and it spans W. Thus, {X1 , X2 } is a basis for
W = span S and dim(W) = 2.

Example 4.4.6

Find a basis for R4 that contains the vectors X1 = (1, 0, 1, 0) and X2 = (−1, 1, −1, 0).

Solution:

Consider the set S = {X1 , X2 , E1 , E2 , E3 , E4 }. The set S spans R4 but it contains some linearly
dependent vectors. In order to delete those, we follow the following procedure:
   
1 −1 1 0 0 0 0  1 0 0 1 1 0 0
0 1 0 1 0 0 0  0 1 0 1 0 0 0
   
 r.r.e.f.
 ≈ ······ ≈ 
  
1 −1 0 0 1 0 0  0 0 1 0 −1 0 0
 

0 0 0 0 0 1 0 0 0 0 0 0 1 0
   

The leading entries pointing on the columns 1, 2, 3, and 6. Therefore, the set {X1 , X2 , E1 , E4 }
is a basis for R4 containing X1 and X2 .

Theorem 4.4.7

If W is a subspace of a finite-dimensional vector space V, then

1. W is finite-dimensional.
2. dim(W) ≤ dim(V).
3. W = V iff dim(W) = dim(V).
100 Chapter 4. General Vector Spaces

Exercise 4.4.1

1. Show that the set S = {X1 , X2 , X3 , X4 } is a basis for R4 , where

X1 = (1, 0, 1, 0), X2 = (0, 1, −1, 2), X3 = (0, 2, 2, 1), and X4 = (1, 0, 0, 1).

2. Let S = {X1 , X2 , X3 , X4 , X5 } be a set of R4 where X1 = (1, 2, −2, 1), X2 = (−3, 0, −4, 3),
X3 = (2, 1, 1, −1), X4 = (−3, 3, −9, 6), and X5 = (9, 3, 7, −6). Find a subset of S that is a
basis for W = span S. Find dim(W). Final answer: {X1 , X2 } is a basis for W and the
dimension is 2.
3. Find the dimension of the subspace of all vectors of the form (a, b, c, d) where c = a − b
and d = a + b (for a, b ∈ R). Final answer: the dimension is 2.
4. Find the dimension of the subspace of all vectors of the form (a + c, a + b + 2c, a + c, a − b)
where a, b, c ∈ R. Final answer: the dimension is 2.
5. Let S = {X1 , X2 , X3 } be a basis for a vector space V. Show that T = {Y1 , Y2 , Y3 } is also
a basis for V, where Y1 = X1 + X2 + X3 , Y2 = X2 + X3 , and Y3 = X3 .
6. Find a standard basis vector for R3 that can be added to the set
{X1 = (1, 1, 1), X2 = (2, −1, 3)} to produce a basis a basis for R3 . Final answer:
any vector of the standard basis will work.
7. The set S = {X1 = (1, 2, 3), X2 = (0, 1, 1)} is linearly independent in R3 . Extend (enlarge)
S to a basis for R3 . Final answer: S = {X1 = (1, 2, 3), X2 = (0, 1, 1), X3 = (1, 0, 0)}
8. Let S = {X1 = (1, 0, 2), X2 = (−1, 0, −1)} be a set of vectors in R3 . Find a basis for R3
that contains the set S. Final answer: {(1, 0, 2), (−1, 0, −1), (0, 1, 0)}.
9. Let S = {X1 , X2 , · · · , Xn } be a set of vectors in a vector space V . Show that S is a
basis for V if and only if every vector in V can be expressed in exactly one way as a linar
combination of the vectors in S. ” ⇒ ” : Use Theorem 4.4.1. And for ” ⇐ ” : Show the
linear independence of S using the uniqueness.
4.5. Row Space, Column Space, and Null Space 101

4.5 Row Space, Column Space, and Null Space

Definition 4.5.1
 
 a11 a12 ··· a1n 
 
a
 21 a22 ··· a2n 
Let A =  . .. .. ..  ∈ Mm×n .

 .. . . . 
 
 
am1 am2 · · · amn
The set of rows of A are:

X1 = [a11 a12 ··· a1n ] 


 
= [a21 a2n ] 

X2 a22 ···

.. .. ∈ Rn
. . 




= [am1 amn ]

Xm am2 ···



These row vectors span a subspace of Rn which is called the row space of A. Moreover, the
row rank of A = dim(row space of A).
Similarly, the columns of A are:
     
 a11   a12   a1n  



     
 a21   a22   a2n  
Y1 =  . ,
 .  Y2 =  . ,
 .  · · · , Yn =  . 
 .  ∈ Rm .
 .   .   . 
     

am1 am2 amn 


These column vectors span a subspace of Rm which is called the column space of A. Moreover,
the column rank of A = dim(column space of A).
Moreover, the solution space of the homogeneous system AX = O (which is a subspace of Rn )
is called the null space of A.

Remark 4.5.1

Let A be any m × n matrix, then

1. the row rank of A = the column rank of A = the rank of A = the rank of AT .
2. n = the nullity of A + the rank of A.
3. m = the nullity of AT + the rank of A.
102 Chapter 4. General Vector Spaces

Theorem 4.5.1

If A and B are two m × n row equivalent matrices, then they have the same row space.

In Example 4.5.1, we illustrate how to find bases for the row and column spaces of a given matrix.

Example 4.5.1
 
 1 −2 0 3 −4
 3 2 8 1 4
 
Let A =  .
 2 3 7 2 3
 

2 0 4
 
−1 −3
1. find a basis for the row space of A,
2. find a basis for the column space of A,
3. find a basis for the row space that contains only rows of A,
4. find a basis for the column space that contains only columns of A.

Solution:
1. To find a basis for the row space of A, we have to find the r.r.e.f. of A, then the set of
non-zero rows of the r.r.e.f. forms a basis for the row space.
 
1 −2 0 3 −4  
1 0 2 0 1 ←

3 2 8 1 4
 
 ≈ · r.r.e.f.

····· ≈ 0 1 1 0 1  ←
  
2 3 7 2 3
 
0 0 0 1 −1 ←
  

2 0 4
 
−1 −3

0 0 0 0 0
Therefore, the set {(1, 0, 2, 0, 1), (0, 1, 1, 0, 1), (0, 0, 0, 1, −1)} forms a basis for the row space
of A. Note that the row rank of A = 3. (That is, nullity of A = 5 - 3 = 2).

2. To find a basis for the column space of A, we have to find a basis for the row space of AT .
Therefore,
 
 1 3 2 −1  
1 0 0 11
−2

2 3 2 24 ←
0 1 0

−49
 
r.r.e.f. ←
A =
T
 0 8 7 0
 
 ≈ ······ ≈  24 
0 0 1 7
←
 
 3 1 2 4
 
3
0 0 0 0
   

−4 4 3 −3  
0 0 0 0
4.5. Row Space, Column Space, and Null Space 103

n o
Therefore, the set (1, 0, 0, 11
24
), (0, 1, 0, −49
24
), (0, 0, 1, 73 ) is a basis for the row space of AT
and it is a basis for the column space of A. The column rank of A = 3. (That is, nullity
of A = 5 - 3 = 2).

3. To find a basis for the row space of A that contains only rows of A, we do as follows:

 
 1 3 2 −1  

−2

2 3 2 1 0 0 11 24 
0 1 0 24 
−49
 
r.r.e.f.
A =
T
 0 8 7 0
 
 ≈ ······ ≈
0 0 1
 
7
 3 1 2 4 3
  
0 0 0 0
   
−4 4 3 −3 
0 0 0 0


↑ ↑ ↑
Then, the leading entries are pointing to column 1, column 2, and column 3 in the r.r.e.f.
of AT which correspond to row 1, row 2, and row 3 in A. Thus,

{(1, −2, 0, 3, −4), (3, 2, 8, 1, 4), (2, 3, 7, 2, 3)}

forms a basis for the row space of A containing only rows of A.

4. To find a basis for the column space of A that only contains columns of A, we do the
following:
 
1 −2 0 3 −4
 
1 0 2 0 1

3 2 8 1 4
 
 ≈ · r.r.e.f.
0 1 1 0 1

····· ≈
  
2 3 7 2 3
   
0 0 0 1 −1
 
2 0 4
  
−1 −3 
0 0 0 0 0


↑ ↑ ↑
Then, the leading entries are pointing to column 1, column 2 , and column 3 in the r.r.e.f.
of A . Thus,
{(1, 3, 2, −1), (−2, 2, 3, 3), (3, 1, 2, 4)}

forms a basis for the column space of A containing only columns of A.

104 Chapter 4. General Vector Spaces

? Rank and Singularity:

Theorem 4.5.2: Equivalent Statements

If A is an n × n matrix, then the following statements are equivalent:

1. A is invertible.
2. AX = O has only the trivial solution.
3. A is row equivalent to In .
4. AX = B has a unique solution for every n × 1 matrix B.
5. det (A) 6= 0.
6. The column vectors of A are: linearly independent; span Rn ; and form a basis for Rn .
7. The row vectors of A are: linearly independent; span Rn ; and form a basis for Rn .
8. A has rank n.
9. A has nullity 0.

If A is an m × n matrix, then the smallest possible rank of A is 0 (when A is the zero matrix),
while the largest possible rank of A:

1. n (if m ≥ n): When every column of the r.r.e.f. of A contains a leading 1.

2. m (if m < n): When every column of the r.r.e.f. of A contains a leading 1.

Also: the largest nullity of A is n (when rank is 0) and the smallest nullity of A is:

1. 0 (if m ≥ n): When every column of the r.r.e.f. of A contains a leading 1.

2. n − m (if m < n): When every column of the r.r.e.f. of A contains a leading 1.

Let A be a 3 × 5 matrix. Then: the largest possible rank of A is 3 and the smallest possible
rank of A is 0 (the zero matrix). This is because, rank of A = row rank = column rank, and we
only have 3 rows. Also, the largest nullity of A is 5 (zero matrix) and the smallest nullity is 2
(when rank of A = 3). Moreover, the largest possible rank of AT is 3, and the largest possible
nullity of AT is 3.
4.5. Row Space, Column Space, and Null Space 105

Example 4.5.2

If A is a 5 × 3 matrix, show that A has linearly dependent rows.

Solution:

The largest possible rank of A is 3 and thus A must has at least two linearly dependent rows.

Example 4.5.3
 
a a12 a13  a a12 a a13 a a13
Show that A =  11 has rank 2 iff at least one of 11 , 11 , 12
a21 a22 a23 a21 a22 a21 a23 a22 a23
is nonzero.

Solution:

We have: rank of A is 2 iff column rank of A is 2 iff basis of column space has two columns iff
two columns are linearly independent iff one of the determinants is nonzero.

Example 4.5.4
 
1 1 4 1 2
0 1 2 1 1


Let A = 
0 0 0 1 2. Find the rank of A and the nullity of A.
 

1 0 0 2
 
 −1 
2 1 6 0 1

Solution:

To find the rank and the nullity of A, we find any basis of any kind of A. So,
 
1 1 4 1 2  

0

1 2 1 1 1 0 2 0 1 ←
0 1 2 0 −1 

←
 
r.r.e.f.
0 0 0 1 2
 
 ≈ ······ ≈ 
0 0 0 1 2 ←
 
1 0 0 2
 
−1 0 0 0 0 0 
   
2 1 6 0 1 
0 0 0 0 0


Therefore {(1, 0, 2, 0, 1), (0, 1, 2, 0, −1), (0, 0, 0, 1, 2)} is a basis for the row space of A and rank(A)
= 3 which implies that nullity of A = 5 − 3 = 2.
106 Chapter 4. General Vector Spaces

Example 4.5.5
 
1 −2 0 3 −4
3 2 8 1 4
 
Let A = .


2 3 7 2 3
 

2 0 4
 
−1 −3
1. Find bases for the row and column spaces of A,
2. Find a basis for the null space of A. Find nullity of A and nullity of AT .
3. Does X = (1, 2, 4, 3, 0) belong to the row space of A? Explain.
4. Express each column of A not in the basis of column space as a linear combination of the
vectors in the basis you got in step 1.

Solution:
1. To get bases for the row space and column spaces of A, we do the following:
 
1 −2 0 3 −4
 
1 0 2 0 1 ←

3 2 8 1 4
 
 ≈ · r.r.e.f. 0 1 1 0 1 

····· ≈ ←
  
2 3 7 2 3
  
0 0 0 1 −1  ←
 
2 0 4
 
−1 −3 
0 0 0 0 0


↑ ↑ ↑
Thus, the set {(1, 0, 2, 0, 1), (0, 1, 1, 0, 1), (0, 0, 0, 1, −1)} forms a basis for the row space of
A, while the set {(1, 3, 2, −1), (−2, 2, 3, 2), (3, 1, 2, 4)} forms a basis for the column space
of A that only contains columns of A, but this is fine since there is no restrictions on the
basis of column space of A mentioned in the question.
2. Using what we got in the previous step, the solution space of the homogeneous system is:

x1 + 2x3 + x5 = 0
x2 + x3 + x5 = 0
x4 − x5 = 0

Let x5 = t, x3 = r, where t, r ∈ R to get

     
−2r − t −2 −1
 −r − t  −1 −1
     

X= =r
 1  + t  0 .
     
 r     
 0   1 
     

 t      
t 0 1
4.5. Row Space, Column Space, and Null Space 107

Therefore, {(−2, −1, 1, 0, 0), (−1, −1, 0, 1, 1)} is a basis for the null space of A. The nullity
of A is 2 while the rank(A) = 3. Also, nullity of AT = 4 (number of rows in A) − 3 = 1.
3. Yes. It is clear that X = (1)(1, 0, 2, 0, 1) + (2)(0, 1, 1, 0, 1) + (3)(0, 0, 0, 1, −1) where those
vectors are the vectors of the basis of the row space that were found in (1). It is also
possible to consider the non-homogenous system X = c1 (1, 0, 2, 0, 1) + c2 (0, 1, 1, 0, 1) +
c3 (0, 0, 0, 1, −1) to find the same answer.
4. Let the columns of A called X1 , · · · , X5 . Then, we will express X3 and X5 (not in the
basis) as a linear combination of the vectors (those in the basis) {X1 , X2 , X4 }. We can do
so by looking at the r.r.e.f. form we got in step 1. For X3 : The third column of the rref
matrix suggest that X3 = 2 X1 + X2 + 0 X4 . For X5 : The fifth column of the rref matrix
suggest that X5 = X1 + X2 − X4 . Can you confirm that!?
108 Chapter 4. General Vector Spaces

Exercise 4.5.1
 
−1 −1 0
1. Let A =  . Find a basis for the null space of A and determine the nullity of
2 0 4
A. Final answer: S = {X1 = (1, 2, 3), X2 = (0, 1, 1), X3 = (1, 0, 0)}
0 0 0 −1
2. Let A = 0

1 0 0

.
 
0 0 0 1
(a) Find rank(A), nullity(A), rank(AT ), and nullity(AT ).
(b) Find a basis for the null space of A.
(c) Find a basis for the row space of AT .
(d) Find a basis for the row space of A.

Final answer:

(a) rank(A) = 2, nullity(A) = 2, rank(AT ) = 2, and nullity(AT ) = 1.

(b) a basis for the null space of A = {(1, 0, 0, 0), (0, 0, 1, 0)}.
(c) a basis for the row space of AT = {(0, 1, 0), (−1, 0, 1)}.
(d) a basis for the row space of A = {(0, 1, 0, 0), (0, 0, 0, 1)}.
 
1 0 −1 1
3. Let A = 
1 1 1 1

.
 
1 2 3 1
(a) Find a basis for the null space of A.
(b) Find a basis for the row space of AT .
(c) Find a basis for the column space of A.

Final answer:

(a) rank(A) = 2, nullity(A) = 2, rank(AT ) = 2, and nullity(AT ) = 1.

(b) a basis for the null space of A = {(1, 0, 0, 0), (0, 0, 1, 0)}.
(c) a basis for the row space of AT = {(0, 1, 0), (−1, 0, 1)}.
(d) a basis for the row space of A = {(0, 1, 0, 0), (0, 0, 0, 1)}.

4. Let S = {X1 , X2 , X3 , X4 , X5 }, where X1 = (1, −2, 0, 3), X2 = (2, −5, −3, 6), X3 =
(0, 1, 3, 0), X4 = (2, −1, 4, −7), and X5 = (5, −8, 1, 2).

(a) Find a subset of S that forms a basis for the subspace span S.
4.5. Row Space, Column Space, and Null Space 109

(b) Express each vector not in the basis as a linear combination of the basis vectors.
(c) If A is the 4 × 5 matrix whose columns are the vectors of S in order, then find a basis
for the row space of A, a basis for the column space of A, and a basis for the null
space of A. Further, what is the nullity of A and the nullity of AT .
110 Chapter 4. General Vector Spaces
 Chapter

5 Eigenvalues and Eigenvectors

5.1 Eigenvalues and Eigenvectors

Definition 5.1.1

Let A ∈ Mn×n . The real number λ is called an eigenvalue of A if there exists a nonzero vector
X ∈ Rn so that
AX = λX X 6= 0.

In this case, X is called an eigenvector corresponding to λ. This is called the eigenproblem.

   
1 −1 1
For example, let A =  and X =  . Then,
−1 1 −1

       
1 −1  1   2  1
AX =  = = 2   = 2X.
−1 1 −1 −2 −1

Therefore, λ = 2 is an eigenvalue of A corresponding to eigenvector X.

Definition 5.1.2

If A is an n × n matrix, then pA (λ) = |λIn − A| is called the characteristic polynomial of A.

Theorem 5.1.1

If A is an n × n matrix, then λ is an eigenvalue of A iff pA (λ) = | (λIn − A) | = 0.

Proof:

λ is an eigenvalue of A iff λ satisfies AX = λX, with X 6= O iff λ satisfies λX − AX = O, with

X 6= O iff (λIn − A)X = O has a nontrivial solutions iff | λ In − A | = 0.

111
112 Chapter 5. Eigenvalues and Eigenvectors

Example 5.1.1
 
1 0 2
Let A = 1 0 0. Find the eigenvalues of A.
 
 
0 0 −1

Solution:

We first compute the characteristic polynomial pA (λ) = |λI3 − A| = 0 as follows:

λ−1 0 −2
λ−1 −2
−1 λ 0 =λ = λ(λ − 1)(λ + 1) = 0.
0 λ+1
0 0 λ+1

which implies that

λ1 = −1 , λ2 = 0 , λ3 = 1 .

Theorem 5.1.2

Let A be an n × n matrix. Then

1. The A is the product of the eigenvalues of A.

2. A is invertible if and only if λ = 0 is not an eigenvalue of A.

Proof:

1. Assume that λ1 , λ2 , · · · , λn are the eigenvalues of A. Then,

pA (λ) = |λIn − A| = (λ − λ1 )(λ − λ2 ) · · · (λ − λn ),

setting λ = 0, pA (0) = |−A| = (−λ1 )(−λ2 ) · · · (−λn ),
(−1) A
n
= (−1)n λ1 λ2 · · · λn ,
A = λ1 λ2 · · · λn .

2. A is invertible iff A 6= 0 iff A = λ1 λ2 · · · λn 6= 0 iff λi 6= 0 for all 1 ≤ i ≤ n.

5.1. Eigenvalues and Eigenvectors 113

Theorem 5.1.3

If A is an n × n triangular matrix (upper triangular, lower triangular, or diagonal), then the

eigenvalues of A are the entries on the main diagonal.

Proof:

If A = [aij ] (where 1 ≤ i, j ≤ n) is a triangular matrix, then λIn − A is also a triangular matrix,

and its main diagonal entries are [λi − aii ] for 1 ≤ i ≤ n. Recall that the determinant of a
triangular matrix is the product of its main diagonal entries. Thus,

pA (λ) = |λIn − A| = 0 ⇒ (λ − a11 )(λ − a22 ) · · · (λ − ann ) = 0.

Therefore, λ1 = a11 , λ2 = a22 , · · · , λn = ann .

Theorem 5.1.4

If A is an n × n matrix, then: λ is an eigenvalue of A iff The system (λIn − A)X = O has

nontrivial solutions iff There is a nonzero vector X such that AX = λX iff λ is a solution of
pA (λ) = | λIn − A | = 0

Definition 5.1.3

Let A be an n × n matrix with an eigenvalue λ. The eigenspace of A corresponding to λ,

denoted Eλ , is defined as the solution space of the homogeneous system (λIn − A)X = O. That
is, Eλ is the null space of the matrix λIn − A.

Example 5.1.2
 
1 0 2
Let A = 1 0 0 . Find bases for the eigenspaces of A. OR: Find the eigenvalues of A and
 
 
0 0 −1
the corresponding eigenvectors.

Solution:
114 Chapter 5. Eigenvalues and Eigenvectors

As in Example 5.1.1, the characteristic polynomial pA (λ) = |λI3 − A| = 0 as follows:

λ−1 0 −2
λ−1 −2
−1 λ 0 =λ = λ(λ − 1)(λ + 1) = 0.
0 λ+1
0 0 λ+1

which implies that

λ1 = −1 , λ2 = 0 , λ3 = 1 .

Thus, there are three eigenspaces of A corresponding to these eigenvalues. To find bases for
these eigenspaces, we solve the homogeneous system (λI3 − A)X = O, for λ1 , λ2 , λ3 . That is:
 
λi − 1 0 −2 0
 −1

λi 0 0

. (5.1.1)
 
0 0 λi + 1 0

1. λ1 = −1 ⇒ (λ1 I3 − A)X1 = 0, X1 = (a, b, c) 6= (0, 0, 0). Substitute λ1 = −1 in Equation

5.1.1 to get:    
−2 0 −2 0 1 0 1 0
−1 −1 0 0 → · · · → 0 −1 1 0 .
   
   
0 0 0 0 0 0 0 0
Thus, a +c = 0, and − b + c = 0. That is, a = −c, and b = c. Let c = t ∈ R\{0} to get
−t
X1 =  t .
 
Choosing t = 1, we get a basis for Eλ1 containing the vector
 
t
 
−1
P1 = 
 1 .

 
1

2. λ2 = 0 ⇒ (λ2 I3 − A)X2 = 0, X2 = (a, b, c) 6= (0, 0, 0). Substitute λ2 = 0 in Equation

5.1.1 to get:    
−1 0 −2 0 1 0 0 0
−1 0 0 0 → · · · → 0 0 1 0 .
   
   
0 0 1 0 0 0 0 0
 
0
Thus, a = c = 0. Let b = t ∈ R\{0} to get X2 =  t . Choosing t = 1, we get a basis for
 
 
0
5.1. Eigenvalues and Eigenvectors 115

Eλ2 containing the vector  

0
P2 = 1 .
 
 
0

3. λ3 = 1 ⇒ (λ3 I3 − A)X3 = 0, X3 = (a, b, c) 6= (0, 0, 0). Substitute λ3 = 1 in Equation

5.1.1 to get:    
 0 0 −2 0 1 −1 0 0
−1 1 0 0 → · · · → 0 0 1 0 .
   
   
0 0 2 0 0 0 0 0
 
t
Thus, a − b = 0, and c = 0. If b = t ∈ R\{0}, then a = t as well and we get X3 =  t .
 
 
0
Choosing t = 1, we get a basis for Eλ3 containing the vector
 
1
P3 = 1 .
 
 
0

Theorem 5.1.5

If k is a positive integer, λ is an eigenvalue of a matrix A, and X is a corresponding eigenvector,

then λk is an eigenvalue of Ak and X is a corresponding eigenvector.

Proof:

If AX = λX, then we have A2 X = A(AX) = A(λX) = λ(AX) = λ2 X. Applying this simple

idea k times, we get
   
Ak X = Ak−1 (AX) = λ Ak−1 X = λ2 Ak−2 X = · · · = λk X.

Theorem 5.1.6

If λ is an eigenvalue of an invertible matrix A, and X is a corresponding eigenvector, then 1

λ
is
an eigenvalue of A−1 and X is a corresponding eigenvector.
116 Chapter 5. Eigenvalues and Eigenvectors

Proof:

If AX = λX and A is invertible, then multiplying with A−1 both sides (from left), we get

1
A−1 · AX = A−1 · λX → X = λ A−1 X → X = A−1 X.
λ
5.1. Eigenvalues and Eigenvectors 117

Exercise 5.1.1

1. Show that A and AT have the same eigenvalues. Hint: |λIn − A| = (λIn − A)T .
2. Suppose that pA (x) = λ2 (λ + 3)3 (λ − 4) is the characteristic polynomial of some matrix
A. Then,

(a) What is the size of A? Explain.

(b) Is A invertible? Why?
(c) How many eigenspaces does A have? Explain.
 
 1 3 3
3. Find the eigenvalues and bases for the eigenspaces of A =  1 −1 −4 Final result:
 
 
−1 −1 2
     
−1  2   0 
λ1 = −2, λ2 = 1, λ3 = 3. And P1 =  1 , P2 = −1, and P3 = −1.
     
     
0 1 1
 
0 1 0
4. Find the eigenvalues of A = 
0 0 1

Final result: λ = k since p(λ) = (λ − k)3 .
 
k3 −3k 2 3k
 
a b
5. Show that if a, b, c, d are integers such that a + b = c + d, then A =  has integer
c d
eigenvalues λ1 = a + b and λ2 = a − c. Hint: Use your algebraic abilities.
118 Chapter 5. Eigenvalues and Eigenvectors

5.2 Diagonalization

Definition 5.2.1

A matrix B is said to be similar to matrix A, denoted by B ≡ A, if there exists a non-singular

matrix P such that B = P −1 AP.

? Properties of Similar Matrices:

1. A ≡ A since A = I −1 AI.
2. if B ≡ A, then A ≡ B.

Proof. If B ≡ A, then ∃ P, P −1 such that B = P −1 AP or P BP −1 = A. Let Q = P −1 to get

A = Q−1 BQ. Thus, A ≡ B.

3. if A ≡ B and B ≡ C, then A ≡ C.

Proof.

A≡B ⇒ ∃ P, P −1 such that A = P −1 BP,

B≡C ⇒ ∃ Q, Q−1 such that B = Q−1 CQ.

Therefore,
A = P −1 BP = P −1 Q−1 C QP = (QP )−1 C (QP ) ⇒ A ≡ C.

4. if A ≡ B, then A = B .

Proof. If A ≡ B, then there exists P, P −1 such that B = P −1 A P with |P | =

6 0. Therefore,

1
B = P −1 A P = |P −1 | A |P | =  A |P| = A .
|P |

5. if A ≡ B, then AT ≡ B T .
5.2. Diagonalization 119

Proof. If A ≡ B, then there exist P, P −1 such that B = P −1 A P . Thus,

B = P −1 A P,
BT = (P −1 A P )T ,
BT = P T AT (P −1 )T ,
BT = P T AT (P T )−1 .

Let Q−1 = P T , to get B T = Q−1 AT Q. Therefore, B T ≡ AT .

Theorem 5.2.1

Similar matrices have the same eigenvalues.

Proof:

Let A and B be two similar n × n matrices. Then, there is an invertible matrix P such that
B = P −1 A P . Then,

pB (λ) = |λIn − B| = λIn − P −1 A P = P −1 (λP P −1 − A)P

= −1
|P
|P| = |λIn − A| = pA (λ).
| |λIn − A| 

The characteristic polynomials of A and B are the same. Hence they have the same eigenvalues.

Definition 5.2.2

An n × n matrix A is diagonalizable if and only if A is similar to a diagonal matrix D, i.e.

D = P −1 A P with |P | =
6 0.

D: its diagonal entries are the eigenvalues of A. That is: D = diag(λ1 , λ2 , · · · , λn ).

P: its columns are the linearly independent eigenvectors of A. That is P = [P1 | P2 | · · · | Pn ].

Theorem 5.2.2

A matrix A has linearly independent eigenvectors if all of its eigenvalues are real and distinct.
120 Chapter 5. Eigenvalues and Eigenvectors

Theorem 5.2.3

An n × n matrix A is diagonalizable if and only if it has n linearly independent eigenvectors.

Definition 5.2.3

If A is an n × n matrix with eigenvalue λ0 , then the number of times that λ − λ0 appears as a

factor in the characteristic polynomial of A is called the algebraic multiplicity of λ0 .

Theorem 5.2.4

Let A be a square matrix. Then A is diagonalizable iff every eigenspace of A corresponding to

eigenvalue λi has its dimension equals to the algebraic multiplicity of λi .

Example 5.2.1
 
1 0 2
Let A = 1 0 0. If possible, find matrices P and D so that A is diagonalizable.
 
 
0 0 −1

Solution:

Recall that in Example 5.1.1, we found λ1 = −1, λ2 = 0, and λ3 = 1 with bases

     
−1 0 1
P1 =  1  , P2 = 1 , P3 = 1 .
     
     
1 0 0

Since, we have real and distinct eigenvalues, the eigenvectors P1 , P2 , and P3 are linearly inde-
pendent. Thus, A is diagonalizable and
   
−1 0 0 −1 0 1
D=
 0 0 0

and P = 
 1 1 1

.
   
0 0 1 1 0 0
5.2. Diagonalization 121

Example 5.2.2

Find a matrix P that diagonalizes  

0 0 1
A = 0 1 2 .
 
 
0 0 1

Solution:

We first compute the characteristic polynomial pA (λ) = |λI3 − A| = 0 as follows:

λ 0 −1
0 λ−1 −2 = λ(λ − 1)(λ − 1) = 0
0 0 λ−1

Thus, λ1 = 1, λ2 = 1, and λ3 = 0. To find the corresponding bases for eigenspaces of A, we

solve the homogeneous system
 
λi 0 −1 0
0 λi − 1 −2 0 . (5.2.1)
 
 
0 0 λi − 1 0

1. λ1 = λ2 = 1 ⇒ (λ1 I3 − A)X1 = 0, X1 = (a, b, c) 6= (0, 0, 0). Substitute λ1 = λ2 = 1 in

Equation (5.2.1) to get:  
1 0 −1 0
0 0 −2 0 .
 
 
0 0 0 0
Thus, we get a −
 c = 0 and −2c = 0 which implies that a = c = 0. If b = t ∈ R\{0},
  then
0 0
we get X1 =  t . We choose t = 1 to get a basis for Eλ1 with one vector P1 = 1.
   
   
0 0

We see here that the dimension of Eλ1 is 1 while the multiplicity of λ1 is 2. Therefore, A is not
diagonalizable.
122 Chapter 5. Eigenvalues and Eigenvectors

Example 5.2.3

Find, if possible, matrices D and P so that D = P −1 A P , where

 
1  0 0
A= 0 0 0 .
 

 
−1 0 0

Solution:

We first compute the characteristic polynomial pA (λ) = |λI3 − A| = 0 as follows:

λ−1 0 0
0 = λ (λ − 1) = 0.
2
0 λ
1 0 λ

Thus, λ1 = 0, λ2 = 0, and λ3 = 1. To find the corresponding bases for eigenspaces of A, we

solve the homogeneous system  
λi − 1 0 0 0
 0

λi 0 0

. (5.2.2)
 
1 0 λi 0

1. λ1 = λ2 = 0 ⇒ (λ1 I3 − A)X1 = 0, X1 = (a, b, c) 6= (0, 0, 0). Substitute λ1 = λ2 = 0 in

Equation 5.2.2 to get:  
−1 0 0 0
 0 0 0 0 .
 
 
1 0 0 0
Thus, we get a = 0. If b = t and c = r (not both zeros) be two real numbers, we get
         
0 0 0 t=1 0 t=0 0
( ) ( )
X1 = t =t 1 + r 0 if ⇒ P1 = 1 , if ⇒ P2 = 0 .
         
      r=0   r=1  
r 0 1 0 1

Here, the dimension of Eλ1 is 2 which equals to the algebraic multiplicity of λ = 0. So, we
continue with the other eigenvalues.
2. λ3 = 1 ⇒ (λ3 I3 − A)X3 = 0, X3 = (a, b, c) 6= (0, 0, 0). Substitute λ3 = 1 in Equation
5.2.2 to get:  
0 0 0 0
0 1 0 0 .
 
 
1 0 1 0
5.2. Diagonalization 123

 
 t 
Thus, we get b = 0 and a + c = 0. If c = t ∈ R\{0}, we get X3 =  0 . We choose t = 1
 
 
−t
 
 1 
to get a basis with one vector P3 =  0 .
 
 
−1

There are three basis vectors in total, so the matrix P = [P1 | P2 | P3 ] diagonalize A and we get
D = P −1 AP = diag(1, 0, 0).
   
1 0 0  1 0 0
D = 0 0 0 and P =  0 1 0 .
   

  
0 0 0 −1 0 1
124 Chapter 5. Eigenvalues and Eigenvectors

Exercise 5.2.1

1. Show that similar matrices have the same trace. Hint: Recall that tr(AB) = tr(BA).
2. Show that A and B in each of the following are not similar matrices:
   
2 1 2 0
(a) A =  and B =  .
3 4 3 3
   
4 2 0 0 3 4
(b) A = 2 1 0 and B = 0 7 2 .
   
   
1 1 7 0 0 4
Hint: Similar
 matrices shares
 some properties like determinants and traces.
 0 −3 −3
3. Let A =  1 4 1 .
 
 
−1 −1 2
(a) Find the eigenvalues of A.
(b) For each eigenvalue λ, find the rank of the matrix λ I3 − A.
(c) Is A diagonalizable? Why?

Hint: For part 3, use what you got in part 2 and recall that for n × n matrix, we have
n = rank − nullity.
4. Show that if A is diagonalizable, then

(a) AT is diagonalizable.
(b) Ak is diagonalizable, for any positive integer k.

Hint: A is
 diagonalizable
 implies that A = P D P .
−1

1 −2 8
5. Let A = 0 −1 0 .
 
 
0 0 −1
(a) Find A10000 .
(b) Find A20021 .
(c) Find A−20021 .

Hint: Write A in the form A = P D P −1 .

6. Show that if A and B are invertible matrices, then AB and BA are similar. Hint: They
are similar if AB = (?)−1 (BA) (?).
5.2. Diagonalization 125

7. Prove: If A and B are n × n invertible matrices, then A B −1 and B −1 A have the same
eigenvalues.
 Hint: Show
 that they have the same characteristic polynomial.
1 2 0
8. Let A = 0 2 0, where a ∈ R.

 
1 a 2
(a) Find all eigenvalues of A.
(b) For a = −2, determine whether A is diagonalizable.
(c) For a 6= −2, find all eigenvectors of A.

Final answer: Eigenvalues: 1 and 2. If a = −2, A is diagonalizable. Otherwise, A is not

diagonalizable.
9. (a) Show that a square matrix A is singular iff it has an eigenvalue
 0. 
2017 −2017 2020
(b) Use part 1 to show that 0 is an eigenvalue of the matrix A = 2018 −2018 2021.
 
 
2019 −2019 2022
126 Chapter 5. Eigenvalues and Eigenvectors

3.5 The Cross Product in Rn

Recall that if 0 ≤ θ ≤ π is an angle between two vectors X and Y in Rn , then

X ·Y
−1 ≤ cos θ = ≤1 or X · Y = k X k k Y k cos θ (3.5.1)
kX kkY k

Definition 3.5.1

Two nonzero vectors X and Y in Rn are said to be orthogonal (or perpendicular) if X ·Y = 0.

Remark 3.5.1

Let X, Y ∈ Rn and 0 ≤ θ ≤ π, then

π
1. X ⊥ Y (orthogonal) ⇔ θ = ⇔ cos θ = 0 ⇔ X · Y = 0.
2
2. X // Y (parallel (same direction)) ⇔ θ = 0 ⇔ cos θ = 1 ⇔ X · Y = kXk kY k ⇔ Y = c X
with c > 0.

3. X // Y (parallel (opposite direction)) ⇔ θ = π ⇔ cos θ = −1 ⇔ X · Y = −kXk kY k ⇔

Y = c X with c < 0.

Definition 3.5.2

Let X = x1 i + x2 j + x3 k and Y = y1 i + y2 j + y3 k be two vectors in R3 , then

the cross product of X and Y , denoted by X × Y , is defined by X ×Y
Y
i j k
x2 x3 x x3 x x2
X × Y = x1 x2 x3 = i− 1 j+ 1 k
y2 y3 y1 y3 y1 y2
y1 y2 y3
X
That is
X × Y = (x2 y3 − x3 y2 , x1 y3 − x3 y1 , x1 y2 − x2 y1 ).
3.5. The Cross Product in Rn 127

Example 3.5.1

Find X × Y where X = 2i + j + 2k and Y = 3i − j − k.

Solution:

i j k
1 2 2 2 2 1
X ×Y = 2 1 2 = i− j+ k = i + 8j − 5k = (1, 8, −5).
−1 −1 3 −1 3 −1
3 −1 −1

Remark 3.5.2

It can be shown (Try it your self) that

i
− −
i×i=0 j×j=0 k×k=0
+ +
i×j=k j×k=i k × j = −i +
k j
i × k = −j j × i = −k k×i=j
−

Theorem 3.5.1: Properties of Cross Product

Let X, Y, Z ∈ R3 and c ∈ R. Then, (Can you prove the following propertes?)

1. X × Y = −(Y × X),
2. X × (Y + Z) = X × Y + X × Z,
3. (X + Y ) × Z = X × Z + Y × Z,
4. c X × Y = X × c Y = c (X × Y ),
5. X × X = O,
6. X × O = O × X = O,
7. X · (X × Y ) = Y · (X × Y ) = 0, (X × Y is orthogonal to X and Y)
8. k X × Y k2 = k X k2 k Y k2 − (X · Y )2 , (Lagrange’s identity)
9. X × (Y × Z) = (X · Z)Y − (X · Y )Z, (triple vector product)
10. (X × Y ) × Z = (X · Z)Y − (Y · Z)X. (triple vector product)
128 Chapter 5. Eigenvalues and Eigenvectors

Example 3.5.2

Let X = x1 i + x2 j + x3 k; Y = y1 i + y2 j + y3 k; Z = z1 i + z2 j + z3 k ∈ R3 . Show that

x1 x2 x3
(X × Y ) · Z = X · (Y × Z) = y1 y2 y3 .
z1 z2 z3

Solution:

i j k
L.H.S. = (X × Y ) · Z = x1 x2 x3 · (z1 i + z2 j + z3 k)
y1 y2 y3
 

=  x2 x3 x
i− 1
x3 x
j+ 1
x2 
k · (z1 i + z2 j + z3 k)
y2 y3 y1 y3 y1 y2

x2 x3 x x3 x x2
= z1 − 1 z2 + 1 z3 = R.H.S.
y2 y3 y1 y3 y1 y2

The proof of X · (Y × Z) is similar.

Example 3.5.3

Let X, Y, Z be in R3 such that (X × Y ) · Z = 6. Find a) X · (Y × Z), b) 2X · (Y × Z), c)

X · (Z × Y ) and d) X × (Y × 4X).

Solution:

1. X · (Y × Z) = 6,
2. 2X · (Y × Z) = 12,
3. X · (Z × Y ) = −6,
4. X · (Y × 4X) = 0.
3.5. The Cross Product in Rn 129

Example 3.5.4

Find a vector of length 12 so that it is perpendicular to both

X = 2i − j − 2k and Y = 2i + j.

Solution:

The vector X × Y is always orthogonal to both X and Y . So, we compute that vector and make
its length equals to 12.

i j k
√
X ×Y = 2 −1 −2 = 2i − 4j + 4k and kX × Y k = 4 + 16 + 16 = 6.
2 1 0
 
Therefore, 16 (X × Y ) is a unit vector and orthogonal to both X and Y ; while 12 1
6
(X ×Y) =
2(X × Y ) is a vector of length 12 and orthogonal to both X and Y .

Theorem 3.5.2

Let X, Y ∈ R3 have an angle θ between them. Then

k X × Y k = k X k k Y k sin θ.

Let X, Y, Z ∈ R3 , then

1. X ⊥ Y ⇐⇒ θ = π
2
⇐⇒ sin θ = 1 ⇐⇒ kX × Y k = kXkkY k,

2. X//Y ⇐⇒ θ = 0 or π ⇐⇒ kX × Y k = 0 ⇐⇒ X × Y = 0,

3. Area of triangle:

1
A∆ = ah
2 kXk
X h
h
sin θ = θ
kxk Y
=⇒ h = kxk sin θ and if kY k = a,
kY k = a
1 1
A∆ = kY kkXk sin θ = kX × Y k.
2 2
130 Chapter 5. Eigenvalues and Eigenvectors

4. Area of parallel gram (two triangles):

A = kX × Y k.

5. Volume of parallel piped:

V olume = |X · (Y × Z)| .
Y
Z

Example 3.5.5

Find the area of the triangle with vertices: P1 (2, 2, 4), P2 (−1, 0, 5), and P3 (3, 4, 3).

Solution:
−−→ −−→
Let X = P1 P2 = P2 − P1 = (−3, −2, 1) and Y = P1 P3 = P3 − P1 = (1, 2, −1). Then,

i j k
X ×Y = −3 −2 1
1 2 −1

−2 1 −3 1 −3 −2
= i− j+ k = 0i − 2j − 4k.
2 −1 1 −1 1 2
√ √ √ √
Therefore, kX × Y k = 4 + 16 = 20 = 2 5 and A∆ = 12 kX × Y k = 5.

Example 3.5.6

Find the volume of the parallel piped with a vertex at the origin and edges X = i − 2j + 3k,
Y = i + 3j + k, and Z = 2i + j + 2k.
3.5. The Cross Product in Rn 131

Solution:

1 −2 3
Volume = X · (Y × Z) = (X × Y ) · Z = 1 3 1
2 1 2
= 1(6 − 1) − 1(−4 − 3) + 2(−2 − 9) = 5 + 7 − 22 = | − 10| = 10.
132 Chapter 5. Eigenvalues and Eigenvectors

Exercise 3.5.1

Show that two nonzero vectors X and Y in R3 are parallel, if and only if, X × Y = 0.

Solution:

X//Y iff Y = c X iff X × Y = X × (cX) = O.

Exercise 3.5.2

If U and V are nonzero vectors in R3 such that k(2U ) × (2V )k = −4U · V , compute the angle
between U and V .
Hint: What is θ if tan (θ) = −1.

Exercise 3.5.3

Find the area of the triangle whose vertices are P (1, 0, −1), Q(2, −1, 3) and R(0, 1, −2).

Exercise 3.5.4

Let U and V be unit vectors in R3 . Show that kU × V k2 + (U · V )2 = 1.

Exercise 3.5.5

Let X and Y be two nonzero vectors in R3 , with angle θ = π

3
between them. Find k X × Y k, if
k X k = 3 and k −2Y k = 4.

Exercise 3.5.6

If X and Y are two vectors in R3 , show that X × Y is orthogonal to X.

Exercise 3.5.7

Find a vector that is orthogonal to both vectors X = (0, 2, −2) and Y = (1, 3, 0).
3.5. The Cross Product in Rn 133

Exercise 3.5.8

Find the are of the parallelogram determined by X = (1, 3, 4) and Y = (5, 1, 2).
√
Final answer: 2 131.
134 Chapter 5. Eigenvalues and Eigenvectors

3.3 Orthogonality

Definition 3.3.1

Two nonzero vectors X and Y in Rn are said to be orthogonal (or perpendicular) if X ·Y = 0.

A nonemoty set of vectors in Rn is called an orthogonal set if all pairs of distinct vectors in
the set are orthogonal. An orthogonal set of unit vectors is called an orthonormal set.

The set of standard unit vectors {i, j, k} in R3 is an orthonormal set.

The set {(1, −2, 0), (2, 1, 2), (4, 2, −5)} is an orthogonal set since the dot product of any pair of
distinct vectors is 0.

Theorem 3.3.1: The Pythagoras Theorem in Rn

If X and Y are orthogonal vectors in Rn , then

k X + Y k2 = k X k2 + k Y k2 X +Y
Y

Proof:

Since X and Y are orthogonal, we have X · Y = 0. Then X

k X + Y k2 = (X + Y ) · (X + Y ) = k X k2 + 2 (X · Y ) + k Y k2 = k X k2 + k Y k2 .

Example 3.3.1

Find all vectors in R4 that are orthogonal to both

X = (1, 1, 1, 1) and Y = (1, 1, −1, −1).

Solution:
3.3. Orthogonality 135

Let Z = (a, b, c, d) ∈ R4 so that Z · X = Z · Y = 0. Therefore, we get the following homogenous

system:
   
a+b+c+d=0 1 1 1 1 0 1 1 0 0 0
=⇒ ∼
a+b−c−d=0 1 1 −1 −1 0 0 0 1 1 0

Solving this system, we get a = −b and c = −d. Let b = r and d = s where r, s ∈ R to get
Z = (−r, r, −s, s) which is the form of any vector in R4 that is orthogonal to both X and Y .

Example 3.3.2

Show that the triangle with vertices P1 (2, 3, −4), P2 (3, 1, 2), and P3 (7, 0, 1) is a right triangle.

Solution:

P3 P3 P1

−−−→ −−−→
P1 P3 P2 P3
−−−→
P1 P2
P1 P2 P1 P2 P3 P2
Figure 1 Figure 2 Figure 3

We start with Figure 1 as we do not know if there is a right angle. We create three vectors,
namely
−−→
X = P 1 P2 = P2 − P1 = (1, −2, 6)
−−→
Y = P 1 P3 = P3 − P1 = (5, −3, 5)
−−→
Z = P 2 P3 = P3 − P2 = (4, −1, −1)
This is draw in Figure 2. Then, we want to find two vectors whose dot product is zero which is
valid by considering X and Z. That is

X · Z = 4 + 2 − 6 = 0.

Therefore, this triangle has a right angle at P2 and it is drawn at Figure 3.

Also, we can use the Pythagoras Theorem to show that k Y k2 = k X k2 + k Z k2 .
136 Chapter 5. Eigenvalues and Eigenvectors

Exercise 3.3.1

1. Find all values of c so that X = (c, 2, 1, c) and Y = (c, −1, −2, −3) are orthogonal.
2. Show that if X and Y are orthogonal unit vectors in Rn , then
√
kaX + bY k = a2 + b 2 .

3. Show that if X and Y are orthogonal unit vectors in Rn , then

k 4 X + 3 Y k = 5.

4. Let X and Y be two vectors in Rn so that k X k = k Y k. Show that X − Y and X + Y

are orthogonal.
5. Verify that the triangle with vertices A(1, 1, 2), B(1, 2, 3), and C(3, 0, 3) is a right triangle.
6. Find all values of a so that X = (a2 − a, −3, −1) and Y = (2, a − 1, 2a) are orthogonal.
7. Find a unit vector that is orthogonal to both X = (1, 1, 0) and Y = (−1, 0, 1).
8.
3.4. Lines and Planes in R3 137

3.4 Lines and Planes in R3

Definition 3.4.1

A line in R3 is determined by a fixed point P0 = (x0 , y0 , z0 ) and

a directional vector U = (a, b, c). The line L through P0 and y

parallel to U consists of the points P (x, y, z) such that P (x, y, z)

L

(x, y, z) = (x0 , y0 , z0 ) + t(a, b, c), where t ∈ R. (3.4.1) U = (a, b, c)

P0 (x0 , y0 , z0 )
X

Such equation is written as X = P0 + tU , where X = (x, y, z). x

O(0, 0, 0)
The parametric equation of line L (Equation 3.4.1):
 z
x = x0 + at 



y = y0 + bt 
t ∈ R,
= +

z z0 ct 


while the symmetric form of L is given by:

x − x0 y − y0 z − z0
t := = = .
a b c

Example 3.4.1

Let P1 (2, −2, 3), P2 (−1, 0, 4), P3 (−4, 2, 5) be three points in R3 .

1. Find the parametric equation and the symmetric form of the line that passes through the
points P1 and P2 .

2. Does P3 lies on the same line? Explain.

Solution:
−−→
1. Let U = P1 P2 = P2 − P1 = (−3, 2, 1) and let P0 = P1 be a fixed point on the line call it
L. Then, the parametric equations of L are:

x = 2 − 3t 



y = −2 + 2t 
t∈R
= 3 +

z t 

138 Chapter 5. Eigenvalues and Eigenvectors

while the symmetric form of L is:

x−2 y − (−2) z−3 2−x y+2

= = ⇐⇒ = = z − 3.
−3 2 1 3 2

2. We have to check if P3 satisfies the parametric equation or the symmetric form of L:

2 − (−4) (2) + 2
= = (5) − 3 =: 2.
3 2

Therefore, t = 2 and P3 lies on L. The same check can be done using parametric equations
of L.

Remark 3.4.1

Let U 1 = (a1 , b1 , c1 ) and U 2 = (a2 , b2 , c2 ) be two vectors associated with L1 and L2 so that

x − x1 y − y1 z − z1 x − x2 y − y2 z − z2
L1 : = = and L2 : = = . Then,
a1 b1 c1 a2 b2 c2

1. L1 ⊥ L2 ⇐⇒ U 1 ⊥ U 2 ⇐⇒ U 1 · U 2 = 0,

2. L1 //L2 ⇐⇒ U 1 //U 2 ⇐⇒ U 1 × U 2 = 0 ⇐⇒ U 2 = c U 1 for c R.

Example 3.4.2

Show that L1 : P1 (4, −1, 4) and U 1 = (1, 1, −3) and L2 : P2 (3, −1, 0) and U 2 = (2, 1, 1) intersect
orthogonally, and find the intersection point.

Solution:

Clearly, U 1 · U 2 = 2 + 1 − 3 = 0. Then, U 1 ⊥ U 2 =⇒ L1 ⊥ L2 . To find the intersection point

P (x, y, z), we look for a point satisfying both parametric equations at the same time:

L1 : x = 4 + t1 , y = −1 + t1 , and z = 4 − 3t1 ,
L2 : x = 3 + 2t2 , y = −1 + t2 , and z = t2 .

Clearly, since y = −1 + t1 = −1 + t2 , we get t1 = t2 . Substituting this in z = 4 − 3t1 = t2 , we

get 4 − 3t1 = t1 which implies that t1 = 1 = t2 . Therefore, the intersection point according to
L1 is P (4 + 1, −1 + 1, 4 − 3), that is P (5, 0, 1).
3.4. Lines and Planes in R3 139

Definition 3.4.2

The equation of a plane Π is determined by a fixed point P0 (x0 , y0 , z0 )

contained in Π and a normal directional vector N = (a, b, c) which is
N

orthogonal to Π. A point P (x, y, z) lies on the plane Π if and only if

−−→ −−→
N ⊥ P0 P ⇐⇒ N · P0 P = 0. L
P
P0

The point-normal equations (general form) of the plane Π that passes through P0 (x0 , y0 , z0 )
and its normal vector is N = (a, b, c) is

a(x − x0 ) + b(y − y0 ) + c(z − z0 ) = 0.

Where the standard form of the plane Π is

ax + by + cz + d = 0.

Remark 3.4.2

Assume that we want to find an equation of a plane Π containing three points P1 (x1 , y1 , z1 ),
P2 (x2 , y2 , z2 ), and P3 (x3 , y3 , z3 ), then we can use either of the following ways:

1. For any point P (x, y, z) ∈ Π, we use the standard form of Π: ax + by + cz + d = 0 and

apply it for the points P1 , P2 , and P3 . This is a homogenous system in a, b, c, and d. This
system has non-trivial solutions if

x y z 1
x1 y1 z1 1
= 0.
x2 y2 z2 1
x3 y3 z3 1

Solving this determinant, we get an equation in the standard form for Π.

−−→ −−→
2. Another way is to compute two contained vectors in Π namely X = P1 P2 and Y = P1 P3
and consider the normal vector to Π which is N = X × Y = (a, b, c) which is orthogonal
to Π. Then, the general form of Π is a(x − x1 ) + b(y − y1 ) + c(z − z1 ) = 0.
140 Chapter 5. Eigenvalues and Eigenvectors

Example 3.4.3

Let P1 (2, −2, 1), P2 (−1, 0, 3), P3 (5, −3, 4), and P4 (4, −3, 7) be four points in R3 . Then,

1. Find an equation of the plane Π that passes through P1 , P2 , and P3 .

2. Is P4 contained in Π? Explain.

Solution:
−−→ −−→
1. Let X = P1 P2 = P2 − P1 = (−3, 2, 2) and Y = P1 P3 = P3 − P1 = (3, −1, 3). These
two vectors are contained in Π while N = X × Y = (8, 15, −3) is a normal vector to Π.
Therefore, a general form of Π is 8(x − 2) + 15(y + 2) − 3(z − 1) = 0. The standard form
of Π is
8x + 15y − 3z + 17 = 0.

2. P4 is contained in Π if it satisfies its equation:

8(4) + 15(−3) − 3(7) + 17 = −17 6= 0.

Therefore, P4 is not on the plane Π.

Remark 3.4.3

Let Π1 : a1 x + b1 y + c1 z + d1 = 0 and Π2 : a2 x + b2 y + c2 z + d2 = 0. Then,

1. Π1 //Π2 ⇐⇒ N 1 //N 2 ⇐⇒ N 1 × N 2 = 0 ⇐⇒ N 2 = c N 1 , where c ∈ R,

2. Π1 ⊥ Π2 ⇐⇒ N 1 ⊥ N 2 ⇐⇒ N 1 · N 2 = 0.
N1 N1

Π2
N2

Π1

N2

Π1 //Π2 Π1 ⊥ Π2
3.4. Lines and Planes in R3 141

Example 3.4.4

Find the parametric equation of the intersection line of the two planes:

Π1 : x − y + 2z = 3 and Π2 : 2x + 4y − 2z = −6.

Solution:

We
 form a non-homogenous
 system
 to solve for the parametric
 equation
 of the intersection
 line:
1 −1 2 3 r2 −2r1 →r2 1 −1 2 3 61 r2 →r2 1 −1 2 3 r1 +r2 →r1
  −−−−−−→   −−−−→   −−−−−−→
2 4 −2 −6 0 6 −6 −12 0 1 −1 −2
 
1 0 1 1
 
0 1 −1 −2
Therefore, the reduced system is:

x + z = 1 and y − z = −2.

Let z = t ∈ R to get the parametric equation of the intersection line:

   
x  1 − t
y  = −2 + t
   
   
z t

Example 3.4.5

Find two equations of two planes whose intersection line is the line L:

x = −2 + 3t; y = 3 − 2t; z = 5 + 4t; where t ∈ R.

Solution:

The symmetric form of L is:

x+2 y−3 z−5
= = .
3 −2 4
Therefore, a first plane is by equating x+2
3
= y−3
−2
, to get −2x−4 = 3y−9. Thus, Π1 : 2x+3y−5 =
0. Another plane is by equating x+2
3
= z−5
4
, to get 4x + 8 = 3z − 15. Thus Π2 : 4x − 3z + 23 = 0.
142 Chapter 5. Eigenvalues and Eigenvectors

Remark 3.4.4

x − x0 y − y0 z − z0
Let U = (a1 , b1 , c1 ) be associated with the line L : = = and N =
a1 b1 c1
(a2 , b2 , c2 ) be associated with the plane Π2 : a2 x + b2 y + c2 z + d2 = 0. Then,

1. L ⊥ Π ⇐⇒ U //N ⇐⇒ U × N = 0 ⇐⇒ U = c N where c ∈ R,

2. L//Π ⇐⇒ U ⊥ N ⇐⇒ U · N = 0.

N1
N1

U
L
U

Π Π

L⊥Π L//Π

Example 3.4.6

Find a plane that passes through the point (2, 4, −3) and is parallel to the plane −2x + 4y −
5z + 6 = 0.

Solution:

Since the two planes parallel, we can choose the normal vector of the second plane. That is
N = (−2, 4, −5). Thus, the equation of the plane is

−2(x − 2) + 4(y − 4) − 5(z + 3) = 0 =⇒ −2x + 4y − 5z − 27 = 0.

Example 3.4.7

Find a line that passes through the point (−2, 5, −3) and is perpendicular to the plane 2x −
3y + 4z + 7 = 0.

Solution:
3.4. Lines and Planes in R3 143

The line L is perpendicular to our plane. So, it is parallel to its normal vector, so we can choose
the normal vector as U . That is U = (2, −3, 4) and hence the parametric equation of L is

x = −2 + 2t 


y = 5 − 3t t∈R


= −3 + 4t

z

Example 3.4.8

Show that the plane Π : 6x − 4y + 2z = 0 and the line L : x−1

3
= −y+4
2
= z − 5 intersect
orthogonally. Find the intersection point.

Solution:

We first have to write the symmetric form as

x−1 y−4 z−5

L: = = .
3 −2 1

Then, the normal vector of Π is N = (6, −4, 2) and the directional vector of L is U = (3, −2, 1).
Clearly, N = 2U which implies that N //U ⇐⇒ Π ⊥ L.
The intersection point with respect to L is

x = 1 + 3t 



y = 4 − 2t 
t∈R
= 5 +

z t 


Therefore, plugin these values into the plane equation, we get

6(1 + 3t) − 4(4 − 2t) + 2(5 + t) = 0

6 + 18t − 16 + 8t + 10 + 2t = 0
28t = 0

Therefore, we get t = 0. Substituting this in the parametric equatio, we get the intersection
point as P (1, 4, 5).
144 Chapter 5. Eigenvalues and Eigenvectors

Exercise 3.4.1

1. Consider the planes:

Π1 : x + y + z = 3, Π2 : x + 2y − 2z = 2k, and Π3 : x + k 2 z = 2.

Find all values of k for which the intersection of the three planes is a line. Hint: Any point
on the intersection of the three planes must satisfies the three equations. This would give
a system of three equation. This system must have infinitely many solutions to describe a
line.
2. Consider the lines:
x+1 y+4 x−3 y−4 z−2
L1 : = = z − 1, and L2 : = = .
3 2 2 −4 2
(a) Show that L1 and L2 are perpendicular and find their point of intersection.

(b) Find an equation of the plane Π that contains both L1 and L2 .

Hint: (a) Show that U 1 · U 2 = 0 and then find a point satisfying both equations of x, y,
and z in terms of t1 and t2 , for instance. (b) Consider N = U 1 × U 2 .
3. Let L be the line through the points P1 (−4, 2, −6) and P2 (1, 4, 3).

(a) Find parametric equations for L.

(b) Find two planes whose intersection is L.

4. Find the parametric equations for the line L which passes through the points P (2, −1, 4)
and Q(4, 4, −2). For what value of k is the point R(k + 2, 14, −14) on the line L?
5. Find the point of intersection of the line x = 1 − t, y = 1 + t, z = t, and the plane
3x + y + 3z − 1 = 0.
6. Find the equations in symmetric form of line of intersection of planes:

Π1 : x + 2y − z = 2, and Π2 : 3x + 7y + z = 11.

7. Find an equation of the plane containing the lines

L1 : x = 3 + t, y = 1 − t, z = 3t, and L2 : x = 2s, y = −2 + s, z = 5 − s.

8. Find a, b ∈ R so that the point P (3, a − 2b, 2a + b) lies on the line

L : x = 1 + 2t, y = 2 − t, z = 4 + 3t.
The Index

A E
addition eigenspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
closed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 eigenvalue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
algebraic multiplicity . . . . . . . . . . . . . . . . . . . . . . 120 eigenvector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
augmented matrix form . . . . . . . . . . . . . . . . . . . . . .3 elementary row operations . . . . . . . . . . . . . . . . . . . 4
Euclidean vector space . . . . . . . . . . . . . . . . . . . . . 63
B
basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 G
general vector space . . . . . . . . . . . . . . . . . . . . . . . . 75
C
Cauchy-Schwarz Inequality . . . . . . . . . . . . . . . . . 69 H
Cauchy-Schwarz inequality . . . . . . . . . . . . . . . . . 69 homogeneous system . . . . . . . . . . . . . . . . . . . . . . . . 1
characteristic polynomial . . . . . . . . . . . . . . . . . . 111
cofactor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 I

cofactor expansion . . . . . . . . . . . . . . . . . . . . . . . . . .51 idempotent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

column rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 inconsistent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

column space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101 initial point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

column vector . . . . . . . . . . . . . . . . . . . . . . . . . . . 3, 13 inner product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

consistent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

cross product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126

L
D leading entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23, 47 line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
determinant of matrix . . . . . . . . . . . . . . . . . . . . . . 51 parametric equation . . . . . . . . . . . . . . . . . . . 137
diagonal matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 linear combination . . . . . . . . . . . . . . . . . . . . . . 18, 65
diagonalizable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 linear independence . . . . . . . . . . . . . . . . . . . . . . . . 88
diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 linearly dependent . . . . . . . . . . . . . . . . . . . . . . . . . . 88
dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 linearly independent . . . . . . . . . . . . . . . . . . . . . . . . 88
dot product . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15, 67 lower triangular matrix . . . . . . . . . . . . . . . . . . . . . 42

145
146 THE INDEX

M O
main diagonal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 orthogonal . . . . . . . . . . . . . . . . . . . . . . . . . . . 126, 134
matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 orthogonal set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 orthogonal vectors . . . . . . . . . . . . . . . . . . . . 126, 134

cofactor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .134

determinant . . . . . . . . . . . . . . . . . . . . . . . . 23, 51 orthonormal set . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

diagonal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
P
inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
parallel gram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
invertable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 parallel piped . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
main diagonal . . . . . . . . . . . . . . . . . . . . . . . . . . 13 permutation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . 15 even . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
nonsingular . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 inversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
similar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 odd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
singular . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 sign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
skew-symmetric . . . . . . . . . . . . . . . . . . . . . . . . 43 perpendicular . . . . . . . . . . . . . . . . . . . . . . . . 126, 134
square . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3, 13 perpendicular vectors . . . . . . . . . . . . . . . . . 126, 134
substraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137, 139
symmetric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 general form . . . . . . . . . . . . . . . . . . . . . . . . . . .139
trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 standard form . . . . . . . . . . . . . . . . . . . . . . . . . 139

transpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 point-normal equation . . . . . . . . . . . . . . . . . . . . . 139

triangular
R
lower . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
upper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
matrix polynomial . . . . . . . . . . . . . . . . . . . . . . . . . . 26 row . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
minor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 rearrangement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
reduced row echelon form . . . . . . . . . . . . . . . . . . . . 6
N row echelon form . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
non-homogeneous system . . . . . . . . . . . . . . . . . . . . 1 row equivalent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 row rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
null space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82, 101 row space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
THE INDEX 147

row vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3, 13 homogenous . . . . . . . . . . . . . . . . . . . . . . . . . 7, 10

inconsistent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
S non-homogeneous . . . . . . . . . . . . . . . . . . . . . . . . 1
scalar multiple of a matrix . . . . . . . . . . . . . . . . . .15 non-homogenous . . . . . . . . . . . . . . . . . . . . . . 7, 8
scalar multiplication
T
closed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
terminal point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
similar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
trace of matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
singularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
transpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
skew-symmetric matrix . . . . . . . . . . . . . . . . . . . . . 43
triangle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
solution
Triangle Inequality . . . . . . . . . . . . . . . . . . . . . . . . . 69
infinite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7, 9
triangular
no . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7, 9
lower . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
non-trivial . . . . . . . . . . . . . . . . . . . . . . . . . . . 7, 11
upper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
trivial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7, 10
unique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7, 8 U

space unit

column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

null . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 upper triangular matrix . . . . . . . . . . . . . . . . . . . . 42

row . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 V
spanned by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 vector
spans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83 column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3, 13
square matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 cross product . . . . . . . . . . . . . . . . . . . . . . . . . . 126
standard form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 directional . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137
standard unit vector . . . . . . . . . . . . . . . . . . . . . . . . 68 dot product . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
subspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 initial point . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
trivial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
symmetric matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 43 norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
system normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
augmented matrix form . . . . . . . . . . . . . . . . . . 3 orthogonal . . . . . . . . . . . . . . . . . . . . . . . .126, 134
consistent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
homogeneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 perpendicular . . . . . . . . . . . . . . . . . . . . .126, 134
148 THE INDEX

row . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3, 13 vector space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63, 75

standard unit . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 general . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
terminal point . . . . . . . . . . . . . . . . . . . . . . . . . . 63
unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Z
zero . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 zero vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64