Lecture notes on Linear Algebra

1.4 Gaussian Elimination

Linear Algebra, Sec. 1.4-1

Review the last lecture

Linear Algebra, Sec. 1.4-2

 Elementary row operations
(ero)
r ↔r
i j

(1) interchange operation, A − − → B indicates row i and row j are

interchanged cr →r
i i
(2) scaling operation, A − −−−→ B indicates the entries of row i
are multiplied by c, where c /=
0
— − j−→ B indicates c times row i is added
− →r
cr +r
i j
row j
(3) row addition, A to
0 1
Example: A = 2 0
0 3

0 1 2 0 1 r →r
1 0 2
1 0
2 0 r−↔r1− → 0 1 −2 −1 −→
1
0 1 −−
(−3)r−− −− → 0 1
+r →r
2 3
0 3 0 3 0 3 0 0
3

Linear Algebra, Sec. 1.3-17

 Example

Consider Rx = c, with

* Solution of [R c] can be easily obtained if it is in

rref.

Linear Algebra, Sec. 1.3-23

 Basic and free variables

► basic variables: the variables corresponding to the leading

entries of R
► free variables: non basic variables.

Example Consider Rx = c, with

[R c] = .

► basic variables: x2, x3, x5

► free variables: x1, x4

Linear Algebra, Sec. 1.3-24

Lecture notes on Linear Algebra

1.4 Gaussian Elimination

Linear Algebra, Sec. 1.4-6

 Some definitions

Let R be the reduced row echelon form of A.

► pivot positions of A are positions that contain the leading
entries of nonzero row of R.
► pivot columns of A are columns of A that contain the
pivot positions.

For example,

The pivot positions are at (1,2), (2,3) and

(3,5). The pivot columns are a2, a3, a5.

Linear Algebra, Sec. 1.4-7

 Procedure of Gaussian elimination

► Step 1. Identify the leftmost nonzero column, which is

pivot column. The topmost position is a pivot position.
► Step 2. Interchange rows to make the entry at the pivot
position nonzero.
► Step 3. Do row additions so that the entries below the pivot
position are zero.
► Step 4. Ignore the row contains the pivot position and all
rows above it. Repeat Steps 1-4 on the remaining
submatrix.
► Step 5. Starting from the last nonzero row, do row scaling so
that the leading entry is equal to one and do row additions so
that the entries above the pivot position are zero.
► Step 6. If Step 5 is applied on the first row, stop. Otherwise
repeat Step 5 on the preceding row.

Linear Algebra, Sec. 1.4-8

 Example of Gaussian elimination

► Step 1. Identify the leftmost nonzero column, which is

pivot column. The topmost position is a pivot position.
Identify pivot column a2 and pivot position (1,2).
► Step 2. Interchange rows to make the entry at the pivot
position nonzero.

Linear Algebra, Sec. 1.4-9

 Example of Gaussian elimination

► Step 3. Do row additions s.t. the entries below the pivot

position are zero.

► Step 4. Ignore the row contains the pivot position and all
rows above it. Repeat Steps 1-4 on the remaining
submatrix.

Linear Algebra, Sec. 1.4-10

 Summary on Gaussian elimination

pivot columns: a2, a3, a5

pivot positions: (1,2), (2,3), (3,5)

► Using the Gaussian elimination, we can obtain the rref of A,

and identify the pivot columns as well as pivot positions.
► We call Steps 1-4 the forward pass and Steps 5-6 the backward
pass.

► The columns of R that contain the pivot positions have only

one nonzero element that is equal to 1. Linear Algebra, Sec. 1.4-11
 Properties of rref
Q. Suppose A, m × n → rref R. Consider the columns of R that
contain the pivot positions.

(1) What can we say about the columns of R that contain the
pivot positions?

standard vectors in Rm
(2) Suppose R has ρ nonzero rows. Describe the vectors in (1).

If R has ρ nonzero rows, its columns that contain the ρ pivot

positions are precisely e1, e2, · · · , eρ in Rm.
2

e.g.,

2
This property is used in slide 20 here. Linear Algebra, Sec. 1.4-12
 Recall: basic and free variables

► basic variables: the variables corresponding to the leading

entries of
R
► free variables: non basic variables.

Example Consider Rx = c, with

[R c] =

► basic variables: ?
► free variables: ?

Linear Algebra, Sec. 1.4-13

 Recall: basic and free variables

► basic variables: the variables corresponding to the leading

entries of
R
► free variables: non basic variables.

Example Consider Rx = c, with

[R c] =

► basic variables: x2, x3, x5

► free variables: x1, x4

Linear Algebra, Sec. 1.4-14

 Number of basic variables
Consider Ax = b, where A is m × n. Suppose
[A b] −→ rref[R c]
Q.
► What is the total number of basic and free variables?
►

number of basic variables

? = number of leading entries of R
? = number of pivot positions of A
? = number of pivot columns of A

Linear Algebra, Sec. 1.4-15

 Number of basic variables
Consider Ax = b, where A is m × n. Suppose
[A b] −→ rref[R c]
Q.
► What is the total number of basic and free variables?
►

number of basic variables

? = number of leading entries of R
? = number of pivot positions of A
? = number of pivot columns of A

number of basic variables

= number of leading entries of R
= number of pivot positions of A
= number of pivot columns of A
= n − number of free variables
Linear Algebra, Sec. 1.4-16
 Rank and Nullity

Let A be m × n.

Def. Rank
The rank of A, denoted by rank A, is the number of nonzero
rows in the rref of A.
(=number of basic variables)

Def. Nullity
The nullity of A, denoted by nullity A, is defined to be
n− rank A.
(=number of free variables)
Example.

eros

rank A=3, nullity A = 5 − 3 = 2

Linear Algebra, Sec. 1.4-17
 Nullity and uniqueness of solutions

Let Ax = b be a consistent system.

Q.
► Suppose the solution is unique. What is nullity A?

The solution is unique as long as there is no free

variable.
► Suppose nullity A = 0. Is the solution unique?

Linear Algebra, Sec. 1.4-18

 Nullity and uniqueness of solutions

Let Ax = b be a consistent system.

Q.
► Suppose the solution is unique. What is nullity A?

The solution is unique as long as there is no free

variable.
► Suppose nullity A = 0. Is the solution unique?

For a consistent system, the solution is unique ⇔ nullity A = 0.

Linear Algebra, Sec. 1.4-19

 rref in the case rank A = n

Let A be m × n. Suppose A → rref R. Consider 2 cases: (1) m ≥ n, (2)

m < n.
(1) m ≥ n:

A= ,R=

* special case m = n:
(2) m < n:

Recall. If R has ρ nonzero rows, its columns that contain the ρ

pivot positions are precisely e1, e2, · · · , eρ in Rm.
When ρ = n, the columns vectors of R are e1, e2, · · · , en in Rm.
Linear Algebra, Sec. 1.4-20
 Maximum of rank A

Let A be m × n and A → rref R.

(1) m and rank A.
rank A is the number of nonzero rows in R, so

rank A ≤ m
(2) n and rank A.
The columns of R that contain the pivot positions are
precisely
e1, e2, · · · , eρ, where ρ = rank A. So

rank A ≤ n

rank A ≤ min(m, n)

Linear Algebra, Sec. 1.4-21

 Test for consistency

Theorem 1.5 Test for consistency

Suppose [A b] → rref[R c].

Ax = b is consistent
⇐⇒ b is a linear combination of a1 , a2 , · · · , an
⇐⇒ [R c] has no row of the form [0 · · · 0 , where d /= 0
d]

Proof.
We already know (a) if and only if (b). We can show that theorem if we can show (a) ⇔ (c).
► ”(c) → (a)”
If [R c] has no row of the form [0 · · · 0 d] , where d /= 0 then Rx = c is
consistent, i.e., Ax = b consistent.
► ”(a) → (c)”, which is equivalent to ”NOT (c) → NOT (a)”.
”Not (c)” statement: [R c] has a row of the form [ 0 ··· 0 d] , where d isn’t 0.
Therefore Rx = c is inconsistent. This means that Ax = b is inconsistent, i.e., ”Not (a)”.

Linear Algebra, Sec. 1.4-22

 Review: Systems of Linear Equations

A system of m linear equations:

a11x1 + a12x2 + · · · + a1nxn = b1.

a21x1 + a22x2 + · · · + a2nxn = b2.

..
am1x1 + am2x2 + · · · + amnxn = bm.

Example. A system of 2 linear equations and 3

variables
x1 + x2 + x3 = 0

x3 = 1

Linear Algebra, Sec. 1.3-8

Review: Matrix forms of Systems of Linear Equations

► [A b], called the augmented matrix, has all the information

needed about the system.
► s is called a solution of the system if s satisfies As = b.
► solution set is the set of all the solutions, i.e.,
n
{s ∈ R : As = b}.

► A system of m linear equations is called consistent if it has

one or more solutions. Otherwise it is inconsistent .

Linear Algebra, Sec. 1.3-11

 Example

Let A be m × n and B = [A b]. Suppose rank A=k. Find possible

values of rank B. Discuss consistency of Ax = b for each
possible possible value of rank B.

Ans.
We can determine rank B from its rref.
Suppose [A b] −→ rrefRB = [R c], then R is the rref of A.
As rank A=k , R has k nonzero rows. It follows that RB has k or k + 1 nonzero rows, i.e, rank
B = k or k + 1.
► rank B = k:
RB has no row of the form [ 0 · · · 0 d], where d is not 0. So Ax = b is consistent.
► rank B = k + 1:
The (k + 1)th row of RB is of the form [0 · · · 0 d], d is not 0.
So Ax = b is inconsistent.

Linear Algebra, Sec. 1.4-25

 Summary on 1.4

► Procedure of Gaussian elimination

► rank A, rank A ≤ min(m, n)
► nullity A
► Properties of rref
The columns of R that contain pivot positions are standard
vectors.
► Test for consistency of Ax = b

Linear Algebra, Sec. 1.4-26

 Guidelines for Submitting Club Ads

► Please email me your materials 1 week before the lecture

► Please include everything in a Google Slide
► 3 Announcements max per lecture. First come first serve

Linear Algebra, Sec. 1.4-27

 Club Announcement #1

https://forms.gle/oaX93vzbaYiH14su9
Koinonia 團契。任何問題可以聯繫 Sherry:0936111237
Linear Algebra, Sec. 1.4-28
 Rules for Kahoot

► Form a group of ~ 5 people

► Introduce yourself to one another
► Come up with a name for your team
► Select your captain
► The captain login to Kahoot https://kahoot.it/

Linear Algebra, Sec. 1.4-30

 Rules for Kahoot

► https://kahoot.it/

Linear Algebra, Sec. 1.4-31

 Questions?

I’ll see you guys next time!