Chapter 1

Systems of Linear Equations

1.1 Introduction to Systems of Linear Equations

1.2 Gaussian Elimination and Gauss-Jordan Elimination
1.3 Applications of Systems of Linear Equations

※ The linear algebra arises from solving systems of linear equations

※ The last section of each chapter illustrates how to apply the
knowledge you learn in that chapter to solving problems in
different fields
1.1
 1.1 Introduction to Systems of Linear Equations
◼ A linear equation in n variables:
a1 x1 + a2 x2 + a3 x3 + + an xn = b
ai : real-number coefficients
xi : variables needed to be solved
b : real-number constant term
a1: leading coefficient
x1: leading variable
◼ Notes:
(1) Linear equations have no products or roots of variables and
no variables involved in trigonometric, exponential, or
logarithmic functions
(2) Variables appear only to the first power
1.2
 ◼ Ex 1: Linear or Nonlinear
1
Linear (a) 3x + 2 y = 7 (b) x + y −  z = 2 Linear
2

Linear (c) x1 − 2 x2 + 10 x3 + x4 = 0 (d) (sin ) x1 − 4 x2 = e 2 Linear

x is the exponent
Nonlinear (e) xy + z = 2 (f) e x − 2 y = 4 Nonlinear
product of variables

Nonlinear (g) sin x1 + 2 x2 − 3x3 = 0 1 1

(h) + = 4 Nonlinear
x y
trigonometric function not the first power

1.3
◼ A solution of a linear equation in n variables:
a1 x1 + a2 x2 + a3 x3 + + an xn = b

x1 = s1 , x2 = s2 , x3 = s3 , , xn = sn

such that a1s1 + a2 s2 + a3 s3 +  + an sn = b

◼ Solution set:
the set of all solutions of a linear equation
※ In most cases, the number of solutions of a linear equation is
infinite, so we need some methods to represent the solution
set.

1.4
◼ Ex 2: Parametric representation of a solution set

x1 + 2 x2 = 4 with a solution (2, 1), i.e., x1 = 2, x2 = 1

※ If you solve for x1 in terms of x2, you obtain

x1 = 4 − 2 x2 (in this form, the variable x2 is free)
By letting x2 = t (the variable t is called a parameter), you can
represent the solution set as
x1 = 4 − 2t , x2 = t , t is any real number
The set representation for solutions: ( 4 − 2t , t ) | t  R

※ If you choose x1 to be the free variable, the parametric

representation of the solution set is ( s, 2 − 12 s ) | s  R
1.5
◼ A system of m linear equations in n variables:

a11 x1 + a12 x2 + a13 x3 + + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + + a2 n xn = b2
a31 x1 + a32 x2 + a33 x3 + + a3n xn = b3

am1 x1 + am 2 x2 + am3 x3 + + amn xn = bm

◼ A solution of a system of linear equations is a sequence of

numbers s1, s2,…,sn that can solve each linear equation in the
system

1.6
◼ Notes:
Every system of linear equations has either
(1) exactly one solution
(2) infinitely many solutions
(3) no solution

◼ Consistent:
A system of linear equations has at least one solution (for
cases (1) and (2))

◼ Inconsistent:
A system of linear equations has no solution (for case (3))
1.7
◼ Ex 4: Solution of a system of linear equations in 2 variables

(1) x + y = 3
x − y = −1 exactly one solution
two intersecting lines
(2) x + y = 3
2x + 2 y = 6 inifinitely many solution
two coincident lines

(3) x + y = 3
x + y = 1 no solution
two parallel lines

1.8
◼ Ex 5: Using back substitution to solve a system in row-
echelon form
x − 2y = 5 (1)
y = −2 (2)
※ The row-echelon form means that it follows a stair-step pattern (two
variables are in Eq. (1) and the first variable x is the leading variable, and one
variable is in Eq. (2) and the second variable y is the leading variable) and
has leading coefficients of 1 for all equations
※ The back substitution means to solve a system in reverse order. That is, you
solve Eq. (2) for y first, and then substitute the solution of y into Eq. (1).
Next, you can solve x directly in Eq. (1) since x is the only unknown variable
Sol: By substituting y = −2 into Eq. (1), you obtain

x − 2(−2) = 5  x = 1
The system has exactly one solution: x = 1, y = −2
1.9
◼ Ex 6: Using back substitution to solve a system in row-
echelon form
x − 2 y + 3z = 9 (1)
y + 3z = 5 (2)
z = 2 (3)
Sol: Substitute z = 2 into (2), y can be solved as follows
y + 3( 2) = 5
y = −1
and substitute y = −1 and z = 2 into (1)
x − 2( −1) + 3( 2) = 9
x = 1
The system has exactly one solution:
x = 1, y = −1, z = 2
1.10
◼ Equivalent:
Two systems of linear equations are called equivalent
if they have precisely the same solution set
◼ Notes:
Each of the following operations on a system of linear
equations produces an equivalent system
O1: Interchange two equations
O2: Multiply an equation by a nonzero constant
O3: Add a multiple of an equation to another equation

◼ Gaussian elimination:
A procedure to rewrite a system of linear equations to be in
row-echelon form by using the above three operations
1.11
◼ Ex 7: Solve a system of linear equations (consistent system)
x − 2 y + 3z = 9 (1)
−x + 3y = −4 (2)
2 x − 5 y + 5 z = 17 (3)
Sol: First, eliminate the x-terms in Eqs. (2) and (3) based
on Eq. (1)
(1) + (2) → (2) (by O3)
x − 2 y + 3z = 9
y + 3z = 5 (4)
2 x − 5 y + 5 z = 17

(1)  (−2) + (3) → (3) (by O3)

x − 2 y + 3z = 9
y + 3z = 5
− y − z = −1 (5) 1.12
Second, eliminate the (-y)-term in Eq. (5) based on Eq. (4)
(4) + (5) → (5) (by O3)
x − 2 y + 3z = 9
y + 3z = 5
2z = 4 (6)

(6)  12 → (6) (by O2)

x − 2 y + 3z = 9
y + 3z = 5
z = 2

Since the system of linear equations is expressed in its row-

echelon form, the solution can be derived by the back
substitution: x = 1, y = −1, z = 2 (only one solution)
1.13
◼ Ex 8: Solve a system of linear equations (inconsistent system)
x1 − 3x2 + x3 = 1 (1)
2 x1 − x2 − 2 x3 = 2 (2)
x1 + 2 x2 − 3x3 = − 1 (3)

Sol:
(1)  ( − 2) + (2) → (2) (by O3)
(1)  ( − 1) + (3) → (3) (by O3)
x1 − 3x2 + x3 = 1
5 x2 − 4 x3 = 0 (4)
5 x2 − 4 x3 = −2 (5)

1.14
(4)  (−1) + (5) → (5) (by O3)
x1 − 3x2 + x3 = 1
5 x2 − 4 x3 = 0
0 = −2 (a false statement)

So, the system has no solution (an inconsistent system)

1.15
◼ Ex 9: Solve a system of linear equations (infinitely many solutions)
x2 − x3 = 0 (1)
x1 − 3x3 = −1 (2)
− x1 + 3x2 = 1 (3)
Sol: (1)  (2) (by O1)
x1 − 3x3 = −1 (1)
x2 − x3 = 0 (2)
− x1 + 3x2 = 1 (3)

(1) + (3) → (3) (by O3)

x1 − 3x3 = −1
x2 − x3 = 0
3x2 − 3x3 = 0 (4) 1.16
Since Equation (4) is the same as Equation (2), it is not
necessary and can be omitted
x1 − 3x3 = −1
x2 − x3 = 0
 x2 = x3 , x1 = −1 + 3x3

Let x3 = t , then x1 = 3t − 1
x2 = t tR
x3 = t

This system has infinitely many solutions.

1.17
 Keywords in Section 1.1:
◼ linear equation
◼ system of linear equations
◼ leading coefficient
◼ leading variable
◼ solution
◼ solution set
◼ free variable
◼ parametric representation
◼ consistent
◼ inconsistent
◼ row-echelon form
◼ equivalent
1.18
 1.2 Gaussian Elimination and Gauss-Jordan Elimination
◼ mn matrix:
 a11 a12 a13  a1n 
 a21 a22 a23  a2 n 
 a31 a32 a33  a3n  m rows
  
 
am1 am 2 am 3  amn 
n columns
◼Notes:
(1) Every entry aij in the matrix is a real number
(2) A matrix with m rows and n columns is said to be with size mn
(3) If m = n, then the matrix is called square matrix of order n
(4) For a square matrix, the entries a11, a22, …, ann are called the
main (or principal) diagonal entries
1.19
◼ Ex 1: Matrix Size
[ 2] 1 1 ※ A scalar (any real number)
can be treated as a 11 matrix.
 0 0 22
0 0

1 − 3 0 1 
 1 4
2 

 e  
2 2 3 2
 − 7 4 

◼ Note:
One very common use of matrices is to represent a system
of linear equations (see the next slide)
1.20
◼ A system of m equations in n variables:
a11 x1 + a12 x2 + a13 x3 + + a1n xn = b1
a21 x1 + a22 x2 + a23 x3 + + a2 n xn = b2
a31 x1 + a32 x2 + a33 x3 + + a3n xn = b3

am1 x1 + am 2 x2 + am3 x3 + + amn xn = bm

Matrix form: 𝐴𝐱 = 𝐛

 a11 a12 a13  a1n   x1   b1 

 a21 a22 a23  a2 n  x  b 
A =  a31 a32 a33  a3n  x =  2 b= 2
     
    
 
am1 am 2 am 3  amn   xn  bm 

1.21
◼ Coefficient matrix:
 a11 a12 a13 a1n 
a a2 n 
 21 a22 a23
 a31 a32 a33 a3n  = A
 
 
 am1 am 2 am3 amn 

◼ Augmented matrix: formed by appending the constant-term

vector to the right of the coefficient matrix of a system of
linear equations
 a11 a12 a13 a1n b1 
a b2 
 21 a22 a23 a2 n
 a31 a32 a33 a3n b3  = [ A b]
 
 
 am1 am 2 am3 amn bm 
1.22
◼ Before introducing the Gaussian elimination and the Gauss-
Jordan elimination, we need some background knowledge,
including the elementary row operations and the rules to
identify the row-echelon or reduced row-echelon forms for
matrices
◼ Elementary row operations:
(1) Interchange two rows: Ri  R j
(2) Multiply a row by a nonzero constant: kRi → Ri
(3) Add a multiple of a row to another row: kRi + R j → R j

◼ Row equivalent:
Two matrices are said to be row equivalent if one can be obtained
from the other by a finite sequence of above elementary row
operations 1.23
◼ Ex 2: Elementary row operations

 0 1 3 4 R1  R2  − 1 2 0 3
 − 1 2 0 3  0 1 3 4
 2 − 3 4 1  2 − 3 4 1
1
2 − 4 6 − 2 2 R1 → R1  1 − 2 3 − 1
 1 3 − 3 0  1 3 − 3 0
 5 − 2 1 2 5 − 2 1 2

 1 2 − 4 3 −2R1 + R3 → R3  1 2 − 4 3
 0 3 − 2 − 1 0 3 − 2 − 1
2 1 5 − 2 0 − 3 13 − 8

1.24
◼ Row-echelon form: (1), (2), and (3)
◼ Reduced row-echelon form: (1), (2), (3), and (4)
(1) All rows consisting entirely of zeros occur at the bottom of the
matrix.
(2) For each row that does not consist entirely of zeros, the first
nonzero entry from the left side is 1, which is called as leading 1.
(3) For two successive nonzero rows, the leading 1 in the higher
row is further to the left than the leading 1 in the lower row.
(4) Every column that contains a leading 1 has zeros everywhere
else.

1.25
◼ Ex 4: Row-echelon form or reduced row-echelon form
 1 2 − 1 4 (row-echelon 0 1 0 5 (reduced row-
0 1 0 3 0 0 1 3
0 0 1 − 2 form) 0 0 0 0 echelon form)

 1 − 5 2 − 1 3 1 0 0 − 1
0 0 1 3 − 2 (row-echelon 0 1 0 2 (reduced row-
0 0 0 1 4 form) 0 0 1 3 echelon form)
0 0 0 0 1 0 0 0 0

 1 2 − 3 4  1 2 − 1 2
0 2 1 − 1  0 0 0 0
0 0 1 − 3 0 1 2 − 4

Violate the Violate the first

second criterion criterion
1.26
◼ Gaussian elimination:
The procedure for reducing a matrix to a row-echelon form
by performing the three elementary row operations
◼ Gauss-Jordan elimination:
The procedure for reducing a matrix to its reduced row-echelon
form by performing the three elementary row operations

◼ Notes:
(1) Every matrix has an unique reduced row-echelon form
(2) A row-echelon form of a given matrix is not unique
(Different sequences of elementary row operations can
produce different row-echelon forms)
※ The above two statements will be verified numerically in Ex. 8
1.27
◼ Ex: Procedure of the Gaussian elimination and Gauss-Jordan elimination
Produce a leading 1
 0 0 − 2 0 7 12 2 4 − 10 6 12 28
2 4 − 10 6 12 28  0 0 − 2 0 7 12
2 4 − 5 6 − 5 − 1 2 4 − 5 6 − 5 − 1
The first nonzero column

Leading 1 Produce a leading 1

 1 2 − 5 3 6 14 1 2 − 5 3 6 14
 0 0 − 2 0 7 12 0 0 − 2 0 7 12
2 4 − 5 6 − 5 − 1 0 0 5 0 − 17 − 29
Eliminate nonzero Submatrix
The first nonzero
entries below the leading
column
1

1.28
 The first nonzero column
Leading 1
 1 2 −5 3 6 14   1 2 −5 3 6 14 
0 0  0 0 − − 
 1 0 −7 2 −6   1 0 7 2 6 
0 0 5 0 −17 −29  0 0 0 0 12 1
Eliminate nonzero entries
below the leading 1 Produce Submatrix
a leading 1
Nonzero entries

 1 2 −5 3 6 14   1 2 0 3 0 7
0 0 1 0 −7 2 −6  0 0 1 0 0 1
  0 0 0 0 1 2
0 0 0 0 1 2 
Leading 1’s
(reduced row-
(row-echelon form)
echelon form)

1.29
◼ Ex 7: Solve a system by the Gauss-Jordan elimination method (only
one solution)
x − 2 y + 3z = 9
− x + 3y = −4
2 x − 5 y + 5z = 17
Sol:
Form the augmented matrix by concatenating A and b
 1 −2 3 9   1 −2 3 9   1 −2 3 9 
 −1 3 0 −4  0 1 3 5  0 1 3 5 
     
 2 −5 5 17  0 −1 −1 −1 0 0 1 2 
(row-echelon form)
 1 0 0 1
0 1 0 −1 x = 1
  y = −1
0 0 1 2  z = 2
(reduced row-echelon form)
1.30
◼ Ex 8：Solve a system by the Gauss-Jordan elimination
method (infinitely many solutions)
2 x1 + 4 x2 − 2 x3 = 0
3x1 + 5 x2 = 1
Sol:
For the augmented matrix,

 2 4 −2 0  1 2 −1 0   1 2 −1 0 
3 5 0  ⎯⎯
→  ⎯⎯ → 
 1 3 5 0 1 0 −1 3 1
1 2 −1 0   1 0 5 2
⎯⎯ →  ⎯⎯→ 
0 1 −3 −1  0 1 −3 −1 
(row-echelon form) (reduced row-echelon form)

1.31
Then the system of linear equations becomes
x1 + 5 x3 = 2
x2 − 3 x3 = −1
It can be further reduced to
x1 = 2 − 5 x3
x 2 = − 1 + 3 x3

Let x3 = t , then x1 = 2 − 5t ,
x2 = −1 + 3t , tR
x3 = t ,
So, this system has infinitely many solutions

1.32
※ In order to show that the reduced row-echelon form is unique, I
conduct an experiment by performing a redundant elementary row
operation of interchanging the first and second rows in advance

 2 4 −2 0   3 5 0 1  1 5 / 3 0 1/ 3
 3 5 0 1 ⎯⎯ →
−  ⎯⎯
→
− 
   2 4 2 0   2 4 2 0 
1 5 / 3 0 1/ 3  1 5 / 3 0 1/ 3
⎯⎯ →  ⎯⎯ → 
 0 2 / 3 −2 − 2 / 3   0 1 −3 − 1
 1 0 5 2 (row-echelon form)
⎯⎯
→ 
 0 1 −3 −1
(reduced row-echelon form)

※ Comparing with the results on Slide 1.31, we can infer that it is

possible to derive different row-echelon forms, but there is a unique
reduced row-echelon form for each matrix

1.33
◼ Homogeneous systems of linear equations
A system of linear equations is said to be homogeneous
if all the constant terms are zero

a11 x1 + a12 x2 + a13 x3 +  + a1n xn = 0

a21 x1 + a22 x2 + a23 x3 +  + a2 n xn = 0
a31 x1 + a32 x2 + a33 x3 +  + a3n xn = 0

am1 x1 + am 2 x2 + am 3 x3 +  + amn xn = 0

1.34
◼ Trivial (obvious) solution:
x1 = x2 = x3 =  = xn = 0
◼ Nontrivial solution:
other solutions
◼ Theorem 1.1
(1) Every homogeneous system of linear equations is consistent.
Furthermore, for a homogeneous system, exactly one of the
following is true:
(a) The system has only the trivial solution
(b) The system has infinitely many nontrivial solutions in addition to the
trivial solution (In other words, if a system has any nontrivial solution, this
system must have infinitely many nontrivial solutions)
(2) If the homogenous system has fewer equations than variables,
then it must have an infinite number of solutions 1.35
◼ Ex 9: Solve the following homogeneous system (also verify
the two statements in Theorem 1.1 numerically)
x1 − x2 + 3x3 = 0
2 x1 + x2 + 3x3 = 0

Sol: For the augmented matrix,

 1 − 1 3 0  1 0 2 0 (reduced row-
2 1 3 0 0 1 − 1 0 echelon form)

x1 + 2 x3 = 0

x2 − x3 = 0
Let x3 = t, then x1 = −2t , x2 = t , x3 = t , t  R
When t = 0, x1 = x2 = x3 = 0 (trivial solution)
When t  0, there are infinitely many nontrivial solutions 1.36
 Keywords in Section 1.2:
◼ matrix
◼ row
◼ column
◼ entry
◼ size
◼ square matrix
◼ order
◼ main diagonal
◼ coefficient matrix
◼ augmented matrix

1.37
◼ elementary row operation
◼ row equivalent
◼ row-echelon form
◼ reduced row-echelon form
◼ leading 1
◼ Gaussian elimination
◼ Gauss-Jordan elimination
◼ homogeneous system
◼ trivial solution
◼ nontrivial solution

1.38
 1.3 Applications of Systems of Linear Equations
◼ Polynomial Curve Fitting
◆ The polynomial curve fitting problem is that given n points
(x1, y1), (x2, y2),…, (xn, yn) on the xy-plane, find a polynomial
function of degree n–1 passing through these n points
◆ Once equipped with this polynomial function, we can predict
the value of y for some missing values of x
◆ A typical application of the curve fitting problem in the field
of finance is to derive the interest rate (y) for different time to
maturity (x)

1.39