⾼斯消去法

Gaussian Elimination
李龍豪 (Lung-Hao Lee)
助教信箱：[email protected]
 Row Echelon Form
• To be of row echelon form (列梯型), a matrix must have the
following properties

① If a row does not consist entirely of zeros, the the rst nonzero
number is the row is 1. We called this is a leading 1 (⾸項1)

② If there are any rows that consist entirely of zeros, then they
are grouped together at the bottom of the matrix. (元素全為0
的列，置於矩陣底部)

③ If any two successive rows that do not consist entirely of

zeros, that leading 1 in the lower row occurs father to the right
than the leading 1 in the higher row (⾸項1較早出現的列應置於
較⾼列)

2
fi
Examples for Row Echelon Form

⎡ 1 4 −3 7 ⎤ ⎡ 1 1 0 ⎤
⎢ ⎥ ⎢ ⎥
⎢ 0 1 6 2 ⎥ ⎢ 0 1 0 ⎥
⎢⎣ 0 0 1 5 ⎥⎦ ⎢⎣ 0 0 0 ⎥⎦

⎡ 0 1 2 6 0 ⎤
⎢ ⎥
⎢ 0 0 1 −1 0 ⎥
⎢⎣ 0 0 0 0 1 ⎥⎦

3
Reduced Row Echelon Form
• A matrix that is in reduced row echelon form (簡約列梯型)
must have the following properties

① It’s identical to row echelon form

② It’s identical to row echelon form

③ It’s identical to row echelon form

④ Each column that contains a leading 1 has zeros

everywhere else in that column (含⾸項1的⾏，該⾏
其他元素皆為0)

4
Examples for Reduced Row Echelon Form

⎡ 1 0 0 4 ⎤ ⎡ 1 0 0 ⎤
⎢ ⎥ ⎢ ⎥
⎢ 0 1 0 7 ⎥ ⎢ 0 1 0 ⎥
⎢⎣ 0 0 1 −1 ⎥⎦ ⎢⎣ 0 0 1 ⎥⎦

⎡ 0 1 −2 0 1 ⎤
⎢ ⎥ ⎡ 0 0 ⎤
⎢ 0 0 0 1 3 ⎥
⎢ ⎥
⎢ 0 0 0 0 0 ⎥ ⎣ 0 0 ⎦
⎢ 0 0 0 0 0 ⎥⎦
⎣

5
 More Examples
• Row echelon form * denotes any real numbers

⎡ 1 * * * ⎤ ⎡ 1 * * * ⎤ ⎡ 1 * * * ⎤ ⎡ 0 1 * * * * * * * * ⎤
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
0 1 * * 0 1 * * 0 1 * * ⎢ 0 0 0 1 * * * * * * ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ 0 0 1 * ⎥ ⎢ 0 0 1 * ⎥ ⎢ 0 0 0 0 ⎥ ⎢ 0 0 0 0 1 * * * * * ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 0 0 0 0 1 * * * * ⎥
⎣ 0 0 0 1 ⎦ ⎣ 0 0 0 0 ⎦ ⎣ 0 0 0 0 ⎦ ⎢ ⎥
⎢⎣ 0 0 0 0 0 0 0 0 1 * ⎥⎦

• Reduced row echelon form

⎡ 0 1 * 0 0 0 * * 0 * ⎤
⎡ 1 0 0 0 ⎤ ⎡ 1 0 0 * ⎤ ⎡ 1 0 * * ⎤ ⎢ ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 0 0 1 0 0 * * 0 * ⎥
⎢ 0 1 0 0 ⎥ ⎢ 0 1 0 * ⎥ ⎢ 0 1 * * ⎥ ⎢ 0 0 0 0 1 0 * * 0 * ⎥
⎢ 0 0 1 0 ⎥ ⎢ 0 0 1 * ⎥ ⎢ 0 0 0 0 ⎥ ⎢ 0 0 0 0 0 1 * * 0 * ⎥
⎢ 0 0 0 1 ⎥ ⎢ 0 0 0 0 ⎥ ⎢ 0 0 0 0 ⎥ ⎢ ⎥
⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎢⎣ 0 0 0 0 0 0 0 0 1 * ⎥⎦

6
 Unique Solution
• Suppose that the augmented matrix for a linear system in
the unknowns x1, x2, x3, and x4 has been reduced by
elementary row operations to ⎡ 1 0 0 0 3 ⎤
⎢ ⎥
⎢ 0 1 0 0 −1 ⎥
⎢ 0 0 1 0 0 ⎥
⎢ 0 0 0 1 5 ⎥
⎣ ⎦
• This matrix is in reduced row echelon form and
correspond to the equations
x1 =3

• Thus, the system has a unique solution, x2 = −1

x1=3, x2=-1, x3=0, x4=5
x3 =0

x4 = 5

7
Linear Systems in Three Unknowns
• Given reduced row echelon form, solve the system
⎡ 1 0 0 0 ⎤ ⎡ 1 0 3 −1 ⎤ ⎡ 1 −5 1 4 ⎤
(a) ⎢ ⎥ (b) ⎢ ⎥ (c) ⎢ ⎥
⎢ 0 1 2 0 ⎥ ⎢ 0 1 −4 2 ⎥ ⎢ 0 0 0 0 ⎥
⎢⎣ 0 0 0 1 ⎥⎦ ⎢⎣ 0 0 0 0 ⎥⎦ ⎢⎣ 0 0 0 0 ⎥⎦

• Solution (a): No solution

The equation that corresponds to the last row of the

augmented matrix is
0x + 0y + 0z = 1
Since this equation is not satis ed bt any values of x, y, and z,
the system is inconsistent

8
fi
 Leading variables vs. Free variables

• (Continued) Solution (b): in nitely many solutions

⎡ 1 0 3 −1 ⎤
⎢
⎢ 0 1 −4 2
⎥
⎥
This equation can be omitted ⎢⎣ 0 0 0 0 ⎥⎦
since it imposes no restrictions on x, y, and z
0x + 0y + 0z = 0
Since x and y correspond to the leading 1’s in the augmented
matrix, we call these leading variables (⾸項變數). The
remaining variables (in this case z) are called free variables (⾃
由變數)
x + 3z = −1 x = −1− 3z
x=-1-3t, y=2+4t, z=t y − 4z = 2 y = 2 + 4z

• Solution (c): in nitely many solutions ⎡ 1 −5 1 4 ⎤

⎢
⎢ 0 0 0 0
⎥
⎥
x=4+5s-t, y=s, z=t ⎢⎣ 0 0 0 0 ⎥⎦
9
fi
fi
 A General Solution
De nition 1

If a linear system has in nitely many solutions, then a

set of parametric equations from which all solutions
can be obtained by assigning numerical values to the
parameters is called a general solution (通解) of the
system

10
fi
fi
 Elimination procedure
• How easy it is to solve a system of linear equations once
its augmented matrix is reduced row echelon form

• To give a step-by-step elimination procedure that can be

used to reduce any matrix to reduced row echelon form

⎡ 0 0 −2 0 7 12 ⎤ ⎡ 1 2 0 3 0 7 ⎤
⎢ ⎥ ⎢ ⎥
⎢ 2 4 −10 6 12 28 ⎥ ⎢ 0 0 1 0 0 1 ⎥
⎢⎣ 2 4 −5 6 −5 −1 ⎥⎦ ⎢⎣ 0 0 0 0 1 2 ⎥⎦

11
Elimination procedure: Step 1
• Locate the leftmost column that dose not consist entirely
of zeros (找出不全為零元素的最左⾏)

⎡ 0 0 −2 0 7 12 ⎤
⎢ ⎥
⎢ 2 4 −10 6 12 28 ⎥
⎢⎣ 2 4 −5 6 −5 −1 ⎥⎦

Leftmost nonzero column

12
 Elimination procedure: Step 2
• Interchange the top row with another row, if necessary, to
bring a nonzero entry to the top of the column in Step 1
(視需要，以另⼀列交換最頂列)

⎡ 2 4 −10 6 12 28 ⎤
⎢ ⎥ The rst and second
⎢ 0 0 −2 0 7 12 ⎥ rows in the preceding
matrix were interchanged
⎢⎣ 2 4 −5 6 −5 −1 ⎥⎦

13
fi
 Elimination procedure: Step 3
• If the entry that is now at the top of the column found in
Step 1 is a, multiply the rst row by 1/a in order to
introduce a leading 1 (若找出的⾏最頂端的元素是a, 則將
第⼀列乘以1/a)

⎡ 1 2 −5 3 6 14 ⎤ The rst row of the

⎢ ⎥ preceding matrix was
⎢ 0 0 −2 0 7 12 ⎥ multiplied by 1/2

⎢⎣ 2 4 −5 6 −5 −1 ⎥⎦

14
fi
fi
Elimination procedure: Step 4
• Add suitable multiples of the top row to the rows below so
that all entries below the leading 1 becomes zero (將頂列乘
以適當倍數加置底下幾列，使其在⾸項1底下的元素皆變為0）

⎡ 1 2 −5 3 6 14 ⎤
⎢ ⎥
⎢ 0 0 −2 0 7 12 ⎥ -2 times the rst row of
⎢⎣ 0 0 5 0 −17 −29 ⎥⎦ the preceding matrix was
added to the third row

15
fi
 Elimination procedure: Step 5
• Now cover the top row in the matrix and begin again with Step
1 applied to the submatrix that remains. Continue in this way
until the entire matrix is in row echelon form (保留矩陣的頂列且
重新使⽤步驟1處裡留下的⼦矩陣，直到整成列梯型）
⎡ ⎤
⎡ 1 2 −5 3 6 14 ⎤ ⎢ ⎥
⎢ ⎥ ⎢ 1 2 −5 3 6 14 ⎥
⎢ 0 0 −2 0 7 12 ⎥
⎢ 7 ⎥
⎢⎣ 0 0 5 0 −17 −29 ⎥⎦ ⎢ 0 0 1 0 −2 −6 ⎥
Leftmost ⎢ ⎥
nonzero column ⎢ 0 0 0 0 1 1 ⎥
in the submatrix ⎢⎣ 2 ⎥⎦
Leftmost
⎡ 1 2 −5 3 6 14 ⎤ nonzero column
⎢ ⎥ in the
⎢ 0 0 1 0 −7 −6 ⎥ Row
new sub matrix
⎢ 2 ⎥ Echelon
⎢ 0 0 0 0 1 2 ⎥⎦
Form
⎣
 Elimination procedure: Step 6
• Beginning with the last nonzero row and working upward, add
suitable multiples of each row to the rows above to introduce
zeros above the leading 1’s (由最後的非零列開始往上做，使得
⾸項1上⾯皆為0）
⎡ 1 2 −5 3 6 14 ⎤ ⎡ 1 2 −5 3 6 14 ⎤ 7/2 times the third row of the
⎢
⎢ 0 0 1 0 −7
⎥
⎢ ⎥
⎢ 2
−6 ⎥
⎥ ⎢ 0 0 1 0 0 1 ⎥ preceding matrix was added
⎢ 0 0 0 0 1 2 ⎥⎦
⎣ ⎢⎣ 0 0 0 0 1 2 ⎥⎦ to the second row

⎡ 1 2 −5 3 0 2 ⎤ -6 times the third row was

⎢ ⎥ added to the rst row
⎢ 0 0 1 0 0 1 ⎥
⎢⎣ 0 0 0 0 1 2 ⎥⎦

Reduced 5 times the second row was

Row ⎡ 1 2 0 3 0 7 ⎤ added to the rst row
Echelon ⎢ ⎥
Form
⎢ 0 0 1 0 0 1 ⎥
⎢⎣ 0 0 0 0 1 2 ⎥⎦

17
fi
fi
 Gauss-Jordan Elimination
• The procedure (algorithm) we have just described for reducing a
matrix to reduced row echelon form is called Gauss-Jordan
Elimination (⾼斯-喬丹消去法). This method consists of two parts

• A forward phase, in which zeros are introduced

below the leading 1’s

• A backward phase, in which zeros are introduced

above the leading 1’s

• If only the forward phase is used, then the procedure produces a

row echelon form only and is called Gauss Elimination

18
Solved by Gauss-Jordan Elimination(1/5)

x1 + 3x2 − 2x3 + 2x5 =0

2x1 + 6x2 − 5x3 − 2x4 + 4x5 − 3x6 = −1
5x3 + 10x4 + + 15x6 = 5
2x1 + 6x2 + 8x4 + 4x5 + 18x6 = 6

• The augmented matrix for the system is

⎡ 1 3 −2 0 2 0 0 ⎤
⎢ ⎥
⎢ 2 6 −5 −2 4 −3 −1 ⎥
⎢ 0 0 5 10 0 15 5 ⎥
⎢ 2 6 0 8 4 18 6 ⎥⎦
⎣
19
Solved by Gauss-Jordan Elimination(2/5)
• Adding -2 times the rst row to the second and fourth
rows gives
⎡ 1 3 −2 0 2 0 0 ⎤
⎢ ⎥
⎢ 0 0 −1 −2 0 −3 −1 ⎥
Step 4
⎢ 0 0 5 10 0 15 5 ⎥
⎢ 0 0 4 8 0 18 6 ⎥⎦
⎣

• Multiplying the second row by -1 and then adding -5

times the new second row to the third row and -4 times
the new second row to the fourth row gives
⎡ 1 3 −2 0 2 0 0 ⎤ Step 3
⎢ ⎥
⎢ 0 0 1 2 0 3 1 ⎥
⎢ 0 0 0 0 0 0 0 ⎥ Step 4
⎢ 0 0 0 0 0 6 2 ⎥
⎣ ⎦
20
fi
Solved by Gauss-Jordan Elimination(3/5)
• Interchanging the third and fourth rows and then
multiplying the third row of the resulting matrix by 1/6
gives the row echelon form

⎡ 1 3 −2 0 2 0 0 ⎤
⎢ ⎥
⎢ 0 0 1 2 0 3 1 ⎥ Step 2
Row
Echelon ⎢ 1 ⎥
⎢ 0 0 0 0 0 1
Form 3 ⎥ Step 3
⎢ ⎥
⎢⎣ 0 0 0 0 0 0 0 ⎥⎦

• This completes the forward phase since there are zeros

below the leading 1’s

21
Solved by Gauss-Jordan Elimination(4/5)
• Adding -3 times the third row to the second row and then
adding 2 times the second row of the resulting matrix to
the rst row yields the reduced row echelon form

⎡ 1 3 0 4 2 0 0 ⎤
⎢ ⎥
Reduced ⎢ 0 0 1 2 0 0 0 ⎥
Row ⎢ 1 ⎥
Echelon ⎢ 0 0 0 0 0 1 ⎥
Form 3
⎢ ⎥
⎢⎣ 0 0 0 0 0 0 0 ⎥⎦

• This completes the backward phase since there are

zeros above the leading 1’s

22
fi
Solved by Gauss-Jordan Elimination(5/5)
• The corresponding system of equations is
⎡ 1 3 0 4 2 0 0 ⎤ x1 + 3x2 +4x4 + 2x5 =0
⎢ ⎥
⎢ 0 0 1 2 0 0 0 ⎥
⎢ 1 ⎥ x3 + 2x4 =0
⎢ 0 0 0 0 0 1 ⎥
3
⎢ ⎥ 1
⎢⎣ 0 0 0 0 0 0 0 ⎥⎦ x6 =
3
• Solving for the leading variables we obtain
1
x1 = −3x2 − 4x4 − 2x5 , x3 = −2x4 , x6 =
3

• Finally, we express the general solution of the system

parametrically by assigning the free variables
1
x1 = −3r − 4s − 2t, x2 = r, x3 = −2s, x4 = s, x5 = t, x6 =
3
23
 Trivial Solution
• A system of linear equations is said to be homogeneous if
the constant terms are all zeros
a11 x1 + a12 x2 +!+ a1n xn = 0
a21 x1 + a22 x2 +!+ a2n xn = 0
!
am1 x1 + am 2 x2 +!+ amn xn = 0

• Every homogeneous system of linear equations is consistent

because all such systems have x1=0, x2=0, …, xn=0 as a
solution. This solution is called the trivial solution (明顯解)

• If there are other solutions, they are called nontrivial solutions

24
 Nontrivial solutions
• Because a homogeneous linear system always has the trivial solution,
there are only two possibilities for its solutions

• The system has only the trivial solution

• The system has in nitely many solutions

in addition to the trivial solution

• There is one case in which a homogeneous system is assured of having

nontrivial solutions, whenever the system involves more unknowns than
equations (未知數個數比⽅程式個數多時，⿑次線性⽅程組保證有非明顯解)

25
fi
Solve the homogeneous linear system

• Using Gauss-Jordan elimination to solve the system

x1 + 3x2 − 2x3 + 2x5 =0
2x1 + 6x2 − 5x3 − 2x4 + 4x5 − 3x6 = 0
5x3 + 10x4 + + 15x6 = 0
2x1 + 6x2 + 8x4 + 4x5 + 18x6 = 0
• Solution
The augmented matrix
⎡ 1 3 −2 0 2 0 0 ⎤
⎢ ⎥
⎢ 2 6 −5 −2 4 −3 0 ⎥
⎢ 0 0 5 10 0 15 0 ⎥
⎢ 2 6 0 8 4 18 0 ⎥
⎣ ⎦
26
 Solution (Continued)
• The reduced row echelon
⎡ 1 3 −2 0 2 0 0 ⎤ ⎡ 1 3 0 4 2 0 0 ⎤
⎢ ⎥ ⎢ ⎥
⎢ 2 6 −5 −2 4 −3 0 ⎥ ⎢ 0 0 1 2 0 0 0 ⎥
⎢ 0 0 5 10 0 15 0 ⎥ ⎢ 0 0 0 0 0 1 0 ⎥
⎢ 2 6 0 8 4 18 0 ⎥ ⎢ ⎥
⎣ ⎦ ⎣ 0 0 0 0 0 0 0 ⎦
• The corresponding system of equations is
x1 + 3x2 +4x4 + 2x5 =0

• Solving for the leading variables we obtain x3 + 2x4 =0

x1=-3x2-4x4-2x5 , x3=-2x4 , x6=0 x6 = 0

• Assigning the free variables x2, x4, and x5, we can express
the solution set parametrically
x1=-3r-4s-2t, x2=r, x3=-2s, x4=s, x5=t, x6=0
Note that the trivial solution results when r=s=t=0

27
More about homogeneous linear systems

• Considering a general homogeneous linear system with

n unknowns, and suppose that the reduced row
echelon form of the augmented matrix has r nonzero
rows, the homogeneous system corresponding to the
reduced row echelon form of the augmented matrix
must have r leading variables and n-r free variables.

X k1 + ∑( )=0
X k1 + ∑( ) = 0
...
X kr + ∑ ( )=0

∑( ) denotes a sum that involves the free variables

28
Theorems for Homogeneous System
Theorem 1.2.1
Free Variable Theorem for Homogeneous System

If a homogeneous linear system has n unknowns, and if

the reduced row echelon form of its augmented matrix has
r nonzero rows, then the system has (n-r) free variables

Theorem 1.2.2

A homogeneous linear system with more unknowns than

equations has in nitely many solutions

29
fi
Solved by Back-Substitution
• For large linear systems that require a computer solution, it’s
generally more e cient to use Gaussian elimination
(reduction to row echelon form) followed by a technique
known as back-substitution (倒回代換法)

• A row echelon form of the augmented matrix is

⎡ 1 3 −2 0 2 0 0 ⎤ x1 + 3x2 −2x3 + 2x5 =0
⎢ ⎥
⎢ 0 0 1 2 0 3 1 ⎥
x3 + 2x4 +3x6 = 1
⎢ 1 ⎥
⎢ 0 0 0 0 0 1
3 ⎥ 1
⎢ ⎥ x6 =
⎢⎣ 0 0 0 0 0 0 0 ⎥⎦ 3
to solve the corresponding system of equations

30
ffi
Back-Substitution: Step 1
• Solving the equations for the leading variables

x1 + 3x2 −2x3 + 2x5 =0 x1 = −3x2 +2x3 − 2x5

x3 + 2x4 +3x6 = 1 x3 = 1− 2x4 − 3x6

1 1
x6 = x6 =
3 3

31
Back-Substitution: Step 2
• Beginning with the bottom equation and working upward,
successively substitute each equation into all the
equations above it

• Substituting x6=1/3 into the second equation yield

x1 = −3x2 +2x3 − 2x5 x1 = −3x2 +2x3 − 2x5
x3 = 1− 2x4 − 3x6 x3 = −2x4
1 1
x6 = x6 =
3 3

• Substituting x3=-2x4 into the rst equation yields

x1 = −3x2 +2x3 − 2x5 x1 = −3x2 −4x4 − 2x5
x3 = −2x4 x3 = −2x4
1 1
x6 = x6 =
3 3
32
fi
Back-Substitution: Step 3
• If we now assign x2, x4, and x5 the arbitrary values r, s,
and t, respectively, the general solution is given by the
formulas

1
x1 = −3r − 4s − 2t, x2 = r , x3 = −2s , x4 = s, x5 = t , x6 =
3

33
More Examples for Back-Substitution
• These matrices are all in row echelon form but not reduced
roe echelon form. Discuss the existence and uniqueness of
solutions to the corresponding linear systems

(a) ⎡ 1 −3 7 2 5 ⎤ (b) ⎡ 1 −3 7 2 5 ⎤ (c) ⎡ 1 −3 7 2 5 ⎤

⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ 0 1 2 −4 1 ⎥ ⎢ 0 1 2 −4 1 ⎥ ⎢ 0 1 2 −4 1 ⎥
⎢ 0 0 1 6 9 ⎥ ⎢ 0 0 1 6 9 ⎥ ⎢ 0 0 1 6 9 ⎥
⎢ 0 0 0 0 1 ⎥⎦ ⎢ 0 0 0 0 0 ⎥⎦ ⎢ 0 0 0 1 0 ⎥⎦
⎣ ⎣ ⎣

• Solution (a): the system is inconsistent

• Solution (b): the system must have in nitely many solutions

• Solution (c): The system must have a unique solution

By back-substitution, x4=0, x3=9, x2=-17, x1=-109
fi
Some Facts about Echelon Forms

• Every matrix has an unique reduced row echelon form

(每⼀個矩陣有⼀個唯⼀的簡約列梯型)

• Row echelon forms are not unique (列梯型不唯⼀)

• The reduced row echelon form and all row echelon forms
of a matrix A have the same number of zero rows, and the
leading 1’s always occur in the same positions in the
row echelon forms of A. Those are called the pivot
positions (軸元位置) of A. The columns containing the
leading 1’s are called the pivot columns (軸元⾏) of A

35
Pivot Positions and Columns
⎡ 0 0 −2 0 7 12 ⎤
• A row echelon form of A = ⎢
⎢ 2 4 −10 6 12 28
⎥
⎥
⎢⎣ 2 4 −5 6 −5 −1 ⎥⎦
⎡ 1 2 −5 3 6 14 ⎤
to be ⎢ ⎥
⎢ 0 0 1 0 −7 −6 ⎥
⎢ 2 ⎥
⎢ 0 0 0 0 1 2 ⎥⎦
⎣

• The leading 1’s occur in positions (row 1, column 1), (row

2, column 3), and (row 3, column 5). These are the pivot
positions. The pivot columns are columns 1, 3, and 5

36
 Chapter 1.2
Concept Review
• Reduced row echelon form / row echelon form

• Leading 1 / leading variables / free variables

• General solution to a linear system

• Gaussian elimination / Gauss-Jordan elimination

• Forward phase / backward phase

• Trivial solution / nontrivial solution

• Back-substitution

37