Mathematics for Economists

Gaussian elimination
Systems of linear equations

You already know how to solve two linear equations with two
variables:
a11 x1 + a12 x2 = b1
a21 x1 + a22 x2 = b2
where all aij and bi are constants, and x1 and x2 are variables.
For instance, the equations

2x1 + 3x2 = 6
2x1 + x2 = 4

have the solution (x1 , x2 ) = ( 32 , 1).

1
Systems of linear equations

The equations
2x1 + x2 = 6
2x1 + x2 = 4
have no solution at all (since 6 ̸= 4). Such a system is called
inconsistent.

2
Systems of linear equations

The equations
2x1 + x2 = 4
4x1 + 2x2 = 8
have many infinitely many solutions.

(x1 , x2 ) = (α, 4 − 2α) is a solution for any real number α.

3
Key operations

You can solve a system of linear equations using three key

operations.

K1. You can multiply any equation with any nonzero real
number.
K2. You can exchange any two equations (i.e., swap places).
K3. You can add any real number α multiplied by one
equation to another equation.

4
We are going to study the following general system of m linear
equations with n variables:

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
.. ..
. .
am1 x1 + am2 x2 + · · · + amn xn = bm

where all aij and bi are constants, x1 , . . . , xn are variables.

Questions:

(a) Does the system have a solution?

(b) If the system admits a solution, is the solution unique?
(c) If there are multiple solutions, how to find all of them?

5
 a11 x1 + a12 x2 + · · · + a1n xn = b1
a21 x1 + a22 x2 + · · · + a2n xn = b2
.. ..
. .
am1 x1 + am2 x2 + · · · + amn xn = bm
This system can be expressed as:

Ax = b (1)

where
     
a11 a12 · · · a1n x1 b1
a21 a22 · · · a2n x2 b2
     
     
A= .. .. .. , x =  .. , b =  .. 
. . ··· . . .
     
     
am1 am2 · · · amn xn bm

6
 Ax = b (1)
where
     
a11 a12 ··· a1n x1 b1
a21 a22 ··· a2n x2 b2
     
     
A= .. .. .. , x =  .. , b =  .. 
. . ··· . . .
     
     
am1 am2 ··· amn xn bm

• A is called the coefficient matrix, x the vector of

variables, and b the right-hand vector.
• If b is equal to an m-vector of zeros, the system (??) is
called homogeneous.

7
 Ax = b (1)
To solve the system (??), we introduce a new m × (n + 1)
matrix Ab which is constructed from A and b:
 
a11 a12 .... a1n b1
 a21 a22 .... a2n b2 
 
Ab =  .. .. .. .. 
. . .... . . 


am1 am2 .... amn bm

We call Ab the augmented matrix of system (??).

8
 Ax = b (1)
 
a11 a12 .... a1n b1
a21 a22 .... a2n b2 
 

Ab =  .. .. . .. 
. . .... .. . 
 

am1 am2 .... amn bm

• Recall the three key operations for solving a system of

linear equations.
• To solve system (??), we can translate the operations
into three row operations for the matrix Ab .
R1. You can multiply any row with any nonzero real number.
R2. You can exchange any two rows (swap places).
R3. You can add any scalar α multiplied by one row to
another row. 9
The row operations R1, R2 and R3 are sufficient for solving
any system of linear equations.
Exercise: Solve the following system of three linear equations
with three variables.

2x2 − x3 = −7
x1 + x2 + 3x3 = 2
−3x1 + 2x2 + 2x3 = −10

10
 2x2 − x3 = −7
x1 + x2 + 3x3 = 2
−3x1 + 2x2 + 2x3 = −10
Solution: First write down the augmented matrix:
 
0 2 −1 −7
Ab =  1 1 3 2 
 
−3 2 2 −10

11
  
1
First, we want the first column to look like this:  0 
 
0
 
0 2 −1 −7
Ab =  1 1 3 2 
 
−3 2 2 −10

Exchange the 1st and 2nd rows:

 
1 1 3 2
 0 2 −1 −7 
 
−3 2 2 −10

12
  
1 1 3 2
 0 2 −1 −7 
 
−3 2 2 −10
Add 3 times the 1st row to the 3rd row:
 
1 1 3 2
0 2 −1 −7
 
 
(−3 + 1 · 3) (2 + 1 · 3) (2 + 3 · 3) (−10 + 2 · 3)
 
1 1 3 2
=  0 2 −1 −7 
 
0 5 11 −4
Now we are done with the first column.

13
  
0
If possible, we now want the second column like this:  1 
 
0
 
1 1 3 2
 0 2 −1 −7 
 
0 5 11 −4

1
Multiply the 2nd row by :
2
 
1 1 3 2
 0 1 −1/2 −7/2 
 
0 5 11 −4

14
  
1 1 3 2
 0 1 −1/2 −7/2 
 
0 5 11 −4
Add −1 times the 2nd row to the 1st row:
 
1 0 7/2 11/2
 0 1 −1/2 −7/2 
 
0 5 11 −4

Add −5 times the 2nd row to the 3rd row:

 
1 0 7/2 11/2
 0 1 −1/2 −7/2 
 
0 0 27/2 27/2

Now we are done with the second column.

15
  
0
If possible, we now want the third column like this:  0 
 
1
 
1 0 7/2 11/2
 0 1 −1/2 −7/2 
 
0 0 27/2 27/2

2
Multiply the 3rd row by :
27
 
1 0 7/2 11/2
 0 1 −1/2 −7/2 
 
0 0 1 1

16
  
1 0 7/2 11/2
 0 1 −1/2 −7/2 
 
0 0 1 1
7
Add − times the 3rd row to the 1st row:
2
 
1 0 0 2
 0 1 −1/2 −7/2 
 
0 0 1 1

17
  
1 0 0 2
 0 1 −1/2 −7/2 
 
0 0 1 1
1
Finally, add times the 3rd row to the 2nd row:
2
 
1 0 0 2
 0 1 0 −3 
 
0 0 1 1

Translate this matrix back into a system of linear equations.

We get: x1 = 2, x2 = −3, x3 = 1

18
  
1 0 0 2
 0 1 0 −3 
 
0 0 1 1

• Note that the left side has become a 3 × 3 identity

matrix, denoted I.
• Using the three operations, we have rewritten the original
expression Ax = b into the form:

Ix = b ′
    
1 0 0 x1 2
 0 1 0  ·  x2  =  −3 
     
0 0 1 x3 1

19
  
1 0 0 2
 0 1 0 −3 
 
0 0 1 1

• Note that the left side has become a 3 × 3 identity

matrix, denoted I.
• Using the three operations, we have rewritten the original
expression Ax = b into the form:
Ix = b ′
x = b′
   
x1 2
 x2  =  −3 
   
x3 1
20
Exercise: Solve the following system of 4 linear equations
with 3 variables:

x1 + 3x2 − x3 = 4
2x1 + x2 + x3 = 7
2x1 − 4x2 + 4x3 = 6
3x1 + 4x2 = 11

Solution: First, write down the augmented matrix:

 
1 3 −1 4
 2 1 1 7 
Ab = 
 
 2 −4

4 6 
3 4 0 11

21
  
1 3 −1 4
 2 1 1 7 
Ab = 
 
 2 −4

4 6 
3 4 0 11
Add −2 times the 1st row to the 2nd row:
 
1 3 −1 4
 0 −5 3 −1 
 
 2 −4
 
4 6 
3 4 0 11
Add −2 times the 1st row to the 3rd row:
 
1 3 −1 4
 0 −5 3 −1 
 
 0 −10 6 −2 
 

3 4 0 11
22
  
1 3 −1 4

 0 −5 3 −1 

0 −10 6 −2 
 

3 4 0 11
We add −3 times 1st row to 4th row so we get
 
1 3 −1 4
 0 −5 3 −1 
 
 0 −10 6 −2 
 

0 −5 3 −1

Now we are done with the first column.

23
  
1 3 −1 4

 0 −5 3 −1 

0 −10 6 −2 
 

0 −5 3 −1
1
Multiply the 2nd row by − :
5
 
1 3 −1 4
 0
 1 −3/5 1/5 

 0 −10 6 −2 
 

0 −5 3 −1

24
  
1 3 −1 4
 0
 1 −3/5 1/5  
−10 6 −2 
 
 0
0 −5 3 −1
Add −3 times the 2nd row to the 1st row:
 
1 0 4/5 17/5
 0
 1 −3/5 1/5  
−10 −2 
 
 0 6
0 −5 3 −1
Add 10 times the 2nd row to the 3rd row:
 
1 0 4/5 17/5
 0
 1 −3/5 1/5  
 
 0 0 0 0 
0 −5 3 −1
25
  
1 0 4/5 17/5

 0 1 −3/5 1/5 
 
 0 0 0 0 
0 −5 3 −1
Add 5 times the 2nd row to the 4th row:
 
1 0 4/5 17/5
 0 1 −3/5 1/5 
 
 
 0 0 0 0 
0 0 0 0

26
  
1 0 4/5 17/5

 0 1 −3/5 1/5 
 
 0 0 0 0 
0 0 0 0
We are now done with the second column.

However, note that the last two equations have completely

disappeared. They are superfluous or redundant. Translate
the matrix back into a system of linear equations.
We have:
17 − 4x3
x1 =
5
1 + 3x3
x2 =
5
27
 17 − 4x3
x1 =
5
1 + 3x3
x2 =
5
x1 and x2 are only determined in terms of x3 . We can let
x3 = α be any real number and then write the set of all
solutions as:

 
17 − 4α 1 + 3α
(x1 , x2 , x3 ) = , , α , for any real number α.
5 5

We say that the solution set has one degree of freedom,

because one of the variables can be any real number.

28
Exercise: Solve the following system of 3 linear equations
with 4 variables:

x1 − 2x2 + x3 + x4 = 1
x1 + x2 − x3 = 1
x1 + 7x2 − 5x3 − 2x4 = −1

Solution: First, write down the augmented matrix:

 
1 −2 1 1 1
Ab =  1 1 −1 0 1 
 
1 7 −5 −2 −1

29
  
1 −2 1 1 1
Ab =  1 1 −1 0 1 
 
1 7 −5 −2 −1
Add −1 times the 1st row to the 2nd row:
 
1 −2 1 1 1
 0 3 −2 −1 0 
 
1 7 −5 −2 −1

Add −1 times the 1st row to the 3rd row:

 
1 −2 1 1 1
 0 3 −2 −1 0 
 
0 9 −6 −3 −2

Now we are done with the first column.

30
  
1 −2 1 1 1
 0 3 −2 −1 0 
 
0 9 −6 −3 −2
1
Multiply the 2nd row by :
3
 
1 −2 1 1 1
 0 1 −2/3 −1/3 0 
 
0 9 −6 −3 −2
Add 2 times the 2nd row to the 1st row:
 
1 0 −1/3 1/3 1
 0 1 −2/3 −1/3 0 
 
0 9 −6 −3 −2

31
  
1 0 −1/3 1/3 1
 0 1 −2/3 −1/3 0 
 
0 9 −6 −3 −2
Add −9 times the 2nd row to the 3rd row:
 
1 0 −1/3 1/3 1
 0 1 −2/3 −1/3 0 
 
0 0 0 0 −2

Now we are done with the second column.

32
  
1 0 −1/3 1/3 1
 0 1 −2/3 −1/3 0 
 
0 0 0 0 −2
Write down the equations above as:
x3 x4
x1 − + = 1
3 3
2x3 x4
x2 − − = 0
3 3
0 = −2

Clearly, the system has no solution since 0 ̸= −2. The system

is inconsistent.

33
Gaussian elimination

The method illustrated in these examples is called the

Gaussian elimination method, named after the German
mathematician Johann Carl Friedrich Gauss (1777—1855).
We summarise the method for solving system (??) as follows:

Apply the three row operations properly to the m × (n + 1)

augmented matrix Ab . In a finite number of steps, one of the
following three cases will occur:

34
Case 1

The first n rows and columns constitute an identity matrix:

 
1 0 0 ··· 0 0 β1
0 1 0 ··· 0 0 β2
 
 
 .. .. .. .. .. .. 
. . . . . .
 
 
 
 0 0 0 ··· 1 0 βn−1 
Ãb =  

 0 0 0 ··· 0 1 βn 

0 0 0 ··· 0 0 0
 
 
 .. .. .. .. .. .. 
. . . . . .
 
 
0 0 0 ··· 0 0 0
with m ≥ n.
• In Case 1, the system has a unique solution:
(x1 , x2 , . . . , xn ) = (β1 , β2 , . . . , βn )
35
Case 2

The resulting m × n matrix à has the form of a staircase with

1 as the first nonzero entry in every row.
• I.e., the 1st element of the first row is 1, the 2nd row has
1 as the first nonzero entry in the second or later position
if the 2nd row has a nonzero element.
• In general, the ith row (2 ≤ i ≤ m) has 1 as the first
nonzero entry in the ith or later position if the ith row
has a nonzero element.
The first nonzero entry (i.e., 1) of any row in à is called a
leading entry.

For every leading entry, every element above or below it in the

same column must be 0. 36
Case 2

That is, we have the following matrix:

 
1 0 ? 0 ··· 0 ? ··· ? 0 ?

 0 1 ? 0 ··· 0 ? ··· ? 0 ? 

0 0 0 1 ··· 0 ? ··· ? 0 ?
 
 

 .. .. .. .. .. .. .. .. .. 

 . . . . . . . . . 
 
(Ã, b̃) = 
 0 0 0 0 ··· 1 ? ··· ? 0 ? 


 0 0 0 0 ··· 0 0 ··· 0 1 ? 

0 0 0 0 ··· 0 0 ··· 0 0 0
 
 
.. .. .. .. .. .. .. .. ..
 
 
 . . . . . . . . . 
0 0 0 0 ··· 0 0 ··· 0 0 0

Here, ? could be zero or a nonzero number.

37
Case 2

In Case 2, the system has a general solution that expresses

the unknowns occurring as leading entries in terms of those
unknowns not occurring as leading entries.

• The later unknowns can be chosen to be any real

numbers.
• The number of these unknowns is the number of degrees
of freedom for the set of solutions.

38
Case 3

The m × n matrix à has the form of a staircase with 1 as the

first nonzero entry in every row.

• I.e., the 1st element of the first row is 1 and the 2nd row
has 1 as the first nonzero entry in the second or later
position if the 2nd row has a nonzero element.
• In general, the ith row (2 ≤ i ≤ m) has 1 as the first
nonzero entry in the ith or later position if the ith row
has a nonzero element.

For every leading entry, every element above or below it in the

same column must be 0.

39
Case 3

Case 3 resembles Case 2 with the exception that βj ̸= 0 for

some k + 1 ≤ j ≤ m:
 
1 0 ? ··· 0 ? ··· ? β1
? ··· ? ···
 
 0 1 0 ? β2 
.. .. .. .. .. .. ..
 
 
 . . . . . . . 
 
(Ã, b̃) = 
 0 0 0 ··· 1 ? ··· ? βk 

0 0 0 ··· 0 0 ··· 0 βk+1
 
 
 .. .. .. .. .. .. .. 
. . . . . . .
 
 
0 0 0 ··· 0 0 ··· 0 βm

• In Case 3, the system has no solution at all.

40