Lecture 1: Linear Systems

Shenghao Yang
Spring 2023

1 Systems of Linear Equations

Examples of Linear and Nonlinear Equations

• Linear equations:
1. x + 3y − 2z = 8
2. 4x1 + 5x2 − x3 + x4 + x5 = 0
3. 9a − 2b + 7c + 2d = −7
• Nonlinear equations:
1. xy + 5yz = 13
2. x1 + x32 − x3 x4 x25 = 0
3. tan(a) + log(b) − cd = 0

Example: Input-Output Analysis

• A society’s economy has three industries: Wood, Rice and Labor.

• Each industry uses the inputs from the other two:
1 1
– 1 unit of Wood needs 2 units of Rice and 4 units of Labor.
1 1
– 1 unit of Rice needs 3 units of Wood and 2 units of Labor.
1 1
– 1 unit of Labor needs 8 units of Wood and 4 units of Rice.
• The outputs of each industry are consumed either as inputs to other industry or as the final demand.

• Let x1 , x2 , x3 be the outputs of Wood, Rice and Labor industries, respectively.

• Suppose that the final demands of the three industries are 0, 1 and 2, respectively.

• We have the following three equations:

1 1
x1 = x2 + x3
3 8
1 1
x2 = x1 + x3 + 1
2 4
1 1
x3 = x1 + x2 + 2
4 2

The LHS of each equation is the input of an industry and the RHS is the consumption. We will use
this example throughout this lecture. Harvard Professor Wassily Leontief first developed such a model
for analysis national/regional economies, and earned the 1973 Nobel Prize in Economics.

1
System of Linear Equations
A linear system of m equations in n unknowns (also called an m × n system) takes the following form:

a11 x1 + a12 x2 + a13 x3 + · · · + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + · · · + a2n xn = b2
..
.
am1 x1 + am2 x2 + am3 x3 + · · · + amn xn = bm

A solution of the system is an assignment of the variables (x1 , x2 , . . . , xn ) that satisfies all the equa-
tions.
The system of linear equations in the input-output example has three unknowns and three equations,
i.e., m = 3 and n = 3, and can be written in the form:
1 1
x1 − x2 − x3 = 0
3 8
1 1
− x1 + x2 − x3 = 1
2 4
1 1
− x1 − x2 + x3 = 2 (1)
4 2

2 Solution Set
We are interested in the following questions about a system of linear equations:
• Does a solution exist? A linear system with solution is called consistent.
• How many solutions does a linear system have?

• How to find the solutions?

Solution Set

• The set of all solutions of a linear system is called the solution set of the system.
• The solution set can either have 0, 1 or infinitely many elements (to be proved in the next lecture).

Thought considered trivial by many, a 1 × 1 system has some properties of a general one:

1 × 1 System
Consider a general 1 × 1 system:
ax = b
• a ̸= 0: exactly one solution x = b/a.
• a = 0, b ̸= 0: no solution.
• a = 0, b = 0: infinitely many solutions.

The solution set of these three cases can be denoted as {b/a}, ∅ and R, respectively.
Example 1. Consider the following three 2 × 2 systems:
The solution sets of these three examples are {(2, 0)}, ∅, {(x, 2 − x) : x ∈ R}, respectively.
In general, a system of linear equations always has either 0, 1 or infinitely many solutions. We will
prove the fact in the next lecture.

2
 x1 + x2 = 2 x1 + x2 = 2 x1 + x2 = 2
x1 − x2 = 2 x1 + x2 = 1 −x1 − x2 = −2

2 2 2

0 0 0

−2 −2 −2
0 1 2 0 1 2 0 1 2
(a) exactly one solution (b) no solution (c) many solutions

3 Equivalent Systems and Equation Operations

Equivalent Systems
Definition 1. Two systems of linear equations involving the same variables are said to be equivalent if
they have the same solution set.
Definition 2. Three equation operations for a linear system:

I. Swap the order of two equations.

II. Multiply by the same nonzero real number on both sides of an equation.
III. Add a multiple of one equation to another.

Proposition 1. If we apply one of the three equation operations to a system of linear equations, then
the original system and the transformed one are equivalent.
Check the equivalence of the following systems:
{ { { { {
x1 + x2 = 4 x1 + x2 = 4 x1 + x2 = 4 x2 = 1 x1 = 3
x1 − x2 = 2 2x1 = 6 x1 = 3 x1 = 3 x2 = 1

Remark 1. • Equation operation I can be obtained by combining equation operation II and equation
operation III.
• There are other equations operations that preserves the equivalence of the systems.
Proof. Let S and S ′ be the solution sets of the original system and the transformed system, respectively.

• When Operation I is applied, the set of equations are the same for both systems, and hence S = S ′ .
• When Operation II is applied, suppose that the first equation is multiplied by ξ ̸= 0 on both sides.
(If it is not the first equation, we can apply Operation I to change it to the first.) So the original
system and the transformed system are different only on the ith equation: The first equation of
the original system is
a11 x1 + a12 x2 + · · · + a1n xn = b1
and the ith equation of the transformed system is

ξa11 x1 + ξa12 x2 + · · · + ξa1n xn = ξb1 .

3
 For (β1 , . . . , βn ) ∈ S, we can verify that (β1 , . . . , βn ) satisfies all the 2nd to the last equations in
the transformed system as they are the same as the ones in the original ∑n system. As (β1 , . . . , βn )
satisfies the first equation in the original system, we know that j=1 aij βj = bi , which implies
∑n
ξ j=1 aij βj = ξbi , i.e., (β1 , . . . , βn ) also satisfies the first equation in the transformed system.
Therefore, (β1 , . . . , βn ) ∈ S ′ .
For (β1 , . . . , βn ) ∈ S ′ , we can verify that (β1 , . . . , βn ) satisfies all the 2nd to the last equations in the
original system as they are the same as the ones in the transformed ∑nsystem. As (β1 , . . . , βn ) satisfies
the first equation in the transformed system, we know that j=1 ξaij βj = ξbi , which implies
∑n
j=1 aij βj = bi by multiplying 1/ξ on both sides (which is feasible as ξ ̸= 0, i.e., (β1 , . . . , βn ) also
satisfies the first equation in the original system. Therefore, (β1 , . . . , βn ) ∈ S.
• (The proof of operation III is left as an exercise.)

How do we solve the first 2 × 2 system in the above example? Your approach can be summarized as
two “equation operations”:
• Add to one equation a multiple of another one.
• Multiply a nonzero number on both sides of a equation.
Now let’s see how to solve the 3 × 3 system (1).

An Example of Solving a 3 × 3 System

1 1
x1 = x2 + x3
3 8
1 1
x2 = x1 + x3 + 1
2 4
1 1
x3 = x1 + x2 + 2
4 2
• Adding multiples of the first equation into the other two equations, we obtain (after combining the
terms of the same variable)
5 5
x2 − x3 = 1
6 16
7 31
− x2 + x3 = 2
12 32
which is a 2 × 2 system.

• We see that x2 = 51 18
20 and x3 = 5 solves the above system. Substituting this solution to the
expression of x1 in the first equation, we get x1 = 17
8 .

The substitution of the first equation into two other equations is called forward substitution. The
substitution of x2 = 3 and x3 = 4 back to the first equation is called backward substitution.
A general idea to solve an n × n system is to first reduce it to a (n − 1) × (n − 1) system by forward
substitution, and then back substituting the solution of the (n−1)×(n−1) system to solve the remaining
variable, which forms a recursion. Eventually, the problem reduces to a 1 × 1 system.
Back to 1949, Leontief had a 500 × 500 system, which cannot be solved by any computers at that
time, so he managed to reduce the system to a 42 × 42 one, which takes the largest computer at that
time 56 hours to solve.

4
4 Augmented Matrix and Row Operations
Matrices

• A matrix is a rectangular array of numbers.

• For an m × n system of linear equations, the m × n matrix
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
 
 ... 
am1 am2 . . . amn

is called the coefficient matrix of the system.

• For an m × n system of linear equations, the m × (n + 1) matrix
 
a11 a12 · · · a1n b1
 a21 a22 · · · a2n b2 
 
 ... 
am1 am2 . . . amn bm

is called the augmented matrix of the system.

An augmented matrix has the complete information to specify a system of linear equations. The
augmented matrix of system (1) is  
1 − 12 − 18 0
 
 1 
− 2 1 − 14 1 .
 
− 14 − 12 1 2

A convention
We use Ri to denote the ith equation of a system of linear equations or the ith row in the corresponding
augmented matrix.
1 1
x1 − x2 − x3 = 0 (R1 )
3 8
1 1
− x1 + x2 − x3 = 1 (R2 )
2 4
1 1
− x1 − x2 + x3 = 2 (R3 )
4 2

Definition 3. Elementary Row Operations:

I. Interchange two rows.
II. Multiply a row by a nonzero real number.
III. Replace a row by its sum with a multiple of another row.

[ ] [ ] [ ] [ ]
2 2 4 R ↔R 1 2 3 R →0.5R 1 2 3 R →R −R 0 1 1
−−1−−−→
2
−−2−−−−−→
2
−−1−−−−
1
−−→
2

1 2 3 2 2 4 1 1 2 1 1 2

(We need only the second two essentially. For the convenience of further discussion, we keep the first
one.) These three row operations correspond to the three equation operations, respectively, and hence
are reversible.
Definition 4. Two matrices are said to be row equivalent if one can be obtained from the other by a
sequence of elementary row operations.

5
Theorem 1. Consider two linear systems. If the augmented matrix of one system can be obtained from
the one of the other system by a sequence of elementary row operations, then the two linear systems are
equivalent.
Example 2. Solve
[ ]
x1 + 2x2 = 5 1 2 5
4x1 + 5x2 = 14 4 5 14

• R2 → R2 − 4R1 : adding (−4) times R1 to R2 yields

[ ]
x1 + 2x2 = 5 1 2 5
−3x2 = −6 0 −3 −6

• R2 → − 13 R2 : multiplying (− 13 ) to R2 yields

[ ]
x1 + 2x2 = 5 1 2 5
x2 = 2 0 1 2

• R1 → R1 − 2R2 : adding (−2) times R2 to R1 gives

[ ]
x1 = 1 1 0 1
x2 = 2 0 1 2

The above process can be expressed by the following row operations:

[ ] [ ]
1 2 5 R2 →R2 −4R1 1 2 5
−−−−−−−−→
4 5 14 0 −3 −6
[ ]
R2 →− 31 R2 1 2 5
−−−−−−−→
0 1 2
[ ]
R →R −2R2 1 0 1
−−1−−−1−−−→ .
0 1 2