KYAMBOGO UNIVERSITY

DEPARTMENT OF MATHEMATICS AND

STATISTICS
SMT 113 LINEAR ALGEBRA.
Lecture Materials by Dr. Hasifa Nampala
Course outline

1. Matrices and systems of equations

2. Determinants

3. Vector spaces

4. Linear Transformation

5. Orthogonality

6. Eigenvalues and Eigenvectors

CHAPTER ONE
Matrices and Systems of Linear Equations

Detailed Content:
Matrix algebra, addition and multiplication of matrices,
types of matrices, partitioned matrices, rank and trace
of a matrix, solving systems of linear equations: elimi-
nation, matrix methods, elementary row operations, row
reduction methods, Cramer’s rule, Echelon form.

1
Objectives
By the end of this topic, you will be able to:

ˆ perform matrix arithmetic on matrices

ˆ perform elementary row operations on a matrix

ˆ reduce a matrix to its reduced row echelon form

ˆ solve a linear system of equations using elementary

row operations

ˆ apply linear algebra to practical problems.

e-References

ˆ Linear Algebra Demystified by David McMahon

ˆ Linear Algebra Problem book by Paul.R. Halmos

ˆ Linear Algebra, Schaum’s outline Fouth Edition

1 Matrices and Matrix Algebra

Summary of matrices and operations:

ˆ Definition of a matrix: can you define the term ma-

trix?
2
 ˆ Size of a matrix: how do write down the size (order)
of a matrix?
m,n
ˆ Notation of a matrix: A = [aij ]i,j=1 = [ai,j ]m×n

ˆ A matrix A = [ai,j ]m×n s.t n = m is called a square

matrix

ˆ Write down more than three matrices with different

orders.

Matrix Operations:

ˆ Addition (and indeed Subtraction):

Note: Two or more matrices can be added or sub-
tracted if they are of the same order. PROPOSI-
TION: (MATRIX ADDITION).
Suppose that A; B; C are m × n matrices. Suppose
further that O represents the m × n matrix with all
entries zero. Then
(a) A + B = B + A;
(b) A + (B + C) = (A + B) + C;
(c) A + O = A; and
(d) ∃ an m × n matrix A∗ such that A + A∗ = O.

3
 Proof:
Parts (a) - (c) are easy consequences of ordinary ad-
dition, as matrix addition is simply entry-wise addi-
tion. For part (d), we can consider the matrix A∗
obtained from A by multiplying each entry of A by
−1. ⊡
Exercise: Build up matrices A, B and C of your
choice and perform the above propositions.

ˆ Scalar Multiplication:
PROPOSITION: (MULTIPLICATION BY
SCALAR).
Suppose that A&B are m × n matrices, and that
c, d ∈ R. Suppose further that O represents the
m × n matrix with all entries zero. Then;
(a) c(A + B) = cA + cB.
(b) (c + d)A = cA + dA.
(c) 0A = O; and
(d) c(dA) = (cd)A.
Proof: These are all easy consequences of ordinary
multiplication, as multiplication by scalar c is simply
entry-wise multiplication by the number c.⊡
4
 EER: Think about any two matrices: A3×4; B3×4
and scalars c = 2; d = −3. Check out the above
propositions.

ˆ Matrix Multiplication:
To multiply any two or more matrices, always check
for compartibility first.

NOTE: With the exception of square matrices (nxn),

given any two matrices A and B, AB ̸= BA PROPO-
SITION: (ASSOCIATIVE LAW):
Suppose that A is an m × n matrix, B is an n × p
matrix and C is an p × r matrix.
Then A(BC) = (AB)C.
PROPOSITION: (DISTRIBUTIVE LAWS):
(a) Suppose that A is an m × n matrix and B and
C are n × p matrices.

5
 Then A(B +C) = AB +AC.
(b) Suppose that A and B are m × n matrices and
C is an n × p matrix.
Then (A+B)C = AC +BC.
PROPOSITION: Suppose that A is an m × n
matrix, B is an n × p matrix, and that c ∈ R. Then
c(AB) = (cA)B = A(cB).

Exercise:
Formulate the following matrices of your own and per-
form the above propositions:
A2×2; B2×3; C3×4
Discussion Exercise:

1. Define the following terms as applied to Matrices

ˆ Transpose of a matrix, AT .

ˆ An inverse Matrix

ˆ A diagonal matrix

ˆ A Symmetric matrix

ˆ A skew symmetric matrix.

6
 ˆ Rank of a matrix

ˆ Trace of a matrix

2. Show that the product of two diagonal matrices is

a diagonal matrix. State the rule for multiplying
diagonal matrices

3. Prove that the inverse of an invertible matrix is unique.

 
3 4
4. If A is an invertible matrix whose inverse is  .
5 6
Find A.
 
−1 2
5. If B is invertible and the inverse of 7B is  .
4 −7
Find B.

2 Linear Equations

Linear Equation of one variable

Consider a linear equation Ax = b; if
(a). a ̸= 0 then x = b/a is a Unique solution to the linear
first degree polynomial
(b). a = 0 but b ̸= 0 then the equation has no solution
(c). a = 0 and b = 0 then every scalar k is a solution.
7
Linear Equation of two variables
Example 1.1.1
Try to draw the two lines

3x + 2y = 5;
6x + 4y = 5 :

It is easy to see that the two lines are parallel and do not
intersect, so that this system of two linear equations has
no solution.
Example 1.1.2.
Try to draw the two lines

3x + 2y = 5;
x+y =2:

It is easy to see that the two lines are not parallel and
intersect at the point (1; 1), so that this system of two
linear equations has exactly one solution.

Example 1.1.3.
8
Try to draw the two lines

3x + 2y = 5;
6x + 4y = 10 :

It is easy to see that the two lines overlap completely,

so that this system of two linear equations has infinitely
many solutions.
In these three examples, we have shown that a system of
two linear equations on the plane R2 may have no solu-
tion, one solution or infinitely many solutions. A natural
question to ask is whether there can be any other con-
clusion. Well, we can see geometrically that two lines
cannot intersect at more than one point without overlap-
ping completely. Hence there can be no other conclusion.

Generally
Consider a system of two nondegenerate linear equations
in two unknowns x and y, which can be put in the stan-

9
dard form

k1x + c1y = b1
k2x + c2y = b2

Then;
1. The system has exactly one solution iff
k1 c1
̸=
k2 c2
2. The system has no solution iff
k1 c1 b1
= ̸=
k2 c2 b2
3. The system has infinitely many solutions iff
k1 c1 b1
= =
k2 c2 b2
In general, we shall study a system of m linear equations
of the form

a11x1 + a12x2 + · · · + a1nxn = b1;

a21x1 + a22x2 + · · · + a2nxn = b2;
... ... ... = ...

am1x1 + am2x2 + · · · + amnxn = bm;

(1)

10
with n variables x1, x2, · · · , xn.
In Equation (1), we may not be able to see geometrically
what is going on (unlike the two dimensional case). We
therefore need to study the problem from a more algebraic
viewpoint.
If we omit reference to the variables, then system (1) can
be represented by the array
 
a11 a12 · · · a1n|b1
 
a
 21 a22 · · · a2n|b2 
(2)

 . ... ... ... 
 .. 
 
am1 am2 · · · amn|bm
of all the coefficients. This is known as the augmented
matrix of the system. Here the first row of the array
represents the first linear equation, and so on.
We also write Ax = b, where
   
a11 a12 · · · a1n b1
   
 21 a22 · · · a2n 
a  b 
 2
A= . ... . . . ...  and b =  . 
 ..   .. 
   
am1 am2 · · · amn bm

11
  
x1
 
x 
 2
and x =  . 
 .. 
 
xn
A represent the coefficient matrix, b constant term and
X represents the variables.
Example 1.1.4.
The array
 
1 3 1 5 1|5
 
0 1 1 2 1|4
 
2 4 0 7 1|3
represents the system of 3 linear equations.
(Can you write down the equations? )
Note that the system can also be written as,
 
x
   1  
1 3 1 5 1   x2 
 5
    
0 1 1 2 1 x3 = 4
    
 
2 4 0 7 1   x4 
 3
x5
A solution to the system (1) is a list of values for the
12
unknown or a vector v = (k1, k2, k3, ....kn) such that x1 =
k1, x2 = k2, ...xn = kn. In that case, if v is a solution
vector, then it satisifies the equation, for example, a11k1 +
a12k2 + · · · + a1nkn = b1

Theorem 1. A system of finite linear equations has

(i) a unique solution, (ii) no solution, or (iii) an in-
finite number of solutions

A linear equation is said to be degenerate if all the co-

efficients are zero, ie 0x1 + 0x2 + 0x3 + ... + 0xn = b. The
solution of such an equation depends only on the value
of the constantb, ie if b ̸= 0, then the equation has no
solution but if b = 0 then every solution vector v is a
solution.
Linear Combination of equations
Consider system of equations (1), let Q be a system ob-
tained by multiplying (1) by a constants c1, c2, c3...cm
then Q is given by

(c1a11+c2a21+...+cmam1)x1+...+(c1a1n+c2a2n+...++cmamn)xn

. Then Q is called a linear combination of equation

(1).
13
The solution to equation (1) is also a solution to Q.

Theorem 2. Two systems of linear equations have

the same solutions if and only if each equation in each
system is a linear combination of the equations in the
other system.

3 Elementary row Operations

Consider a system of equations with rows numbers L1, L2, ..., Lm,
then, the following operations on a system of linear equa-
tions are called elementary operations.
1. Interchange two of the equations, ie Li → Lj
2. Replace an equation by a nonzero multiple of itself ie
kLi → Li
3. Replace an equation by the sum of a multiple of an-
other equation and itself kLi + Lj → Lj . Also kLi +
nLj → Lj applies.

Theorem 3. Suppose a system Q of linear equations

is obtained from a system S of linear equations by a
finite sequence of elementary operations. Then Q and
S have the same solution.

14
Let us continue with Example 1.1.4.
Example 1.2.1.
Let us interchange the first and second rows of the array
to obtain  
0 1 1 2 1|4
 
1 3 1 5 1|5
 
2 4 0 7 1|3
Then this represents the system of equations

x2 + x3 + 2x4 + x5 = 4;
x1 + 3x2 + x3 + 5x4 + x5 = 5;
2x1 + 4x2 + 7x4 + x5 = 3;
(3)

essentially the same as the system of Example 1.1.4, the

only difference being that the first and second equations
have been interchanged. Any solution of the system of
Example 1.1.4 is a solution of the system (3), and vice
versa.

Example 1.2.2.
Let us add 2 times the second row to the first row of the

15
array of Example 1.1.4 to obtain
 
1 5 3 9 3|13
 
0 1 1 2 1|4 
 
2 4 0 7 1|3
This then represents the system of equations

x1 + 5x2 + 3x3 + 9x4 + 3x5 = 13;

x2 + x3 + 2x4 + x5 = 4;
2x1 + 4x2 + 7x4 + x5 = 3;
(4)

essentially the same as the system of Example 1.1.4, the

only difference being that we have added 2 times the sec-
ond equation to the first equation. Any solution of the
system of Example 1.1.4 is a solution of the system (4),
and vice versa.

Example 1.2.3.
Let us now multiply the second row of the array of Ex-
ample 1.1.4 by 2 to obtain
 
1 3 1 5 1|5
 
0 2 2 4 2|8
 
2 4 0 7 1|3
16
This then represents the system of equations

x1 + 3x2 + x3 + 5x4 + x5 = 5;
2x2 + 2x3 + 4x4 + 2x5 = 8;
2x1 + 4x2 + 7x4 + x5 = 3;
(5)

essentially the same as the system of Example 1.1.4, the

only difference being that the second equation has been
multiplied through by 2. Any solution of the system of
Example 1.1.4 is a solution of the system (5), and vice
versa.

4 Row Echelon Form

Triangular Form
Consider the following system of equation

2x1 + 3x2 − 5x3 + 7x4 = 10

−x2 + 5x3 − 5x4 = −4
9x3 + 3x4 = 5
x4 = −1
17
The solution to this system is attained with backword
substitution.
Definition:
A system, is said to be in Echelon form if no equation
is degenerate and the leading unknown in each equation
other than the first is to the right of the leading unknown
in the preceding equation. E.g

2x1 + 3x2 − 5x3 + 7x4 + x5 = 10

5x3 − 5x4 + 3x5 = −4
9x4 + 3x5 = 5

The leading unknowns in the system, x1, x3, x4, are called
pivot variables, and the other unknowns, x2 and x5, are
called free variables.

Definition:
A rectangular array of numbers is said to be in row
echelon form if the following conditions are satisfied:
(1) The left-most non-zero entry of any non-zero row has
value 1. These are called the pivot entries.
(2) All zero rows are grouped together at the bottom of
18
the array.
(3) The pivot entry of a non-zero row occurring lower in
the array is to the right of the pivot entry of a non-zero
row occurring higher in the array.

A system of linear equations in echelon form, may have

say with n equations in m unknowns. If m = n then the
system has unique solution, otherwise, when n < m ie,
more unknowns than equations, then, we either
i. assign arbitrary unknowns to m − n free variables and
solve uniquely for the n pivot variables (Parameteric
form).
OR
ii. Use back-substitution to solve for the pivot variables
directly in terms of the free variables (Free-variables
form)

Next, we investigate how we may reduce a given array to

row echelon form.
PROPOSITION 1A. (ELEMENTARY ROW
OPERATIONS)

19
Consider the array (2) corresponding to the system (1).
(a) Interchanging the ith and j th rows of (2) corre-
sponds to interchanging the ith and j th equations in
(1).
(b) Adding a multiple of the ith row of (2) to the j th
row corresponds to adding the same multiple of the ith
equation in (1) to the j th equation.
(c) Multiplying the ith row of (2) by a non-zero con-
stant corresponds to multiplying the ith equation in
(1) by the same non-zero constant.
In all three cases, the collection of solutions to the system
(1) remains unchanged.

Let us investigate how we may use elementary row oper-

ations to help us solve a system of linear equations.

4.1 Gaussian elimination

This is one of the main methods used to solve systems

of equations. It involves two steps that include Forward
and Backword elimination
Forward eliminataion involves reducing the system

20
to either yield a degenerate equation with no solution
or an equivalent simpler system in triangular or echelon
form.
Backword eliminataion which is a back-substitution
to find the solution of the simpler system.
It involves the following steps
1. Find the first unknown in the system with a nonzero
coefficient (maybe x1)
2. If the first unknown (say x1) has zero coefficient in
the first equation, interchange with another equation so
that the first unknown (say x1) appears with a nonzero
coefficient in the first equation.
3. Use x1 as a pivot variable with coefficeint say (a11)
and eliminate x1 from all other equations except the first
one.
4. Examine each new equation that do not have x1. If any
equations takes the form 0x1 + 0x2 + ... + 0xn = b ∀b ̸= 0
then the sytem has no solution so stop the computation.
Otherwise, if b = 0 or if any equation is a product of the
other, then delete that equation from the system.
Example:

21
Solve the system using Gaussian elimination

x − 3y − 2z = 6
2x − 4y − 3z = 8
−3x + 6y + 8z = −5

−2L1 + L2 → L2 and 3L1 + L3 → L3

x − 3y − 2z = 6
2y + z = −4
−3y + 2z = 13

3/2L2 + L3 → L3

x − 3y − 2z = 6
2y + z = −4
7z = 14

By backword substitution we have z = 2, y = −3 and

x=1
Example

22
Solve

x + 2y − 3z = 1
2x + 5y − 8z = 4
3x + 8y − 13z = 7

−2L1 + L2 → L2 and −3L1 + L3 → L3

x + 2y − 3z = 1
y − 2z = 2
2y − 4z = 4

Since the third equation is a multiple of the second equa-

tion, it can be deleted to have

x + 2y − 3z = 1
y − 2z = 2

Note: We have more variables than the number of equa-

tions,
The system is a echelon form
we therefore make use of the free variable z, thus let z = k
then y = 2 − 2k and x = −3 − k. Therefore the system
has infinitely many solution given by the solution vector

23
U = (−3 − k, 2 − 2k, k) ∀k ∈ R.

Let us continue again with Example 1.1.4.

Example 1.2.4.
Consider again the system of linear equations

x1 + 3x2 + x3 + 5x4 + x5 = 5;
x2 + x3 + 2x4 + x5 = 4;
2x1 + 4x2 + 7x4 + x5 = 3;
(6)

represented by the array

 
1 3 1 5 1|5
 
0 1 1 2 1|4
  (7)
2 4 0 7 1|3
Let us now perform elementary row operations on the
array (7). At this point, do not worry if you do not un-
derstand why we are taking the following steps. Adding
−2 times the first row of (7) to the third row, we obtain
 
1 3 1 5 1| 5
 
0 1 1 2 1| 4 
 
0 −2 −2 −3 −1| − 7
24
From here, we add 2 times the second row to the third
row to obtain  
1 3 1 5 1|5
 
0 1 1 2 1|4
  (8)
0 0 0 1 1|1
Next, we add −3 times the second row to the first row to
obtain  
1 0 −2 −1 −2| − 7
 
0 1 1 2 1|4 
 
0 0 0 1 1|1
Next, we add the third row to the first row to obtain
 
1 0 −2 0 −1| − 6
 
0 1 1 2 1|4 
 
0 0 0 1 1|1
Finally, we add −2 times the third to row to the second
row to obtain
 
1 0 −2 0 −1| − 6
 
0 1 1 0 −1| 2 
  (9)
0 0 0 1 1| 1
We remark here that the array (8) is said to be in row ech-
elon form, while the array (9) is said to be in reduced row
25
echelon form, precise definitions will follow later. Let us
see how we may solve the system (6) by using the arrays
(8) or (9). First consider (8). Note that this represents
the system

x1 + 3x2 + x3 + 5x4 + x5 = 5;
x2 + x3 + 2x4 + x5 = 4;
x4 + x5 = 1;
(10)

First of all, take the third equation

x4 + x5 = 1.

If we let x5 = t, then x4 = 1 − t. Substituting these

into the second equation, we obtain (you must do the
calculation here)

x2 + x3 = 2 + t.

If we let x3 = s, then x2 = 2 + t − s. Substituting all

these into the first equation, we obtain (you must do the
calculation here)

x1 = −6 + t + 2s.
26
Hence

x = (x1, x2, x3, x4, x5) = (−6+t+2s, 2+t−s, s, 1−t, t)

is a solution of the system (10) for every s, t ∈ R. In

view of Proposition 1A, these are also precisely the
solutions of the system (6).
Alternatively, consider (9) instead. Note that this repre-
sents the system

x1 − 2x3 − x5 = −6;
x2 + x3 − x5 = 2;
x4 + x5 = 1;
(11)

First of all, take the third equation

x4 + x5 = 1.

If we let x5 = t, then x4 = 1 − t. Substituting these

into the second equation, we obtain (you must do the
calculation here)

x2 + x3 = 2 + t.

If we let x3 = s, then x2 = 2 + t − s. Substituting all

these into the first equation, we obtain (you must do the
27
calculation here)

x1 = −6 + t + 2s.

Hence

x = (x1, x2, x3, x4, x5) = (−6+t+2s, 2+t−s, s, 1−t, t)

is a solution of the system (11) for every s, t ∈ R. In

view of Proposition 1A, these are also precisely the
solutions of the system (6). However, if you have done
the calculations as suggested, you will notice that the cal-
culation is easier for the system (11) than for the system
(10). This is clearly a case of the array (10) in reduced
row echelon form having more 0’s than the array (9) in
row echelon form, so that the system (11) has fewer non–
zero coefficients than the system (10).

4.2 Reduced Row Echelon Form

Definition:
A rectangular array of numbers is said to be in reduced
row echelon form if the following conditions are satisfied:
(1) The left-most non-zero entry of any non-zero row has
28
value 1. These are called the pivot entries.
(2) All zero rows are grouped together at the bottom of
the array.
(3) The pivot entry of a non-zero row occurring lower in
the array is to the right of the pivot entry of a non-zero
row occurring higher in the array.
(4) Each column containing a pivot entry has 0’s every-
where else in the column.
We now investigate how we may reduce a given array to
reduced row echelon form. Here, we basically take an
extra step to convert an array from row echelon form to
reduced row echelon form. We shall illustrate the ideas
by continuing on an earlier example.
Example 1.4.1:
Consider  
0 0 5 0 15 5
 
0 2 4 7 1 3
.
 

0 1 2 3 0 1
 
0 1 2 4 1 2

29
This can be reduced to row echelon form
 
0 1 2 4 1 2
 
0 0 1 0 3 1
.
 

0 0 0 1 1 1
 
0 0 0 0 0 0
Step 1: Cover all zero rows at the bottom of the array.
We now have
 
0 1 2 4 1 2
 
0 0 1 0 3 1
.
 

0 0 0 1 1 1
 
× × × × × ×
Step 2: We now try to make all the entries above the
pivot entry on the bottom row zero (here again we do not
count any covered rows). This can be achieved by adding
suitable multiples of the bottom row to the other rows.
So let us add −4 times row 3 to row 1 to obtain
 
0 1 2 0 −3 −2
 
0 0 1 0 3 1
.
 

0 0 0 1 1 1
 
× × × × × ×

30
Step 3: Now cover the bottom row. We then obtain
 
0 1 2 0 −3 −2
 
0 0 1 0 3 1 
.
 

× × × × × × 
 
× × × × × ×
Step 4: Repeat Steps 2–3 on the uncovered array, and
as many times as necessary so that eventually the whole
array gets covered. So let us continue. Following Step 2,
we add −2 times row 2 to row 1 to obtain
 
0 1 0 0 −9 −4
 
0 0 1 0 3 1
.
 

× × × × × ×
 
× × × × × ×
Following Step 3, we cover row 2 to obtain
 
0 1 0 0 −9 −4
 
× × × × × × 
.
 

× × × × × × 
 
× × × × × ×

31
Following Step 2, we do nothing!! Following Step 3, we
cover row 1 to obtain
 
× × × × × ×
 
× × × × × ×
.
 

× × × × × ×
 
× × × × × ×
Step ∞ :
Uncover everything! We then have
 
0 1 0 0 −9 −4
 
0 0 1 0 3 1
.
 

0 0 0 1 1 1
 
0 0 0 0 0 0
in reduced row echelon form.
Again, in practice, we do not actually cover any entries of
the array, so repeat the same argument without covering
anything!!
The method described above is called the Gauss-Jordan
elimination.

32
5 Solving a System of Linear Equations

Let us first summarize what we have done so far. We

study a system (1) of m linear equations in n variables
x1, · · · , xn. If we omit reference to the variables, then
the system (1) can be represented by the array (2), with
m rows and n + 1 columns. We next reduce the array
(2) to row echelon form or reduced row echelon form by
elementary row operations.
By Proposition 1A, the system of linear equations
represented by the array in row echelon form or reduced
row echelon form has the same solution set as the system
(1). It follows that to solve the system (1), it remains to
solve the system represented by the array in row echelon
form or reduced row echelon form. We now describe a
simple way to obtain all solutions of this system.
Definition: Any column of an array (2) in row echelon
form or reduced row echelon form containing a pivot entry
is called a pivot column.
First of all, let us eliminate the situation when the system
has no solutions. Suppose that the array (2) has been

33
reduced to row echelon form, and that this contains a
row of the form

0| ·{z
·· 0} 1
n of them

corresponding to the last column of the array being a

pivot column. This row represents the equation

0x1 + · · · + 0xn = 1,

clearly the system cannot have any solution.

Definition: Suppose that the array (2) in row echelon
form or reduced row echelon form satisfies the condition
that its last column is not a pivot column. Then any vari-
able xi corresponding to a pivot column is called a pivot
variable. All other variables are called free variables.
Example 1.5.1:
Consider the array
 
0 1 0 0 −9 | − 4
 
0 0 1 0 3 |1 
.
 

0
 0 0 1 1 |1 

0 0 0 0 0 |0

34
representing the system

x2 − 9x5 = −4;
x3 + 3x5 = 1;
x4 + x5 = 1

Note that the zero row in the array represents an equa-

tion which is trivial! Here the last column of the array is
not a pivot column. Now columns 2, 3, 4 are the pivot
columns, so that x2, x3, x4 are the pivot variables and
x1, x5 are the free variables.
To solve the system, we allow the free variables to take
any values we choose, and then solve for the pivot vari-
ables in terms of the values of these free variables.
Example 1.5.2:

35
Consider the system of 4 linear equations

5x3 + 15x5 = 5;
2x2 + 4x3 + 7x4 + x5 = 3;
x2 + 2x3 + 3x4 = 1;
x2 + 2x3 + 4x4 + x5 = 2;
(12)

in the 5 variables x1, x2, x3, x4, x5.

If we omit reference to the variables, then the system can
be represented by the array
 
0 0 5 0 15|5
 
0 2 4 7 1|3 
(13)
 
 
0 1 2 3 0|1 
 
0 1 2 4 1|2
As in Example 1.3.1, we can reduce the array (13) to row
echelon form  
0 1 2 4 1|2
 
0 0 1 0 3|1
(14)
 
 
0 0 0 1 1|1
 
0 0 0 0 0|0

36
representing the system

x2 + 2x3 + 4x4 + x5 = 2;
x3 + 3x5 = 1;
x4 + x5 = 1;
(15)

Alternatively, we can reduce the array (13) to reduced

row echelon form
 
0 1 0 0 −9| − 4
 
0 0 1 0 3| 1 
(16)
 
 
0 0 0 1 1| 1 
 
0 0 0 0 0| 0
representing the system

x2 − 9x5 = −4;
x3 + 3x5 = 1;
x4 + x5 = 1;
(17)

By Proposition 1A, the three systems (12), (15) and

(17) have exactly the same solution set. Note that x2, x3, x4
are the pivot variables and x1, x5 are the free variables.
37
So we assign values x1 = s and x5 = t, then we have,
from (15) (harder) or (17) (easier), that

(x1, x2, x3, x4, x5) = (s, 9t−4, −3t+1, −t+1, t); s, t ∈ R

EER:
Solve the system

2x1 + x2 + 3x3 + 2x4 = 5;

x1 + 3x2 + 2x3 + 4x4 = 1;
3x1 + 2x2 = 2;

The brains behind the solution methods described above

are:

38
6 Homogeneous Systems

Consider a homogeneous system of m linear equations of

the form

a11x1 + a12x2 + · · · + a1nxn = 0;

a21x1 + a22x2 + · · · + a2nxn = 0;
... ... ... = ...

am1x1 + am2x2 + · · · + amnxn = 0;

(18)

with n variables x1, x2, · · · , xn. If we omit reference to

the variables, then system (18) can be represented by the
array  
a11 a12 · · · a1n|0
 
 21 a22 · · · a2n|0 
a 
 . ... . . . ...  (19)
 .. 
 
am1 am2 · · · amn|0
of all the coefficients. Note that the system (18) always
has a solution, namely the trivial solution

x1 = x2 = · · · = xn = 0.

39
Indeed, if we reduce the array (19) to row echelon form
or reduced row echelon form, then it is not difficult to see
that the last column is a zero column and so cannot be
a pivot column.
On the other hand, if the system (18) has a non-trivial so-
lution, then we can multiply this solution by any non-zero
real number different from 1 to obtain another non-trivial
solution. We have therefore proved the following simple
result.
PROPOSITION 1B:
The homogeneous system (18) either has the trivial
solution as its only solution or has infinitely many
solutions.
The purpose of this section is to discuss the following
stronger result.
PROPOSITION 1C:
Suppose that the system (18) has more variables than
equations; in other words, suppose that n > m. Then
there are infinitely many solutions.
To see this, let us consider the array (19) representing the
system (18). Note that (19) has m rows, corresponding
40
to the number of equations. Also (19) has n + 1 columns,
where n is the number of variables. However, the col-
umn of (19) on the extreme right is a zero column, cor-
responding to the fact that the system is homogeneous.
Furthermore, this column remains a zero column if we
perform elementary row operations on the array (19). If
we now reduce (19) to row echelon form by elementary
row operations, then there are at most m pivot columns,
since there are only m equations in (18) and m rows in
(19). It follows that if we exclude the zero column on the
extreme right, then the remaining n columns cannot all
be pivot columns. Hence at least one of the variables is
a free variable. By assigning this free variable arbitrary
real values, we end up with infinitely many solutions for
the system (18).

7 Application to Network Flow

Systems of linear equations arise when we investigate the

flow of some quantity through a network. Such networks
arise in science, engineering and economics. Two such

41
examples are the pattern of traffic flow through a city
and distribution of products from manufacturers to con-
sumers through a network of wholesalers and retailers.
A network consists of a set of points, called the nodes, and
directed lines connecting some or all of the nodes. The
flow is indicated by a number or a variable. We observe
the following basic assumptions:

ˆ The total flow into a node is equal to the total flow

out of a node.

ˆ The total flow into the network is equal to the total

flow out of the network.

Example:
The picture below represents a system of one way streets
in a particular part of some city and the traffic flow along
the streets between the junctions:

We first equate the total flow into each node with the
total flow out of the same node:

node A: 200 + x3 = x1 + x2,

node B: 200 + x2 = 300 + x4,
node C: 400 + x5 = 300 + x3,
42
 Figure 1: Traffic Flow 1

node D: 500 + x4 = 300 + x5.

We then equate the total flow into and out of the network:

400 + 200 + 200 + 500 = 300 + 300 + x1 + 300

These give rise to a system of 5 linear equations:

x1 + x2 − x3 = 200;
x2 − x4 = 100;
x3 − x5 = 100;
x4 − x5 = −200;
x1 = 400;
(20)

43
From which you can find the general solution (x1, · · · , x5) =
(400, t − 100, t + 100, t − 200, t), where t ≥ 200.
EER:
The picture below represents the quantities of a partic-
ular product that flow from manufacturers M1, M2, M3,
through wholesalers W1, W2, W3 and retailers R1, R2, R3, R4,
to consumers:

Figure 2: Goods Flow

Equating the total flow into each node with the total flow
out of the same node:

node W1 : 200 + x1 = x4 + x5,

44
 node W2 : 300 + x2 = 300 + x6,
node W3 : x3 = 100 + x7,
node R1 : x4 = 400,
node R2 : 300 + x5 = x8,
node R3 : 100 + x6 = 200,
node R4 : x7 = 500.

We then equate the total flow into and out of the network:

200 + x1 + x2 + 300 + x3 = 400 + x8 + 200 + 500.

Where the general solution (x1, · · · , x8) = (t−100, 100, 600, 400, t−
300, 100, 500, t), where t ≥ 300.

8 Application to Electrical Networks

A simple electric circuit consists of two basic components,

electrical sources where the electrical potential V is mea-
sured in volts (V), and resistors where the resistance R is
measured in ohms (Ω). We are interested in determining
the current I measured in amperes (A).
The electrical potential between two points is sometimes
called the voltage drop between these two points. Cur-
rents and voltage drops can be positive or negative. The
45
current flow in an electrical circuit is governed by three
basic rules:

ˆ Ohm’s law: The voltage drop E across a resistor

with resistance R with a current I passing through it
is given by V = IR.

ˆ Current law: The sum of the currents flowing into

any point is the same as the sum of the currents
flowing out of the point.

ˆ Voltage law: The sum of the voltage drops around

any closed loop is equal to zero.

Around any loop, we select a positive direction – clock-

wise or anticlockwise as we see fit. We have the following
convention:

ˆ The voltage drop across a resistor is taken to be posi-

tive if the current flows in the positive direction of the
loop, and negative if the current flows in the negative
direction of the loop.

ˆ The voltage drop across an electrical source is taken

46
 to be positive if the positive direction of the loop is
from + to −, and negative if the positive direction
of the loop is from − to +.

Example 1.8.1:
Consider the electric circuit shown in the diagram below:

Figure 3: Electric Flow 1

We wish to determine the currents I1, I2 and I3. Applying

the Current law to the point A, we obtain I1 = I2 + I3.
Applying the Current law to the point B, we obtain the
same. Hence we have the linear equation
I1 − I2 − I3 = 0.
Next, let us consider the left hand loop, and let us take
47
the positive direction to be clockwise. By Ohm’s law,
the voltage drop across the 8Ω resistor is 8I1, while the
voltage drop across the 4Ω resistor is 4I2. On the other
hand, the voltage drop across the 20V electrical source is
negative, since the positive direction of the loop is from
− to +. The Voltage law applied to this loop now gives
8I1 + 4I2 − 20 = 0, and we have the linear equation

8I1 + 4I2 = 20, or 2I1 + I2 = 5.

Next, let us consider the right hand loop, and let us take
the positive direction to be clockwise. By Ohm’s law,
the voltage drop across the 20Ω resistor is 20I3, while the
voltage drop across the 4Ω resistor is −4I2. On the other
hand, the voltage drop across the 16V electrical source is
negative, since the positive direction of the loop is from
− to +. The Voltage law applied to this loop now gives
20I3 − 4I2 − 16 = 0, and we have the linear equation

4I2 − 20I3 = −16, or I2 − 5I3 = −4.

48
We now have a system of three linear equations

I1 − I2 − I3 = 0;
2I1 + I2 = 5;
I2 − 5I3 = −4.
(21)

The augmented matrix is given by

   
1 −1 −1| 0 1 0 0|2
   
 with reduced row echelon form
2 1 0| 5  0 1 0|1 .
  
0 1 −5| − 4 0 0 1|1
Hence I1 = 2 and I2 = I3 = 1.
EER:
Consider the electric circuit shown in the diagram below:

We wish to determine the currents I1, I2 and I3. Applying

the Current law to the point A, we obtain I1 + I2 = I3.
Applying the Current law to the point B, we obtain the
same. Hence we have the linear equation

I1 + I2 − I3 = 0.

Next, let us consider the left hand loop, and let us take
the positive direction to be clockwise. By Ohm’s law,
49
 Figure 4: Electric Flow 2

the voltage drop across the 8Ω resistor is 8I1, the voltage

drop across the 6Ω resistor is −6I2, while the voltage
drop across the 8Ω resistor is 5I1. On the other hand, the
voltage drop across the 20V electrical source is negative,
since the positive direction of the loop is from − to +.
The Voltage law applied to this loop now gives 8I1 −6I2 +
5I1 − 20 = 0, and we have the linear equation

13I1 − 6I2 = 20.

Next, let us consider the outer loop, and let us take the
positive direction to be clockwise. By Ohm’s law, the
voltage drop across the 8Ω resistor on the top is 8I1,
50
the voltage drop across the 8Ω resistor on the right is
8I3, while the voltage drop across the 8Ω resistor is 5I1.
On the other hand, the voltage drop across the 30V and
20V electrical sources are both negative, since the positive
direction of the loop is from − to + in each case. The
Voltage law applied to this loop now gives 8I1 + 8I3 +
5I1 − 50 = 0, and we have the linear equation

13I1 + 8I3 = 50.

We now have a system of three linear equations

I1 + I2 − I3 = 0;
13I1 − 6I2 = 20;
13I1 + 8I3 = 50.
(22)

Solving (22) gives I1 = 2, I2 = 1andI3 = 3.

Note that Linear Algebra can be applied to many other
fields like Economics, Chemistry, Mechanics, Engineering
to solve resulting linear equations.

51
EER:

1. Find the minimum value for x4 in the following sys-

tem of one way streets:

Figure 5: Traffic Flow 2

2. Consider the system of linear equations

2x1 + 5x2 + 8x3 = 2;

x1 + 2x2 + 3x3 = 4;
3x1 + 4x2 + 4x3 = 1 :

a) Write down the augmented matrix for this system.

b) Reduce the augmented matrix by elementary row
operations to row echelon form.

52
 c) Use your answer in part (b) to solve the system of
linear equations.

3. Consider a system of linear equations in five vari-

ables x = (x1, x2, x3, x4, x5) and expressed in matrix
form Ax = b, where x is written as a column ma-
trix. Suppose that the augmented matrix (A|b) can
be reduced by elementary row operations to the row
echelon form
 
1 3 2 0 6|4
 
0 0 1 1 2|1
 
 
0 0 0 1 1|7
 
0 0 0 0 0|0
a) Which are the pivot variables and which are the
free variables?
b) Determine all the solutions of the system of linear
equations.

53
4. Solve the following system of linear equations

x1 − x2 − 7x3 + 7x4 = 5;
−x1 + x2 + 8x3 − 5x4 = −7;
3x1 − 2x2 − 17x3 + 13x4 = 14;
2x1 − x2 − 11x3 + 8x4 = 7.

5. Determine the currents I1, I2 and I3 in the following

current flow diagram:

Figure 6: Current Flow Circuit

54
9 Machine Learning Application

See the book, Basics of Linear Algebra for Machine Learn-

ing by Jason Brownlee for more details.
Note that machine learning is now a fast growing area of
study, (you may need to pick interest as well.)

10 Applications of Linear Algebra to Statis-

tics

Some clear fingerprints of linear algebra on statistics and

statistical methods include: Use of vector and matrix no-
tation, especially with multivariate statistics. Solutions
to least squares and weighted least squares, such as for
linear regression. Estimates of mean and variance of data
matrices. Linear Algebra is largely applied to nearly all
fields of Statistics, both theory and applied.

55