Linear Algebra with Applications

Samat Kassabek

Department of Mathematics
School of Sciences and Humanities
Nazarbayev University
Kazakhstan

Week 1

Samat Kassabek ([email protected])

Linear Algebra with Applications 1 / 22
 What’s up in Week-1

System of Linear Equations (Sec. 1.1)

Row Reduction and Echelon Forms (Sec. 1.2)

Samat Kassabek ([email protected])

Linear Algebra with Applications 2 / 22
 Section 1.2
Row Reduction and Echelon Forms

Samat Kassabek ([email protected])

Linear Algebra with Applications 3 / 22
 Row reduction

In fact, to solve the system of equations we have used the row reduction
or Gaussian elimination.
The point of Gaussian Elimination method is to use the elementary
operations to reduce a given system of linear equations and reach a simple
form, which is called echelon form and then set up back-substitution to
find unknowns (if it has a solution).
A nonzero row or column is a row or column with at least one nonzero
entry.
The leading entry of a row is the leftmost nonzero entry (in a nonzero
row).
A matrix is called a triangular matrix if the entries below or above the
diagonal are zeros.

Samat Kassabek ([email protected])

Linear Algebra with Applications 4 / 22
 Echelon Form
Definition
A rectangular matrix is in echelon form (or row echelon form) if it has the
following three properties:
1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading
entry of the row above it.
3. All entries in a column below a leading entry are zeros.

Example (1)
The following matrices are in echelon form. The leading entries (■) may have
any nonzero value; the starred entries (∗) may have any value (including zero).

Samat Kassabek ([email protected])

Linear Algebra with Applications 5 / 22
 Echelon Form

Definition
A system of linear equation is in echelon form, if it has the following form:


 a11 x1 + a12 x2 + a13 x3 + . . . + a1n xn = b1
a2j2 xj2 + a2j3 xj3 + . . . + a2n xn = b2


 . . . . . . . ...................................
arjr xjr + . . . + arn xn = br


where 1 < j2 < · · · < jr and a11 , a2j2 , . . . , arjr are not zero. x1 , x2 , . . . , xr are
called pivots or leading unknowns. The unknows that are not leading in the
echelon form are called free.

Example

x1 − 2x2 + x3 = 5
x2 − 2x3 = 3.
The system is in echelon form. x1 and x2 are leading unknowns, x3 is free.

Samat Kassabek ([email protected])

Linear Algebra with Applications 6 / 22
 Special case: Triangular Form

Special case of echelon form is triangular form. It is the case when r = n.

Definition
A system of linear equation is in triangular form, if it has the following form:

 a11 x1 + a12 x2 + a13 x3 + . . . . . . + a1n xn = b1

a22 x2 + a23 x3 + . . . + a2n xn = b2


 . . . . . . . . . . . . . ................................
ann xn = bn


where a11 , a22 , . . . , ann are not zero.

Example

 x1 − 2x2 + x3 = 5
x2 − 2x3 = 3
4x3 = 4.


Samat Kassabek ([email protected])

Linear Algebra with Applications 7 / 22
 Gaussian Elimination Method

Gaussian Elimination Method essentially consists of two parts:

1 (Forward Elimination) Step-by-step reduction of the system yielding
either an equation that has no solution or an equivalent simpler system in
a triangular or echelon form.
1 Input: a system of linear equations with m equations in n unknowns
2 Elimination Step
3 Recursion Step
4 Output: a triangular or echelon form, or an equation with no solution
2 (Backward Elimination) Step-by-step back-substitution to find the
solution of the simpler system.

Samat Kassabek ([email protected])

Linear Algebra with Applications 8 / 22
 Algorithm for Forward Elimination

1 (Input) A system of linear equations with m equations in n unknowns:



 a11 x1 + a12 x2 + . . . + a1n xn = b1
a21 x1 + a22 x2 + . . . + a2n xn = b2


 ....................................
am1 x1 + am2 x2 + . . . + amn xn = bm


Samat Kassabek ([email protected])

Linear Algebra with Applications 9 / 22
 Continued: Algorithm for Forward Elimination
2 (Elimination Step) Find the first unknown in the system with a nonzero
coefficient (which now must be x1 ).
Arrange so that a11 ̸= 0. That is, if necessary, interchange equations so
that the first unknown x1 appears with a nonzero coefficient in the first
equation.
Use a11 as a pivot to eliminate x1 from all equations except the first
equation. That is, for i > 1 : perform Ri → λR1 + Ri , where
λ = −ai1 /a11 . The system now has the following form:


 a11 x1 + a12 x2 + a13 x3 + . . . + a1n xn = b1
a2j2 xj2 + . . . + a2n xn = b2


 .......................................
amj2 xj2 + . . . + amn xn = bm


where x1 does not appear in any equation except the first, a11 ̸= 0, and xj2
the first unknown with a nonzero coefficient in any equation other than the
first.
Examine each equation whether
any of it has the form 0x1 + 0x2 + . . . + 0xn = b with b ̸= 0, if so, then we
stop and the system has no solution.
any of it has the form 0x1 + 0x2 + . . . + 0xn = 0, if so, then delete that
equation from the system.
Samat Kassabek ([email protected])
Linear Algebra with Applications 10 / 22
 Continued: Algorithm for Forward Elimination

3 (Recursion Step) Repeat the Elimination Step with each new ”smaller”
subsystem formed by all the equations excluding the first equation.
4 (Output) Finally, the system is reduced to triangular or echelon form, or
an equation with no solution.
From Gauss Elimination Method we can conclude the following theorem.

Theorem
Consider a system of linear equations in echelon form, say with r equations in
n unknowns. Then there are two cases:
(i) r = n. That is, there are as many equations as unknowns (triangular
form). Then the system has a unique solution.
(ii) r < n. That is, there are more unknowns than equations. Then we can
arbitrarily assign values to the n − r free unknowns and solve uniquely for
the r unknowns, obtaining a solution of the system.

Samat Kassabek ([email protected])

Linear Algebra with Applications 11 / 22
 Example
Example
Consider the case when we have infinitely many solutions.

 x1 + x2 − 2x3 + 4x4 = 5
2x1 + 2x2 − 3x3 + x4 = 3
3x1 + 3x2 − 4x3 − 2x4 = 1


We apply Gaussian Elimination method.

Samat Kassabek ([email protected])

Linear Algebra with Applications 12 / 22
 Example
Example
Consider the case when we have infinitely many solutions.

 x1 + x2 − 2x3 + 4x4 = 5
2x1 + 2x2 − 3x3 + x4 = 3
3x1 + 3x2 − 4x3 − 2x4 = 1


We apply Gaussian Elimination method.

First we perform R2 → −2R1 + R2 and then R3 → −3R1 + R3 .
Then we have 
 x1 + x2 − 2x3 + 4x4 = 5
x3 − 7x4 = −7
2x3 − 14x4 = −14

Now we perform R3 → −2R2 + R3 and have

 x1 + x2 − 2x3 + 4x4 = 5 
x1 + x2 − 2x3 + 4x4 = 5
x3 − 7x4 = −7 ⇔
x3 − 7x4 = −7
0x3 + 0x4 = 0


It implies that x1 and x3 are pivots, and x2 and x4 are free unknowns.
Samat Kassabek ([email protected])
Linear Algebra with Applications 12 / 22
 Using the the second equation we express x3 by x4 and have
x3 = −7 + 7x4 .
Then we substitute it into the first equation and we derive
x1 + x2 − 10x4 = −9 ⇔ x1 = −9 − x2 + 10x4 .
So x1 and x3 depend on x2 and x4 .
We can substitute arbitrary numbers for x2 and x4 and find x1 and x3 .
Since K is infinite and x2 , x4 ∈ K, the system has infinitely many sol.
In this case, the solution set can be written as For example, if we set
x2 = x4 = 0, then x1 = −9, x3 = −7 or if we set x2 = 0 and x4 = 1, then
x1 = 1, x3 = 0.
These are examples of particular solutions of the system and they are
infinite.
The general answer is
{(−9 − x2 + 10x4 , x2 , −7 + 7x4 , x4 ) | x2 and x4 are any numbers from K} ,
that’s why, it is called a general solution of the system.
Important remark
As we wish, we can declare the unknowns x3 and x1 as free unknowns and
express x2 and x4 by them. However, again we have the same set of solutions.
Samat Kassabek ([email protected])
Linear Algebra with Applications 13 / 22
 In Lecture 1 we studied how to solve a system of linear equations by
Gaussian Elimination method.
The main idea was to derive an echelon form matrix from the system or
from a augmented matrix by elementary operations and then use
back-substitution to find values of unknowns.
However, echelon form matrices are not unique, namely, a system or
matrix may have different echelon form matrices.

Samat Kassabek ([email protected])

Linear Algebra with Applications 14 / 22
 In Lecture 1 we studied how to solve a system of linear equations by
Gaussian Elimination method.
The main idea was to derive an echelon form matrix from the system or
from a augmented matrix by elementary operations and then use
back-substitution to find values of unknowns.
However, echelon form matrices are not unique, namely, a system or
matrix may have different echelon form matrices.

Example
 
2 2
Let A = . Then we note that
4 3
     
2 2 1 1 2 0
, ,
0 −1 0 −1 0 −1

are all echelon matrices of A.

To have a unique type of echelon form we introduce reduced echelon form
matrix of a matrix and Gauss-Jordan reduction which allows to get
reduced echelon form matrix.
We will observe that Gauss-Jordan reduction is just an extension of
Gaussian Elimination method and it has some advantages.
Samat Kassabek ([email protected])
Linear Algebra with Applications 14 / 22
 Echelon Form

Definition
A rectangular matrix is in echelon form (or row echelon form) if it has the
following three properties:
1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading
entry of the row above it.
3. All entries in a column below a leading entry are zeros.

Samat Kassabek ([email protected])

Linear Algebra with Applications 15 / 22
 Reduced Echelon Form
Definition
If a matrix in echelon form satisfies the following additional conditions, then
it is in reduced echelon form (or reduced row echelon form):
4. The leading entry in each nonzero row is 1.
5. Each leading 1 is the only nonzero entry in its columns.

Example
The following matrices are in reduced echelon form because the leading entries
are 1’s, and there are 0 ’s below and above each leading 1.

Samat Kassabek ([email protected])

Linear Algebra with Applications 16 / 22
 Example
Every matrix in reduced echelon form is also a matrix in echelon form.

Example
The matrix
 
2 −3 2 1
 0 1 −4 8 
5
0 0 0 2

is in echelon form.

Example
The matrix
 
1 0 0 29
 0 1 0 16 
0 0 1 3

is in reduced echelon form.

Samat Kassabek ([email protected])
Linear Algebra with Applications 17 / 22
 Example
Consider the system which was solved in Lecture 1.

 x1 + 2x2 + x3 = −1
3x1 − x2 − x3 = −1
−2x1 + 2x2 + 3x3 = 5.


After performing some elementary operations we obtained

   
1 2 1 −1 1 2 1 −1
 3 −1 −1 −1  ∼ · · · ∼  0 −7 −4 2 .
−2 2 3 5 0 0 11 33
The last matrix is in echelon form.
To be reduced echelon form matrix the entries in red circles should be
zero and the entries in the main diagonal which are circled by blue should
be one.  
1 2 1 −1
 
 0 –7 –4 2 .
 
0 0 11 33
Samat Kassabek ([email protected])
Linear Algebra with Applications 18 / 22
 1
We perform the elementary operation R3 → 11 R3 and have
 
1 2 1 −1
 0 −7 −4 2 .
0 0 1 3
From R2 → 4R3 + R2 and R1 → −R3 + R1 follows the matrix
 
1 2 0 −4
 0 −7 0 14  .
0 0 1 3

Now we make the leading coefficient -7 into 1 by performing R2 → − 17 R2

 
1 2 0 −4
 0 1 0 −2  .
0 0 1 3
We perform R1 → −2R2 + R1 and have
 
1 0 0 0
 0 1 0 −2  .
0 0 1 3

Samat Kassabek ([email protected])

Linear Algebra with Applications 19 / 22
 Thus, the obtained matrix
 
1 0 0 0
 0 1 0 −2 
0 0 1 3

is now in reduced echelon form.

From the matrix one can find solution of the given system of linear
equations without back-substitution.
If we write the matrix as system of linear equations we have

 x1 = 0
x2 = −2
x3 = 3.


Samat Kassabek ([email protected])

Linear Algebra with Applications 20 / 22
 Questions for Self-Study

1 Find a polynomial f (x) in degree 3 , if f (−2) = 1, f (−1) = 3, f (1) = 13,

f (2) = 33.

 λx1 + x2 + x3 = 1
2 Let λ ∈ K be a parameter in the system x1 + λx2 + x3 = 1
x1 + x2 + λx3 = 1

(a) Solve the system.
(b) Find all possible values of λ for which the given system has (1) unique
solution, (2) infinitely many solutions, (3) no solutions.
3 Find the general solutions of the systems whose augmented matrix is
given:  
1 −7 0 6 5
 0 0 1 −2 −3 
−1 7 −4 2 7

Samat Kassabek ([email protected])

Linear Algebra with Applications 21 / 22
 The End

Samat Kassabek ([email protected])

Linear Algebra with Applications 22 / 22