MA102: Linear Algebra

Lecture 02

Shiv Prakash Patel

AT IIT Dharwad

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Useful thoughts

• Don’t just read it! Ask your own questions, look for your own
examples, discover your own proofs. Is the hypothesis necessary? Is
the converse true? What happens in the classical special case?
Where does the proof use the hypothesis?
[Paul Halmos]
• ”for example” is not a proof.
[Jewish proverb]

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Announcement

• Quiz-1 on 19th January 2024 from 3:00pm to 4:00pm

• Office hours: Wednesdays 3:00pm-4:00pm (or by appointment)
• Office location: 615, Block-2
• Appointment contact: [email protected]

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Last Lecture

• Operations on vectors and their properties

(addition, scalar multiplication, . . . )
• Operations on matrices and their properties
(addition, scalar multiplication, matrix multiplication, . . . )

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Our notation

• F is either Q, R or C.
• Mm×n (F) the set of all m × n matrices with entries from F
• For i = 1, · · · , n, let ei ∈ Mn×1 (F) be such that its i-th entry is 1
and other entries are 0. The column vector ei is called i-th basic
column vectors.  
0
 .. 
.
 
0
t
 
ei = [0, · · · , 0, 1, 0, · · · 0] = 
1

0
 
 .. 
.
0

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Important Observations

Proposition
Let A := [aij ] ∈ Mm×n (F), and let e1 , . . . , en ∈ Mn×1 (F) be the basic
column vectors. Then, for k = 1, . . . , n,
 
a1k
 .. 
 . 
 
 ajk , which is the k-th column of A.
A ek =  
 .. 
 . 
amk

Corollary
A = B if and only if Aek = Bek for each k = 1, . . . , n.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Properties of Matrix Multiplication

Proposition
1 Associative law: For A ∈ Mm×n (F), B ∈ Mn×p (F), C ∈ Mp×k (F), we
have A(BC ) = (AB)C .
2 Distributive law: (1) For A, B ∈ Mm×n (F) and C ∈ Mn×p (F) we
have
(A + B)C = AC + BC .
(2) For A, B ∈ Mn×p (F) and C ∈ Mm×n (F) we have
C (A + B) = CA + CB.
3 For A ∈ Mm×n (F), B ∈ Mn×p (F) and α ∈ F we have
(αA)B = α(AB) = A(αB).

Proof: Exercise

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Linear combination of vectors
We often identify Fn with M1×n (F) and Mn×1 (F) in an obvious way.
Definition
Let x1 . . . , xk be vectors in Fn and α1 , . . . , αk ∈ F. Then

α1 · x1 + · · · + αk · xk

is called a (finite) linear combination of x1 , . . . , xk .

We also talk about linear combinations of row vectors / column vectors.

Example
Recall the basic column vectors e1 , e2 , · · · , en ∈ Mn×1 (F).
 t
For b = b1 · · · bk · · · bn ∈ Mn×1 (F), we get

b = b1 · e1 + · · · + bk · ek + · · · + bn · en .
That is b is a linear combination of e1 , . . . , en .
Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02
 Linear combination of vectors, contd....

Proposition
Let A ∈ Mm×n (F) and B ∈ Mn×p (F). Then
1 The i-th row of AB is a linear combination of the n row vectors of B
with coefficients ai1 , . . . , ain provided by the i-th row of A.
2 The j-th column of AB is a linear combination of the n column
vectors of A with coefficients b1j , . . . , bnj provided by the jth column
of B.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Proof

(1) Let AB = [cij ] and Rk be the k-th row of B. Then we want to show
 
ci1 ci2 . . . cip = ai1 R1 + ai2 R2 + · · · + ain Rn .

We prove that their

 j-th entry are equal
 for j = 1, . . . , p.
The j-th entry of ci1 ci2 . . . cip is cij and by definition of matrix
multiplication
Pn
cij = k=1 aik bkj .

On the other hand, the j-th entry of ai1 R1 + ai2 R2 + · · · + ain Rn equals

ai1 b1j + ai2 b2j + · · · + ain bnj

which equals cij .

The proof of (2) is similar and left as an exercise.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Example

Example
 
  1 6 0 2
2 1 −1
Let A = and B = 2 −1 1 −2. Then
0 3 1
2 0 −1 1
 
2 11 2 1
AB = .
8 −3 2 −5

We have
     
2 11 2 1 = 2 1 6 0 2 + 1 2 −1 1 −2
 
−1 2 0 −1 1 ,
       
11 2 1 −1
= 6 −1 +0 , etc.
−3 0 3 1
.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 System of Linear Equation

A linear system of m equations in the n unknowns x1 , ..., xn is given by

a11 x1 + a12 x2 + · · · + a1n xn = b1 (1)

a21 x1 + a22 x2 + · · · + a2n xn = b2 (2)
.. .. .. .. ..
. . . . .
am1 x1 + am2 x2 + · · · + amn xn = bm , (m)

where aij ∈ F for i = 1, . . . , m; j = 1, . . . , n and also bj ∈ F for

j = 1, . . . , m are given.
Question? What is the meaning of a solution to the above system of
equations?
Goal: Solve the equations (1) to (m) simultaneously

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 System of Linear Equation, contd....

The system of equations can be written as follows

    
a11 a12 · · · a1n x1 b1
 a21 a22 · · · a2n  x2   b2 
..   ..  =  .. 
    
 .. .. ..
 . . . .  .   . 
am1 am2 ··· amn xn bm
 t
Let A := [aij ] ∈ Mm×n (F), x := x1 · · · xn and
 t
b := b1 · · · bm ∈ Mm×1 (F).
Using matrix multiplication, the linear system can be written as

Ax = b

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Notations and terminology

For the system of equations

Ax = b

• Coefficient Matrix: the m × n matrix A

• Solution of the above linear system of equations: A column vector
x0 ∈ Mn×1 (F) such that Ax0 = b
• Homogeneous linear system: b = 0
• Non-homogeneous linear system: b 6= 0
• Note: 0 is always a solution of a homogeneous system
• Caution !: A Non-homogeneous linear system of equations may not
have a solution, may have only one solution or may have (infinitely)
many solutions.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Example-1

The following system of linear equations

x1 + x2 = 1
2x1 + 2x2 = 4
does not have a solution.

2x1 + 2x2 = 4

x1 + x2 = 1

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Example-2

The following system of linear equations

x1 + x2 = 1
x1 − x2 = 0
has a unique solution, namely x1 = 1/2 = x2 .

x1 − x2 = 0

x1 + x2 = 1

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Example-3
The following system of linear equations
x1 + x2 = 1
2x1 + 2x2 = 2
has (infinitely) many solutions, namely x1 = α, x2 = 1 − α, α ∈ R.

2x1 + 2x2 = 2

x1 + x2 = 1

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Homogeneous system

Proposition
Consider the homogeneous linear system Ax = 0. If r ∈ N and x1 , . . . , xr
are solutions of such a system, then so is their linear combination
α1 x1 + · · · + αr xr for any α1 , . . . , αr ∈ F.

Proof:
A(α1 x1 + · · · + αr xr ) = α1 Ax1 + · · · + αr Axr
= α1 0 + · · · + αr 0
= 0

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Non-homogeneous system

Proposition
Consider the system of equations Ax = b with a particular solution x0 .
Let S denote the set of all solutions of the corresponding homogeneous
linear system Ax = 0. Then, the set of all solutions of Ax = b is given by

{x0 + s : s ∈ S}

Proof: First note that A(x0 + s) = Ax0 + As = b + 0 = b.

Moreover, if x is any solution of (∗), then

A(x − x0 ) = Ax − Ax0 = b − b = 0

=⇒ x − x0 ∈ S. Let x − x0 = s for some s ∈ S. Then x = x0 + s.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Upshot

To find all solutions of Ax = b:

Step-1: Find all the solutions of the corresponding Ax = 0.

Step-2: Find a particular solution x0 of Ax = b.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Examples

Question: Solve the following system of linear equations

x1 − x2 + x3 = b1
x2 + x3 = b2
x3 = b3

Solution:

x3 = b3
x2 = b2 − x3 = b2 − b3
x1 = b1 + x2 − x3 = b1 + b2 − 2b3

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Back Substitution method

Suppose the coefficient matrix A is upper triangular and its diagonal

elements are nonzero. Then the linear system

a11 x1 + a12 x2 + · · · + · · · + · · · + · · · + a1n xn = b1 (1)

a22 x2 + a23 x3 + · · · + · · · + · · · + a2n xn = b2 (2)
.. .. .. ..
. . . .
a(n−1)(n−1) xn−1 + a(n−1)n xn = bn−1 (n − 1)
ann xn = bn (n)

can be solved by back substitution as follows.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 xn = bn /ann
xn−1 = (bn−1 − a(n−1)n xn )/a(n−1)(n−1), where xn = bn /ann
.. .. ..
. . .
x2 = (b2 − a2n xn − · · · − a23 x3 )/a22 , where xn = · · ·, x3 = · · ·,
x1 = (b1 − a1n xn − · · · − · · · − a12 x2 )/a11 , where xn = · · ·, x2 = · · ·.

In this case, the homogeneous system Ax = 0 has only the zero solution
and the general system Ax = b has a unique solution.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Upshot

Goal: To transform any m × n matrix to an “upper triangular form”.

Question: How can this be done?

Attempt: We successively attempt to eliminate

the unknown x1 from the equations (2), . . . , (m),
the unknown x2 from the equations (3), . . . , (m),
and so on.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Example

Consider the linear system

x1 − x2 + x3 = 0
−x1 + x2 − x3 = 0
10x2 + 25x3 = 90
20x1 + 10x2 = 80.

Eliminating x1 from the 4th, 3rd and 2nd equations,

x1 − x2 + x3 = 0
0 = 0
10x2 + 25x3 = 90
30x2 − 20x3 = 80.

Interchanging the 2nd and the 3rd equations,

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 x1 − x2 + x3 = 0
10x2 + 25x3 = 90
0 = 0
30x2 − 20x3 = 80.

Eliminating x2 from the 4th equation, and then interchanging the 3rd
and the 4th equations,

x1 − x2 + x3 = 0
10x2 + 25x3 = 90
−95x3 = −190
0 = 0.

Now back substitution gives x3 = 2, x2 = (90 − 25x3 )/10 = 4 and

 t
x1 = −x3 + x2 = 2, that is, x = 2 4 2 .

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

 Augmented matrix
The above process can be carried out without writing down the entire
linear system by considering the augmented matrix

a11 a12 ··· a1n b1

 
 a12 a22 ··· a2n b2 
[A|b] :=  .. .. .. .. ..  .
 
 . . . . .
am1 am2 ··· amn bn

This m × (n + 1) matrix completely describes the linear system.

In the above example,
 
1 −1 1 0
−1 1 −1 0
[A|b] = 
0
.
10 25 90
20 10 0 80

Subtracting 20 times the first row from the 4th row, and adding the first
row to the second row, we obtain
Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02
 Revisit example

For the following system of linear equations

x1 − x2 + x3 = 0
−x1 + x2 − x3 = 0
10x2 + 25x3 = 90
20x1 + 10x2 = 80.

the augmented matrix is

 
1 −1 1 0
−1 1 −1 0
[A|b] = 
0
.
10 25 90
20 10 0 80

Subtracting 20 times the first row from the 4th row, and adding the first
row to the second row, we obtain

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02

  
1 −1 1 0
R4 −20R1 , R2 +R1 , 0 0 0 0
−−−−−−−−−−→  0 10
.
25 90
0 30 −20 80
Interchanging the 2nd and the 3rd rows, we obtain
 
1 −1 1 0
R2 ←→R3  0 10 25 90
−− −−−→  0 0
.
0 0
0 30 −20 80
Finally, subtracting 3 times the 2nd row from the 4th row and
interchanging the3rd and the 4th rows, wearrive at
1 −1 1 0
R4 −3R2 , R3 ←→R4 0 10 25 90 
−−−−−−−−−−−→  0 0 −95
.
−190
0 0 0 0
The upper triangular nature of the 3 × 3 matrix on the top left enables
back substitution.

Shiv Prakash Patel, IIT Dharwad Linear Algebra: Lecture 02