MA-106 Linear Algebra

Ananthnarayan H.

Department of Mathematics
Indian Institute of Technology Bombay
Powai, Mumbai - 76

8th January, 2015

D2 - Lecture 3

Ananthnarayan H. D2 - Lecture 3
Recall

Last time: We saw a system of linear equations, and the

elimination algorithm, in the matrix form.
• A matrix is a collection of numbers arranged in a fixed
number of rows and columns.
• Two matrices can be added if they have the same size.
• We can multiply a matrix A and a column vector x if
no. of columns of A = length of x.
 
A1 · x
 A2 · x 
In this case: Ax =  ..  (row version)
 
 . 
Am · x
or Ax = x1 A1 + · · · + xn An (column version)

Ananthnarayan H. D2 - Lecture 3
An Example

 
  2
1 3 −3 −1 1
Let A = 1 2 0 −2 and x =  1 .

1 0 −2 0

2

1
A = 1 3 −3 −1 , A = 1 2 0 −2 A3 =?.
2
 
0
Then A1 · x =?, A2 · x = 0, A3 · x = 0, hence Ax = 0 .
0
Question: What can you say about the solutions of Ax = 0?
Observe: No. of rows of Ax = No. of rows of A,
and No. of columns of Ax = No. of columns of x.

Ananthnarayan H. D2 - Lecture 3
Operations on Matrices: Matrix Multiplication

Two matrices A and B can be multiplied if and only if

no. of columns of A = no. of rows of B .
If A is m × n and B is n × r, then AB is m × r.

Key Idea: We know how to multiply a matrix and a vector.

Column wise: Write

B column-wise, i.e., let
B = B1 . . . Br . Then


AB = AB1 . . . ABr

Note: Each Bi is a column vector of length n. Hence, ABi is

a column vector of length m. So, the size of AB is m × r.

Ananthnarayan H. D2 - Lecture 3
Operations on Matrices: Matrix Multiplication

Row wise: Write A row-wise, i.e., let A1 , . . . , Am be the rows

of A. Then
   
A1 A1 B
AB =  ...  B =  ... 
   
Am Am B

Note: Each Ai is a row vector of size 1 × n. Hence, Ai B is a

row vector of size 1 × r. So, the size of AB is m × r.
Working Rule:
The entry in the ith row and jth column of AB is Ai · Bj , i.e.,
the dot product of the ith row of A with the jth column of B.

Ananthnarayan H. D2 - Lecture 3
Properties of Matrix Multiplication

If A is m × n and B is n × r.
a) (AB)ij = Ai · Bj ., i.e., (ith row of A) · (jth column of B)
b) jth column of AB =A · (jth column of B)
c) ith row of AB =(ith row of A) · B
Properties of Matrix Multiplication:
(associativity) (AB)C = A(BC)
(distributivity) A(B + C) = AB + AC
(B + C)D = BD + CD
(non-commutativity) AB 6= BA, in general.
Find examples.

Ananthnarayan H. D2 - Lecture 3
Matrix Multiplication: Examples
 
2 1 1  
1 2 3
Examples: A =  4 −6 0 , B =
2 2 4
−2 7 2
   
1 0 0 1 0 0
E = −2 1 0 , I = 0 1 0 (Identity Matrix).
0 0 1 0 0 1
   
2 1 1 0 1 1
Then EA =  0 −8 −2 , AE =  16 −6 0
−2 7 2 − 16 7 2
IA = A = AI, AB does not make sense. Find BA.

Ananthnarayan H. D2 - Lecture 3
Elementary Matrices: Row Operations
    
1 0 0 u u
Example: Ex := −2 1 0  v  = v − 2u .
0 0 1 w w



If A = A1 A2 A3 , then EA = EA1 EA2 EA3 .
Thus, EA has the same effect on A as the row operation
R2 7→ R2 + (−2)R1 on the matrix A.
Note: E is obtained from the identity matrix I by the row
operation R2 7→ R2 + (−2)R1 .
Such a matrix (diagonal entries 1 and atmost one off-diagonal
entry non-zero) is called an elementary matrix.
Notation: E := E21 (−2). Similarly define Eij (λ).

Ananthnarayan H. D2 - Lecture 3
Permutation Matrices: Row Interchange

    
0 1 0 u v
P12 x = 1 0 0  v  =  u 
0 0 1 w w



If A = A1 A2 A3 , then P12 A = P12 A1 P12 A2 P12 A3 .
Thus P12 A has the same effect on A as the interchange of the
first and second rows of A.
Note: We get P12 from the identity matrix I by interchanging
first and second rows.
P12 is called a permutation (or row exchange) matrix.
Remark: Row operations correspond to multiplication by
elementary matrices Eij (λ) or permutation matrices Pij on
the left.

Ananthnarayan H. D2 - Lecture 3
Elementary and Permutation Matrices
For any n × n matrix A, observe that the row operations
R2 7→ R2 − 2R1 , R2 7→ R2 + 2R1 leave the matrix unchanged.
In matrix terms, E21 (2)E21 (−2)A = IA = A since
    
1 0 0 1 0 0 1 0 0
E21 (−2) E21 (2) = −2 1 0 2 1 0 = 0 1 0 .
0 0 1 0 0 1 0 0 1

Also notice that the row interchange R1 ↔ R2 followed by

R1 ↔ R2 leaves a matrix unchanged.
In matrix terms, P12 P12 A = IA = A, since
    
0 1 0 0 1 0 1 0 0
P12 P12 = 1 0 0 1 0 0 = 0 1 0 .
0 0 1 0 0 1 0 0 1
In general, Eij (λ)Eij (−λ) = I = Pij Pij .
Ananthnarayan H. D2 - Lecture 3
Elimination: Elementary Matrices

    
2 1 1 u 5
Consider  4 −6 0  v  = −2 (Ax = b)
−2 7 2 w 9

Step 1 Eliminate u by R2 7→ R2 + (−2)R1 , R3 7→ R3 + R1 .

This corresponds to multiplying both sides on the left first by
E21 (−2) and then by E31 (1). The equivalent system is:

E31 (1)E21 (−2)Ax = E31 (1)E21 (−2)b, i.e.,

    
2 1 1 u 5
0 −8 −2  v  = −12 .
0 8 3 w 14

Ananthnarayan H. D2 - Lecture 3
Elimination: Elementary Matrices

Step 2 Eliminate v by R3 7→ R3 + R2 ,
i.e., multiply both sides by E32 (1) to get
 U x = c, 
2 1 1
where U = E32 (1)E31 (1)E21 (−2)A = 0 −8 −2 and

  0 0 1
5
c = E32 (1)E31 (1)E21 (−2)b = −12 .
2
Elimination changed A to an upper triangular matrix
and reduced the problem to solving U x = c.
Observe: The pivots of the system Ax = b are the diagonal
entries of U .

Ananthnarayan H. D2 - Lecture 3
Triangular Factorization

Thus Ax = b is equivalent to Ux = c .
where
E32 (1) E31 (1) E21 (−2) A = U
Multiply both sides by E32 (−1) on the left:
E31 (1) E21 (−2) A = E32 (−1)U
Multiply first by E31 (−1) and then E21 (2) on the left:

A = E21 (2) E31 (−1) E32 (−1) U

where U is upper triangular, which is obtained by forward

elimination, with diagonal entries as pivots.

Ananthnarayan H. D2 - Lecture 3
Triangular Factorization

Note that each Eij (a) is a lower triangular. Product of lower

triangular matrices is lower triangular. In particular L is lower
triangular, where
L := E21 (2) E31 (−1) E32 (−1) =
     
1 0 0 1 0 0 1 0 0 1 0 0
2 1 0  0 1 0 0 1 0 =  2 1 0
0 0 1 −1 0 1 0 −1 1 −1 −1 1

Observe: L is lower triangular with diagonal entries 1 and

below the diagonals are the multipliers.
(2, −1, −1 in the earlier example).

Ananthnarayan H. D2 - Lecture 3
LU Decomposition

If A is an n × n matrix, with no row interchanges needed in

the Gaussian elimination of A, then A = L U , where
• U is an upper triangular matrix, which is obtained by
forward elimination, with non-zero diagonal entries as pivots.
• L is a lower triangular with diagonal entries 1 and with the
multipliers needed in the elimination algorithm below the
diagonals.
Q: What happens if row exchanges are required?

Ananthnarayan H. D2 - Lecture 3
LU Decomposition: with Row Exchanges
 
0 2
Example: A = . A can not be factored as LU .
3 4
(Check this!) The 1st step in the Gaussian elimination of A is
a row exchange.
    
0 1 0 2 3 4
P12 A = =
1 0 3 4 0 2

Now elimination can be carried out without row exchanges.

• If A is an n × n non-singular matrix, then there is a matrix
P which is a product of permutation matrices (needed to take
care of row exchanges in the elimination process) such that
P A = LU , where L and U are as defined earlier.
Q: What happens when A is an m × n matrix? A: Patience!

Ananthnarayan H. D2 - Lecture 3