2/16/2021.

A Choudhury
LINEAR ALGEBRA
Linear Algebra
Mathematics for AI

Boot Camp
1
Sections

2/16/2021. A Choudhury
• Matrices
• Matrix operations
• Transpose of a matrix
• Inverse of a matrix

LINEAR ALGEBRA
• Elementary matrices
• Invertibility and system of linear equations
• Special matrices

2
 Matrices

2/16/2021. A Choudhury
Algebra of matrices
1. Addition, subtraction and scalar multiplication –
dimension /size compatibility
2. Product of two matrices-Rule :

LINEAR ALGEBRA
3
Matrix multiplication with a column/row
matrix

2/16/2021. A Choudhury
LINEAR ALGEBRA
4
 Matrix Product

2/16/2021. A Choudhury
Linear Combination: If A1,A2,…Ar are matrices of the same size, and
c1,c2…c r are scalars , then an expression of the form
c1A1+c2A2 +….crAr is called the linear combination of the matrices
A1,A2,…Ar with coefficients c1,c2…c r
Matrix products can be viewed as a linear combination:

LINEAR ALGEBRA
Theorem 1:If A is an m × n matrix, and if x is an n × 1 column vector,
then the product Ax can be expressed as a linear combination of the
column vectors of A in which the coefficients are the entries of x.
Example 1:

5
 Matrix Product

2/16/2021. A Choudhury
Example 2:

Column-row expansion

LINEAR ALGEBRA
Example 3:

6
Transpose and trace of a matrix

2/16/2021. A Choudhury
Transpose of a matrix :If A is any m × n matrix, then the transpose of A, denoted
by AT , is defined to be the n × m matrix that results by interchanging the rows and
columns of A; that is, the first column of AT is the first row of A, the second column
of AT is the second row of A, and so forth.
Example:

LINEAR ALGEBRA
Trace of a matrix: If A is a square matrix, then the trace of A, denoted by tr(A), is
defined to be the sum of the entries on the main diagonal of A. The trace of A is
undefined if A is not a square matrix.

7
If A and B are n × n matrices, then tr(A + B) = tr(A) + tr(B)
Properties of Matrix Operations

2/16/2021. A Choudhury
Assuming that the sizes of the matrices A,B,C are such that the indicated
operations can be performed, the following rules of matrix arithmetic are
valid ;a,b,c are scalars.
(a) A + B = B + A [Commutative law for matrix addition]
(b) A + (B + C) = (A + B) + C [Associative law for matrix addition]
(c) A(BC) = (AB)C [Associative law for matrix multiplication]

LINEAR ALGEBRA
(d ) A(B + C) = AB + AC [Left distributive law]
(e) (B + C)A = BA + CA [Right distributive law]
(f) a(B + C) = aB + aC
( g ) (a + b)C = aC + bC
(h) a(bC) = (ab)C
(i) a(BC) = (aB)C = B(aC)
Properties of matrix multiplication:
(a) AB [matrix multiplication is not commutative] 8
(b) AB = 0, but A 0 and B 0(Zero product with non-zero factors)
Properties of Zero matrix

2/16/2021. A Choudhury
Theorem 2:If c is a scalar, and if the sizes of the matrices A ,O
are such that the operations can be performed, then:
(a) A + O = O + A = A
(b) A − A = A + (−A) = O
(d) 0A = O

LINEAR ALGEBRA
(e) If c A = O, then c = 0 or A = O.
Theorem 3:If R is the reduced row echelon form of an n × n
matrix A, then either R has a row of zeros or R is the identity
matrix I of order n.

9
Inverse of a matrix

2/16/2021. A Choudhury
Inverse of a matrix :If A is a square matrix, and if a matrix B of the
same size can be found such that AB = BA = I , then A is said to be
invertible (or nonsingular) and B is called an inverse of A(B=A-1).
If no such matrix B can be found, then A is said to be singular.
Properties of inverse :

LINEAR ALGEBRA
Theorem 4: If B and C are both inverses of the matrix A, then B
= C.[Uniqueness and existence of the inverse of matrix A]
Theorem 5:The matrix is invertible if and only if ,in which case
the inverse is given by

Theorem 6: If A and B are two invertible matrices with same

2/16/2021. A Choudhury
Theorem 7:If A is invertible and n is a nonnegative integer, then:
(a) A−1 is invertible and (A−1)−1 = A.
(b) An is invertible and (An)−1 = A−n = (A−1)n.
(c) kA is invertible for any nonzero scalar k, and (kA)−1 = k−1A−1
Theorem 8: If the sizes of the matrices are such that the stated

LINEAR ALGEBRA
operations can be performed, then:
(a) (AT )T = A
(b) (A + B)T = AT + BT
(c) (A − B)T = AT − BT
(d) (kA)T = k AT
(e) (AB)T = BTAT
Theorem 9:If A is an invertible matrix, then AT is also invertible and 11
(AT )−1 = (A−1) T
Elementary Matrices

2/16/2021. A Choudhury
Row equivalent: Matrices A and B are said to be row
equivalent if either (hence each)can be obtained from the
other by a sequence of elementary row operations.
Elementary matrix: A matrix E is called an elementary
matrix if it can be obtained from an identity matrix by

LINEAR ALGEBRA
performing a single elementary row operation.
Examples of elementary matrices:

12
Elementary matrices

2/16/2021. A Choudhury
Theorem 10:If the elementary matrix E results from
performing a certain row operation on I of order m and if A is
an m × n matrix, then the product EA is the matrix that results
when this same row operation is performed on A.
Example:

LINEAR ALGEBRA
Theorem 11:Every elementary matrix is invertible, and the
inverse is also an elementary matrix.

13
Equivalence theorem

2/16/2021. A Choudhury
Theorem12: If A is an n × n matrix, then the following
statements are equivalent.
(a) A is invertible.
(b) Ax = 0 has only the trivial solution.
(c) The reduced row echelon form of A is In.

LINEAR ALGEBRA
(d) A is expressible as a product of elementary matrices.
(e) Ax =b is consistent for every nx1 matrix b.
(f) Ax=b has exactly one solution for every n x 1 matrix b.
(g) The rank of matrix A is n.

14
Inversion Algorithm

2/16/2021. A Choudhury
Inversion Algorithm To find the inverse of an invertible
matrix A, find a sequence of elementary row operations that
reduces A to the identity and then perform that same
sequence of operations on In to obtain A −1.
Example:

LINEAR ALGEBRA
15
 Inversion Algorithm

LINEAR ALGEBRA 2/16/2021. A Choudhury

2/16/2021. A Choudhury
The inversion algorithm can be used to check whether a
matrix a is invertible.
Example:

LINEAR ALGEBRA
On the left side we obtained a
row of zeros, so A is not
invertible.

17
Analysis of homogeneous system of equations

2/16/2021. A Choudhury
Determine whether the following system of homogeneous
equations have non-trivial solutions:

LINEAR ALGEBRA
• The coefficient matrix A in system (a) is invertible and in
system (b) is not invertible.
• By, the second part of Equivalence theorem, the system of
homogeneous equations in (a) has a trivial solution and (b)
have non-trivial solutions.

18
Consistency by elimination

2/16/2021. A Choudhury
What conditions must b1, b2, and b3 satisfy in order for a
system of equations to be consistent?
Example 1:

LINEAR ALGEBRA
Example2:

What does the result in Example 2 tell you about the

2/16/2021. A Choudhury
Diagonal matrices: A square matrix in which all the entries off the main
diagonal are zero is called a diagonal matrix.

LINEAR ALGEBRA
A diagonal matrix is invertible if and only if all of its diagonal entries are
nonzero.

The powers of a diagonal matrix can be computed easily. If D is a diagonal

2/16/2021. A Choudhury
Triangular Matrices –Upper triangular and lower triangular
matrices

LINEAR ALGEBRA
Properties:
• A square matrix A = [aij ] is upper triangular if and only if all
entries to the left of the main diagonal are zero; if i>j then aij=0.
• A square matrix A = [aij ] is lower triangular if and only if all
entries to the right of the main diagonal are zero; if i<j then aij=0.
• A square matrix A = [aij ] is upper triangular if and only if the i th
row starts with at least i − 1 zeros for every i. 21
• A square matrix A = [aij ] is lower triangular if and only if the j th
column starts with at least j − 1 zeros for every j.
Triangular Matrices

2/16/2021. A Choudhury
Theorem 13:
(a) The transpose of a lower triangular matrix is upper triangular,
and the transpose of an upper triangular matrix is lower triangular.
(b) The product of lower triangular matrices is lower triangular, and
the product of upper triangular matrices is upper triangular.

LINEAR ALGEBRA
(c) A triangular matrix is invertible if and only if its diagonal entries
are all nonzero.
(d) The inverse of an invertible lower triangular matrix is lower
triangular, and the inverse of an invertible upper triangular matrix is
upper triangular.
Computation with Triangular matrices:
a) Which of the matrices, A or B is
invertible? Compute the inverse. 22
b) Compute AB and BA
Special matrix : Symmetric Matrix

2/16/2021. A Choudhury
Symmetric matrix:A square matrix A is said to be symmetric if A
= AT . Equivalently, (A)ij=(A)ji
Theorem 14:If A and B are symmetric matrices with the same size,
and if k is any scalar, then:
(a) AT is symmetric.

LINEAR ALGEBRA
(b) A + B and A − B are symmetric.
(c) kA is symmetric.
(d) The product of a matrix and its transpose is symmetric.
Theorem 15:The product of two symmetric matrices is symmetric if
and only if the matrices commute.
Theorem 16:If A is an invertible symmetric matrix, then A−1 is
symmetric.
Theorem 17:If A is an invertible matrix, then AAT and ATA are also
23
invertible.
References:

2/16/2021. A Choudhury
1) Howard Anton (2014). Elementary Linear Algebra, 11 th Edition.
Wiley Publication.
2) Gilbert Strang (2016 ).Introduction to Linear Algebra , 5th Edition ,
Wellesley Publishers

LINEAR ALGEBRA
24