Linear Algebra

For Machine
Learning

Copyright Intellipaat. All rights

 Agenda

01 What is a Matrix?

Number of Rows
02 Matrix Operations 5 2 9
03 Eigen Values
4 6 7
1 3 8
04 Eigen Vectors

Copyright Intellipaat. All rights

What is a Matrix?

A Matrix is a combination of arrays, numbers or expressions that are

represented in a rectangular format in the form of rows and columns.

Number of Columns

Number of Rows
5 2 9
4 6 7
1 3 8

Copyright Intellipaat. All rights

What is a Matrix?

Number of Columns(n)
Number of Rows(m)

The rows and columns of a matrix are

5 2 9 associated with the dimensions of a
matrix and immensely help while
4 6 7 working with linear expressions. The
matrix with m rows and n columns will
1 3 8 have the m x n dimensions.

Copyright Intellipaat. All rights

What is a Matrix?

Number of Columns(n)
Number of Rows(m)

5 2 9 Let us go through some of the Matrix

operations like addition, subtraction,
4 6 7 multiplication, transpose, inverse of a
matrix, etc.
1 3 8

Copyright Intellipaat. All rights

Hands on - Python

Output

In Python, we use the list of lists in a numpy array to declare a list. As seen in
the example we have declared A and B matrices with dimensions 2x2.

Copyright Intellipaat. All rights

Addition

5 2 2 4 5 4 9 7 6
4 6 7 2 3 2 6 9 9
1 3 2 1 2 4 2 5 6

In the addition operation, the corresponding values of the elements are added
to the resultant matrix like shown in the example.

Copyright Intellipaat. All rights

Addition

5 2 2 4 5 4 9 7 6
4 6 7 2 3 2 6 9 9
1 3 2 1 2 4 2 5 6

In the addition operation, it is mandatory that the dimensions of the

matrices are equivalent to each other.

Copyright Intellipaat. All rights

Subtraction

5 2 2 4 5 4 1 -3 -2
4 6 7 2 3 2 2 3 5
1 3 2 1 2 4 0 1 -2

Similar to the addition operation, in subtraction we will subtract the

corresponding element in both the matrices to get the resultant matrix.

Copyright Intellipaat. All rights

Subtraction

5 2 2 4 5 4 1 -3 -2
4 6 7 2 3 2 2 3 5
1 3 2 1 2 4 0 1 -2

In the subtraction operation, it is mandatory for the matrices to be of

equivalent dimensions as well.

Copyright Intellipaat. All rights

Multiplication

5 2 9 10 4 18
4 6 7 2 8 12 14
1 3 8 2 6 16

Multiplying a matrix with a scalar will yield the resultant matrix that will have
each element multiplied with the scalar.

Copyright Intellipaat. All rights

Vector Multiplication

5 2 9 1 While multiplying a matrix with

a column vector, we must
4 6 7 2 figure out the dimensions of the
resultant matrix. For this, the
1 3 8 4 rows of the first matrix and the
columns of the second matrix
will be the dimensions of the
resultant matrix.

Copyright Intellipaat. All rights

Vector Multiplication

5 2 9 1
4 6 7 2 The dimensions of the first
matrix is 3x3 and that of the
1 3 8 4 second matrix is 3x1. Therefore,
the dimensions of the resultant
matrix will be 3x1.

Copyright Intellipaat. All rights

Vector Multiplication

5 2 9 1 X
4 6 7 2 Y
1 3 8 4 Z

For finding the elements of the resultant matrix, we multiply the rows of the first matrix to the
column of the second matrix.

Copyright Intellipaat. All rights

Vector Multiplication

5 2 9 1 45
4 6 7 2 44
1 3 8 4 39

X = (5x1) + (2x2) + (9x4) = 5+4+36 = 45

Y = (4x1) + (6x2) + (7x4) = 4+12+28 = 44

Z = (1x1) + (3x2) + (8x4) = 1+6+32 = 39

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.dot() method to multiply the two
matrices.

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 1 2 Similar to the vector

multiplication, we will
4 6 7 2 1 determine the dimension of
the resultant matrix from
these two matrices.
1 3 8 1 2

Copyright Intellipaat. All rights

Matrix Multiplication

The dimension of the first

5 2 9 1 2 matrix is 3x3 and that of the
second matrix is 3x2.
4 6 7 2 1 Therefore the dimension of
the resultant matrix will be
1 3 8 1 2 3x2.

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 1 2 x1 x2
4 6 7 2 1 y1 y2
1 3 8 1 2 z1 z2

Copyright Intellipaat. All rights

Matrix Multiplication

X1 = (5x1) + (2x2) + (9x1) = 5+4+9 = 18

X2 = (5x2) + (2x1) + (9x2) = 10+2+18 = 30

5 2 9 1 2
X1 = (4x1) + (6x2) + (7x1) = 4+12+7 = 23
4 6 7 2 1
X2 = (4x2) + (6x1) + (7x2) = 8+6+14 = 28
1 3 8 1 2
X1 = (1x1) + (3x2) + (8x1) = 1+6+8 = 15

X2 = (1x2) + (3x1) + (8x2) = 2+3+16 = 21

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 1 2 18 30
4 6 7 2 1 23 28
1 3 8 1 2 15 21

There is a limitation with matrix multiplication that states that the columns of the first matrix should
always be equal to the rows of the second matrix. For example – if the dimensions of the above matrices
were 3x3 and 2x2, the multiplication would not have been possible.

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.dot() method to multiply the two
matrices.

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 5 2 9 The dimension of the first

matrix is 3x3 and that of the
4 6 7 4 6 7 vector is 3x3. Therefore, the
dimension of the resultant
1 3 8 1 3 8 matrix will be 3x3.

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 1 2 1 The approach would be

same here, we will multiply
4 6 7 3 1 2 the rows of the first matrix
to the column of the second
1 3 8 4 2 1 matrix.

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 1 2 1 47 30
4 6 7 3 1 2
1 3 8 4 2 1

X1 = (5x1) + (2x3) + (9x4) = 5+6+36 = 47

X2 = (5x2) + (2x1) + (9x2) = 10+2+18 = 30

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 1 2 1 47 30 ?
4 6 7 3 1 2 ? ? ?
1 3 8 4 2 1 ? ? ?

Similarly, you can find the rest of the elements of the resultant matrix.

Copyright Intellipaat. All rights

Matrix Multiplication

5 2 9 1 2 1 47 30 18
4 6 7 3 1 2 50 28 23
1 3 8 4 2 1 42 21 15

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.dot() method to multiply the two
matrices.

Copyright Intellipaat. All rights

Cross Product

3 2 i j k
2 9 3 2 8
8 7 2 9 7

2 8 3 8 3 2
i 9 7
j 2 7
k 2 9

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.cross() method to find the cross
product of the two matrices.

Copyright Intellipaat. All rights

Transpose

T 1 2
1 1 0
1 1
2 1 2
0 2

In the transpose of a matrix, the columns and rows are interchanged like
shown in the example.

Copyright Intellipaat. All rights

Properties of Transpose of a Matrix

Transpose of a Transposed matrix is the Additive property of transpose of a

original matrix matrix
(A’)’ = A (A+B)’ = A’ + B’

Multiplication by a Constant Multiplication property of the transpose:

K x A’ = KA’, Where K is a constant (AB)’ = B’A’

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.transpose() method to obtain the

transpose of the matrix E.

Copyright Intellipaat. All rights

Inverse

-1
5 2 9
4 6 7 To understand the inverse of a
matrix, there are several
concepts that you must be
1 3 8 familiar with like determinant
of a matrix, adjoint of a matrix,
etc.

Copyright Intellipaat. All rights

Inverse – Determinant of a Matrix

5 2 9
4 6 7 We can follow the laplacian
expansion method to find the
1 3 8 determinant of the matrix. It
helps us in finding the inverse
of the matrix.

Copyright Intellipaat. All rights

Inverse – Determinant of a 2x2
Matrix

a b
|Matrix| = a x d – b x c
c d

1 4
|Matrix| = 1 x 9 – 4 x 6 = 9 – 24 = -15
6 9

Copyright Intellipaat. All rights

Inverse – Determinant of a 3x3
Matrix

a b c
|Matrix| = a(e x i – f x h) – b(d x i – f x g) + c(d x h – e x g)
d e f
g h i
2 2 -3
|Matrix| = 2(2 x 2 – 2 x 1) – 2(1 x 2 – 2 x 1) – 3(1 x 1 – 2 x 1)
1 2 2
= 2(2) – 2(0) – 3(-1)
= 4 – 0 – (-3)
=4+3=7
1 1 2

Copyright Intellipaat. All rights

Inverse – Adjoint of a Matrix

5 2 9
4 6 7 To find the adjoint of a given
matrix, we have to find the
transpose of the cofactors of
1 3 8 the elements in the given
matrix. It is also known as
adjugate of a matrix.

Copyright Intellipaat. All rights

Inverse – Adjoint of a Matrix

5 2 9 m11 m12 m13

4 6 7 m21 m22 m23

1 3 8 m31 m32 m33

The minor (m11, m12,… mij) are the determinants of the square matrix that is formed after
eliminating the ith row and jth column of the matrix. We use the minors to calculate the cofactors
of the given matrix.

Copyright Intellipaat. All rights

Inverse – Adjoint of a Matrix

5 2 9 m11 m12 m13

4 6 7 m21 m22 m23

1 3 8 m31 m32 m33

Formulae to find the cofactors = Cij = (-1)i+j|mij|

We will use the above formulae to calculate the cofactors of the given matrix.

Copyright Intellipaat. All rights

Inverse – Adjoint of a Matrix

m11 = 48 – 21 = 27 m22 = 40 – 9 = 31

m12 = 32 – 7 = 25 m23 = 15 -2 = 13

m13 = 12 – 6 = 6 m31 = 14 – 54 = -40

m21 = 16 – 27 = -11 m32 = 35 – 36 = -1

m33 = 30 – 8 = 22

Copyright Intellipaat. All rights

Inverse – Adjoint of a Matrix

T
27 -25 6 27 11 -40
11 31 -13 -25 31 1
-40 1 22 6 -13 22

Let’s take a look at the formulae to find the inverse of the matrix.

Copyright Intellipaat. All rights

Inverse – Formulae

-1
2 1 The formulae to calculate the inverse
of a matrix is as follows:
4 3 A-1 = 1/|A|. Adj A

Copyright Intellipaat. All rights

Inverse – Formulae

-1
2 1 Now, to calculate the inverse, we will
find the determinant and adjoint of
4 3 the given matrix.

Copyright Intellipaat. All rights

Inverse – Formulae

3 -1
Adjoint of the given matrix is

-4 2

Determinant of the given matrix is => 2x3 – 4x1 = 2

Copyright Intellipaat. All rights

Inverse – Formulae

-1
2 1 3/2 -1/2
4 3 -2 1

To verify the result we can use the inverse property that states that:
AA-1 = I , where I is the identity matrix.

Copyright Intellipaat. All rights

Inverse – Formulae

2 1 3/2 -1/2 1 0

4 3 -2 1 0 1

Since we are getting an identity matrix when we did AA-1, we have calculated the inverse of the
matrix successfully. We can follow the same formulae for the 3x3 matrix as well – where the adjoint
and the determinant will determine the inverse of the given matrix.

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.linalg.inv() method to calculate the

inverse of a 2x2 and a 3x3 matrices F and G respectively.

Copyright Intellipaat. All rights

Rank of a Matrix

1 0 0 The number of linearly

ρ(A) 0 1 0 independent columns in a
matrix is known as the rank of
the matrix. There are several
0 0 1 ways we can use to calculate
the rank of the matrix. One
such method is to reduce the
matrix to its row-echelon form.

Copyright Intellipaat. All rights

Rank of a Matrix

Things to Remember:

1 0 0 1. If the matrix A is of the

order mxm, and if the
0 1 0 determinant of |A| is not
equal to zero, then the
0 0 1 rank of the matrix will be
m.
2. If the determinant is equal
to zero, the rank will be
less than m.

Copyright Intellipaat. All rights

Rank of a Matrix

Row Echelon Form

1 0 0 1. The pivots go from top-left

to bottom right. Pivots are
0 1 0 the first non-zero element
in the row.
0 0 1 2. There must be a zero
below the pivots in the row
echelon form.
3. The rows will all zero
elements must be at the
end of the matrix.

Copyright Intellipaat. All rights

Rank of a Matrix

1 2 2 1 2 2
0 1 4 0 1 4
2 4 4 0 0 0

We can use the gaussian elimination method where we interchange, multiply and
add the rows to reach the row echelon form. Here, we have used the following
operation to get the row echolen form.
1. R3 = R3 – 2R1
Since, the non-zero rows are 2, hence the rank of the matrix is 2.

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.linalg.matrix_rank() method to

calculate the rank of the given matrix.

Copyright Intellipaat. All rights

Characteristic Polynomial

A characteristic polynomial is used to derive the Eigen values of a given matrix. The
characteristic polynomial of a matrix A is denoted by the function given below.

The characteristic polynomial of the square matrix A is given by the function f(λ), and the I is
the identity matrix. The whole objective of using the characteristic polynomial is to find the
Eigen values of the given matrix.

Copyright Intellipaat. All rights

Eigen Value

Eigen values are the roots of the characteristic polynomial of the matrix A. Let’s understand
this with a simple example.

To calculate the Eigen values of the given

2 1 matrix, we will first compute the
characteristic polynomial and then derive
the Eigen values by finding the roots of
4 3 the Characteristic polynomial.

Copyright Intellipaat. All rights

Eigen Value

To find the characteristic polynomial, we will

5 2 calculate the determinant of (A - λIn ).

The computation will get the resultant

2 1 characteristic polynomial as:

f(λ ) = λ2 - 6 λ +1

Now, to find the eigen values, we will calculate the roots of the characteristic polynomial.

Copyright Intellipaat. All rights

Eigen Values

5 2 f(λ ) = λ2 - 6 λ +1
2 1 -> λ2 - 6 λ +1 = 0
Using the quadratic formula, we can get the roots of the
equation as 3 + 2√2 and 3 - 2√2.

Therefore, the Eigen values for the given matrix are 3 + 2√2 and 3 - 2√2

Copyright Intellipaat. All rights

Eigen Vectors

Now to calculate the Eigen vectors, we will do the

5 2 following:
1. We will use the equation, Av = λv where v is the eigen
vector.
2 1

5 2 X X
λ .
2 1 Y Y

Copyright Intellipaat. All rights

Eigen Vectors

5 2 X X
λ .
2 1 Y Y

Using the equations that we get by solving the above equation for both the values of λ, we will
get the Eigen Vectors of the given matrix.

Copyright Intellipaat. All rights

Hands on - Python

Output

In this example, we have used the numpy.linalg.eig() method to calculate the

eigen values and eigen vectors of the given matrix M.

Copyright Intellipaat. All rights

Correlation Matrix

Collinearity is a concept in statistics that resonates with the linear relationship between the
predictor variables(independent variable) in a regression model. If the independent variables
are correlated, they won’t be able to independently predict the value of the dependent
variable in the regression model.

A correlation matrix is a table that has the correlated

coefficients between the variables. It is mostly used to
deduce the patterns in a large dataset.

Copyright Intellipaat. All rights

Hands on - Python

In this example, we have plotted the

correlation coefficients of the titanic data
on a heatmap using seaborn in python. By
the value of the correlation coefficient, we
can figure out the collinearity of the
predictor variables.

Copyright Intellipaat. All rights

India: +91-
7847955955
US: 1-800-216-8930 (TOLL
FREE)

support@intellipaat.
com

24/7 Chat with Our Course

Advisor

Copyright Intellipaat. All rights