Linear Algebra

Tushar B. Kute,
http://tusharkute.com
Linear Algebra

• Linear algebra is a branch of mathematics that

deals with linear equations and their
representations in the vector space using
matrices.
• In other words, linear algebra is the study of
linear functions and vectors.
• It is one of the most central topics of
mathematics.
• Most modern geometrical concepts are based
on linear algebra.
Linear Algebra

• Linear algebra facilitates the modeling of many

natural phenomena and hence, is an integral
part of engineering and physics.
• Linear equations, matrices, and vector spaces
are the most important components of this
subject.
Linear Algebra

• The branch of mathematics that deals with

vectors, matrics, finite or infinite dimensions as
well as a linear mapping between such spaces is
defined as linear algebra.
• It is used in both pure and applied mathematics
along with different technical forms such as
physics, engineering, natural sciences, etc.
Linear Algebra: Branches

• Linear algebra can be categorized into three

branches depending upon the level of difficulty
and the kind of topics that are encompassed
within each.
• These are
– elementary,
– advanced, and
– applied linear algebra.
Elementary Linear Algebra

• Elementary linear algebra introduces students

to the basics of linear algebra.
• This includes simple matrix operations, various
computations that can be done on a system of
linear equations, and certain aspects of vectors.
Elementary Linear Algebra

• Scalars - A scalar is a quantity that only has

magnitude and not direction. It is an element that is
used to define a vector space. In linear algebra,
scalars are usually real numbers.
• Vectors - A vector is an element in a vector space. It
is a quantity that can describe both the direction
and magnitude of an element.
• Vector Space - The vector space consists of vectors
that may be added together and multiplied by
scalars.
Elementary Linear Algebra

• Matrix - A matrix is a rectangular array wherein

the information is organized in the form of rows
and columns. Most linear algebra properties can
be expressed in terms of a matrix.
• Matrix Operations - These are simple arithmetic
operations such as addition, subtraction, and
multiplication that can be conducted on
matrices.
Advanced Linear Algebra

• Once the basics of linear algebra have been

introduced to students the focus shifts on more
advanced concepts related to linear equations,
vectors, and matrices.
• Certain important terms that are used in
advanced linear algebra are as follows:
– Linear Transformations - The transformation of a
function from one vector space to another by
preserving the linear structure of each vector
space.
Advanced Linear Algebra

• Inverse of a Matrix - When an inverse of a matrix

is multiplied with the given original matrix then
the resultant will be the identity matrix. Thus, A-
1
A = I.
• Eigenvector - An eigenvector is a non-zero vector
that changes by a scalar factor (eigenvalue) when
a linear transformation is applied to it.
• Linear Map - It is a type of mapping that preserves
vector addition and vector multiplication.
Advanced Linear Algebra

• Applied linear algebra is usually introduced to

students at a graduate level in fields of applied
mathematics, engineering, and physics.
• This branch of algebra is driven towards integrating
the concepts of elementary and advanced linear
algebra with their practical implications.
• Topics such as the norm of a vector, QR
factorization, Schur's complement of a matrix, etc.,
fall under this branch of linear algebra.
Applications

• Signal Processing - Linear algebra is used in encoding and

manipulating signals such as audio and video signals.
Furthermore, it is required in the analysis of such signals.
• Linear Programming - It is an optimizing technique that is
used to determine the best outcome of a linear function.
• Computer Science - Data scientists use several linear
algebra algorithms to solve complicated problems.
• Prediction Algorithms - Prediction algorithms use linear
models that are developed using concepts of linear
algebra.
Scaler / Real Numbers

• Real number is any number that can be found in the

real world. We find numbers everywhere around us.
• Natural numbers are used for counting objects,
rational numbers are used for representing
fractions, irrational numbers are used for
calculating the square root of a number, integers
for measuring temperature, and so on.
• These different types of numbers make a collection
of real numbers.
Scaler / Real Numbers

• Any number that we can think of, except complex

numbers, is a real number. For example, 3, 0, 1.5,
3/2, √5, and so on are real numbers.
• Definition of Real Numbers
– Real numbers include rational numbers like positive
and negative integers, fractions, and irrational
numbers.
– Now, which numbers are not real numbers? The
numbers that are neither rational nor irrational are
non-real numbers, like, √-1, 2 + 3i, and -i. These
numbers include the set of complex numbers, C.
Vector

• Vectors are geometrical entities that have magnitude

and direction.
• A vector can be represented by a line with an arrow
pointing towards its direction and its length
represents the magnitude of the vector.
• Therefore, vectors are represented by arrows, they
have initial points and terminal points.
• The concept of vectors was evolved over a period of
200 years.
• Vectors are used to represent physical quantities such
as displacement, velocity, acceleration, etc.
Vector

• A vector is a Latin word that means carrier. Vectors

carry a point A to point B.
• The length of the line between the two points A
and B is called the magnitude of the vector and the
direction of the displacement of point A to point B
is called the direction of the vector AB.
• Vectors are also called Euclidean vectors or Spatial
vectors.
• Vectors have many applications in maths, physics,
engineering, and various other fields.
Vector
Vector in Euclidean Geometry

• Vectors in math is a geometric entity that has

both magnitude and direction.
• Vectors have an initial point at the point where
they start and a terminal point that tells the
final position of the point.
• Various operations can be applied to vectors
such as addition, subtraction, and
multiplication.
Vector Representation

• Vectors are usually represented in bold lowercase

such as a or using an arrow over the letter as →a.
• Vectors can also be denoted by their initial and
terminal points with an arrow above them, for
example, vector AB can be denoted as −−→AB.
• The standard form of representation of a vector is
→A=a^i+b^j+c^k.
• Here, a,b,c are real numbers and ^i,^j,^k are the unit
vectors along the x-axis, y-axis, and z-axis
respectively.
Vector Representation
Matrices

• Matrices, the plural form of a matrix, are the

arrangements of numbers, variables, symbols, or
expressions in a rectangular table that contains
various numbers of rows and columns.
• They are rectangular-shaped arrays, for which
different operations like addition, multiplication, and
transposition are defined.
• The numbers or entries in the matrix are known as its
elements.
• Horizontal entries of matrices are called rows and
vertical entries are known as columns.
Matrix

• A matrix is a rectangular array of numbers,

variables, symbols, or expressions that are
defined for the operations like subtraction,
addition, and multiplications.
• The size of a matrix (which is known as the
order of the matrix) is determined by the
number of rows and columns in the matrix.
• The order of a matrix with 6 rows and 4 columns
is represented as a 6 × 4 and is read as 6 by 4.
Matrix

• For example, the given matrix B is a 3 × 4 matrix

and is written as [B]3×4
Matrix
Matrix Notations

• If a matrix has m rows and n columns, then it

will have m × n elements.
• A matrix is represented by the uppercase letter,
in this case, 'A', and the elements in the matrix
are represented by the lower case letter and
two subscripts representing the position of the
element in the number of row and column in
the same order, in this case, 'aij', where i is the
number of rows, and j is the number of columns.
Matrix Notations

• For example, in the given matrix A, element in

the 3rd row and 2nd column would be a32, can
be verified in the matrix given below:
Matrix Operations

• The basic operations that can be performed on

matrices are:
– Addition of Matrices
– Subtraction of Matrices
– Scalar Multiplication
– Multiplication of Matrices
– Transpose of Matrices
Matrix Addition

• The addition of matrices can only be possible if

the number of rows and columns of both the
matrices are the same.
• While adding 2 matrices, we add the
corresponding elments. i.e.,
– (A + B) = [aij] + [bij] = [aij + bij],
where i and j are the number of rows and
columns respectively.
Matrix Addition

• For example:
Matrix Substraction

• Matrices subtraction is also possible only if the

number of rows and columns of both the
matrices are the same.
• While subtracting 2 matrices, we subtract the
corresponding elements. i.e.,
– (A - B) = [aij] - [bij] = [aij - bij],
where i and j are the row number and column
number respectively.
Matrix Substraction

• For example:
Scalar Multiplication

• The product of a matrix A with any number 'c' is obtained by

multiplying every entry of the matrix A by c, is called scalar
multiplication. i.e., (cA)ij = c(Aij)
• Properties of scalar multiplication in matrices:
• The different properties of matrices for scalar multiplication of
any scalars K and l, with matrices A and B are given as,
– K(A + B) = KA + KB
– (K + l)A = KA + lA
– (Kl)A = K(lA) = l(KA)
– (-K)A = -(KA) = K(-A)
– 1·A = A
– (-1)A = -A
Matrix Multiplication

• Matrices multiplication is defined only if the

number of columns in the first matrix and rows
in the second matrix are equal.
Matrix Multiplication
Matrix Multiplication
Matrix Multiplication: Properties

• There are different properties associated with the

multiplication of matrices.
• For any three matrices A, B, and C:
– AB ≠ BA
– A(BC) = (AB)C
– A(B + C) = AB + AC
– (A + B)C = AC + BC
– AIm = A = AIn, for identity matrices Im and In.
– Am×nOn×p = Om×p, where O is a null matrix.
Matrix Multiplication: Example

import numpy as np

# Create two matrices

A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])

# Calculate the product of the two matrices

C = np.matmul(A, B)

# Print the product

print(C)
Matrix Multiplication: Applications

• Matrix multiplication is a powerful operation that

can be used to perform a variety of tasks, such as:
– Solving systems of linear equations
– Transforming vectors and matrices
– Calculating statistics
– Performing machine learning algorithms
Matrix Transpose

• The transpose of a matrix is done when we

replace the rows of a matrix to the columns and
columns to the rows.
• Interchanging of rows and columns is known as
the transpose of matrices. In the matrix given
below, we have row elements as row-1: 2, -3, -4,
and row-2: -1, 7, -7.
• On transposing, we will get the elements in
column-1: 2, -3, -4, and column-2: -1, 7, -7,
Matrix Transpose
Matrix Transpose: Properties

• There are various properties associated with

transposition. For matrices A and B, given as,
– (AT)T = A
– (A + B)T = AT + BT, A and B being of the same
order.
– (KA)T= KAT, K is any scalar(real or complex).
– (AB)T= BTAT, A and B being conformable for the
product AB. (This is also called reversal law.)
Matrix Transpose: Example

import numpy as np

# Create a matrix
A = np.array([[1, 2], [3, 4]])

# Calculate the transpose of the matrix

B = np.transpose(A)

# Print the transpose

print(B)
Matrix Trace

• The trace of any matrix A, Tr(A) is defined as the

sum of its diagonal elements. Some properties of
trace of matrices are,
tr(AB) = tr(BA)
tr(A) = tr(AT)
tr(cA) = c tr(A), for a scalar 'c'
tr(A + B) = tr(A) + tr(B)
Matrix Trace

import numpy as np

# Create a matrix
A = np.array([[1, 2], [3, 4]])

# Calculate the trace of the matrix

trace = np.trace(A)

# Print the trace

print(trace)
Determinant of Matrix

• The determinant of a matrix is a number defined only

for square matrices.
• It is used in the analysis of linear equations and their
solution.
• The determinant formula helps calculate the
determinant of a matrix using the elements of the
matrix.
• Determinant of a matrix is equal to the summation of
the product of the elements of a particular row or
column with their respective cofactors. Determinant
of a matrix A is denoted as |A|.
Determinant of Matrix

• Let say we want to find the determinant of the

matrix:
Determinant of Matrix

import numpy as np

# Create a matrix
A = np.array([[1, 2], [3, 4]])

# Calculate the determinant of the matrix

determinant = np.linalg.det(A)

# Print the determinant

print(determinant)
Inverse of Matrix

• The inverse of any matrix is denoted as the matrix

raised to the power (-1), i.e. for any matrix "A", the
inverse matrix is denoted as A-1.
• The inverse of a square matrix, A is A-1 only when: A
× A-1 = A-1 × A = I.
• There is a possibility that sometimes the inverse of
a matrix does not exist if the determinant of the
matrix is equal to zero(|A| = 0).
• The inverse of a matrix is shown by A-1.
Inverse of Matrix

• Matrices inverse is calculated by using the following

formula:
A-1 = (1/|A|)(Adj A)
• Where,
|A| is the determinant of the matrix A and |A| ≠ 0.
Adj A is the adjoint of the given matrix A.
Inverse of Matrix

import numpy as np

# Create a matrix
A = np.array([[1, 2], [3, 4]])

# Calculate the inverse of the matrix

A_inv = np.linalg.inv(A)

# Print the inverse of the matrix

print(A_inv)
Rank of matrix

• The rank of a matrix A is defined as the

maximum number of linearly independent
row(or column) vectors of the matrix.
• That means the rank of a matrix will always be
less than or equal to the number of its rows or
columns.
• The rank of a null matrix is zero since it has no
independent row or column vectors.
Rank of matrix

import numpy as np

# Create a matrix
A = np.array([[1, 2], [3, 4]])

# Calculate the rank of the matrix

rank = np.linalg.matrix_rank(A)

# Print the rank

print(rank)
Types of Matrix

• There are various types of matrices based on the

number of elements and the arrangement of
elements in them.
• Row matrix: A row matrix is a matrix having a single
row is called a row matrix. Example: [1, −2, 4].
• Column matrix: A column matrix is a matrix having a
single column is called a column matrix.
• Example: [−1, 2, 5]T.
Types of Matrix

• Square matrix: A matrix having equal number of

rows and columns is called a square matrix. For
example:

• Rectangular Matrix: A matrix having unequal

number of rows and columns is called a rectangular
matrix. For example:
Types of Matrix

• Diagonal matrices: A matrix with all non-diagonal

elements to be zeros is known as a diagonal matrix.

• Identity matrices: A diagonal matrix having all the

diagonal elements equal to 1 is called an identity
matrix.
Types of Matrix

• Symmetric matrices: A square matrix D of size n×n

is considered to be symmetric if and only if D T= D.

• Skew-symmetric matrices-A square matrix F of size

n×n is considered to be skew-symmetric if and only
if FT= - F.
Types of Matrix

• Invertible Matrix: Any square matrix A is called

invertible matrix, if there exists another matrix B,
such that, AB = BA = In, where In is an identity matrix
with n × n.
• Orthogonal Matrix: Any square matrix A is
orthogonal if its transpose is equal to its inverse.
i.e., AT = A-1
Types of Matrix

• An orthogonal matrix is a square matrix whose

transpose is equal to its inverse. In other words, if Q
is an orthogonal matrix, then QT=Q−1.

• Orthogonal matrices are also known as unitary

matrices, because they preserve the magnitude of
vectors.
• This means that if v is a vector and Q is an
orthogonal matrix, then ∣∣Qv∣∣=∣∣v∣∣.
Orthogonal Matrix

• Orthogonal matrices have many important

properties, including the following:
– The determinant of an orthogonal matrix is
either 1 or -1.
– The eigenvalues of an orthogonal matrix are all
of magnitude 1.
– The columns of an orthogonal matrix form an
orthonormal basis, which means that they are all
of magnitude 1 and are orthogonal to each
other.
Orthonormal Matrix

• An orthonormal matrix is a square matrix whose

columns (and rows) are orthonormal vectors.
• In other words, the columns of an orthonormal matrix
are orthogonal (perpendicular) to each other and have
a magnitude of 1.
• Orthonormal matrices have many important properties,
including:
– Their inverse is equal to their transpose.
– Their determinant is equal to 1 or -1.
– They can be used to represent rotations and
reflections in Euclidean space.
Orthonormal Matrix

• Here is an example of an orthonormal matrix in two

dimensions:

[[1/sqrt(2), 1/sqrt(2)]
[-1/sqrt(2), 1/sqrt(2)]]

• This matrix represents a rotation of 45 degrees

counterclockwise.
• Orthonormal matrices are a powerful tool for representing
and manipulating vectors and matrices. They have many
important applications in a wide range of fields.
Eigen values

• The eigenvalues of matrix are scalars by which

some vectors (eigenvectors) change when the
matrix (transformation) is applied to it.
• In other words, if A is a square matrix of order n x n
and v is a non-zero column vector of order n x 1
such that Av = λv (it means that the product of A
and v is just a scalar multiple of v), then the scalar
(real number) λ is called an eigenvalue of the matrix
A that corresponds to the eigenvector v.
Eigen values

• The word "eigen" is from the German language

and it means "characteristic", "proper", or
"own".
• Thus, eigenvalues are also known as
"characteristic values" (or) "characteristic roots"
(or) "proper values", etc.
Eigen values
Eigen values

• From the definition of eigenvalues, if λ is an

eigenvalue of a square matrix A, then
Av = λv
• If I is the identity matrix of the same order as A,
then we can write the above equation as
Av = λ (Iv) (because v = Iv)
Av - λ (Iv) = 0
Taking v as common factor,
v (A - λI) = 0
Eigen values

• This represents a homogeneous system of linear

equations and it has a non-trivial solution only when
the determinant of the coefficient matrix is 0.

i.e., |A - λI| = 0

• This equation is called the characteristic equation

(where |A - λI| is called the characteristic
polynomial) and by solving this for λ, we get the
eigenvalues.
Eigen values

• Here is the step-by-step process used to find

the eigenvalues of a square matrix A.
– Take the identity matrix I whose order is the
same as A.
– Multiply every element of I by λ to get λI.
– Subtract λI from A to get A - λI.
– Find its determinant.
– Set the determinant to zero and solve for λ.
Eigen values

• Let us see the process of finding the

eigenvalues of a 2x2 matrix with an example
where we will find the eigenvalues of A

• Let λ represents its eigenvalue(s). The identity

matrix of order 2x2 is, I =
Eigen values

• Its determinant is,

• |A - λI| = (5 - λ) (2 - λ) - (1)(4)
= 10 - 5λ - 2λ + λ2 - 4
= λ2 - 7λ + 6
• The characteristic equation is,
|A - λI| = 0
λ2 - 7λ + 6 = 0
(λ - 6)(λ - 1) = 0
λ - 6 = 0; λ - 1 = 0
λ = 6; λ = 1
• Thus, the eigenvalues of matrix
A are 1 and 6.
Eigen Vector and Values : Example

• Example: For this matrix

• an eigenvector is:

• with a matching eigenvalue of 6

Eigen Vector and Values : Example

• Av gives us:

• λv gives us :

• Yes they are equal! So Av = λv as promised.

Eigne values: Applications

• Eigenvalues are used in electric circuits,

quantum mechanics, control theory, etc.
• They are used in the design of car stereo
systems.
• They are also used to design bridges.
• It is not surprising to know that eigenvalues are
also used in determining Google's page rank.
• They are used in geometric transformations.
Eigne values: Example

import numpy as np

# Create a matrix
A = np.array([[1, 2], [3, 4]])

# Calculate the eigenvalues and eigenvectors of the matrix

w, v = np.linalg.eig(A)

# Print the eigenvalues

print(w)

# Print the eigenvectors

print(v)
 Thank you
This presentation is created using LibreOffice Impress 7.4.1.2, can be used freely as per GNU General Public License

@mITuSkillologies @mitu_group @mitu-skillologies @MITUSkillologies

Web Resources
https://mitu.co.in
@mituskillologies http://tusharkute.com @mituskillologies

[email protected]
[email protected]