The document discusses key linear algebra concepts for machine learning, including vectors, systems of linear equations, matrices, matrix operations, determinants, matrix inverse, and solving systems of linear equations using Gaussian elimination. It also covers linear dependence and independence of vectors, basis, rank, and the image and kernel of transformations.
The document discusses key linear algebra concepts for machine learning, including vectors, systems of linear equations, matrices, matrix operations, determinants, matrix inverse, and solving systems of linear equations using Gaussian elimination. It also covers linear dependence and independence of vectors, basis, rank, and the image and kernel of transformations.
The document discusses key linear algebra concepts for machine learning, including vectors, systems of linear equations, matrices, matrix operations, determinants, matrix inverse, and solving systems of linear equations using Gaussian elimination. It also covers linear dependence and independence of vectors, basis, rank, and the image and kernel of transformations.
The document discusses key linear algebra concepts for machine learning, including vectors, systems of linear equations, matrices, matrix operations, determinants, matrix inverse, and solving systems of linear equations using Gaussian elimination. It also covers linear dependence and independence of vectors, basis, rank, and the image and kernel of transformations.
Linear Algebra DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Linear algebra is the study of vectors and certain rules to manipulate vectors DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Manipulating vectors DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Systems of Linear Equations DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Matrix
M. P. Deisenroth, A. A. Faisal and Ch. S. Ong, Mathematics for Machine Learning, p. 22
DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Matrix DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Matrix Addition
A+B=B+A DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Matrix Multiplication
AB≠BA matrix multiplication is not commutative DAMA50 - MATHEMATICS FOR MACHINE LEARNING
(Back to) Systems of Linear Equations DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Identity Matrix DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Inverse Matrix
Consider a square matrix A, if there is a matrix B so that
AB = I = BA
then B is called the inverse of A and denoted by A-1.
Not every matrix A possesses an inverse A−1.
DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Inverse of a 2x2 matrix DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Inverse of a 2x2 matrix DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Inverse of a 2x2 matrix
A matrix is invertible if and only if its determinant is non-zero.
DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Determinants Determinant of a 2x2 matrix
Determinant of a 3x3 matrix
DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Multiplication by a Scalar Multiply a matrix A by a scalar λ λ A = K, where Kij =λ Aij.
Transpose Matrix For a matrix A, the matrix B with bij = aji is called the transpose of A. We write B = AT. DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Transpose Matrix The transpose of a matrix A (denoted by ) is obtained by interchanging the rows and columns of A.
Example:
A matrix B is called symmetric if
Example: DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Matrix inverse The inverse of a square matrix denoted by is defined as follows provided We can calculate the inverse of a matrix using the following formula
where the adjugated matrix is defined as follows:
Example For a 2x2 matrix
For a 3x3 matrix A, we first calculate the cofactor matrix C.
The adjugated matrix is the transpose of C DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Why do we need the inverse of a square matrix?
If a system of linear equations is described by a square matrix A,
Ax = b
Then by applying the inverse of A in both sides we get:
A-1A x = A-1b Where, A-1A ≡ Ι, thus: x = A-1b
Because we can use it to solve the system of linear equations.
DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Solving Systems of Linear Equations (Gaussian Elimination) DAMA50 - MATHEMATICS FOR MACHINE LEARNING
Solving Systems of Linear Equations (Gaussian Elimination) DAMA50 - MATHEMATICS FOR MACHINE LEARNING Linearly (In)dependent Vectors DAMA50 - MATHEMATICS FOR MACHINE LEARNING Linearly (In)dependent Vectors (Example) DAMA50 - MATHEMATICS FOR MACHINE LEARNING Linearly (In)dependent Vectors (Example) DAMA50 - MATHEMATICS FOR MACHINE LEARNING Basis Consider a vector space V and a generating set A of V is called minimal if there exists no smaller set A that spans V. Every linearly independent generating set of V is minimal and is called a basis of V.
Rank The number of linearly independent columns of a matrix A equals the number of linearly independent rows and is called the rank of A and is denoted by rk(A). DAMA50 - MATHEMATICS FOR MACHINE LEARNING Basis (Example) DAMA50 - MATHEMATICS FOR MACHINE LEARNING Rank (Example) DAMA50 - MATHEMATICS FOR MACHINE LEARNING Image and Kernel of a transformation DAMA50 - MATHEMATICS FOR MACHINE LEARNING Kernel of a transformation (exercise).
