Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Review of Mathematical Techniques and Probability: Introduction to Linear Algebra

Chong It Tan

October 21, 2013

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Table of contents
1 2

Overview Vectors and Matrices Denitions Operations Special Matrices Determinant and Inverse Linear Independence System of Linear Equations General Form Matrix Solution Cramers Rule Eigenvalues and eigenvectors Denitions Properties
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Overview

Vectors and Matrices System of Linear Equations Eigenvalues and Eigenvectors

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Denitions
Vector Ordered n-tuple of objects is called a n-dimensional vector y = (y1 , y2 , . . . , yn ) Matrix A rectangular array with m rows and n columns is called a matrix with dimension m n a11 a12 a1n a21 a22 a2n A m n = . . . .. . . . . . . . am1 am2 amn
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Operations
Addition and Subtraction Two matrices can be added or subtracted if and only if they have the same dimension. (Conformability) The additions and subtractions are performed for corresponding elements. Example 1 5 2 4 1 + 1 0 6 2 3 8 7 2 4 5 1 0
Chong It Tan

= =

9 3 5 2 4 6 3 5

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Operations

Scalar multiplication There is no comformability condition. The scalar is used to multiply every element of the matrix. Example 2 3 4 2 5 1 = 12 6 15 3

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Operations
Matrix multiplication Given two matrices Amn and Bpq , if we want to nd the product of AB , the conformability condition is that the column dimension of A must be equal to the row dimension of B , i.e. n = p . Example 3.1 Given A23 = 2 1 6 0 3 4 and B33 8 1 2 = 2 3 0, 0 5 7 but not B33 A23 .

We can nd the product of A23 B33

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Operations
Matrix multiplication (cont.) The resulting product is of dimension m q . The operation is performed by multiplying the row elements of A with the column elements of B and summing up. Example 3.2 8 1 2 2 1 6 2 3 0 0 3 4 0 5 7 14 35 46 32 11 28

A23 B33 = =

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Operations
Laws of Matrix Operations Both matrix addition and multiplication are associative. (A + B ) + C = A + (B + C ) (AB )C = A(BC ) Matrix addition is commutative; matrix multiplication is not. A+B =B +A AB = BA Matrix multiplication is distributive. A(B + C ) = AB + AC
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Operations
Transpose The transpose of a matrix is obtained by interchanging its rows and columns, (AT )ij = (A)ji Example 4 1 7 3 2 0 1
T

1 2 = 7 0 3 1

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Operations
Properties of Transpose (AT )T = A (A + B )T = AT + B T (A)T = AT (AB )T = B T AT Example 5 1 3 2 4 2 5 0 8
T

2 5 0 8 1 3 2 4
T

= =

2 29 4 42

= =

2 4 29 42 2 4 29 42

2 0 5 8

1 2 3 4

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Special Matrices
Special Matrices Square Matrix: equal number of rows and columns. Diagonal Matrix: a square matrix where all the non-diagonal elements are 0. 3 0 0 e.g., 0 2 0 0 0 5 Triangular Matrix: a square matrix is called lower (upper) triangular if all the entries above (below) the main diagonal are 0. 4 1 0 1 0 0 e.g., 0 10 2 , 3 0 0 0 0 3 2 6 7
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Special Matrices
Special Matrices (cont.) Identity Matrix: a square matrix and 0s elsewhere. 1 I3 = 0 0 with 1s on the main diagonal 0 0 1 0 0 1 square matrix) with all 0 0 0

Null Matrix: a matrix (not neccesarily entries equal to 0. 0 0 0 034 = 0 0 0 0 0 0

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Special Matrices
Special Matrices (cont.) Symmetric Matrix: AT = A.

3 5 7 e.g., 5 1 2 7 2 0

Invertible Matrix: a square matrix having a multiplicative inverse B such that AB = BA = I . We denote the inverse of A as A1 , that is, A1 = B . Singular Matrix: a square matrix that is not invertible. Orthogonal Matrix: a matrix whose inverse equals to its transpose, A1 = AT .
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Properties of Inverse Not every square matrix has an inverse. Squareness is a necessary but not sucient condition. If A has an inverse, A is said to be non-singular. If A has an inverse A1 , then it is unique. A and A1 are inverses of each other. (A1 )1 = A (AB )1 = B 1 A1 (AT )1 = (A1 )T

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Finding Determinant and Inverse Given matrix A, Minor Mij is the determinant of submatrix obtained by deleting row i and column j from matrix A. Cofactor Cij = (1)i +j Mij . The matrix of cofactors is called cofactor matrix C, in which the transpose of C is the adjoint of A (adj A). The determinant of matrix A, det(A) is computed by summing the product of entries with its cofactor across any row OR any column. Inverse of A can be found analytically via A1 =
Chong It Tan Introduction to Linear Algebra

adj A det (A) .

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Example 6 Given A = M11 a11 a12 , a21 a22 = a22 , M12 = a21 ,

M21 = a12 ,

M22 = a11

C11 = a22 , C12 = a21 , C21 = a12 , C22 = a11 a22 a12 a22 a21 , adj A = C T = C= a21 a11 a12 a11 det (A) = A 1 = a11 a12 = a11 |a22 | a12 |a21 | = a11 a22 a12 a21 a21 a22

adj A det (A)

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Example 7 a11 a12 a13 Given A = a21 a22 a23 , a31 a32 a33 M11 = C11 = a22 a23 , a32 a33 M12 =

a21 a23 ... a31 a33

a22 a23 a a , C12 = 21 23 . . . a32 a33 a31 a33 C11 C21 C31 adj A = C T = C12 C22 C32 C13 C23 C33
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Example 7 (cont.) a11 a12 a13 a a a a a a a21 a22 a23 = a11 22 23 a12 21 23 + a13 21 22 a32 a33 a31 a33 a31 a32 a31 a32 a33 = a11 a22 a23 a a a a a21 12 13 + a31 12 13 a32 a33 a32 a33 a22 a23

= a11 a22 a33 + a12 a23 a31 + a13 a21 a32 a11 a23 a32 a12 a21 a33 a13 a22 a31

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 3 7 1 Given A = 2 4 0, compute the inverse of A. 0 0 2

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 3 7 1 Given A = 2 4 0, compute the inverse of A. 0 0 2 3 7 1 2 4 0 0 0 2

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 3 7 1 Given A = 2 4 0, compute the inverse of A. 0 0 2 3 7 1 2 4 2 0 4 0 2 4 0 = 3 = 52 +1 7 0 0 0 2 0 2 0 0 2

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 3 7 1 Given A = 2 4 0, compute the inverse of A. 0 0 2 3 7 1 2 4 2 0 4 0 2 4 0 = 3 = 52 +1 7 0 0 0 2 0 2 0 0 2 =0+0+2 3 7 = 52 2 4

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 (cont.) 3 7 1 Given A = 2 4 0, compute the inverse of A. 0 0 2

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 (cont.) 3 7 Given A = 2 4 0 0 8 14 adj A = 4 6 0 0

1 0, compute the inverse of A. 2 4 2, 26

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 (cont.) 3 7 Given A = 2 4 0 0 8 14 adj A = 4 6 0 0

1 0, compute the inverse of A. 2 2 7 1 4 13 26 13 3 1 1 26 2, A1 = 13 26 1 26 0 0 2

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Determinant and Inverse

Practice 1 (cont.) 3 7 Given A = 2 4 0 0 8 14 adj A = 4 6 0 0 Check: 3 AA1 = 2 0 .

1 0, compute the inverse of A. 2 2 7 1 4 13 26 13 3 1 1 26 2, A1 = 13 26 1 26 0 0 2 2 7 1 13 1 4 0 13 0 2 0

Chong It Tan

7 26 3 26

1 13 1 0 0 1 26 = 0 1 0 1 0 0 1 2

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Linear Independence
Denition A sequence of distinct vectors v1 , v2 , , vn is said to be linearly independent if and only if the only solution set of scalars ci s for c1 v1 + c2 v2 + + cn vn = 0 is c1 = c2 = = cn = 0, i.e. there is only a trivial solution. Equivalent denition: If we cannot express the following v1 = a2 v2 + a3 v3 + + an vn for some scalars a2 , a3 , , an , then the sequence of distinct vectors v1 , v2 , , vn is said to be linearly independent.
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Linear Independence
Alternative Method Form a new matrix by taking distinct vectors v1 , v2 , , vn as the columns and nd its determinant. If the determinant is non-zero, then they are said to be linearly independent. Otherwise they are linearly depedent. Example 8 1 0 0 It is easy to check that 0 , 2 , 0 are linearly 0 0 5 independent.
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Linear Independence
Practice 2 2 0 1 Determine if 3 , 1 , 9 are linearly independent. 8 2 7

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Linear Independence
Practice 2 2 0 1 Determine if 3 , 1 , 9 are linearly independent. 8 2 7 1 0 2 1 9 3 1 +0+2 =0 3 1 9 =1 2 8 7 2 7 2 8

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Operations Special Matrices Determinant and Inverse Linear Independence

Linear Independence
Practice 2 2 0 1 Determine if 3 , 1 , 9 are linearly independent. 8 2 7 1 0 2 1 9 3 1 +0+2 =0 3 1 9 =1 2 8 7 2 7 2 8 They are linearly dependent.

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

General Form Matrix Solution Cramers Rule

System of Linear Equations

Solving System of Linear Equations Suppose we want to solve the following a11 x1 + a12 x2 + a13 x3 = b1 a21 x1 + a22 x2 + a23 x3 = b2 a31 x1 + a32 x2 + a33 x3 = b3 We can express it in matrix a11 a12 a21 a22 a31 a32 form a13 x1 b1 a23 x2 = b2 a33 x3 b3
Introduction to Linear Algebra

In matrix notation, it can be written as Ax = b.

Chong It Tan

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

General Form Matrix Solution Cramers Rule

System of Linear Equations

Solving System of Linear Equations If A is invertible, we can nd the unique solution for x by simply pre-multiply both sides by A1 , so x = A1 b. If A is not invertible (with determinant equals to 0), we have more than one solution set for x if the system is consistent. If the system is inconsistent, it has no solution. Practice 3 Solve the following system 3x1 + 5x2 + 2x3 = 15 2x1 + 6x2 + 3x3 = 11 9x1 2x2 + 4x3 = 48
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

General Form Matrix Solution Cramers Rule

System of Linear Equations

Practice 3 (cont.) 3 5 2 A = 2 6 3 , A 1 = . . . 9 2 4 2 x = 1 7

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

General Form Matrix Solution Cramers Rule

System of Linear Equations

Cramers Rule The solution for a system of linear equations Ax = b can be solved by the following step: dene matrix Bi the same as the matrix A except that the i -th column is replaced by the vector b . the solution can be written as xi = det Bi det A

Practice 4 Redo Practice 3 by using Cramers Rule.

Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

General Form Matrix Solution Cramers Rule

System of Linear Equations

Practice 4 (cont.) 15 5 2 11 6 3 48 2 4 3 5 2 2 6 3 9 2 4 3 15 2 2 11 3 9 48 4 3 5 2 2 6 3 9 2 4 3 5 15 2 6 11 9 2 48 3 5 2 2 6 3 9 2 4

x1 =

, x2 =

, x3 =

x1 = 2, x2 = 1, x3 = 7

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Denitions A vector v is an eigenvector of a square matrix A if and only if Av = v where is the eigenvalue corresponding to v. The eigenvalues can be determined by solving the characteristic polynomial p (), dened as det (A I) = 0. For each eigenvalue i , we have a specic eigenvalue equation (A i I)v = 0.

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Properties The product of eigenvalues equals to the determinant of A. The eigendecomposition of A can be written as A = VV1 , where the i -th column of V is the eigenvector vi and is a diagonal matrix whose diagonal element ii is the eigenvalue i . Practice 5 Given A = 3 1 , nd the eigendecomposition of matrix A. 2 4

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Practice 5 (cont.) p () = 3 1 = (2 )(5 ) 2 4

Hence we have 1 = 2, 2 = 5. 1 1 x1 0 For 1 = 2, we have = , so if we let x1 = t , 2 2 x2 0 1 then x2 = t , so v1 = . 1 2 1 x1 0 , so if we let x1 = t , For 1 = 5, we have = 0 2 1 x2 1 . then x2 = 2t , so v2 = 2
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Practice 5 (cont.) Hence we can write the following eigendecomposition form 3 1 2 4 = 1 1 1 2 2 0 0 5 1 1 1 2

1

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Practice 6 2 1 1 2 , nd the eigendecomposition of A. Given A = 0 5 0 4 1

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Practice 6 2 1 1 2 , nd the eigendecomposition of A. Given A = 0 5 0 4 1 2 1 1 0 5 2 = (2 )(3 )(1 ) 0 4 1 Hence we have 1 = 2, 2 = 3, 3 = 1.

Chong It Tan

Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Practice 6 (cont.) 1 for 1 = 2, we have v1 = 0 0 0 for 2 = 3, we have v2 = 1 1 1 for 3 = 1, we have v3 = 1 2
Chong It Tan Introduction to Linear Algebra

Overview Vectors and Matrices System of Linear Equations Eigenvalues and eigenvectors

Denitions Properties

Eigenvalues and eigenvectors

Practice 6 (cont.) Hence we can write the following eigendecomposition form 1 2 1 1 1 0 1 2 0 0 1 0 1 0 5 2 = 0 1 1 0 3 0 0 1 1 0 4 1 0 1 2 0 0 1 0 1 2

Chong It Tan

Introduction to Linear Algebra