Linear Algebra

Dr. M. Tanveer

Department of Mathematics

Indian Institute of Technology Indore

January 27, 2023

Dr. M. Tanveer Linear Algebra January 27, 2023 1 / 164

Notations
N stands for the set of natural numbers.
Z stands for the set of all integers.
Q stands for the set of all rational numbers.
R stands for the set of all real numbers.
C stands for the set of all complex numbers.
Rn = {(x1 , x2 , ..., xn ) : xi ∈ R}.
Cn = {(z1 , z2 , ..., zn ) : zi ∈ C}.
F = R or C.

Dr. M. Tanveer Linear Algebra January 27, 2023 2 / 164

Definition: Matrix
A matrix is an m × n rectangular array of numbers called the entries or
elements of the matrix: A = [aij ]m×n , 1 ≤ i ≤ m; 1 ≤ j ≤ n.
 
a11 a12 . . . a1n
 a21 a22 . . . a2n 
A= . .. .. 
 
 .. . . 
am1 am2 . . . amn

where aij is the entry at the intersection of i th row and j th column.

The number of rows and columns in a matrix is called its
Dimension or Order.
Two matrices are said to be equal if they have the same order and
their corresponding entries are equal.
If m = n, then the matrix A is called Square Matrix.
The matrix A is said to be Diagonal Matrix if aij = 0, i 6= j.

Dr. M. Tanveer Linear Algebra January 27, 2023 3 / 164

Addition, Subtraction and Scalar Multiplication With Matrix
If A = [aij ] and B = [bij ] are matrices of the same size, then the
sum A + B is the matrix obtained by adding the entries of B to the
corresponding entries of A, and the difference A − B is the matrix
obtained by subtracting the entries of B from the corresponding
entries of A.
A + B = [cij ], cij = aij + bij and A − B = [cij ], cij = aij − bij .
Let A = [aij ] be an m × n matrix, and let k ∈ F be a scalar. Then
the matrix kA, the scalar multiplication of k and A, is the m × n
matrix. we define kA = [kaij ]   
6 −3 2 5 −6 −3
Examples: Let A = and B =
−7 0 4 −4 −2 −4
   
11 −9 −1 12 −6 4
A+B = , 2A = .
−11 −2 0 −14 0 8

Dr. M. Tanveer Linear Algebra January 27, 2023 4 / 164

Note:
The set of all m × n matrices is denoted by Mm,n (F).
In case where m = n, we write Mn (F) to denote the matrices of size
n × n.

Dr. M. Tanveer Linear Algebra January 27, 2023 5 / 164

Fundamental Properties of Addition and Scalar Multiplication
Let A, B, and C ∈ Mm,n (F) and c, d ∈ F, then
1 A + B = B + A. Commutative Law of Addition
2 A + (B + C ) = (A + B) + C . Associative Law of Addition
3 Om×n + A = A + Om×n = A. Existence of Identity Element For
Addition
4 A + (−A) = (−A) + A = Om×n . Existence of Inverse Element For
Addition
5 c(A + B) = (cA) + (cB). Distributive Laws For Scalar
6 (c + d)A = (cA) + (dA). Multiplication over Addition
7 (cd)A = c(dA). Associativity of Scalar Multiplication
8 1A = A. Identity Property For Scalar Multiplication

Dr. M. Tanveer Linear Algebra January 27, 2023 6 / 164

Matrix Multiplication
If A = [aij ] is an m × n matrix and B = [bij ] is an n × p matrix, their
matrix product C = AB is theP m×p
n
matrix: C = AB = [cij ], cij = k=1 aik bkj .

Fundamental Properties of Matrix Multiplication

Suppose that A, B, and C are matrices for which the following sums and
products are defined. Let c be a scalar. Then
1 A(BC ) = (AB)C . Associative Law of Multiplication
2 A(B + C ) = AB + AC . Distributive Laws of Matrix Multiplication
Over Addition
3 (A + B)C = AC + BC .
4 c(AB) = (cA)B = A(cB). Associative Law of Scalar and Matrix
Multiplication

Dr. M. Tanveer Linear Algebra January 27, 2023 7 / 164

Types of Matrices
The square matrix A is said to be a diagonal matrix if aij = 0, i 6= j.
It is denoted as diag (d1 , d2 , ..., dn ) where,

aii = di aij = 0 j 6= i.

The diagonal matrix diag

 (1, 1, ..., 1)
is called the identity matrix
1 0 ... 0
0 1 . . . 0
and is denoted by In =  . . .. 
 
 .. .. .
0 0 ... 1
The diagonal matrix 0n = diag (0, 0, ..., 0) is called the zero matrix.
A matrix having only one row is called row matrix (or row vector)
(A = [aij ]1×n ).
A matrix having only one column is called column matrix ( or
column vector) (A = [aij ]m×1 ).

Dr. M. Tanveer Linear Algebra January 27, 2023 8 / 164

Lower Triangular and Upper Triangular Matrix
The square matrix A is said to be a lower triangular matrix if

aij = 0, i < j.
 
4 0 0
Example: A = 1 1 0
4 9 1
The square matrix A is said to be a upper triangular matrix if

aij = 0, i > j.
 
9 3 3
Example: A = 0 1 0
0 0 5
A matrix is said to be triangular if it is either lower or upper
triangular.

Dr. M. Tanveer Linear Algebra January 27, 2023 9 / 164

The Transpose of a Matrix and Its Properties
If A is an m × n matrix, then the transpose, AT , is the n × m matrix
whose (i, j) entry is the same as the (j,
 i) entry
 of A.
  9 3
9 7 5
Example: A = and AT = 7 1
3 1 0
5 0
Let A and B both be m × n matrices, and let c be a scalar. Then
1 (AT )T = A.
2 (A + B)T = AT + B T .
3 (cA)T = c(AT ).
If A is an m × n matrix and B is an n × p matrix, then
(AB)T = B T AT .

Dr. M. Tanveer Linear Algebra January 27, 2023 10 / 164

Symmetric and Skew-Symmetric Matrix
A matrix A is symmetric if and only if A = AT . A matrix A is
skew-symmetricif and only if A = −AT .
2 6 5
Example: A = 6 1 4  is symmetric matrix and
5 4 −2
 
0 2 −5
B = −2 0 −4 is skew-symmetric matrix.
5 4 0
In Skew-symmetric matrices, diagonal entries are zero. Since
aij = −aji for all i, j =⇒ aii = −aii =⇒ (1+1)aii = 0 =⇒ aii = 0.
T
A−AT
For any square matrix A, S = A+A
2 is symmetric, T = 2 is
skew-symmetric, and A = S + T .
If A and B both are symmetric matrices, then AB is symmetric if
and only if AB = BA.

Dr. M. Tanveer Linear Algebra January 27, 2023 11 / 164

Trace of a Matrix
For a square matrix A of order n, we define trace of A, denoted as tr (A)
as
Xn
tr (A) = aii .
i=1

Let A and B both be of same order, then

tr (A + B) = tr (A) + tr (B).
tr (AB) = tr (BA).

Dr. M. Tanveer Linear Algebra January 27, 2023 12 / 164

Multiplicative Inverse of a Matrix
Let A be an n × n matrix. Then an n × n matrix B is a
(multiplicative inverse) of A if and only if AB = BA = In . Inverse
of A is denoted as A−1 .
Uniqueness of the inverse matrix: if B and C are both inverses
of n × n matrix A, then B = C .
A square matrix A is singular if and only if it does not have an
inverse. A square matrix A is nonsingular if and only if it has an
inverse.
Let A and B be nonsingular n × n matrices. Then
1 A−1 is nonsingular, and (A−1 )−1 = A.
2 Ak is nonsingular, and (Ak )−1 = (A−1 )k = A−k , for any integer k.
3 AB is nonsingular, and (AB)−1 = B −1 A−1 .
4 AT is nonsingular, and (AT )−1 = (A−1 )T .

Dr. M. Tanveer Linear Algebra January 27, 2023 13 / 164

Conjugate Transpose of a Matrix
Let A = [aij ] be an m × n matrix over C. Then the conjugate of A,
denoted by Ā, is the matrix B = [bij ] with bij = a¯ij and the conjugate
transpose of A, denoted
 by A∗ , is thematrix B = [bij ] with bij = a¯ji . 
1 4 + 3i i 1 4 − 3i −i
Example: Let A = , then Ā =
0 1 i −2 0 1 −i − 2
 
1 0
and A∗ = 4 − 3i 1 
−i −i − 2

Dr. M. Tanveer Linear Algebra January 27, 2023 14 / 164

Hermitian and skew-Hermitian matrix
The square matrix A over C is said to be a Hermitian matrix if
A∗ = A.
The square matrix A over C is said to be a skew-Hermitian matrix if
A∗ = −A.  
1 1−i 2
Example: A = 1 + i 3 i  is Hermitian matrix.
2 −i 0
 
i 1−i 2
B = −1 − i 3i i  is skew-Hermitian Matrix.
−2 i 0
∗
A−A∗
For any square matrix A, S = A+A
2 is Hermitian, T = 2 is
skew-Hermitian, and A = S + T .

Dr. M. Tanveer Linear Algebra January 27, 2023 15 / 164

Orthogonal and Unitary Matrix
The matrix A is said to be orthogonal matrix if

AT A = AAT = In
 
1 0
Example: A =
0 −1
The square matrix A is said to be Unitary matrix if

A∗ A = AA∗ = In
 
1 1+i 1−i
Example: A = 2 1−i 1+i

Dr. M. Tanveer Linear Algebra January 27, 2023 16 / 164

Determinant of a Matrix
Let A be an n × n matrix, with n ≥ 2. The (i, j) submatrix, Aij , of
A is the (n − 1) × (n − 1) matrix obtained by deleting all entries of
the i th row and all entries of the j th column of A. The (i, j)
minor,|Aij |, of A is the determinant of the submatrix Aij of A.
Let A be an n × n, with n ≥ 2.The (i, j) cofactor of A, Aij , is
(−1)i+j times the (i, j) minor of A. That is Aij = (−1)i+j |Aij |
Let A be an n × n matrix. The determinant of A, denoted |A| or
det(A), is defined as follows:
If n = 1, then det(A) = a11 .
If n ≥ 2, then ai1 Ai1 + ai2 Ai2 + ... + ain Ain = |A|, for each i,
1 ≤ i ≤ n.
If n ≥ 2, then a1j A1j + a2j A2j + ... + anj Anj = |A|, for each j,
1 ≤ j ≤ n.

Dr. M. Tanveer Linear Algebra January 27, 2023 17 / 164

Note:
If A is n × n matrix, and k is any scalar, then det(kA) = k n det(A).
An n × n matrix A is nonsingular if and only if det(A) 6= 0.
If A and B are both n × n matrices, then det(AB) = det(A)det(B).
If A is nonsingular, then det(A−1 ) = 1
det(A) .

If A is an n × n matrix, then det(A) = det(AT ).

Dr. M. Tanveer Linear Algebra January 27, 2023 18 / 164

The Adjoint Matrix
Let A be an n × n matrix, with n ≥ 2. The Adjoint A of A is the n × n
matrix whose (i, j) entry is Aji , the (j, i) cofactor of A.

Theorem
If A is a nonsingular n × n matrix with adjoint matrix A, then
1
A−1 = ( )A.
|A|

Dr. M. Tanveer Linear Algebra January 27, 2023 19 / 164

Theorem
If A is an n × n matrix with adjoint matrix A, then

AA = AA = (|A|)In

Proof:  
a11 a12 ... a1n A11 A21 ... An1
 a21 a22 ... a2n  A12 A22 ... An2 
AA =  . .. ..   .. .. .. 
  
 .. . .  . . . 
am1 am2 . . . amn A1n A2n ... Ann
and (i, j)th element of AA will be,

ai1 Aj1 + ai2 Aj2 + ... + ain Ajn = |A| if i = j, otherwise 0 if i 6= j.

Therefore, AA = (|A|)In . Similarly we can show AA = (|A|)In

Dr. M. Tanveer Linear Algebra January 27, 2023 20 / 164

Linear Combination of Vectors
A vector w is called a linear combination of the vectors u1 , u2 , ..., un if it
can be expressed in the form

w = a1 u1 + a2 u2 + ... + an un

where ai ,1≤ i ≤ n are scalars.

Examples: Let u1 = (1, −1, 0), u2 = (1, 0, 2) and u3 = (0, −2, 5) be the
vectors in R3 , then the vector (1, −2, −2) is a linear combination of
these vectors,

(1, −2, −2) = 2(1, −1, 0) + (−1)(1, 0, 2)

Linearly Independent (L.I.) and Linearly Dependent (L.D.) Vectors

A set S = {u1 , ..., um } of vectors in Cn is called linearly independent if
the only choice of a1 , ..., am ∈ C that makes a1 u1 + ... + am um equal 0 is
a1 = ... = am = 0.
A set S = {u1 , ..., um } of vectors is called linearly dependent if it is not
linearly independent.
Dr. M. Tanveer Linear Algebra January 27, 2023 21 / 164
Elementary Operations
The following types of elementary operations can be performed on a
matrix A :
1 Interchange any two rows (columns) of A,i.e (Ri ↔ Rj or Ci ↔ Cj )
2 Multiply any row (column) by a nonzero constant,i.e (kRi or kCi ).
3 Adding in any row (column) to any multiple of any other row
(column),i.e (Ri + kRj or Ci + kCj ) .
If these operations are applied on rows only the they are called row
operations.
If these operations are applied on columns only then they are called
column operations

Dr. M. Tanveer Linear Algebra January 27, 2023 22 / 164

Row Echelon Form
A m × n matrix A is in row echelon form if it satisfies the following
properties:
1 Zero rows, if any, occurs at the bottom of the matrix.
2 First non-zero entry in any non-zero row is 1 which is known as
leading entry (pivot entry).
3 The number of zeros before leading entry (pivot entry) in i th row is
less than the number of zeros before leading entry in (i + 1)th row.

Example
  
0 1 2 3 0 1 0 1
The matrices 0 0 1 −2 3,0 1 5 are in row echelon form.
0 0 0 1 5 0 0 1
 
0 1 2 3 0
The matrix 0 1 1 −2 3 is not in row echelon form.
0 0 0 1 5

Dr. M. Tanveer Linear Algebra January 27, 2023 23 / 164

Row Reduced Echelon Form
A row echelon form matrix is called row reduced echelon form if all
the entries in the column in which leading entry (pivot entry) occurs are
zero.  
1 0 2 3
Example: The matrix 0 1 1 0 is in the row reduced echelon
0 0 0 0
form.  
1 0 2 3
The matrix 0 1 1 0 is not in the row reduced echelon form
0 0 0 1
though it is in row echelon form.

Dr. M. Tanveer Linear Algebra January 27, 2023 24 / 164

Note:
An matrix A is row equivalent to a matrix B if and only if A is
obtained from B by a finite number of row operations.
If a matrix A is row equivalent to a matrix B, then B is row
equivalent to A.

Theorem
Every matrix is row equivalent to a unique matrix in row reduced echelon
form.

Dr. M. Tanveer Linear Algebra January 27, 2023 25 / 164

Rank of a Matrix
Let A be a matrix. Then the rank of A is the number of nonzero
rows in the unique row reduced echelon form matrix that is row
equivalent to A 
and is denoted by rank(A)
 or ρ(A).
1 −2 3 9
Example: A = −1 3 0 −4 using row operations, we
2 −5 5 17
 
1 0 0 1
obtain row reduced echelon form as 0 1 0 −1. Here we have
0 0 1 2
3 non zero rows. Therefore, ρ(A) = 3.

Dr. M. Tanveer Linear Algebra January 27, 2023 26 / 164

Important Properties on the Rank
1 ρ(A) ≤ min(m, n), where A = [aij ]m×n .
2 ρ(AB) ≤ min(ρ(A), ρ(B)).
3 ρ(A + B) ≤ ρ(A) + ρ(B).
4 ρ(A) = ρ(AT ) = ρ(A∗ ).

Dr. M. Tanveer Linear Algebra January 27, 2023 27 / 164

Proof (1)
The rank of any matrix is the number of linearly independent rows,
which is the same as the number of linearly independent columns. The
maximum this value can be is therefore the maximum of the minimum of
the dimensions of the matrix, or ρ(A) ≤ min(m, n).

Proof (2)
The product AB can be viewed in two ways. The first is as a set of
linear combinations of the rows of B, and the other is as a set of linear
combinations of the columns of A. In either case the number of linear
independent rows (or columns as the case may be) In other words, the
rank of the product AB cannot be greater than the number of linearly
independent columns of A nor greater than the number of linearly
independent rows of B. Another way to express this is as
ρ(AB) ≤ min(ρ(A), ρ(B)).

Note: Similarly you can see other properties also.

Dr. M. Tanveer Linear Algebra January 27, 2023 28 / 164

Theorem
Let A ∈ Mm,n (F). The following statements are equivalent
1 ρ(A) = k.
2 There exists exactly k linearly independent columns of A.
3 There exists exactly k linearly independent rows of A.

Dr. M. Tanveer Linear Algebra January 27, 2023 29 / 164

Row Space and Column Space and Null Space of a Matrix
Let A be an m × n matrix. The subset of Rn consisting of all vectors
that are linear combinations of the rows of A is called the Row
Space of A.
Let A be an m × n matrix. The subset of Rm consisting of all vectors
that are linear combinations of the columns of A is called the
Column Space of A.
Suppose that A and B are row equivalent matrices. Then the row
space of A equals the row space of B.
Let A be an m × n matrix. The Null Space of A is defined to be

Null(A) = {x : Ax = 0}

Dr. M. Tanveer Linear Algebra January 27, 2023 30 / 164

  
3 1 −2
Examples: Consider the matrix A= 4 0 1 . We want to deter-
−2 4 −3
mine whether (5,17,-20) is in the row space of A. If so, (5,17,-20) can be
expressed as a linear combination of the rows of A, as follow:

(5, 17, −20) = x(3, 1, −2) + y (4, 0, 1) + z(−2, 4, −3).

Equating the coordinates on each side leads to the following system:

3x + 4y − 2z = 5
x + 4z = 17
−2x + y − 3z = −20

Hence, the solution of this system is x = 5, y = −1 and z = 3.

Therefore, (5,17,-20) is in the row space of A.
Similarly, we can do for column space of A also.

Dr. M. Tanveer Linear Algebra January 27, 2023 31 / 164

Systems of Linear Equations
A linear equation in the n variables x1 , x2 , ..., xn is an equation that
can be written in the form

a1 x1 + a2 x2 + ...an xn = b

where the coefficients a1 , a2 , ..., an and the constant term b are

constants.
A system of linear equations is a finite set of linear equations, each
with same variables.
A solution of a system of linear equations is a vector that is
simultaneously a solution of each equation in the system.
The solution set of a system of linear equations is the set of all
solutions of the system.

Dr. M. Tanveer Linear Algebra January 27, 2023 32 / 164

Systems of Linear Equations
The following system has m linear equations and n variables:

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
.. .. .. ..
. . . .
am1 x1 + am2 x2 + · · · + amn xn = bm

Example:

x1 + 2x2 + 9x3 = 8
7x1 + 3x2 + 8x3 = 2
x1 − x2 − 2x3 = −1

Dr. M. Tanveer Linear Algebra January 27, 2023 33 / 164

System of Linear Equation in Matrix Form
we can write the system of linear equations in matrix form as AX = b
where,
     
a11 a12 . . . a1n x1 b1
 a21 a22 . . . a2n   x2   b2 
A= . .. .. , X= .. , and b= .. 
     
 .. . .   .   . 
am1 am2 ... amn xm bm
We can abbreviate the system by writing only the rectangular array of
numbers:
 
a11 a12 . . . a1n b1
 a21 a22 . . . a2n b2 
[A|b]= . .. .. .. 
 
 .. . . . 
am1 am2 ... amn bm
This is called the Augmented Matrix for the system.

Dr. M. Tanveer Linear Algebra January 27, 2023 34 / 164

Note:
A system of linear equation Ax = b is called Homogeneous
System if b = 0 (zero column matrix), otherwise it is called Non
Homogeneous System (b 6= 0).
The zero n-tuple (0,0,...,0) is always a solution of the system Ax=0,
and is called the Trivial Solution.
A non-zero n-tuple x, if it satisfies Ax=0, is called a Non-Trivial
Solution.
A system of linear equations has trivial solution, unique solution, or
infinitely many solutions.There are no other possibilities.
Two systems of m linear equations in n variables are equivalent
system if and only if they have exactly the same solution set.

Dr. M. Tanveer Linear Algebra January 27, 2023 35 / 164

Consistent and Inconsistent System of Linear Equation
A system of linear equations is called consistent if it has at least
one solution.
A system of linear equations with no solutions is called
inconsistent.
Examples: The system

x −y =1
x +y =3

is consistent and the solution is (2,1).

The system

x −y =1
x −y =3

is inconsistent,

Dr. M. Tanveer Linear Algebra January 27, 2023 36 / 164

 Figure: Types of Solution

Source:https://www.slideserve.com/terri/examples
Dr. M. Tanveer Linear Algebra January 27, 2023 37 / 164
 Figure: Types of Solution

Source:https://slideplayer.com/slide/14894118/

Dr. M. Tanveer Linear Algebra January 27, 2023 38 / 164

Gauss Elimination Method
1 Write the augmented matrix of the system of linear equation.
2 Use elementary row operation to reduce the augmented matrix to
row echelon form.
3 Using back substitution, solve the equivalent system to get the
solution of the system of linear equation.

Note:
After Gaussian elimination, the column having no pivot entries are
often referred to as nonpivot columns, while those with pivots are
called pivots columns.
The variables for nonpivot columns are called independent
variables or free variables, while those for pivots columns are
dependent variables

Dr. M. Tanveer Linear Algebra January 27, 2023 39 / 164

Example
Solve the system

2x2 + 3x3 = 8
2x1 + 3x2 + x3 = 5
x1 − x2 − 2x3 = −5

Dr. M. Tanveer Linear Algebra January 27, 2023 40 / 164

Solution:

The augmented matrix is

 
0 2 3 8
2 3 1 5 
1 −1 −2 −5

Now  we reduce this matrix

  Interchange R1 ↔
to row echelon form.
0 2 3 8 1 −1 −2 −5
R3 ,2 3 1 5  → 2 3 1 5 
1 −1 −2 −5 0 2 3 8

Dr. M. Tanveer Linear Algebra January 27, 2023 41 / 164

cont...

Make second zero in the first column so apply R2 → R2 + (−2)R1

 
1 −1 −2 −5
0 5 5 15 
0 2 3 8

Multiply R2 by 51 i.e( 51 R2 ) will make a leading 1,

 
1 −1 −2 −5
0 1 1 3 
0 2 3 8

Dr. M. Tanveer Linear Algebra January 27, 2023 42 / 164

Cont...

Now apply R3 → R3 + (−2)R2

 
1 −1 −2 −5
0 1 1 3 
0 0 1 2

The augmented matrix is now in row echelon form. The corresponding

system is

x1 − x2 − 2x3 = −5
x2 + x3 = 3
x3 = 2

then using back substitution we get solution as x1 = 0, x2 = 1 and

x3 = 2. Here x1 , x2 and x3 are dependent variables.

Dr. M. Tanveer Linear Algebra January 27, 2023 43 / 164

Examples for Linear Combination
Question: Is the vector (1,2,3) a linear combination of the vectors
(1,0,3) and (-1,1,-3).
Solution: We want to find the scalar x and y such that
     
1 −1 1
x 0 + y  1  = 2
3 −3 3

Now we obtain the system of linear equation:

x −y =1
y =2
3x − 3y = 3

Solve this system by Gaussian elimination method and we get x=3 and
y=2.

Dr. M. Tanveer Linear Algebra January 27, 2023 44 / 164

Example
1 Consider the system x + y + z = 3, x + 4y + 2z = 7 and
4x + 10y − z = 7. The unique solution to this system is
(x, y , z) = (1, 1, 1);i.e The three planes intersect at a point.
2 Consider the system x + y + z = 3, x + 2y + 2z = 5 and
3x + 4y + 4z = 11. The set of solution to this system is
(x, y , z) = (1, 2 − z, z), with z is arbitrary.i.e the three planes
intersect on a line.
3 The system x + y + z = 3, x + 2y + 2z = 5 and 3x + 4y + 4z = 13
has no solution.

Dr. M. Tanveer Linear Algebra January 27, 2023 45 / 164

Theorem: Homogeneous System and Rank
Let AX = O be a homogeneous system in n variables.
(1) If ρ(A) < n, then the system has a nontrivial solution.
(2) If ρ(A) = n, then the system has only the trivial solution

Result
Let AX = O be a homogeneous system of m linear equations in n
variables. If m < n, then the system has a nontrivial solution.

Dr. M. Tanveer Linear Algebra January 27, 2023 46 / 164

Example: Consider the following homogeneous system:

x1 − 3x2 + 2x3 − 4x4 + 8x5 + 17x6 = 0

3x1 − 9x2 + 6x3 − 12x4 + 24x5 + 49x6 = 0
−2x1 + 6x2 − 5x3 + 11x4 − 18x5 − 40x6 = 0

Because this homogeneous system has fewer equations than variables, it

has a nontrivial solution.

Dr. M. Tanveer Linear Algebra January 27, 2023 47 / 164

Theorem: Existence and Non-Existence
Consider a linear system Ax = b, where A is a m × n matrix, and x, b are
vectors with orders n × 1, and m × 1, respectively. Then exactly one of
the following statement holds:
1 If ρ(A) = ρ[A|b] < n, then the linear system has Infinite Solution.
2 If ρ(A) = ρ[A|b] = n, then the linear system has Unique Solution.
3 If ρ(A) < ρ[A|b], then the linear system has No Solution.

Dr. M. Tanveer Linear Algebra January 27, 2023 48 / 164

Example: For what value of the system

x +y +z =1
x + 2y + 4z = a
x + 4y + 10z = a2

is consistent.  
1 1 1 1
Solution: Augmented matrix, [A|b] = 1 2 4 a 
1 4 10 a2
Apply R2 → R2 + R1 (−1) and R3 → R3 + R1 (−1),
we get,  
1 1 1 1
0 1 3 a − 1 
0 3 9 a2 − 1

Dr. M. Tanveer Linear Algebra January 27, 2023 49 / 164

Cont...

 
1 1 1 1
Now apply R3 → R3 + R2 (−3), we get 0 1 3 a−1 
0 0 0 a2 − 3a + 2
This implies that,
0x + 0y + 0z = a2 − 3a + 2 = 0 ⇒ a = 2, 1.
Now for a = 1, we have y + 3z = 0 ⇒ y = −3z and x + y + z = 1 ⇒
x = 1 + 2z.
Here x and y are dependent variable and z is free variable.Take z = r ,
then y = −3r and x = 1 + 2r for any r .
Therefore, given system has infinite number of solutions.

Dr. M. Tanveer Linear Algebra January 27, 2023 50 / 164

Note:
For an n × n matrix A, the following conditions are equivalent.
1 A is singular.
2 ρ(A) 6= n.
3 |A| = 0.
4 The linear system Ax = 0 has a non-trivial solution.
5 The linear system Ax = b doesn’t have a unique solution.

Dr. M. Tanveer Linear Algebra January 27, 2023 51 / 164

Question1: Find the rank of the given matrix
 
1 0 −2 1 0
 0 −1 −3 1 3 
A= −2 −1 1 −1
.
3 
0 3 9 0 −12

Question2: Check whether the given system is consistent or inconsis-

tent:

2x − y + z = 4
3x − y + z = 6
4x − y + 2z = 7
−x + y − z = 9

Dr. M. Tanveer Linear Algebra January 27, 2023 52 / 164

Definition: Vector Space
A non empty set V is said to be a vector space over F, denoted V (F),
together with an operation called vector addition (a rule that takes two
elements of V to obtain a third element of V ) and another operation
called scalar multiplication (a rule that takes an element of V and
another element from F to obtain a new element of V ), satisfying the
following properties:
For every u, v , and w in V and for every a and b in F,
u + v ∈ V . Closure Property of Addition
au ∈ V . Closure Property of Scalar Multiplication
u + v = v + u. Commutative Law of Addition
u + (v + w ) = (u + v ) + w . Associative Law of Addition
There is an element 0 of V so that for every u in V we have
0 + u = u + 0 = u. Existence of Identity Element for Addition

Dr. M. Tanveer Linear Algebra January 27, 2023 53 / 164

cont....
There is an element −u ∈ V such that u + (−u) = 0 = (−u) + u.
Existence of Additive Inverse
a(u + v ) = (au) + (av ). Distributive Laws for Scalar
(a + b)u = (au) + (bu). Multiplication over Addition
(ab)u = a(bu). Associativity of Scalar Multiplication
1u = u. Identity Property for Scalar Multiplication

Dr. M. Tanveer Linear Algebra January 27, 2023 54 / 164

cont....
There is an element −u ∈ V such that u + (−u) = 0 = (−u) + u.
Existence of Additive Inverse
a(u + v ) = (au) + (av ). Distributive Laws for Scalar
(a + b)u = (au) + (bu). Multiplication over Addition
(ab)u = a(bu). Associativity of Scalar Multiplication
1u = u. Identity Property for Scalar Multiplication

Note:
1 The Elements of V , F are called vectors and scalars respectively.
2 A Vector Space has a unique additive identity.
3 Every element in a Vector Space has a unique additive inverse.
4 If F = R, the vector space is called a Real Vector Space.
5 If F = C, the vector space is called a Complex Vector Space.

Dr. M. Tanveer Linear Algebra January 27, 2023 54 / 164

Examples:
1 Let V = R and F = R, then R(R) is a vector space under the usual
operation of addition and scalar multiplication.
2 Let V = Rn = {(x1 , ..., xn ) | xi ∈ R, 1 ≤ i ≤ n}, be the set of
n-tuples of real numbers. For

x = (x1 , ..., xn ), y = (y1 , ..., yn ) and a ∈ R

Define
x + y := (x1 + y1 , ..., xn + yn ) and ax := (ax1 , ..., axn )
(called component wise or coordinate wise operations)
Then Rn (R) is a vector space.

Dr. M. Tanveer Linear Algebra January 27, 2023 55 / 164

Example:
Let V = R2 and F = R, with addition defined by

(x1 , x2 ) + (y1 , y2 ) = (x1 + y1 + 1, x2 + y2 − 2)

and scalar multiplication defined by

a(x1 , x2 ) = (ax1 + a − 1, ax2 − 2a + 2).

Question: Is it form a Vector Space. If yes, then find its identity

element and additive inverse for addition operation.

Dr. M. Tanveer Linear Algebra January 27, 2023 56 / 164

Examples:
Consider the set C = {x + iy : x, y ∈ R} of complex numbers.
For z1 = x1 + iy1 , z2 = x2 + iy2 ∈ C and a ∈ R, define,

z1 + z2 = (x1 + x2 ) + i(y1 + y2 ) and az1 = (ax1 ) + i(ay1 ).

Then C(R) is a real vector space.

Let V = R and F = C, it is not a vector space because for all
u ∈ R and a ∈ C, au ∈
/R
Note: Closure property of scalar multiplication fail here.
For z1 = x1 + iy1 , z2 = x2 + iy2 ∈ C and a = a1 + ia2 ∈ C, define,

z1 +z2 = (x1 +x2 )+i(y1 +y2 ) and az1 = (a1 x1 −a2 y1 )+i(a1 y1 +a2 x1 ).

Then C(C) is a complex vector space.

Dr. M. Tanveer Linear Algebra January 27, 2023 57 / 164

Examples:
The set Cn = {(z1 , ..., zn ) | zi ∈ C, 1 ≤ i ≤ n} For z = (z1 , ..., zn )
and w = (w1 , ..., wn ) ∈ Cn and a ∈ F, define,

z + w = (z1 + w1 , ..., zn + wn )

and
az := (az1 , ..., azn )
n
Then C (F) is a vector space.
Let Mn (R) denotes the set of all n × n matrices with real entries.
Then Mn (R) is a real vector space with vector addition and scalar
multiplication defined by

A + B = [aij + bij ] and cA = [caij ]

Dr. M. Tanveer Linear Algebra January 27, 2023 58 / 164

Examples: Polynomial Vector Space
A function P : F → F is called a polynomial with coefficients in F if
there exist a0 , ..., am ∈ F such that

P(x) = a0 + a1 x + a2 x 2 + ... + am x m for all x ∈ F.

P is said to have degree m if am 6= 0.

For m is nonnegative integer, Pm (F) denotes the set of all
polynomials with coefficients in F and degree at most m.

Dr. M. Tanveer Linear Algebra January 27, 2023 59 / 164

Cont...
Let

P(x) = a0 + a1 x + a2 x 2 + ... + am x m
q(x) = b0 + b1 x + b2 x 2 + ... + bm x m

Addition of P(x) and q(x)

P(x) + q(x) := (a0 + b0 ) + (a1 + b1 )x + ... + (an + bn )x m
Scalar multiplication of P(x) by a ∈ F
aP(x) := (aa0 ) + (aa1 )x + ... + (aam )x m .
Pm (F) is a vector space with above two operations over F.

Dr. M. Tanveer Linear Algebra January 27, 2023 60 / 164

Definition: Subspace
Let V be a vector space over the field F. A subspace of V is a subset W
of V which is itself a vector space over F with the operations of vector
addition and scalar multiplication on V .

Dr. M. Tanveer Linear Algebra January 27, 2023 61 / 164

Definition: Subspace
Let V be a vector space over the field F. A subspace of V is a subset W
of V which is itself a vector space over F with the operations of vector
addition and scalar multiplication on V .

Note:
Any vector space has two improper subspace: {0} and the vector
space V itself. Other subspaces are called proper subspaces.

Dr. M. Tanveer Linear Algebra January 27, 2023 61 / 164

Theorem
If W is a set of one or more vectors from a vector space V , then W is a
subspace of V if and only if the following conditions holds:
1 If u, v ∈ W , then u + v ∈ W .
2 If a ∈ F and u ∈ W , then au ∈ W .

Dr. M. Tanveer Linear Algebra January 27, 2023 62 / 164

Theorem
If W is a set of one or more vectors from a vector space V , then W is a
subspace of V if and only if the following conditions holds:
1 If u, v ∈ W , then u + v ∈ W .
2 If a ∈ F and u ∈ W , then au ∈ W .

Proof:
If W is a subspace of V , then all the vector space properties are
satisfied, in particular, above two properties (1) and (2) are satisfied.
Conversely, assume above two condition (1) and (2) holds. Let u be any
vector in W . By condition (2),au ∈ W for all scalar a. Setting
a = 0, ou = 0 ∈ W , and setting a = −1, (−1)u = −u ∈ W . All other
properties are automatically satisfied by the vectors in W since they are
satisfied by all vectors in V .

Dr. M. Tanveer Linear Algebra January 27, 2023 62 / 164

Theorem
A non-empty subset W of V is a subspace of V if and only if for each
pair of vectors u, v in W and each scalar a in F the vector au + v is again
in W .

Dr. M. Tanveer Linear Algebra January 27, 2023 63 / 164

Theorem
A non-empty subset W of V is a subspace of V if and only if for each
pair of vectors u, v in W and each scalar a in F the vector au + v is again
in W .

Proof:
Suppose that W is a non-empty subset of V such that au + v belongs to
W for all vectors u, v in W and all scalars a in F. Since W is non-empty,
there is a vector p in W , and (−1)p + p = 0 is in W . Then if u is any
vector in W and a any scalar, the vector au = au + 0 is in W . In
particular, (−1)u = −u is in W . Finally, If u and v are in W , then
u + v = 1u + v is in W . Thus W is a subspace of V .
Conversely, if W is a subspace of V , u and v are in V , and a is a scalar,
then au + v is in W .

Dr. M. Tanveer Linear Algebra January 27, 2023 63 / 164

Examples:
1 W = {(x1 , x2 , 0) : x1 , x2 ∈ R} is a subspace of R3 (R).
2 W = {(x1 , x2 , 12 x1 − 2x2 ) : x1 , x2 ∈ R} is a subspace of R3 (R).
3 All lines in R2 through the origin are subspaces of R2 (R).
4 The solution set of a homogeneous linear system is a subspace of
Rn .
5 Question: Whether the solution set of a non homogeneous system
of linear equation {Ax = b} form a subspace ?.

Dr. M. Tanveer Linear Algebra January 27, 2023 64 / 164

Examples:
1 W = {(x1 , x2 , 0) : x1 , x2 ∈ R} is a subspace of R3 (R).
2 W = {(x1 , x2 , 12 x1 − 2x2 ) : x1 , x2 ∈ R} is a subspace of R3 (R).
3 All lines in R2 through the origin are subspaces of R2 (R).
4 The solution set of a homogeneous linear system is a subspace of
Rn .
5 Question: Whether the solution set of a non homogeneous system
of linear equation {Ax = b} form a subspace ?.

Question:
Q1. Is W = {(x1 , x2 ) ∈ R2 : 2x1 + x2 = 0, x1 + x2 = 1} is a subspace of
R2 ? (  )
a b
Q2. Is W = : a, b ∈ R is a subspace of R2×2 (R) ?.
b 0

Dr. M. Tanveer Linear Algebra January 27, 2023 64 / 164

Questions:
Check whether the following subsets are subspaces of Mn (R) or not.
W = {A ∈ Mn (R) : trace(A) = 0}.
W = {A ∈ Mn (R) : A = AT }.
W = {A ∈ Mn (R) : A + AT = 0}.

Dr. M. Tanveer Linear Algebra January 27, 2023 65 / 164

Sum of Subsets
Let S1 , ..., Sm be the subsets of a vector space V , Then the sum of
S1 , ..., Sm is defined as:

S1 + ... + Sm = {s1 + ... + sm : s1 ∈ S1 , ..., sm ∈ Sm }

Dr. M. Tanveer Linear Algebra January 27, 2023 66 / 164

Sum of Subsets
Let S1 , ..., Sm be the subsets of a vector space V , Then the sum of
S1 , ..., Sm is defined as:

S1 + ... + Sm = {s1 + ... + sm : s1 ∈ S1 , ..., sm ∈ Sm }

Examples:
S1 = {(x, 0, 0) ∈ R3 : x ∈ R} and S2 = {(0, y , 0) ∈ R3 : y ∈ R}. Then

S1 + S2 = {(x, y , 0) : x, y ∈ R}.

Dr. M. Tanveer Linear Algebra January 27, 2023 66 / 164

Sum of Subsets
Let S1 , ..., Sm be the subsets of a vector space V , Then the sum of
S1 , ..., Sm is defined as:

S1 + ... + Sm = {s1 + ... + sm : s1 ∈ S1 , ..., sm ∈ Sm }

Examples:
S1 = {(x, 0, 0) ∈ R3 : x ∈ R} and S2 = {(0, y , 0) ∈ R3 : y ∈ R}. Then

S1 + S2 = {(x, y , 0) : x, y ∈ R}.

Intersection of Subsets
Let S1 , ..., Sm be the subsets of a vector space V , Then the intersection
of S1 , ..., Sm is defined as:

∩Si = S1 ∩ ... ∩ Sm = {s : s ∈ Si , 1 ≤ i ≤ m}

Dr. M. Tanveer Linear Algebra January 27, 2023 66 / 164

Theorem
Let V be a vector spacePover F and Wi ‘s are subspaces of V then ∩Wi
n
is a subspace of V and i=1 Wi is also a subspace of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 67 / 164

Theorem
Let V be a vector spacePover F and Wi ‘s are subspaces of V then ∩Wi
n
is a subspace of V and i=1 Wi is also a subspace of V .

Proof:
Let u, v ∈ ∩Wi and a ∈ F then u, v ∈ Wi for all i.
⇒ u + v ∈ Wi for all i and au ∈ Wi for all i.
⇒ u + v ∈ ∩Wi and au ∈ ∩Wi .
⇒ ∩Wi is aPsubspace of V .
n
Let u, v ∈ i=1 Wi
0 0 0
⇒ u = w1 + ... + wn and v = w1 + ... + wn where wi , wi ∈ Wi
0 0 0 Pn
⇒ u + v = w1 + w1 + wP 2 + w2 ... + wn + wn ∈ i=1 Wi and
n
= aw1 + ... + awn ∈ i=1 Wi
au P
n
⇒ i=1 Wi is a subspace of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 67 / 164

Union of Two Subspace
Let W1 and W2 be two subspace of a vector space V , then W1 ∪ W2 is
not a subspace of V .
Example:
Let W1 = {(x, 0, 0) : (x, 0, 0) ∈ R3 } and
W2 = {(0, y , 0) : (0, y , 0) ∈ R3 },
(x, 0, 0) + (0, y , 0) = (x, y , 0) ∈
/ W1 ∪ W2 . Therefore, W1 ∪ W2 is not a
subspace of R3 (R).

Dr. M. Tanveer Linear Algebra January 27, 2023 68 / 164

Definition: Linear Combination of Vectors
Let S be a nonempty (possibly infinite) subset of a vector space V .
Then a vector u in V is a finite linear combination of the vectors in S if
and only if there exist a finite subset {u1 , ..., um } of S such that
u = a1 u1 + a2 u2 + ... + am um for some scalar a1 , ..., am ∈ F.

Dr. M. Tanveer Linear Algebra January 27, 2023 69 / 164

Definition: Linear Combination of Vectors
Let S be a nonempty (possibly infinite) subset of a vector space V .
Then a vector u in V is a finite linear combination of the vectors in S if
and only if there exist a finite subset {u1 , ..., um } of S such that
u = a1 u1 + a2 u2 + ... + am um for some scalar a1 , ..., am ∈ F.

Example:
Let S = {(1, −1, 0), (1, 0, 2), (0, −2, 5)} be a subset of R3 , then the
vector (1, −2, −2) is a linear combination of the vectors in S,

(1, −2, −2) = 2(1, −1, 0) + (−1)(1, 0, 2)

Q1: Whether the vector (17, −4, 2) can be written in a linear

combination of (2, 1, −3), (1, −2, 4)?.

Dr. M. Tanveer Linear Algebra January 27, 2023 69 / 164

Linear Span of a set in a Vector Space
Let S be a nonempty subset of a vector space V . Then the linear span
of S in V is the set of all possible (finite) linear combinations of the
vectors in S. In other words,
m
nX o
L(S) = Span(S) = ai ui : ui ∈ S and ai ∈ F, 1 ≤ i ≤ m
i=1

In case V = L(S), we say that S spans V or generates V .

Dr. M. Tanveer Linear Algebra January 27, 2023 70 / 164

Linear Span of a set in a Vector Space
Let S be a nonempty subset of a vector space V . Then the linear span
of S in V is the set of all possible (finite) linear combinations of the
vectors in S. In other words,
m
nX o
L(S) = Span(S) = ai ui : ui ∈ S and ai ∈ F, 1 ≤ i ≤ m
i=1

In case V = L(S), we say that S spans V or generates V .

Theorem
Let S be a nonempty subset of a vector space V . Then:
1 S ⊆ Span(S).
2 Span(S) is a subspace of V (under the same operation as V ).
3 If W is a subspace of V with S ⊆ W , then Span(S) ⊆ W .
4 Span(S) is the smallest subspace of V containing S.

Dr. M. Tanveer Linear Algebra January 27, 2023 70 / 164

Examples:
1 The polynomials 1, x, x 2 , ..., x n span the vector space Pn (F)

Pn (F) = Span{1, x, x 2 , ..., x n }

2 The linear span of S = {(1, 1, 1), (2, 1, 3)} over R is

L(S) = {a(1, 1, 1) + b(2, 1, 3) : a, b ∈ R}

= {(a + 2b, a + b, a + 3b) : a, b ∈ R}
= {(x, y , z) ∈ R3 : 2x − y = z}

as 2(a + 2b) − (a + b) = a + 3b, and if z = 2x − y , take a = 2y − x

and b = x − y .

Dr. M. Tanveer Linear Algebra January 27, 2023 71 / 164

Note:
Let S1 and S2 be two subsets of a vector space V .
Span(S1 ) = Span(S2 ) if and only if S1 ⊆ Span(S2 ) and
S2 ⊆ Span(S1 ).
If S1 ⊆ S2 then Span(S1 ) ⊆ Span(S2 ).
Span(S1 ∩ S2 ) ⊆ Span(S1 ) ∩ Span(S2 ).
Span(S1 ) ∪ Span(S2 ) ⊆ Span(S1 ∪ S2 ).

Dr. M. Tanveer Linear Algebra January 27, 2023 72 / 164

Linearly Independent (L.I.) Vectors
A set S = {u1 , ..., um } of vectors in V is called Linearly Independent if
the only choice of a1 , ..., am ∈ F that makes a1 u1 + ... + am um equal 0 is
a1 = ... = am = 0.

Dr. M. Tanveer Linear Algebra January 27, 2023 73 / 164

Linearly Independent (L.I.) Vectors
A set S = {u1 , ..., um } of vectors in V is called Linearly Independent if
the only choice of a1 , ..., am ∈ F that makes a1 u1 + ... + am um equal 0 is
a1 = ... = am = 0.

Examples
1 A set of two vectors in V is L.I. if and only if neither vector is a
scalar multiple of the other.
2 (1, 0, 0, 0), (0, 1, 0, 0) and (0, 0, 1, 0) are L.I. in R4 .

Dr. M. Tanveer Linear Algebra January 27, 2023 73 / 164

Linearly Dependent (L.D.) Vectors
A set S = {u1 , ..., um } of vectors in V is called Linearly dependent if it is
not linearly independent. In other words, the set S = {u1 , ..., um } of
vectors in V is L.D. if there exit a1 , ..., am ∈ F, not all 0, such that
a1 u1 + ... + am um = 0.

Dr. M. Tanveer Linear Algebra January 27, 2023 74 / 164

Linearly Dependent (L.D.) Vectors
A set S = {u1 , ..., um } of vectors in V is called Linearly dependent if it is
not linearly independent. In other words, the set S = {u1 , ..., um } of
vectors in V is L.D. if there exit a1 , ..., am ∈ F, not all 0, such that
a1 u1 + ... + am um = 0.

Examples:
1 (2, 3, 1), (1, −1, 2), (7, 3, 8) is L.D. in R3 because

2(2, 3, 1) + 3(1, −1, 2) + (−1)(7, 3, 8) = (0, 0, 0).

2 The vectors u1 = 1 − x, u2 = 5 + 3x − 2x 2 , and u3 = 1 + 3x − x 2 are

L.D in P2 (R), since 3u1 − u2 + 2u3 = 0.

Dr. M. Tanveer Linear Algebra January 27, 2023 74 / 164

Linearly Dependent (L.D.) Vectors
A set S = {u1 , ..., um } of vectors in V is called Linearly dependent if it is
not linearly independent. In other words, the set S = {u1 , ..., um } of
vectors in V is L.D. if there exit a1 , ..., am ∈ F, not all 0, such that
a1 u1 + ... + am um = 0.

Examples:
1 (2, 3, 1), (1, −1, 2), (7, 3, 8) is L.D. in R3 because

2(2, 3, 1) + 3(1, −1, 2) + (−1)(7, 3, 8) = (0, 0, 0).

2 The vectors u1 = 1 − x, u2 = 5 + 3x − 2x 2 , and u3 = 1 + 3x − x 2 are

L.D in P2 (R), since 3u1 − u2 + 2u3 = 0.

Note:
If S is a linearly dependent subset of V then every set containing S
is also linearly dependent.
If S is a linearly independent subset of a vector space V , then every
subset of S is also linearly independent.
Dr. M. Tanveer Linear Algebra January 27, 2023 74 / 164
Theorem
let V be a vector space then {u1 , ..., un } is linearly dependent if and only
if one of the ui ‘s is a linear combination of the other uj ‘s.

Dr. M. Tanveer Linear Algebra January 27, 2023 75 / 164

Theorem
let V be a vector space then {u1 , ..., un } is linearly dependent if and only
if one of the ui ‘s is a linear combination of the other uj ‘s.

Proof:

Pn {u1 , ..., un } is L.D there exists ai ∈ R, not all zero, such that
Since
i=1 ai ui = 0. Suppose ai 6= 0 for some i. Then
a1 u1 + ... + ai ui + ... + an un = 0. Hence ui = −ai−1 ( j6=i aj uj ) and
P

therefore ui = j6=i bj uj where bj = −ai−1 aj ∈ R. Hence ui is a linear

P
combination ofPthe other uj ‘s.
Suppose ui = j6=i aj uj , Then we have

a1 u1 + ... + (−1)ui + ai+1 ui+1 + ... + an un = 0

Pn
That is, there exist a1 , ..., an , with ai = −1 6= 0 such that i=1 ai ui = 0.
Hence {u1 , ..., un } is linearly dependent.

Dr. M. Tanveer Linear Algebra January 27, 2023 75 / 164

Lemma:
Let S be a linearly independent subset of a vector space V . If
u ∈ V \L(S), then S ∪ {u} is Linearly independent

Dr. M. Tanveer Linear Algebra January 27, 2023 76 / 164

Lemma:
Let S be a linearly independent subset of a vector space V . If
u ∈ V \L(S), then S ∪ {u} is Linearly independent

Proof
Suppose there is a linear dependent relation in the elements of S ∪ {u} of
the form
Xn
ai ui + bu = 0, ui ∈ S.
i=1
P 
n
If b 6= 0 then u = −1b i=1 ai ui ∈ L(S). Therefore, b = 0. But then
Pn
i=1 ai ui = 0 and this implies that all ai = 0, since S is L.I. Therefore,
S ∪ {u} is Linearly independent.

Dr. M. Tanveer Linear Algebra January 27, 2023 76 / 164

Definition: Basis
A subset S of a vector space V is called a basis of V if the following two
conditions holds:
1 L(S) = V .
2 S is Linearly Independent.

Dr. M. Tanveer Linear Algebra January 27, 2023 77 / 164

Definition: Basis
A subset S of a vector space V is called a basis of V if the following two
conditions holds:
1 L(S) = V .
2 S is Linearly Independent.

Examples:
let V = Rn and B = {e1 , e2 , ..., ek }, 1 ≤ k ≤ n, where
ej = (0, ..., 0, 1, 0, ..., 0), 1 ≤ j ≤ k (1 is at the jth place). Then B is
called the Standard basis of Rn . Any vector u ∈ Rn can be written
as
n
X
u = (x1 , x2 , ..., xn ) = e i xi
i=1
i
Let Pi (x) = x for i = 0, 1, 2, ..., m, where m is some positive
integer. Then the set B = {P0 (x), P1 (x), ..., Pm (x)} is called the
standard basis for Pm (R)

Dr. M. Tanveer Linear Algebra January 27, 2023 77 / 164

Theorem
Let S be a finite subset of a vector space V such that V = L(S).
Suppose S1 is a subset of S which is L.I. Then there exist a basis B of V
such that S1 ⊆ B ⊆ S.

Dr. M. Tanveer Linear Algebra January 27, 2023 78 / 164

Theorem
Let S be a finite subset of a vector space V such that V = L(S).
Suppose S1 is a subset of S which is L.I. Then there exist a basis B of V
such that S1 ⊆ B ⊆ S.

Proof:
If S ⊆ L(S1 ), then L(S1 ) = L(S) = V and we can take B = S1 .
0
Otherwise there exist u1 ∈ S\S1 and by above lemma, S1 = S1 ∪ {u1 } is
0
L.I. Replace S1 by S1 and repeat the above process. Since S is finite set,
this process will terminate in a finite number of steps and we have the
desired result.

Dr. M. Tanveer Linear Algebra January 27, 2023 78 / 164

Theorem
Let V be a vector space and B = {u1 , ..., un } is a subset of V , then
following statements are equivalent.
1 B is a basis.
2 B is a minimal spanning subset of V.
3 B is maximal linearly independent subset of V.

Dr. M. Tanveer Linear Algebra January 27, 2023 79 / 164

Theorem
Let V be a vector space and B = {u1 , ..., un } is a subset of V , then
following statements are equivalent.
1 B is a basis.
2 B is a minimal spanning subset of V.
3 B is maximal linearly independent subset of V.

Proof: (1) ⇒ (2)

Let A be any proper subset of B then there exist ui ∈ B such that
ui 6∈ A for at least one i. Now suppose L(A) = V then
ui = ai1 ui1 + ... + air uir , uij ∈ A. This implies that {ui1 , ..., uir , ui } is L.D
but A ∪ {ui } ⊆ B and B is L.I and subset of L.I is L.I. Therefore, there is
contradiction ⇒ B is a minimal spanning subset of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 79 / 164

Cont...

Proof: (2) ⇒ (3)

Pn
Suppose i=1 P ai ui = 0 and let ai 6= 0 for P
some i, then
ui = −ai−1 ( j6=i aj uj ) and therefore ui = j6=i bj uj where bj = −ai−1 aj .
But B is a spanning subset of V ⇒ for every u ∈ V ,

u = c1 u1 + c2 u2 + ... + cn un
X 
= c1 u1 + c2 u2 + ... + ci bj uj + ... + cn un
j6=i

This implies that {u1 , u2 , ..., ui−1 , ui+1 , ..., un } is a spanning set for
V ,which is contradictions that B is minimal spanning set of V .
Therefore, all ai = 0. Hence,
Pn B is L.I.
Now for any u ∈ V , u = i=1 di ui where di ∈ F ⇒ {u1 , u2 , .., un , u} is
L.D ⇒ B is maximal L.I subset of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 80 / 164

Cont...

Proof: (3) ⇒ (1)

Now given that B is maximal L.I, therefore , for any u ∈ V , B ∪ {u} is
L.D. Hence, u is a linear combination of the elements of B.
Therefore, B is a spanning set for V ⇒ B is a basis of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 81 / 164

Cont...

Proof: (3) ⇒ (1)

Now given that B is maximal L.I, therefore , for any u ∈ V , B ∪ {u} is
L.D. Hence, u is a linear combination of the elements of B.
Therefore, B is a spanning set for V ⇒ B is a basis of V .

Corollary:
If B is a L.I subset of vector space V , then B can be extended to a basis
of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 81 / 164

Theorem
If S = {u1 , u1 , ..., un } is a basis for vector space V , then every vector u
in V can be expressed in the form u = a1 u1 + ... + an un in exactly one
way.

Dr. M. Tanveer Linear Algebra January 27, 2023 82 / 164

Theorem
If S = {u1 , u1 , ..., un } is a basis for vector space V , then every vector u
in V can be expressed in the form u = a1 u1 + ... + an un in exactly one
way.

Proof:
Since S span V , every vector in V can be expressed as a linear
combination of the vectors in S. Suppose that some vector u can be
written as

u = a1 u1 + ... + an un and also as u = b1 u1 + ... + bn un

We get
0 = (a1 − b1 )u1 + ... + (an − bn )un
Since S is linearly independent set, it implies that

a1 = b1 , ..., an = bn

Thus the two expressions for u are the same.

Dr. M. Tanveer Linear Algebra January 27, 2023 82 / 164
Lemma
Let S and T be subsets of a vector space V such that L(S)=V, S is
finite, and T is L.I. Then T is finite and |T| ≤ |S|.

Dr. M. Tanveer Linear Algebra January 27, 2023 83 / 164

Lemma
Let S and T be subsets of a vector space V such that L(S)=V, S is
finite, and T is L.I. Then T is finite and |T| ≤ |S|.

Theorem
Let V be a vector space, and let B1 and B2 be bases for V such that B1
has finite elements. Then B2 also has finite many elements, and
|B1 | = |B2 |.

Dr. M. Tanveer Linear Algebra January 27, 2023 83 / 164

Lemma
Let S and T be subsets of a vector space V such that L(S)=V, S is
finite, and T is L.I. Then T is finite and |T| ≤ |S|.

Theorem
Let V be a vector space, and let B1 and B2 be bases for V such that B1
has finite elements. Then B2 also has finite many elements, and
|B1 | = |B2 |.

Proof
Because B1 and B2 are bases for V , L(B1 )) = V and B2 is L.I. Hence,
above lemma shows that B2 has finitely many elements and |B2 | ≤ |B1 |.
Now, since B2 is finite and span V and B1 is L.I. Then using above
lemma, |B1 | ≤ |B2 |.
Therefore, the number of elements in B1 and B2 are same, i.e
|B1 | = |B2 |.

Dr. M. Tanveer Linear Algebra January 27, 2023 83 / 164

Example:
B = {1, x, x 2 , x 3 } is the standard basis for P3 (R) and |B| = 4.
Every other basis of P3 (R), such as
S = {(x 3 − x, x 2 + x + 1, x 3 + x − 5, 2x 3 + x 2 + x − 3)} also has four
elements, i.e. |B| = |S|.

Dr. M. Tanveer Linear Algebra January 27, 2023 84 / 164

Dimension of a Vector Space
Let V be a vector space. If V has a basis B containing a finite number
of elements, then V is said to be finite dimensional. In this case the
dimension of V , dim(V ), is the number of elements in the basis for V ,
i.e. dim(V ) = |B|.
If V has no finite basis then V is infinite dimensional.

Examples:
dim(Rn ) = n.
dim(Mm,n ) = mn.
dim(Pn ) = n + 1.

Dr. M. Tanveer Linear Algebra January 27, 2023 85 / 164

Examples:
Consider the complex vector space V = Cn (C). Then,

(a + ib, c + id) = (a + ib)(1, 0) + (c + id)(0, 1).

So {(1, 0), (0, 1)} is a basis of Cn (C) and thus dim(V ) = 2.

Consider the real vector space V = Cn (R). Then,

(a + ib, c + id) = a(1, 0) + b(i, 0) + c(0, 1) + d(0, i).

So {(1, 0), (i, 0), (0, 1), (0, i)} is a basis of Cn (R) and thus
dim(V ) = 4.

Dr. M. Tanveer Linear Algebra January 27, 2023 86 / 164

Theorem: Dimension of a Subspace
If W is a subspace of a finite dimensional vector space V . Then:
1 dim(W ) ≤ dim(V ).

2 dim(V ) = dim(W ) if and only if W = V .

Dr. M. Tanveer Linear Algebra January 27, 2023 87 / 164

Theorem
Let U1 and U2 be subspaces of vector space V then

dim(U1 + U2 ) = dim(U1 ) + dim(U2 ) − dim(U1 ∩ U2 ).

Dr. M. Tanveer Linear Algebra January 27, 2023 88 / 164

Theorem
Let U1 and U2 be subspaces of vector space V then

dim(U1 + U2 ) = dim(U1 ) + dim(U2 ) − dim(U1 ∩ U2 ).

Proof
Let {u1 , u2 , ..., ur } be a basis of U1 ∩ U2 , then as U1 ∩ U2 is a subspace
of U1 and U2 , {u1 , u2 , ..., ur } can be extended to bases
{u1 , u2 , ..., ur , v1 , v2 , ..., vt } and {u1 , u2 , ..., ur , w1 , w2 , ..., wp } of U1 and
U2 , respectively.

Dr. M. Tanveer Linear Algebra January 27, 2023 88 / 164

Cont...
Let u ∈ U1 + U2 ,then u = x + y , x ∈ U1 , y ∈ U2 ,
r
X t
X r
X p
X
=⇒ u = λ i ui + βi v i + αi ui + δi wi
i=1 i=1 i=1 i=1
r
X X t Xp
= (λi + αi )ui + βi v i + δi wi
i=1 i=1 i=1
=⇒ S = {u1 , ..., ur , v1 , ..., vt , w1 , ..., wp } spans U1 + U2 .

Now we will show that the set S is L.I. Let ai , bi and ci are some scalars
such that
r
X t
X p
X
ai ui + bi vi + ci wi = 0
i=1 i=1 i=1
r
X Xp t
X
=⇒ ai ui + ci wi = − bi vi ∈ U1 ∩ U2
i=1 i=1 i=1

Dr. M. Tanveer Linear Algebra January 27, 2023 89 / 164

Cont...
Pt Pr
=⇒ − i=1 bi vi = i=1 di ui as {u1 , u2 , ..., ur } is a basis of U1 ∩ U2 .
Pt Pr
=⇒ i=1 bi vi + i=1 di ui = 0 =⇒ bi and di are 0 for all i, as
{u1 , uP 2 , ..., vt } is linearly independent.
2 , ..., ur , v1 , vP
r p
=⇒ a
i=1 i i u + i=1 ci wi = 0 but we know {u1 , ..., ur , w1 , ..., wp } is
L.I.
=⇒ ai = 0 and ci = 0 for all i.
=⇒ {u1 , ..., ur , v1 , ..., vt , w1 , ..., wp } is linearly independent.
Hence,

dim(U1 + U2 ) = r + t + p = (r + t) + (r + p) − (r )
= dim(U1 ) + dim(U2 ) − dim(U1 ∩ U2 )

Dr. M. Tanveer Linear Algebra January 27, 2023 90 / 164

Examples
Let U1 = {(x, y ) ∈ R2 | y = mx} and
U2 = {(x, y ) ∈ R2 | y = tx, t 6= m} be subspaces of vector space
R2 (R). Then find dim(U1 + U2 ).
Solution:
we know that dim(U1 ) = dim(U2 ) = 1 and U1 ∩ U2 = {0} then

dim(U1 + U2 ) = 1 + 1 − 0 = 2

Let U1 = {(x, y , z) ∈ R3 | 2x + y − z = 0} and

U2 = {(x, y , z) ∈ R3 | x + 2y + 2z = 0} then find a basis of U1 ∩ U2
and dim(U1 ∩ U2 ).
Solution
Let (a, b, c) ∈ R3 . Then (a, b, c) ∈ U1 ∩ U2 if 2a + b − c = 0 and
a + 2b + 2c = 0. After solving this system of equations, we get
a = −4r , b = 5r and c = −3r for r 6= 0, r ∈ R. Therefore,
{(−4, 5, −3)} is a basis of U1 ∩ U2 .
Hence, dim(U1 ∩ U2 ) = 1.

Dr. M. Tanveer Linear Algebra January 27, 2023 91 / 164

Example
Let U1 = {(x, y , z) ∈ R3 | x + y − z = 0},
U2 = {(x, y , z) ∈ R3 | 3x + y − 2z = 0} and
U3 = {(x, y , z) ∈ R3 | x − 7y + 3z = 0}. Find dim(U1 + U2 ) and
dim(U1 + U3 ).
Solution
we know that U1 ∩ U2 = {(x, x, 2x) ∈ R3 | x ∈ R} and
U1 ∩ U3 = {(x, x, 2x) ∈ R3 | x ∈ R} then,
dim(U1 + U2 ) = dim(U1 ) + dim(U2 ) − dim(U1 ∩ U2 ) = 2 + 2 − 1 = 3.
dim(U1 + U3 ) = dim(U1 ) + dim(U3 ) − dim(U1 ∩ U3 ) = 2 + 2 − 1 = 3.

Dr. M. Tanveer Linear Algebra January 27, 2023 92 / 164

Ordered Basis
An ordered basis for a vector space V is an ordered n-tuple of vectors
(u1 , u2 , ..., un ) such that the set {u1 , ..., un } is a basis of V .

Coordinates with respect to an Ordered Basis

Let B = (u1 , u2 , ..., un ) be an ordered basis for a vector space V .
Suppose that u = a1 u1 + a2 u2 + ... + an un .Then [u]B , the coordinate of u
with respect to B, is the n-vector (a1 , a2 , ..., an ).

Dr. M. Tanveer Linear Algebra January 27, 2023 93 / 164

Example:
Let V = P2 (R) and B1 = {1 − x, 1 + x, x 2 } is an ordered basis of P2 (R)
For any element u = a0 + a1 x + a2 x 2 ∈ P2 (R), we have,
a0 − a1 a0 + a1
a0 + a1 x + a2 x 2 = (1 − x) + (1 + x) + a2 x 2 .
2 2
a − a a + a 
0 1 0 1
[u]B1 = , , a2 .
2 2
If we take B2 = {1 + x, 1 − x, x 2 } as an ordered basis of P2 (R), then,
a + a a − a 
0 1 0 1
[u]B2 = , , a2 .
2 2

Dr. M. Tanveer Linear Algebra January 27, 2023 94 / 164

Linear Transformation (L.T.)
Let V and W be vector spaces over a field F, and let T : V → W be a
function. Then T is called linear transformation if
(1) T (v1 + v2 ) = T (v1 ) + T (v2 ), for all v1 , v2 ∈ V
(2) T (av ) = aT (v ), for all a ∈ F and all v ∈ V
Note: If V = W , then T is called Linear Operator on V .

Dr. M. Tanveer Linear Algebra January 27, 2023 95 / 164

Linear Transformation (L.T.)
Let V and W be vector spaces over a field F, and let T : V → W be a
function. Then T is called linear transformation if
(1) T (v1 + v2 ) = T (v1 ) + T (v2 ), for all v1 , v2 ∈ V
(2) T (av ) = aT (v ), for all a ∈ F and all v ∈ V
Note: If V = W , then T is called Linear Operator on V .

Theorem
Let V and W be vector spaces, and let T : V → W be a function. Then
T is called linear transformation if and only if

T (au + bv ) = aT (u) + bT (v ), ∀ u, v ∈ V , a, b ∈ F

Dr. M. Tanveer Linear Algebra January 27, 2023 95 / 164

Proof
(⇒) Since T is L.T., we have

T (au + bv ) = T (au) + T (bv ) ∀u, v ∈ V , a, b ∈ F

= aT (u) + bT (v )

(⇐) Since T (au + bv ) = aT (u) + bT (v ), ∀ u, v ∈ V , a, b ∈ F.

Take a = b = 1, we have

T (1u + 1v ) = 1T (u) + 1T (v )
= T (u) + T (v )

Therefore, T (u + v ) = T (u) + T (v ), ∀u, v ∈ V . Now take b = 0,

T (au + 0v ) = aT (u) + 0T (v )
= aT (u) + 0
= aT (u)

Therefore, T (au) = aT (u), ∀u ∈ V , a ∈ F.

Dr. M. Tanveer Linear Algebra January 27, 2023 96 / 164
Example:
Consider the mapping T : Mmn → Mnm , given by T (A) = AT for any
m × n matrix A. We will show that T is a linear transformation.
(1) We must show that T (A1 + A2 ) = T (A1 ) + T (A2 ), for matrices
A1 , A2 ∈ Mmn . However,
T (A1 + A2 ) = (A1 + A2 )T = AT T
1 + A2 = T (A1 ) + T (A2 )
(2) We must show that T (aA) = aT (A), for all a ∈ R and for all
A ∈ Mmn . However, T (aA) = (aA)T = a(AT ) = aT (A).
Hence, T is a linear transformation.

Dr. M. Tanveer Linear Algebra January 27, 2023 97 / 164

Example:
Let T : R3 → R3 given by T (x, y , z) = (x, y , −z). This mapping
reflects the vector (x, y , z) through the xy -plane.

T ((x1 , y1 , z1 ) + (x2 , y2 , z2 )) = T ((x1 + x2 , y1 + y2 , z1 + z2 ))

= (x1 + x2 , y1 + y2 , −(z1 + z2 ))
= (x1 , y1 , −z1 ) + (x2 , y2 , −z2 )
= T (x1 , y1 , z1 ) + T (x2 , y2 , z2 )

and

T (a(x, y , z) = T (ax, ay , az)

= (ax, ay , −az)
= a(x, y , −z)
= aT (x, y , z)

Therefore, T is a linear transformation.

Dr. M. Tanveer Linear Algebra January 27, 2023 98 / 164

Examples:
1 Let A be a given m × n matrix.Then T : Rn → Rm , defined by

T (x) = Ax,

for all x ∈ Rn , is a linear transformation. Let x1 , x2 ∈ Rn . Then,

T (x1 + x2 ) = A(x1 + x2 ) = Ax1 + Ax2 = T (x1 ) + T (x2 ). And,
T (ax) = A(ax) = a(Ax) = aT (x), a ∈ R. Therefore, T is a linear
transformation.
2 Let T : R → R be a function defined as

T (x) = x 2

For all x1 , x2 ∈ R, T (x1 + x2 ) = (x1 + x2 )2 6= (x1 )2 + (x2 )2 .

Therefore, T is not a linear transformation.

Dr. M. Tanveer Linear Algebra January 27, 2023 99 / 164

Theorem
Let V and W be vector spaces, and let T : V → W be a linear
transformation. Then
(1) T (0) = 0
(2) T (−u) = −T (u), for all u ∈ V
(3) T (a1 v1 + a2 v2 + ... + an vn ) = a1 T (v1 ) + a2 T (v2 ) + ... + an T (vn ), for
all a1 , a2 , ..., an ∈ F, and v1 , v2 , ...vn ∈ V .

Proof
Proof(1)
Since T is L.T. Then, T (au) = aT (u), ∀a ∈ F, u ∈ V ⇒ Take a = 0
we have, T (0) = 0T (u) = 0.
Proof(2)
Since T is L.T, Then, T (−u) = T (−1u) = (−1)T (u) = −T (u).
Proof(3) Prove it by induction method.

Dr. M. Tanveer Linear Algebra January 27, 2023 100 / 164

Zero Transformation
Let V and W be vector spaces and let T : V → W be the function
defined by
T (u) = 0, ∀u ∈ V
Then T is a linear transformation. Such a linear transformation is called
the zero transformation.

Identity Transformation
Let V be a vector space and let T : V → V be the function defined by

T (u) = u, ∀u ∈ V

Then T is a linear transformation. Such a linear transformation is called

the identity transformation and denoted by I .

Dr. M. Tanveer Linear Algebra January 27, 2023 101 / 164

Inverse Linear Transformation
Let T : V → W be a one-one and onto linear transformation then the
function T −1 : W → V defined by

T −1 (w ) = u, whenever T (u) = w , u ∈ V and w ∈ W

is a linear transformation and is called the inverse of the linear

transformation T .

Proof
Let w1 , w2 ∈ W . Since T is one-one and onto, then there exist unique
vectors u1 , u2 ∈ V such that T −1 (w1 ) = u1 and T −1 (w2 ) = u2 . . Or
equivalently, T (u1 ) = w1 and T (u2 ) = w2 . So,
T (au1 + bu2 ) = aT (u1 ) + bT (u2 ) = aw1 + bw2 , ∀a, b ∈ F.
Thus for any a, b ∈ F,

T −1 (aw1 + bw2 ) = au1 + bu2

= aT −1 (w1 ) + bT −1 (w2 )

Hence T −1 is a linear transformation.

Dr. M. Tanveer Linear Algebra January 27, 2023 102 / 164
Note:
Let T : R2 → R2 be a linear transformation given by
cos(θ) − sin(θ)
A= , defined as T (u) = Au, for some (fixed)
sin(θ) cos(θ)
angle θ, represents the counterclockwise rotation of any vector
(x, y ) ∈ R2 with initial point at the origin through the angle θ.
Solution:
Let u = (x, y ) = (r cos(α), r sin(α))

  
cos(θ) − sin(θ) r cos(α)
T (u) = Au =
sin(θ) cos(θ) r sin(α)
 
r cos(α + θ)
=
r sin(α + θ)

Dr. M. Tanveer Linear Algebra January 27, 2023 103 / 164

Note:
Let T : V → W be a linear transformation, then T maps linearly
dependent subsets of V into linearly dependent subsets of W.
Suppose S = {u1 , u2 , ..., un } is a L.D. subset P
of V .
n
⇒ There exist at least one ai 6= 0, such that i=1 ai ui = 0.

T (a1 u1 + a2 u2 + ...ai ui + ... + an un ) = T (0) = 0

a1 T (u1 ) + a2 T (u2 ) + ... + ai T (ui ) + ... + an T (un ) = 0

⇒{ T(u1 ), T (u2 ), ..., T (un )} is L.D. in W .

Dr. M. Tanveer Linear Algebra January 27, 2023 104 / 164

Note:
Let T : V → W be a linear transformation, then T maps linearly
dependent subsets of V into linearly dependent subsets of W.
Suppose S = {u1 , u2 , ..., un } is a L.D. subset P
of V .
n
⇒ There exist at least one ai 6= 0, such that i=1 ai ui = 0.

T (a1 u1 + a2 u2 + ...ai ui + ... + an un ) = T (0) = 0

a1 T (u1 ) + a2 T (u2 ) + ... + ai T (ui ) + ... + an T (un ) = 0

⇒{ T(u1 ), T (u2 ), ..., T (un )} is L.D. in W .

Note:
Let T : V → W be a linear transformation, then T may not map
L.I. subsets of V into L.I. subsets of W .
Example:
Let T : R2 → R2 , such that T (x, y ) = (x, 0) is a L.T. and
{(1, 0), (0, 1)} is L.I. subset of R2 , But
{T (1, 0), T (0, 1)} = {(1, 0), (0, 0)} is not L.I subset of R2 .

Dr. M. Tanveer Linear Algebra January 27, 2023 104 / 164

Matrix Of a Linear Transformation
Let V and W be finite dimensional vector spaces over F with
dim(V ) = n and dim(W ) = m. Also, Let T : V → W be a linear
transformation. Suppose B1 = (u1 , u2 , ..., un ) is an ordered basis of V
and B2 = (w1 , w2 , ..., wm ) is an ordered basis of W .
Now for each 1 ≤ j ≤ n, the vectors T (uj ) ∈ W and we can express
these vectors in term of ordered basis B2 of W . So, for each 1 ≤ j ≤ n,
there exist unique scalars a1j , a2j , ..., amj ∈ F such that

T (u1 ) = a11 w1 + a21 w2 + ... + am1 wm

T (u2 ) = a12 w1 + a22 w2 + ... + am2 wm
..
.
T (un ) = a1n w1 + a2n w2 + ... + amn wm

In short we can write it as :

[T (uj )]B2 = [a1 j, a2 j, ..., am j]T , 1 ≤ j ≤ n

Dr. M. Tanveer Linear Algebra January 27, 2023 105 / 164

Cont...

Let [u]B1 = [a1 , a2 , ..., an ]T be the coordinates of a vector u ∈ V . Then

n
X  Xn
T (u) = T aj uj = aj T (uj )
j=1 j=1
n
X m
X 
= aj aij wi
j=1 i=1
m
X X  n 
= aij aj wi .
i=1 j=1

Dr. M. Tanveer Linear Algebra January 27, 2023 106 / 164

Cont...

 
a11 a12 ... a1n
 a21 a22 ... a2n 
Define a matrix A =  . .. ..  . Then the coordinate of
 
 .. ..
. . . 
am1 am2 . . . amn
the vector T (u) with respect to the ordered basis B2 is
 Pn 
a1j aj
Pj=1
 nj=1 a2j aj 
[T (u)]B2 = 
 
.. 
Pn .
 
j=1 amj aj

Dr. M. Tanveer Linear Algebra January 27, 2023 107 / 164

Cont...

  
a11 a12 ... a1n a1
 a21 a22 ... a2n  a2 
[T (u)]B2 = . .. ..   .. 
  
 .. ..
. . .  . 
am1 am2 ... amn an
= A[u]B1 .

Here the matrix A is called the matrix of the linear transformation T

with respect to the ordered basis B1 and B2 , and is denoted by T [B1 , B2 ].

Dr. M. Tanveer Linear Algebra January 27, 2023 108 / 164

Theorem
Let V and W be finite dimensional vector spaces with dim(V ) = n and
dim(W ) = m. Let T : V → W be a linear transformation. If B1 is an
ordered basis of V and B2 is an ordered basis of W , then there exists an
m × n matrix A = T [B1 , B2 ] such that

[T (u)]B2 = A[u]B1

Dr. M. Tanveer Linear Algebra January 27, 2023 109 / 164

Example
Let T : R2 → R2 be a linear transformation, given by

T (x, y ) = (x + y , x − y )

and B1 = {(1, 0), (0, 1)} and B2 = {(1, 1), (1, −1)} are ordered basis of
R2
For any vector u = (x, y ) ∈ R2 , [u]B1 = [x, y ]T . Now,
T (1, 0) = (1, 1) = 1(1, 1) + 0(1, −1). So, T [(1, 0)]B2 = [1, 0]T .
T
and T (0, 1) = (1,−1) =  0(1, 1) + 1(1, −1). So, T [(0, 1)]B2 = [0, 1] .
1 0
So, T [B1 , B2 ] =
0 1
 
x
Notice: [T (x, y )]B2 = [(x + y , x − y )]B2 = x(1, 1) + y (1, −1) = .
y
And     
1 0 x x
T [B1 , B2 ][(x, y )]B1 = = = [T (x, y )]B2
0 1 y y

Dr. M. Tanveer Linear Algebra January 27, 2023 110 / 164

Null Space and Range Space of A Linear Transformation
Let V and W be vector spaces and T : V → W be a linear
transformation. Then
1 The set N(T ) = {u ∈ V : T (u) = 0} is called the kernel of T .
2 The set R(T ) = {T (u) : u ∈ V } is called the range of T .

Dr. M. Tanveer Linear Algebra January 27, 2023 111 / 164

Null Space and Range Space of A Linear Transformation
Let V and W be vector spaces and T : V → W be a linear
transformation. Then
1 The set N(T ) = {u ∈ V : T (u) = 0} is called the kernel of T .
2 The set R(T ) = {T (u) : u ∈ V } is called the range of T .

Theorem
Let V and W be vector spaces and T : V → W be a linear
transformation. Then
1 N(T ) is a subspace of V and it is called null space of T or the
kernel of T .
2 R(T ) is a subspace of W and it is called range space of T .

Dr. M. Tanveer Linear Algebra January 27, 2023 111 / 164

Proof(1)
Let u, v ∈ N(T ), then T (u) = 0 and T (v ) = 0. Now ∀a, b ∈ F,
T (au + bv ) = aT (u) + bT (v ) = 0
This implies that au + bv ∈ N(T ). Therefore, N(T ) is a subspace of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 112 / 164

Proof(1)
Let u, v ∈ N(T ), then T (u) = 0 and T (v ) = 0. Now ∀a, b ∈ F,
T (au + bv ) = aT (u) + bT (v ) = 0
This implies that au + bv ∈ N(T ). Therefore, N(T ) is a subspace of V .

Proof(2)
Let w1 , w2 ∈ R(T ), then w1 = T (u1 ) and w2 = T (u2 ) for some
u1 , u2 ∈ V . Now,∀a, b ∈ F

aw1 + bw2 = aT (u1 ) + bT (u2 )

= T (au1 ) + T (bu2 )
= T (au1 + bu2 )

as au1 + bu2 ∈ V ⇒ aw1 + bw2 ∈ R(T ).

Therefore, R(T ) is a subspace of W .

Dr. M. Tanveer Linear Algebra January 27, 2023 112 / 164

Proposition
Let V and W be vector spaces and let T : V → W be a linear
transformation.Then
1 T is one-one if and only if N(T ) = {0}.
2 T is onto if and only if ρ(T ) = dim(W ).

Dr. M. Tanveer Linear Algebra January 27, 2023 113 / 164

Proof (1)
Suppose T is one-one. Let u ∈ N(T ) then T (u) = 0 but we know
T (0) = 0 for any L.T. Thus T (u) = T (0). Since T is one-one ⇒ u = 0.
Therefore, N(T ) = {0}. Conversely,
Let N(T ) = {0}. Let us assume that for some u, v ∈ V , T (u) = T (V ).
Since T is L.T, so T (u − v ) = 0.
⇒ u − v ∈ N(T ) = {0}. ⇒ u = v . Therefore, T is one-one.

Proof (2)
Let T is onto ⇒ ∀w ∈ W , there exist u ∈ V such that T (u) = w . It
means that R(T ) = W ⇒ ρ(T ) = dim(W ).
conversely, we know that R(T ) ⊆ W and
ρ(T ) = dim(W ) ⇒ R(T ) = W . Therefore, T is onto.

Dr. M. Tanveer Linear Algebra January 27, 2023 114 / 164

Note:
We denote ρ(T ) = dim(R(T )) and is called the rank of the linear
transformation T .
We denote η(T ) = dim(N(T )) and is called the nullity of the
linear transformation T .

Dr. M. Tanveer Linear Algebra January 27, 2023 115 / 164

Rank Nullity Theorem
Let T : V → W be a linear transformation and V be a finite dimensional
vector space. Then
ρ(T ) + η(T ) = dim(V )

Dr. M. Tanveer Linear Algebra January 27, 2023 116 / 164

Proof
Let dim(V ) = n and η(T ) = r . Suppose B = {u1 , u2 , ..., ur } is a basis of
N(T ). Since B is L.I set of V , we can extend it to form a basis of V .
So, there exist vectors {ur +1 , ..., un } such that
S = {u1 , u2 , ..., ur , ur +1 , ..., un } is a basis of V . Therefore,

R(T ) = L(T (u1 ), T (u2 ), ..., T (ur ), T (ur +1 ), ..., T (un ))

= L(0, 0, ..., 0, T (ur +1 ), ..., T (un ))
= L(T (ur +1 ), T (ur +2 ), ..., T (un ))

We now prove that the set S1 = {T (ur +1 ), T (ur +2 ), ..., T (un } is L.I.
Suppose that S1 is not L.I. then, there exists scalars, ar +1 , ar +2 , ..., an ,
not all zero, such that

ar +1 T (ur +1 ) + ar +2 T (ur +2 ) + ... + an T (un ) = 0

T (ar +1 ur +1 + ar +2 ur +2 + ... + an un ) = 0

Dr. M. Tanveer Linear Algebra January 27, 2023 117 / 164

Cont...
Therefore,

ar +1 ur +1 + ar +2 ur +2 + ... + an un ∈ N(T ).
Hence, we can express this vector with the help of basis B so there exist
scalars ai , 1 ≤ i ≤ r such that
r
X
ar +1 ur +1 + ar +2 ur +2 + ... + an un = ai ui .
i=1

That is,
a1 u1 + ... + ar ur − ar +1 ur +1 − ar +2 ur +2 − ... − an un = 0
But the set {u1 , u2 , ..., un } is a basis of V and so L.I. Thus,
ai = 0, ∀i, 1 ≤ i ≤ n
In particular, all scalars ar +1 , ar +2 , ..., an are zero. So we get contradic-
tion. Therefore, The set S1 is L.I and a basis of R(T ). Hence,

ρ(T ) + η(T ) = (n − r ) + r = n = dim(V )

Dr. M. Tanveer Linear Algebra January 27, 2023 118 / 164
Example:
1 Let T : M2 (R) → R defined by
 
a b
T =a+b
c d

is a linear transformation. Find its rank and nullity.

α 0
for any α ∈ R, there exist ∈M2 (R) such that
  c d
 α 0 
T =α
c d
⇒ T is onto. Therefore, ρ(T ) = 1 and η(T ) = 3.

Dr. M. Tanveer Linear Algebra January 27, 2023 119 / 164

Example
Let T : P2 (R) → P2 (R) be a linear transformation, defined by

T (ax 2 + bx + c) = (a − c)x + (b − c)

Find basis of the null space of T , rank and nullity of T .

Solution:
By definition, N(T ) = {ax 2 + bx + c : a = b = c}
⇒ N(T ) = {a(x 2 + x + 1) : a ∈ R}
⇒ N(T ) is spanned by {x 2 + x + 1}
Therefore, η(T ) = 1, and ρ(T ) = 3 − 1 = 2.

Dr. M. Tanveer Linear Algebra January 27, 2023 120 / 164

Question
Let T : R3 → R3 be a linear transformation, defined by

T (x, y , z) = (x − y , x + z, 2y − z)

Find η(T ) and ρ(T ).

Let T : R4 → R3 be a linear transformation, defined by

T (x, y , z, w ) = (x − y , z + w , x + z)

Find the bases of N(T ) and R(T ), η(T ) and ρ(T ).

Dr. M. Tanveer Linear Algebra January 27, 2023 121 / 164

Note
Let T : V → W be a map and B = {u1 , u2 , ..., un } is a basis of V , then
∀u ∈ V there exist unique a1 , a2 , ..., an ∈ F, such that
n
X
u= ai ui
i=1

Now define,
n
X
T (u) = ai T (ui ).
i=1

then T is a well defined map and is a linear transformation. So every

linear transformation T : V → W is uniquely determined by the value
T (u1 ), T (u1 ), ..., T (un ) of the basis B of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 122 / 164

Example
Let T : R2 → R2 be a map such that T (1, 0) = (2, 1) and
T (0, 1) = (1, 2) where B = {(1, 0), (0, 1)} is the standard basis of R2 .
we know for any (x, y ) ∈ R2 ,

(x, y ) = x(1, 0) + y (0, 1)

T (x, y ) = xT (1, 0) + yT (0, 1)
= (2x + y , x + 2y ).

Dr. M. Tanveer Linear Algebra January 27, 2023 123 / 164

Similar Matrices
Two square matrices A and B of same order are said to be similar if
there exists a non-singular matrix P such that A = PBP −1 or
equivalently AP = PB.
Note: A is similar to B if and only if B is similar to A.
Note: If A and B are similar matrices then |A| = |B|.

Dr. M. Tanveer Linear Algebra January 27, 2023 124 / 164

Similar Matrices
Two square matrices A and B of same order are said to be similar if
there exists a non-singular matrix P such that A = PBP −1 or
equivalently AP = PB.
Note: A is similar to B if and only if B is similar to A.
Note: If A and B are similar matrices then |A| = |B|.

Example
     
2 −3 1 0 3 1
Let A= and B= and P= , then AP = PB.
1 −2 0 −1 1 1
It mean
 that A is similar
 to B.

1 2 2 1
A= and B = are not similar matrices, since
2 1 1 2
|A| =
6 |B|.

Dr. M. Tanveer Linear Algebra January 27, 2023 124 / 164

Theorem
Let V be a finite dimensional vector space with ordered bases
B1 = {u1 , u2 , ..., un } and B2 = {v1 , v2 , ..., vn }. Let T : V → V be a L.T
with A = T [B1 , B1 ] and B = T [B2 , B2 ] as matrix representations of T in
bases B1 and B2 . Then A and B are similar matrices.

Dr. M. Tanveer Linear Algebra January 27, 2023 125 / 164

Theorem
Let V be a finite dimensional vector space with ordered bases
B1 = {u1 , u2 , ..., un } and B2 = {v1 , v2 , ..., vn }. Let T : V → V be a L.T
with A = T [B1 , B1 ] and B = T [B2 , B2 ] as matrix representations of T in
bases B1 and B2 . Then A and B are similar matrices.

Proof
Pn Pn
Let T (ui ) = j=1 aji uj and T (vi ) = j=1 bji vj . Let P = (pij ) be the
transition
Pn matrix from B1 to B2 , then
ui = j=1 pji vj
n
X
T (ui ) = pji T (vj )
j=1
Xn n
X
= pji bkj vk
j=1 k=1
Xn X
n 
= bkj pji vk
k=1 j=1
Dr. M. Tanveer Linear Algebra January 27, 2023 125 / 164
Cont...

Also Since,

n
X
T (ui ) = aji uj
j=1
Xn n
X
= aji pkj vk
j=1 k=1
Xn X n 
= pkj aji vk
k=1 j=1

Since B2 is a basis then the coordinate of T (ui ) will be unique. There-

fore, we get

Dr. M. Tanveer Linear Algebra January 27, 2023 126 / 164

Cont...

n
X n
X
bkj pji = pkj aji
j=1 j=1

but the left hand side is (k, i)th element of BP and right hand side is
(k, i)th element of PA and since it is true for all k = 1, 2, ..., n and i =
1, 2, ..., n
⇒ BP = PA ⇒ B = PAP −1 .

Dr. M. Tanveer Linear Algebra January 27, 2023 127 / 164

Example
Let T : R2 → R2 be a linear transformation defined by

T (x, y ) = (x + 3y , x − y )
2
and let B1 = {(1, 0), (0, 1)}
 and B2 = {(1, 2), (2, 1)}
 be thebases of R .
1 3 −3 −1
A = T [B1 , B1 ] = and B = T [B2 , B2 ] =
1 −1  5  3
1 2
Now the transition matrix from B2 to B1 is P = then we can see
2 1
that,
PB = AP
⇒ A and B are similar matrices.

Dr. M. Tanveer Linear Algebra January 27, 2023 128 / 164

Example
Let T : C2 (R) → C2 (R) be a linear transformation given by

T (a + ib, c + id) = (a − ib, c − id)

Now write the matrix of T with respect to the standard basis

B = {(1, 0), (0, 1), (i, 0), (0, i)} of C2 (R). For writing the matrix find the
images of the vectors in B and express it in terms of B

T (1, 0) = (1, 0) = 1(1, 0) + 0(0, 1) + 0(i, 0) + 0(0, i)

T (0, 1) = (0, 1) = 0(1, 0) + 1(0, 1) + 0(i, 0) + 0(0, i)
T (i, 0) = (−i, 0) = 0(1, 0) + 0(0, 1) + (−1)(i, 0) + 0(0, i)
T (i, 0) = (0, −i) = 0(1, 0) + 0(0, 1) + 0(i, 0) + (−1)(0, i)
 
1 0 0 0
0 1 0 0
Therefore, the matrix is  
0 0 −1 0 
0 0 0 −1

Dr. M. Tanveer Linear Algebra January 27, 2023 129 / 164

Example
Let
 
1 −1 2
A = 0 3 1
1 −2 5

be a given matrix then with respect to the standard basis of R3 find the
linear transformation T : R3 → R3 .
Let (x, y , z) ∈ R3 ,

(x, y , z) = xe1 + ye2 + ze3

T (x, y , z) = xT (e1 ) + yT (e2 ) + zT (e3 )
= x(1, 0, 1) + y (−1, 3, −2) + z(2, 1, 5)
= (x − y + 2z, 3y + z, x − 2y + 5z)

Dr. M. Tanveer Linear Algebra January 27, 2023 130 / 164

Cont...
If we take B = {(0, 1, 1), (1, 1, 0), (1, 0, 1)} as basis of R3 then what
would be the linear transformation.
Let (x, y , z) ∈ R3 , then there exist a, b and c ∈ R such that
(x, y , z) = a(0, 1, 1) + b(1, 1, 0) + c(1, 0, 1) now we have to find a, b and
c in terms of x, y and z.

(x, y , z) = a(0, 1, 1) + b(1, 1, 0) + c(1, 0, 1)

T (x, y , z) = aT (0, 1, 1) + bT (1, 1, 0) + cT (1, 0, 1)
y −x +z x +y −z x −y +z
= (1, 1, 2) + (1, 2, −3) + (6, 3, 7)
2 2 2
= (3x − 2y + 3z, 2x + z, x − 4y + 6z)

Dr. M. Tanveer Linear Algebra January 27, 2023 131 / 164

Inner Product
Let V be a vector space over F. An inner product over V (F), denoted
by h, i, is a map,
h, i : V × V → F
such that ∀ u, v , w ∈ V and ∀ a, b ∈ F
1 hu + v , w i = hu, w i + hv , w i,
2 hau, v i = ahu, v i.
3 hu, v i = hv , ui, and
4 hu, ui ≥ 0 ∀ u ∈ V and equality holds if and only if u = 0.

Inner Product Space (I.P.S)

Let V be a vector space with an inner product h, i. Then (V , h, i) is
called an inner product space.

Dr. M. Tanveer Linear Algebra January 27, 2023 132 / 164

Example:
Let V = Rn be a real vector space and let u = (u1 , u2 , ..., un ),
v = (v1 , v2 , ..., vn ) ∈ V , define
n
X
hu, v i = u i vi .
i=1

This inner product is called the standard inner product or dot

product on Rn .
Let u, v and w ∈ Rn where w = (w1 , w2 , ..., wn ), and a, b ∈ R, then

n
X n
X n
X
hau + bv , w i = (aui + bvi )wi = a ui wi + b vi wi
i=1 i=1 i=1
= ahu, w i + bhv , w i
Pn Pn Pn
hu, ui = i=1 ui2 ≥ 0, and hu, v i = i=1 ui vi = i=1 vi ui = hv , ui.

Dr. M. Tanveer Linear Algebra January 27, 2023 133 / 164

Examples:
Let V = Cn be a complex vector space and let u = (u1 , u2 , ..., un ),
v = (v1 , v2 , ..., vn ) ∈ V , define
n
X
hu, v i = ui v¯i = u(v̄ )T = uv ∗ .
i=1

This inner product is called the standard inner product or dot

product on Cn .

Dr. M. Tanveer Linear Algebra January 27, 2023 134 / 164

Example:
Let V = Mn (R) be a vector space of all real square matrices of order n.
For A, B ∈ Mn (R), define

hA, Bi = tr (AB T )

hA+B, C i = tr ((A+B)C T ) = tr (AC T )+tr (BC T ) = hA, C i+hB, C i.

hA, Bi = tr (AB T ) = tr (AB T )T = tr (BAT ) = hB, Ai
haA, Bi = tr (aAB T ) = atr (AB T ) = ahA, Bi
Let A = [aij ]. Then P
n Pn Pn
hA, Ai = tr (AAT ) = i=1 (AAT )ii = i=1 j=1 aij2 ≥ 0.
Therefore, hA, Bi is an inner product on Mn (R).

Dr. M. Tanveer Linear Algebra January 27, 2023 135 / 164

Example
Let Pn (R) be a real vector space and P1 (x) = an x n + an−1 x n−1 + ... + a0 ,
P2 (x) = bn x n + bn−1 x n−1 + ... + b0 be in Pn (R). define,

hP1 , P2 i = an bn + an−1 bn−1 + ... + a0 b0 .

Then it is an inner product on Pn (R).

Dr. M. Tanveer Linear Algebra January 27, 2023 136 / 164

Length (Norm) of a Vector
Let V be a vector space with an inner product h, i. Let u ∈ V , define
the length (norm) of u, denoted ||u||, by
p
||u|| = hu, ui

Dr. M. Tanveer Linear Algebra January 27, 2023 137 / 164

Cauchy- Schwartz inequality
Let V be a vector space with an inner product h, i. Then for any
u, v ∈ V ,
|hu, v i| ≤ ||u|||v ||.

Proof
If u = 0 or v = 0, then the inequality holds. Let u 6= 0 and v 6= 0, For
any scalar a,

0 ≤ hu − av , u − av i
= aāhv , v i − ahv , ui − āhu, v i + hu, ui
= hu, ui − ahv , ui − ā{hu, v i − ahv , v i}
hu,v i
Choose a = hv ,v i , then

hu, v i
0 ≤ ||u||2 − hv , ui − 0
hv , v i

Dr. M. Tanveer Linear Algebra January 27, 2023 138 / 164

Cont...
⇒ 0 ≤ ||u||2 − hu,v||vihu,v
||2
i

⇒ |hu, v i|2 ≤ ||u||2 |v ||2

⇒ |hu, v i| ≤ ||u|||v ||.

Dr. M. Tanveer Linear Algebra January 27, 2023 139 / 164

Mutually Orthogonal Set of Vectors
Let V be an inner vector space. Then,
The vectors u and v in V are said to be orthogonal if hu, v i = 0
Let S = {u1 , u2 , ..., un } be a subset of V . Then S is called
mutually orthogonal set if

hui , uj i = 0, 1 ≤ i 6= j ≤ n.

Orthonormal Set of Vectors

Let V be an inner vector space. A set of non zero, mutually orthogonal
vectors {u1 , u2 , ..., un } in V is called an orthonormal set if

||ui || = 1, 1 ≤ i ≤ n.

Note: A vector u with norm (length) one is called unit vector.

Dr. M. Tanveer Linear Algebra January 27, 2023 140 / 164

Examples
Let B = {e1 , e2 , ..., en } be the standard basis of Rn . Then with
respect to the standard inner product on Rn , B is orthonormal set
and is called orthonormal basis of Rn
Let u = (1, 0, −1, 0) and v = (3, 0, 3, 0) be two vectors in R4 . Then
u and v are orthogonal vectors because,

hu, v i = 1.3 + 0.0 + (−1).3 + 0.0 = 0

√ √
Here, ||u|| = 2 and ||v || = 3 2, so u and v is not unit vectors
and the set {(1, 0, −1, 0), (3, 0, 3, 0)} is an orthogonal set, not
orthonormal set.

Dr. M. Tanveer Linear Algebra January 27, 2023 141 / 164

Note
A basis for an inner product space V is an orthogonal [orthonormal]
basis if the vectors in the basis form an orthogonal [orthonormal] set.

Dr. M. Tanveer Linear Algebra January 27, 2023 142 / 164

Note
A basis for an inner product space V is an orthogonal [orthonormal]
basis if the vectors in the basis form an orthogonal [orthonormal] set.

Theorem
Let V be an inner product space. Let S = {u1 , u2 , ..., un } be a set of
non-zero, mutually orthogonal vectors of V . Then,
1 The set S is linearly independent.
2 Let dim(V ) = n and also let ||ui || = 1, for all i = 1, 2, ..., n. Then
for any v ∈ V ,
Xn
v= hv , ui iui .
i=1

Dr. M. Tanveer Linear Algebra January 27, 2023 142 / 164

Proof (1)
Let S is orthogonal. Suppose there exist scalars a1 , a2 , ..., an , not all zero,
such that
X n
ai ui = 0.
i=1

Then for 1 ≤ i P ≤ n, we have P

n n
0 = h0, ui i = h j=1 aj uj , ui i = j=1 aj huj , ui i = ai hui , ui i as
hui , ui i =
6 0 ⇒ ai = 0 for all i, but it contradict our assumption.
Therefore, the set S is L.I.

Dr. M. Tanveer Linear Algebra January 27, 2023 143 / 164

Proof (2)
Since dim(V ) = n and S is L.I in V then it form a basis of V . Thus, for
everyPvector v ∈ V , there exist scalars ai , 1 ≤ i ≤ n, such that
n
v = i=1 ai ui . Hence,

Xn n
X
hv , uj i = h ai ui , uj i = ai hui , uj i = aj huj , uj i.
i=1 i=1
hv ,uj i
Now, aj = huj ,uj i and given that ||uj || = 1, for all j = 1, 2, ..., n,
Therefore,
n
X
v= hv , ui iui .
i=1

Dr. M. Tanveer Linear Algebra January 27, 2023 144 / 164

Theorem
Let B = (u1 , u2 , ..., un ) be an orthogonal ordered basis for an inner
product space V , and if u is any vector in V , then
 hu, u i hu, u i hu, un i 
1 2
[u]B = , , ..., .
hu1 , u1 i hu2 , u2 i hun , un i

In particular, If B is an orthonormal ordered basis for V , then

[u]B = (hu, u1 i, hu, u2 i, ..., hu, un i).

Dr. M. Tanveer Linear Algebra January 27, 2023 145 / 164

Example
Let V = R2 be a vector space. For x = (x1 , x2 ) and y = (y1 , y2 ) ∈ R2
define a map
hx, y i = x1 y1 − x1 y2 − x2 y1 + 2x2 y2 .
It is an inner product on R2 . Let B = {(2, 1), (0, 1)} be an ordered
orthogonal basis of R2 . Let u1 = (2, 1) and u2 = (0, 1), then
hu1 , u1 i = 2 and hu2 , u2 i = 2
Suppose that,
u = (a, b) is any vector in R2 . Now,

hu, u1 i = a and hu, u2 i = −a + 2b.

Then
 a −a + 2b 
[u]B = ,
2 2
Notice that:
a −a + 2b
u = (a, b) = (2, 1) + (0, 1)
2 2
Dr. M. Tanveer Linear Algebra January 27, 2023 146 / 164
Gram-Schmidt Process
Let B = {w1 , w2 , ..., wk } be a linearly independent subset of an inner
product space V . we can create a new set {u1 , u2 , ..., uk } of vectors as
follows:
Let u1 = w1 .
 
u2 = w2 − hw 2 ,u1 i
hu1 ,u1 i u1 .
   
u3 = w3 − hw 3 ,u1 i hw3 ,u2 i
hu1 ,u1 i u1 − hu2 ,u2 i u2 .
     
hwk ,uk−1 i
uk = wk − hw k ,u1 i
hu1 ,u1 i u 1 − hwk ,u2 i
hu2 ,u2 i u2 − ... − huk−1 ,uk−1 i uk−1 .

Dr. M. Tanveer Linear Algebra January 27, 2023 147 / 164

Theorem
Let V be an inner product space. Suppose B = {w1 , w2 , ..., wn } is a
subset of L.I. vectors of V. Then the set {u1 , u2 , ..., un } obtained by
applying the Gram-Schmidt process to B is an orthogonal subset in V .

Dr. M. Tanveer Linear Algebra January 27, 2023 148 / 164

Proof
Step(1) Let u1 = w1 .
Step(2)Obtain a vector u2 that is orthogonal to u2 as
 hw , u i 
2 1
u2 = w2 − u1 .
hu1 , u1 i

If u2 = 0, then w2 can written as a scalar multiple of w1 ⇒ w1 and w2

are L.D, that is contradiction. Therefore, u2 6= 0.

 hw , u i 
2 1
hu1 , u2 i = hu1 , w2 − u1 i
hu1 , u1 i
= hu1 , w2 i − hw2¯, u1 i
=0

⇒ {u1 , u2 } is orthogonal set.

Dr. M. Tanveer Linear Algebra January 27, 2023 149 / 164

Cont...
Step(3) Construct a vector u3 that is orthogonal to both u1 and u2 as
 hw , u i   hw , u i 
3 1 3 2
u3 = w 3 − u1 − u2 .
hu1 , u1 i hu2 , u2 i
Here again u3 6= 0 because the set B is L.I.

hu1 , u3 i = 0 and hu2 , u3 i = 0

 hw , u i   hw , u i 
3 1 3 2
hu2 , u3 i = hu2 , w3 − u1 − u2 i
hu1 , u1 i hu2 , u2 i
hw3¯, u1 i hw3¯, u2 i
= hu2 , u3 i − hu2 , u1 i − hu2 , u2 i
hu1 , u1 i hu2 , u2 i
=0

Similarly, we can show that

hu1 , u3 i = 0.
Dr. M. Tanveer Linear Algebra January 27, 2023 150 / 164
Cont...
⇒ {u1 , u2 , u3 } is orthogonal set.
Continuing this way,
An orthogonal set of vectors {u1 , u2 , ..., un } will be obtained after n step.
Note: This preceding step-by-step construction for converting an
arbitary L.I set into an orthogonal set is called the Gram-Schmidt
process.

Dr. M. Tanveer Linear Algebra January 27, 2023 151 / 164

Theorem
Let B = {u1 , u2 , ..., un } be a basis for a finite dimensional inner product
space V . Then the set {v1 , v2 , ..., vn } obtained by applying the
Gram-Schmidt process to B is an orthogonal basis for V .
Hence, every nontrivial finite dimensional inner product space has
an orthogonal basis.

Note
If B = {u1 , u2 , ..., un } is an orthogonal basis of V , then
{ ||uu11 || , ||uu22 || , ..., ||uunn || } will be an orthonormal basis of V .
Therefore, every finite dimensional inner product space has an
orthonormal basis.

Dr. M. Tanveer Linear Algebra January 27, 2023 152 / 164

Example
Let S = {(2, 2, 1), (0, 4, 1), (8, 3, 5)} be a L.I subset of R3 , then S is a
basis of R3 . Find the orthonormal basis corresponding to S.
Solution:
Let u1 = (2, 2, 1), u2 = (0, 4, 1) and u3 = (8, 3, 5), then
v1 = ||uu11 || = 13 (2, 2, 1), define

w2 = u2 − hu2 , v1 iv1 = (−2, 2, 0).

w2 √1 (−1, 1, 0).
Let v2 = ||w2 || = 2
Define,

w3 = u3 − hu3 , v1 iv1 − hu3 , v2 iv2

5 −5
= (8, 3, 5) − 3(2, 2, 1) − ( , , 0)
2 2
1
= (−1, −1, 4)
2
w3 1
⇒ v3 = ||w 3 ||
= 3√ 2
(−1, −1, 4).
Therefore, {v1 , v2 , v3 } is the orthonormal basis of R3 .
Dr. M. Tanveer Linear Algebra January 27, 2023 153 / 164
Questions:
1 Let A, B ∈ M2 (R) and hA, Bi= tr (ATB)is the inner
 product on
n o
1 0 1 1 1 1 1 1
M2 (R). The set B = , , , is a
0 0 0 0 1 0 1 1
basis of M2 (R). Find the orthogonal (or orthonormal) basis of
M2 (R) corresponding to the set B.
2 Let u = a0 + a1 x + a2 x 2 , v = b0 + b1 x + b2 x 2 ∈ P2 (R) with inner
product
hu, v i = a0 b0 + a1 b1 + a2 b2 ,
let B = {1 − x, 1 + x, 1 − x 2 } be a basis of P2 (R). Then find the
orthonormal basis of P2 (R) corresponding to B.

Dr. M. Tanveer Linear Algebra January 27, 2023 154 / 164

Orthogonal Complement
Let W be a subspace of an inner product space V . Then the
orthogonal complement W T of W in V is the set of all vectors u ∈ V
with the property that

hu, w i = 0, ∀w ∈ W .

i.e.,
W T = {u ∈ V : hu, w i = 0, ∀w ∈ W }

Dr. M. Tanveer Linear Algebra January 27, 2023 155 / 164

Theorem
Let W be a subspace of an inner product space V , Then,
1 W T is a subspace of V .
2 W ∩ W T = {0}.

Dr. M. Tanveer Linear Algebra January 27, 2023 156 / 164

Proof(1)
we know that 0 ∈ W T ⇒ W T 6= ∅. Now, let u, v ∈ W T and a, b ∈ F,
then

hu, w i = hv , w i = 0, ∀w ∈ W .
Now, for all w ∈ W ,

hau + bv , w i = ahu, w i + bhv , w i

= a.0 + b.0 = 0

⇒ au + bv ∈ W T . Therefore, W T is a subspace of V .

Dr. M. Tanveer Linear Algebra January 27, 2023 157 / 164

Proof(1)
we know that 0 ∈ W T ⇒ W T 6= ∅. Now, let u, v ∈ W T and a, b ∈ F,
then

hu, w i = hv , w i = 0, ∀w ∈ W .
Now, for all w ∈ W ,

hau + bv , w i = ahu, w i + bhv , w i

= a.0 + b.0 = 0

⇒ au + bv ∈ W T . Therefore, W T is a subspace of V .

Proof(2)
Let u ∈ W ∩ W T ⇒ u ∈ W and u ∈ W T
⇒ hu, ui = 0 ⇒ u = 0.
Therefore, W ∩ W T = {0}.

Dr. M. Tanveer Linear Algebra January 27, 2023 157 / 164

Orthogonal Projection
Let V be an inner product space. Let u ∈ V be a unit vector and v ∈ V
be an arbitrary element of V . Then the orthogonal projection Pu (v ) of v
along u is defined by
Pu (v ) = hu, v iu
If u is nonzero vector, then
u u hu, v i
Pu (v ) = h , vi = u
||u|| ||u|| hu, ui

Dr. M. Tanveer Linear Algebra January 27, 2023 158 / 164

Note:
v − Pu (v ) ⊥ u for any nonzero u, v ∈ V (R).
Proof:

hv − Pu (v ), ui = hv , ui − hPu (v ), ui
hu, v i
= hv , ui − hu, ui
hu, ui
= hv , ui − hv , ui
= hv , ui − hv , ui (since inner product is real)
= 0.

Dr. M. Tanveer Linear Algebra January 27, 2023 159 / 164

Proposition
Let V be an inner product space. For a unit vector u and any v ∈ V , we
let
Pu (v ) = hu, v iu,
Then ||v − Pu (v )|| ≤ ||v − au|| for any a ∈ R.

Dr. M. Tanveer Linear Algebra January 27, 2023 160 / 164

Proposition
Let V be an inner product space. For a unit vector u and any v ∈ V , we
let
Pu (v ) = hu, v iu,
Then ||v − Pu (v )|| ≤ ||v − au|| for any a ∈ R.

Proof
We know that v − Pu (v ) ⊥ u . Hence, v − Pu (v ) ⊥ au, for all a ∈ R and
therefore
(v − Pu (v )) ⊥ (Pu (v ) − u) ∀ a ∈ R

||v − au||2 = ||v − Pu (v ) + Pu (v ) − au||2 = ||v − Pu (v )||2 + ||Pu (v ) − au||2 .

Thus we get,
||v − Pu (v )|| ≤ ||v − au|| and equality hold if and only if au = Pu (v ).

Dr. M. Tanveer Linear Algebra January 27, 2023 160 / 164

Theorem: Orthogonal Decomposition of V With Respect To The
Subspace W
Let W be a subspace of a finite dimensional inner product space V .
Then,
V = W ⊕ WT.

Dr. M. Tanveer Linear Algebra January 27, 2023 161 / 164

Theorem: Orthogonal Decomposition of V With Respect To The
Subspace W
Let W be a subspace of a finite dimensional inner product space V .
Then,
V = W ⊕ WT.

Proof
Since W is a subspace of V and V is an inner product space, W is also
an inner product space. Then there exist an orthonormal basis, say
B = {w1 , w2 , ..., wn } of P
W.
n
Let u ∈ V and x = v − i=1 hv , wi iwi , then hx, wi i = 0 for i = 1, 2, ..., n.
Since any element of W can be written as a linear of w1 , w2 , ..., wn ; we
get hx, w i = 0, ∀w ∈ W ⇒ x ∈ W T .
Hence v ∈ W + W T ⇒ V ⊆ W + W T ⇒ V = W + W T .
Now, since W ∩ W T = {0}, therefore

V = W ⊕ WT.

Dr. M. Tanveer Linear Algebra January 27, 2023 161 / 164

Note:
If W is a subspace of inner product space V and let {wi }ri=1 be an
orthonormal basis of W and {ui }sj=1 be an orthonormal basis of W T ,
then {wi } ∪ {uj } is an orthonormal basis of V and therefore

dim(V ) = r + s = dim(W ) + dim(W T ).

Dr. M. Tanveer Linear Algebra January 27, 2023 162 / 164

Example
Let V = R2 be an inner product space with the standard inner product
and W = {(x, 0) ∈ R2 |x ∈ R}. Then find the orthogonal complement of
W.
Solution:
Let u = (a, b) ∈ W T and
w = (x, 0) ∈ W ⇒ hu, w i = a.x + b.0 = 0 ⇒ x = 0 and b is free
variable.
Then W T = {(0, b) ∈ R2 | b ∈ R}.
dim(W ) = 1 and dim(W T ) = 1 and dim(V ) = dim(W ) + dim(W T ) = 2.

Dr. M. Tanveer Linear Algebra January 27, 2023 163 / 164

 THANK YOU

Dr. M. Tanveer Linear Algebra January 27, 2023 164 / 164