Review of Linear Algebra

Dr. Manas Kumar Bera

Associate Professor
Department of EE
NIT Rourkela

February 11, 2025

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 1 / 82
 Outline
1 Field
2 Vector Space
3 Vector Subspace
4 Linear Combination
5 Spanning Set
6 Linear Independence & Dependence
7 Basis
8 Linear Operator
9 Normed linear space
10 Eigenvalues and eigenvectors
11 Definiteness of Matrix
12 Cayley- Hamilton Theorem

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 2 / 82
 Field

Definition
Field: A field F is an object consisting of a set of elements, and two
binary operations
addition(+)
multiplication(.)
such that the following axioms are obeyed. Addition is:
1 associative (α + β) + γ = α + (β + γ) ∀ α, β, γ ∈ F.
2 commutative α + β = β + α.
3 ∃ identity element 0 : α + 0 = α.
4 ∃ inverse element : ∀ α ∃ − α s.t. α + (−α) = 0.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 3 / 82
 Field (contd.)

Definition (contd..)
Similarly, Multiplication is:
5 associative (α.β).γ = α.(β.γ)∀α, β, γ ∈ F.
6 commutative α.β = β.α.
7 ∃ identity element 1 : α.1 = α.
8 ∃ inverse element : ∀ α 6= 0 ∃ α−1 s.t. α.α−1 = 1.
and finally, (.) distributes over (+)
9 α.(β + γ) = α.β + α.γ.
10 (β + γ).α = β.α + γ.α.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 4 / 82
 Field (contd.)

Examples
First of all, it’s easy to see that the rational numbers satisfy all the
field axioms, so Q is a field. In fact, verifying the field axioms for Q
simply boils down to the basic arithmetic properties of the integers:
associativity, commutativity and distributivity and the existence of 0
and 1.
Similarly, the set of real numbers R and complex numbers C are the
fields.
The integers Z are not a field, since field axiom 8 isn’t satisfied by Z.
Indeed, the only integers which have multiplicative inverses are ±1.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 5 / 82
 Vector Space

Definition
Vector Spaces (or linear spaces)
A vector space (V, F) is a set of vectors V, and a field of scalars F, and
two binary operations:
vector addition (+)
scalar multiplication (.) (multiplication of vectors by scalars)
such that:
Addition + : V × V : (x, y ) → x + y ∀ x, y ∈ V
1 associative (x + y ) + z = x + (y + z).
2 commutative x + y = y + x.
3 ∃ identity 0 (zero vector) : x + 0 = 0 + x = x.
4 ∃ inverse, ∀x ∈ V, ∃(−x) s.t. x + (−x) = 0.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 6 / 82
 Vector Space (contd.)

Definition (contd..)
Scalar multiplication . : F × V : (α, x) → αx ∀α ∈ F, x ∈ V
5 (αβ)x = α(βx).
6 1.x = x.
7 0.x = 0.
a and, distributive laws:
8 (α + β)x = αx + βx ∀x ∈ V, ∀ α, β ∈ F .
9 α(x + y ) = αx + αy αx, y ∈ V, ∀α ∈ F.
a
Here 1 and 0 are the multiplicative and additive identities of the field.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 7 / 82
 Vector Space (contd.)

Examples
The first example is the obvious one: if n ≥ 1, then Rn with the usual
component-wise addition and scalar multiplication is a real vector
space, that is a vector space over the field R. We usually call Rn real
n-space.
More generally, for any field F and n ≥ 1, the set F n of all n-tuples
(x1 , x2 , · · · , xn )> of elements of F can be made into a vector space
over F in exactly the same way. That is,

(x1 , x2 , · · · , xn )> + (y1 , y2 , · · · , yn )> = (x1 + y1 , x2 + y2 , · · · , xn + yn )>

and, for all α ∈ F,

α(x1 , x2 , · · · , xn )> = (αx1 , αx2 , · · · , αxn )> .

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 8 / 82
 Vector Space (contd.)

Examples(contd.)
The set Pn of all polynomials

p(x) = an x n + an−1 x n−1 + · · · + a1 x + a0

with real coefficients having degree at most n is a real vector space.

Consider the differential equation

ÿ + aẏ + by = 0 (1)

where a and b are real constants. This is an example of a

homogeneous linear second order differential equation with constant
coefficients. The set of twice differentiable functions on R which
satisfy (1) is a real vector space.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 9 / 82
 Vector Space (contd.)

Examples(contd.)
Let R2×2 denotes the set of 2 × 2 matrices. If addition and scalar
multiplication are defined to be the standard matrix operations, then
R2×2 is a vector space with the following operations: Consider
 
a11 a12
A= A ∈ R2×2
a21 a22
 
b11 b12
B= B ∈ R2×2 (2)
b21 b22

;where aij and bij are the real constants.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 10 / 82
 Vector Space (contd.)

Examples(contd.)
Closure under addition:
 
a11 + b11 a12 + b12
A+B =
a21 + b21 a22 + b22

which is also belongs to R2×2 .

Closure under scalar multiplication:
 
αa11 αa12
αA = ∈ R2×2
αa21 αa22

In general Rm×n , the set of m × n matrices are also a vector space with
standard matrix addition and scalar multiplication.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 11 / 82
 Vector Space (contd.)

Examples(contd.)
a set that is not a vector space:
Let

V = {A ∈ R2×2 : |A| =
6 0}

with the standard matrix addition and multiplication.

Reason
This is not a vector space, it is not closed under matrix addition. Both I
and −I are in V . Yet I + (−I ) ∈ / V.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 12 / 82
 Vector Subspace

Definition
Let (V, F) be a linear space and W a subset of V. Then (W, F) is called
a subspace of (V, F) if (W, F) is itself a vector space.
How to check if something is a subspace:
Verify that W is a subset of V (thus W inherits the vector space
axioms of V).
Verify closure under vector addition and scalar multiplication, i.e.,
∀w1 , w2 ∈ W, ∀α1 , α2 ∈ F then α1 w1 + α2 w2 ∈ W.
Notice that every subspace of a vector space contains the zero vector
0 (why?). In fact, 0 is itself a subspace, called the trivial subspace.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 13 / 82
 Vector Subspace

Example
Consider the vector space R2 of all points (x, y ), where x, y ∈ R.
Let L be a line through the origin 0.
Suppose that L is represented by the equation αx + βy = 0, in other
words

L = {(x y )> ∈ R2 : αx + βy = 0}

Is L is a vector space?
YES! The L forms a vector space over R. In fact every line in R2
through the origin is vector space over R.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 14 / 82
 Vector Subspace

Example
A plane passing through origin is a subspace of R3 . In other words,
W ⊂ R3 such that

W = {(x y z)> ∈ R3 : ax + by + cz = 0}

Test

0
Suppose  0  ∈ W . Now, a.0 + b.0 + c.0 = 0. So W is nonempty.
0
   
x1 x2
Closure under addition: Let  y1  and  y2  ∈ W .
z1 z2

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 15 / 82
 Vector Subspace

Example
Then,

ax1 + by1 + cz1 = 0

ax2 + by2 + cz2 = 0

Now,

a(x1 + x2 ) + b(y1 + y2 ) + c(z1 + z2 )

= (ax1 + by1 + cz1 ) + (ax2 + by2 + cz2 ) = 0

Closure under scalar multiplication:

a(αx1 ) + b(αy1 ) + c(αz1 ) = α(ax1 + by1 + cz1 ) = 0.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 16 / 82
 Vector Subspace

Example
Let F be a field and n be an integer with n > 2, and B ∈ Fn×n , show that

W = {A ∈n×n : AB = BA}

is a subspace.

Solution
To show null vector included in subspace W

0∈W

because 0B = B0 for zero matrix 0.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 17 / 82
 Vector Subspace

Solution
To show subspace W is closure under addition and scalar multiplication.
Give A1 , A2 ∈ W , c ∈ F. We have

(A1 + cA2 )B
= A1 B + cA1 B
= BA1 + CBA2
= B(A1 + cA2 )

so , A1 + cA2 ∈ W . The W is a subspace.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 18 / 82
 Linear Combination

Definition
Suppose (V, F) is a linear space. Let S = {v1 , v2 , · · · vr } be a finite set of
vectors in V. Any vector of the form

α1 v1 + α2 v2 + · · · + αr vr (3)

is called a linear combination of the vectors in S.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 19 / 82
 Linear Combination

Example
   
1 1
Let V = R3 . S = {u1 , u2 }, where u1 =  −1  and u2 =  0 .
2 1
 
1
Then u =  −1  is a linear combination of vector space in S.
3
It is easy to verify

u = 2u1 + (−1)u2 .

Here α1 = 2 and α2 = −1.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 20 / 82
 Linear Combination

Example
Let V = R2×3 . S = {A1 , A2 }, where
   
1 −1 1 1 1 1
A1 = ; A2 =
0 1 1 0 1 0

Then
 
1 −3 1
A=
0 1 2

is a linear combination of vectors in S. Since

A = 2A1 + (−1)A2 .

Here α1 = 2 and α2 = −1.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 21 / 82
 Spanning Set

Definition
Suppose (V, F) is a linear space. Let S = {v1 , v2 , · · · vn } be a subset of V.
We say S is spanning set of V or that S spans V if for every vector v ∈ V
can be written as a linear combination of vectors in S.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 22 / 82
 Spanning Set

Example
   
1 0
The set S = {e1 , e2 }, where e1 = and e2 = is a spanning set
0 1
 
2 x
of R because every vector v = in R2 could be written as a linear
y
combination of e1 and e2 as follows:

v = xe1 + ye2 .

It is called the standard spanning set of R2 .

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 23 / 82
 Spanning Set

Example
Prove that the matrices
   
−2 0 3 0
,
0 1 0 −1

span the set of all diagonal 2 × 2 matrix.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 24 / 82
 Spanning Set
Solution
 
a 0
We have to show that if is an arbitrary diagonal matrix, the
0 b
equation
     
−2 0 3 0 a 0
c +d =
0 1 0 −1 0 b

has a solution for c and d. Rewriting,

   
−2c + 3d 0 a 0
=
0 c −d 0 b

which gives

−2c + 3d = a
c −d =b
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 25 / 82
 Spanning Set

Solution
Solving this system we see that there will always be a solution for any a
and b. Thus any diagonal matrix can be written as a linear combination of
the matrices above so theses two matrices span the set of all diagonal
2 × 2 matrices.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 26 / 82
 Linear Independence & Dependence
Discussion
Suppose (V, F) is a linear space. The set of vectors {v1 , v2 , · · · vp }, vi ∈ V
is a subset of V. Now the linear combination of these vectors is

α1 v1 + α2 v2 + · · · + αp vp

In particular we can choose

=⇒ α1 = α2 = αp = 0

where αi ∈ F to get the

0.v1 + 0.v2 + · · · + 0.vp = 0

This linear combination is called the ‘trivial linear combination’. The linear
combination will be a nontrivial linear combination if at least one of the αi
is non zero to yield 0.
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 27 / 82
 Linear Independence & Dependence

Definition
Suppose (V, F) is a linear space. The set of vectors {v1 , v2 , · · · vp }, vi ∈ V
is said to be linearly independent iff

α1 v1 + α2 v2 + · · · + αp vp = 0 (4)

=⇒ α1 = α2 = αp = 0 (5)
where αi ∈ F.
The set of vectors is said to be linearly dependent iff there ∃ scalars
α1 , α2 , · · · , αp ∈ F at least one of which is not zero such that

α1 v1 + α2 v2 + · · · + αp vp = 0. (6)

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 28 / 82
 Linear Independence & Dependence

Example
Let R3 is the underlying vector space. Is the set S = {v1 , v2 , v3 }, where
     
1 1 0
v1 =  1  , v2 =  0  , v3 =  1  (7)
0 −1 −1

linearly independent?

Answer
For the linear combination

α1 v1 + α2 v2 + α3 v3 = 0. (8)

find α’s.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 29 / 82
 Linear Independence & Dependence

solution
       
1 1 0 0
α1  1  + α2  0  + α3  1  =  0 
0 −1 −1 0
    
1 1 0 α1 0
⇒ 1 0 −1
   α2 = 0 
 
0 −1 −1 α3 0
⇒ α1 = α2 = α3 = 0.

The set S is linearly independent.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 30 / 82
 Linear Independence & Dependence

Example
Let R3 is the underlying vector space. Is the set S = {v1 , v2 , v3 }, where
     
1 1 3
v1 =  1  , v2 =  0  , v3 =  2  (9)
0 −1 −1

linearly independent?

Answer
For the linear combination

α1 v1 + α2 v2 + α3 v3 = 0. (10)

find α’s.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 31 / 82
 Linear Independence & Dependence

solution
       
1 1 3 0
α1  1  + α2  0  + α3  2  =  0 
0 −1 −1 0
    
1 1 3 α1 0
⇒ 1 0
 2   α2 = 0 
 
0 −1 −1 α3 0
⇒ α1 = 2, α2 = 1, α3 = −1.

The set S is linearly dependent.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 32 / 82
 Basis

Definition
Suppose (V, F) is a linear space. Then a set of vectors
B = {b1 , b2 , · · · , bn } is called a basis of V if
1 B spans V.
2 B is a linearly independent set.

Example
 
1
We have already seen that the set S = {e1 , e2 }, where e1 = and
0
 
0
e2 = is a spanning set of R2 . It is also linear independent set.
1
Therefore S is basis for R2 .

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 33 / 82
 Basis

Example
 
1
We have already seen that the set B = {e1 , e2 , e3 }, where e1 =  0 ,
0
   
0 0
e2 =  1  and e3 =  0  is a spanning set of R3 . It is also linear
0 1
independent set. Therefore B is basis for R3 .

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 34 / 82
 Coordinate Representation using basis vectors

Coordinate Representation
Coordinate Representation: Any vector x ∈ V may be written as a linear
combination of the basis vectors:
e.g.,

x =ξ1 b1 + ξ2 b2 + · · · + ξn bn (11)
Xn
= ξi bi
i=1
 
ξ1
 .. 
and here, ξ =  .  ∈ F n is called the coordinate vector of x with respect
ξn
to the {bi }.
The ξi ’s are called the coordinates of x with respect to {bi }.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 35 / 82
 Basis (contd.)

Facts
Fact: The ξi ’s are uniquely defined in terms of x and the {bi }.
Proof: Suppose not. Then ∃ξ, ξ 0 such that

x = ξ1 b1 + ξ2 b2 + · · · + ξn bn (12)
x= ξ10 b1 + ξ20 b2 + ··· + ξn0 bn (13)

Subtracting (13) from (12)

⇒ 0 = (ξ10 − ξ1 )b1 + (ξ20 − ξ2 )b2 + · · · + (ξn0 − ξn )bn (14)

which for ξi 6= ξi0 implies that the {bi } are linearly dependent contradicting
the assumption that {bi } is a basis.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 36 / 82
 Basis (contd.)

Notes:
A basis 3
 of
 avector
 space is not unique,
 i.e.,
in R
1
    
 1 0 0   1 0 0 
0 1 0 , 1 1 0 are both bases.
0 0 1 0 1 1
   
2 A vector space (V, F) is said to be finite dimension n or to be n
dimensional, written dimV = n, if there exists linearly independent
vectors {b1 , b2 , · · · , bn } which span V . The set {b1 , b2 , · · · , bn } is
then called a basis for V .

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 37 / 82
 Basis

Example
Consider the vectors α1 = (1, 0, −1), α2 = (0, −3, 4), α3 = (1, 2, −1) in R3

1 Show that B = {α1 , α2 , α3 } is a basis for R3

2 Given a vector γ ∈ R3 with coordinate vector γ = (1 − 2 5)T with
respect to the ordered basis B = {α1 , α2 , α3 }, compute its coordiante
vector with respect to the ordered standard basis E = {e1 , e2 , e3 }.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 38 / 82
 Basis

Solution
 
a
a) To show that B is a basis, let γ =  b  ∈ R3 . If
c
c1 α1 + c2 α2 + c3 α3 = γ has a unique solution, then {α1 , α2 , α3 } is a basis.
Augmented matrix for systems
     
c1 a 1 0 1 a
A  c2  =  b  →  0 −3 2 b 
c3 c −1 4 −1 c
   
1 0 1 a 1 0 1 a
≈  0 −3 2 b  ≈  0 −3 2 b 
0 4 0 a+c 0 0 8/3 a + c + 4/3b

pivots in every columns corresponds to c1 , c2 , c3 ⇒ unique solution.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 39 / 82
 Basis

Solution
b)

γ = 1α1 + (−2)α2 + 5α3

     
1 0 1
=  0  − 2  −3  + 5  2 
−1 4 −1
       
6 1 0 0
=  16  =6 0
  + 16 1
  +2 0 

2 0 0 1

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 40 / 82
 Linear Operator

Definition
Let V and W be vector spaces over the same field F . A linear
operator is a mapping M : V → W such that
I (a) M(v1 + v2 ) = M(v1 ) + M(v2 ) (additivity)
I (b)M(αv1 ) = αM(v1 ) (homogeneity)
for all v1 , v2 ∈ V and all α ∈ F
Example: The following operator is linear:
M : Rn → Rm : v → Av where A ∈ R m×n
Linear operators on finite dimensional vector spaces are matrices with
the action of matrix-vector multiplication.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 41 / 82
 Range Spaces and Null Spaces

Definition: Null Space

Let the matrix A represent a linear transformation from vector space
V into the vector space W . The important vector subspace of V and
W can be associated with the transformation A
I The Null space or kernal of A is defined by the set
N (A) = Kar(A) = {v ∈ V : Av = 0}
Example: suppose A is the zero map from V to W ; in other words
Av = 0 for every v ∈ V . Then N (A) = V .
Suppose ϕ ∈ L(R3 , R) is defined by ϕ(z1 , z2 , z3 ) = z1 + 2z2 + 3z3 .
The N (ϕ) = {(z1 , z2 , z3 ) ∈ R3 : z1 + 2z2 + 3z3 = 0}. This means
z1 = 0, z2 = 0, z3 = 0.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 42 / 82
 Range Spaces and Null Spaces

Definition: Range Space

Let the matrix A represent a linear transformation from vector space
V into the vector space W . The important vector subspace of V and
W can be associated with the transformation A
I The range space or image of A is defined by the set
R(A) = Im(A) = {w ∈ W : Av = w for some v ∈ V }
Example: suppose A is the zero map from V to W ; in other words
Av = 0 for every v ∈ V . Then R(A) = Im(A) = {0}. The 0 is a
vector in W .
Suppose A ∈ L(R2 , R3 ) is defined by A(x, y ) = (2x, 5y , x + y ). Then
R(A) = {(2x, 5y , x + y ) : x, y ∈ R}.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 43 / 82
 Rank and Nullity

Let A ∈ Rm×n be a linear operator (matrix). Then R(A) is simply the

set of all linear combinations of the columns of A and N (A) is the set
of vectors x ∈ Rn such that Ax = 0.
The rank of a matrix A (or a linear operator A) is the dimension of
R(A).
The nullity of a matrix A (or a linear operator A) is the dimension of
N (A).
Suppose V is a finite dimensional and A ∈ L(V , W ) . Then
dimV = dimN (A) + dimR(A).

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 44 / 82
 Range Space and Null Space

Example
Find the bases for the null space and the range space where
 
1 −1 1 −1
A =  4 −4 5 −2 
2 −2 −1 −8

Solution:
rref(A)
     
1 −1 1 −1 1 1 1 −1 1 −1 0 −3
 0 0 1 2 → 0 0 1 2 → 0 0 1 2 
0 0 −3 −6 0 0 0 0 0 0 0 0

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 45 / 82
 Range Space and Null Space

Example
Now

x1 − x2 − 3x4 = 0
x3 + 2x4 = 0

So the free variables are x2 and x4 . Let x2 = t and x4 = s.

The solution can be written as:

x1 = t + 3s
x2 = t
x3 = −2s
x4 = s

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 46 / 82
 Range Space and Null Space

Example
Now, above can be written as
     
x1 1 3
 x2   1 
  0 
 =t +s 
 x3   0   −2 
x4 0 1
   
1 3
( )
 1   0 
Basis for N (A) = 
 0  ,  −2
   ;

0 1
( 1   1 )

Basis for R(A) =  4  ,  5 

2 −1

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 47 / 82
 Rank

FACT
The dimension of the row space and of the column space are called
the row rank of A and the column rank A respectively.
The rank of A is defined as the number of linearly independent
columns. It also equals the number of linearly independent rows.
Because of this fact, if A is m × n, then

rank(A) 6 min(m, n)

If A is an arbitrary m × n matrix over a field R, then row space of A is

the subspace of Rn generated by the rows A, and the column space of
A is the subspace of Rm generated by the columns of A.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 48 / 82
 Rank

Example
 
−3 6 −1 1 −7
Example:  1 −2 2 3 −1  ∼
2 −4 5 8 −4 3×5
 
1 −2 0− 1 3
 0 0 1 2 −2  = rref(A)
0 0 0 0 0 3×5
Row Space:
u1 = 1 −2 0 −1 3 ∈ R5 ; u2 = 0 ∈ R5 .
   
0 1 2 −2
   
−3 −1
Column space: V1 =  1  ∈ R3 ; V2 =  2  ∈ R3 .
2 5
R(A) = span{V1 , V2 }. rank(A) = 2.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 49 / 82
 Linear Algebraic Equations

Consider the set of linear algebraic equations

Ax = y

where x ∈ Rn , y ∈ Rm and A = : Rn → Rm .


a1 · · · an
The matrices A and y are given and x is the unknown to be solved.
Thus the set actually consists of m equations and n unknowns.
The number of equations can be larger than, equal to, or smaller than
the number of unknowns.
What is the existence condition and general form of solution of
Ax = y ?

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 50 / 82
 Linear Algebraic Equations

Revision of some facts!

The range space of A is defined as all possible linear combinations of
all columns of A.
The rank of A is defined as the dimension of the range space or
equivalently, the number of linearly independent columns in A.
A vector x is called a null vector of A if Ax = 0. The null space of A
consists of all its null vectors.
The nullity is defined as the maximum number of linearly independent
null vectors of A and is related to the rank by : Nullity (A)= no. of
columns of A − rank (A).

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 51 / 82
 Linear Algebraic Equations

Existence of the solution

Given an m × n matrix A and an m × 1 vector y , an n × 1 solution x
exists in Ax = y if and only if y lies in the range space of A or
equivalently

rank(A) = rank([A y ])

where [A y ] is an m × (n + 1) matrix with y append to A as an

additional column.
Given A, a solution x exists in Ax = y for every y , if and only if A has
rank m (full row rank).

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 52 / 82
 Summary

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 53 / 82
 Linear Algebraic Equations

General Solution
Given an m × n matrix A and an m × 1 vector y , let xp be a solution
of Ax = y .
let k := n − rank(A) be the nullity of A. If A has rank n (full column
rank) or k = 0, then the solution xp is unique.
If k > 0, then for every real αi , i = 1, 2, · · · k, the vector

x = xp + α1 n1 + · · · + αk nk

is a solution of Ax = y , where {n1 , · · · , nk } is a basis of the null

space of A.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 54 / 82
 Linear Algebraic Equations

Example
Find the solution of the following systems

4x + 2y − 2z + 3w = 1
2x + y − 2z + 5w = 1

The equations can be written as

 
  x  
4 2 −2 3  y 
 = 1
2 1 −2 5  z  1
w

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 55 / 82
 Linear Algebraic Equations

Example
The augmented matrix for this system is
 
4 2 −2 3 1
2 1 −2 5 1

Putting this into echelon form we get

 
4 2 −2 3 1
0 0 2 −7 1

Free variables are y and w . Let y = s and w = t

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 56 / 82
 Linear Algebraic Equations

Example
The solution    
x −1/2s + t + 1
 y   s 
 z  =  7/2t + 3/2
   

w t
which also can be written as:
     
1 −1/2 1
 0   1   0 
 3/2  + s  0  + t  7/2
     

0 0 1

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 57 / 82
 Determinant
Some useful properties of determinants will now be listed
FACT
1 If A has a row (column) of zeros, then det(A) = 0.
2 If A has a two identical rows (columns), then det(A) = 0.
3 If any row (column) of a matrix A is multiplied by a scalar k to
produce a new matrix B then det(B) = kdet(A).
4 Interchanging any two rows (columns) of matrix A to produce matrix
B, then det(B) = −det(A).
5 Adding a scalar multiple of a row (column) of A to another to yield
B, det(B) = det(A).
6 If A is invertible then det(A) is non zero.
7 The determinant of a product of two matrices A and B is equal to the
product of their determinants, det(AB) = det(A)det(B).
8 The matrix A has rank equal to its dimension if it has a nonzero
determinant.
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 58 / 82
 Equilibrium Points

FACT
Let us consider a unforced system

ẋ(t) = Ax(t) (15)

Here, the equilibrium state x ∗ is obtained by solving Ax ∗ (t) = 0

1 An equilibrium point could be isolated; that is, there are no other
equilibrium points in its vicinity, or there could be a continuum of
equilibrium points
2 x ∗ = 0 if A is nonsingular (isolated equilibrium point at x = 0)
3 x ∗ ∈ N (A), if A is singular ⇒ a continuum of equilibrium points.
4 For ẋ = f (x), where f (x) is nonlinear function of x and f (x ∗ ) = 0, x ∗
can belong to a set of multiple isolated equilibrium points.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 59 / 82
 Normed linear space

Norm of Vectors
A norm is a function which assigns to every vector x in a vector space
a real positive number which is represented as kxk.
For example, the length of a vector u ∈ R2 is obtained from the
Pythagorean theorem by computing the length of the hypotenuse of a
right triangle as shown in figure.
p
This measured length kuk = x 2 + y 2 is called the Euclidean norm
in R2 and there is an obvious extension to the higher dimension.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 60 / 82
 Normed linear space

Definition
Let the field F be R or C. A linear space (V, F) is said to be a normed
linear space if ∃ a map : k.k : V → R+ satisfying the following axioms.
1 kv1 + v2 k ≤ kv1 k + kv2 k ∀v1 , v2 ∈ V.
2 kαv k = |α|kv k ∀α ∈ F, v ∈ V.
3 kv k = 0 ⇔ v = 0.
>
Examples: (F n , F) with x = x1 x2 · · ·

xn
kxk1 = ni=1 |xi | → `1 -norm.
P
 1
Pn 2
kxk2 = 2
i=1 |xi | → `2 -norm.
 1
Pn p
kxkp = i=1 |xi |p → `p -norm.

kxk∞ = maxi |xi | → `∞ -norm.

Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 61 / 82
 Normed linear space

Example
 
1
 1 
Let x = 
 
.. 
 . 
1
Pn×1
n
kxk1 = i=1 |xi |
=n
1
√

Pn 2
kxk2 = 2
i=1 |xi | = n

kxk∞ = maxi |xi | = 1

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 62 / 82
 Normed linear space

Example
 
3
Let x =  4 − 3i 
1 3×1
Pn
kxk1 = i=1 |xi | = 3 + 5 + 1 = 9
1
√ √

Pn 2
kxk2 = 2
i=1 |xi | = 9 + 25 + 1 = 35

kxk∞ = maxi |xi | = max(3, 5, 1) = 5

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 63 / 82
 Orthonormalization

Definition
A vector x is said to be normalized if its Euclidean norm is 1 or
x T x = 1.
Note that x T x is scalar and xx T is n × n.
Two vectors x1 and x2 are said to be orthogonal if x1T x2 = x2T x1 = 0.
A set of vectors xi , i = 1, 2, · · · m, is said to be orthonormal if
(
T 0, if i =6 j
xi xj =
1, ifi = j.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 64 / 82
 Eigenvalues and eigenvectors

Concepts
Notice that in ẋ = Ax , the matrix A operates on the vector x to give
the vector ẋ. This is basic function of any matrix-mapping one vector
into another vector. Generally the derived vector is different from the
source vector, both in magnitude and direction.
But there may be some special directions in the state space such that
if the vector x is in that direction, the resultant vector ẋ also lies
along the same direction. It only gets stretched or squeezed.
Any vector along these special directions are called eigenvectors and
the factor by which any eigenvector expands or contracts when it is
operated on by the matrix A, is called the eigenvalue.
Figure shows this definition of eigenvectors. If Ax is not collinear with
x after the transformation, as in Figure (a), x is not an eigenvector. If
Ax is collinear with x after the transformation, as in Figure (b), x is
an eigenvector.
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 65 / 82
 Eigenvalues and eigenvectors

Concepts

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 66 / 82
 Eigenvalues and eigenvectors

Definitions
Eigenvectors: The eigenvectors of the matrix A are all vectors, xi 6= 0,
which under the transformation A become multiples of themselves;
that is,

Axi = λi xi (16)

where λi ’s are constants.

Eigenvalues: The eigenvalues of the matrix A are the values of λi
that satisfy Eq.16 for xi 6= 0.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 67 / 82
 Eigenvalues and eigenvectors

How to find Eigenvalues and eigenvectors?

To find the eigenvectors, we rearrange Eq. (16). Eigenvectors, xi ,
satisfy

(λi I − A)xi = 0 (17)

Solving for xi by premultiplying both sides by (λi I − A)−1 yields

adj(λi I − A)
xi = (λi I − A)−1 0 = 0 (18)
det(λi I − A)

since xi 6= 0, a nonzero solution exists if

det(λi I − A) = 0 (19)

from which λi , the eigenvalues, can be found.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 68 / 82
 Eigenvalues and eigenvectors
How to find Eigenvalues and eigenvectors ?
We are now ready to show how to find the eigenvectors, xi . First we
find the eigenvalues, λi , using Eg. (19), and then we use Eq. (16) to
find the eigenvectors.
Consider the following matrix
 
−3 1
A=
1 −3

Eigenvalues:
   
λ 0 −3 1
det(λI − A) = −
0 λ 1 −3
 
λ + 3 −1
=
−1 λ + 3
= λ2 + 6λ + 8 = (λ + 2)(λ + 4) (20)
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 69 / 82
 Eigenvalues and eigenvectors
How to find Eigenvalues and eigenvectors
The eigenvalues are λ = −2, −4. Now we find an eigenvector for each
of these by finding a vector in N (λI − A).
With eigenvalue λ = −2, following can be written as (16),
    
−3 1 x1 x1
= −2
1 −3 x2 x2
or

−3x1 + x2 = −2x1
x1 − 3x2 = −2x2

from which x1 = x2 .
 
c
Thus, x = .
c
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 70 / 82
 Eigenvalues and eigenvectors

How to find Eigenvalues and eigenvectors

 
c
Using the other eigenvalue,−4, we have x = .
−c
   
1 1
one choice of eigenvectors is x1 = and x2 =
1 −1

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 71 / 82
 Repeated Eigenvalues

Algebraic and Geometric Multiplicity

algebraic multiplicity: number of times eigenvalue λ appears as a root
of characteristics equation (mi ).
geometric multiplicity: dimension of eigenspace ⇒ the number of
linearly independent eigenvectors qi associated with an eigenvalue λi
repeated with an algebraic multiplicity mi is equal to the nullity of
(λi In − A). This dimension is given by qi = n − rank(λi In − A)

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 72 / 82
 Algebraic and Geometric Multiplicity
Example
Let
 
1 3 3
A =  −3 −5 −3 
3 3 1

First we find the eigenvalues of A.

The characteristic polynomial of A: λ3 + 3λ2 − 4 = (λ − 1)(λ + 2)2 .
So its eigen values are λ1 = −1 and λ2 = −2 (repeated).
The algebraic multiplicity is of λ2 is 2. Now, we find bases for the
eigenspace of A.
For λ = 1, we get that the echelon form of I − A is
 
3 6 3
A =  0 −3 −3 
0 0 0
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 73 / 82
 Algebraic and Geometric Multiplicity

Example
Finding a basis for the null space is given by
 
1
 −1 
1

For λ = −2, the echelon form of (−2I − A) is

 
−3 −3 −3
A= 0 0 0 
0 0 0
   
−1 −1
The basis for the null space is  1  ,  0 .
0 1

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 74 / 82
 Algebraic and Geometric Multiplicity

Example
The geometric multiplicity λ2 = −2 is q2 = 3 − 1 = 2.
Fact: GM of λi ≤ AM of λi .

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 75 / 82
 FACTS

Consider a system ẋ = Ax; x ∈ Rn .

Let A have λ1 , λ2 , · · · , λn real and distinct eigenvalues and a
complete set of linearly independent eigenvectors {v1 , v2 , · · · , vn }.
The general solution can be constructed as

x(t) = c1 e λ1 t v1 + c2 e λ2 t v2 + · · · + cn e λn t vn (21)

which implies
n
X
x(0) = ci vi (22)
i=1

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 76 / 82
 Example

 
−5 3
Consider a system ẋ = Ax; x ∈ Rn . Here A =
−2 2
The characteristics polynomial: λ2 + 3λ − 4 = (λ + 4)(λ − 1).
The eigenvalues are −4 and 1.
   
3 1
The eigenvectors are and
1 2
   
3 1
The general form of solution is x(t) = c1 e −4t + c2 et .
1 2

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 77 / 82
 Hermitian Matrix

A Hermitian matrix H ∈ Cn×n (or self-adjoint matrix) is a square matrix

with complex entries which is equal to its own conjugate transpose, i.e.,
H = H T . If H ∈ Rn×n , then H is symmetrical. Facts are:
All the eigenvalues of a hermitian matrix H are real.
The Jordan from representation of a hermitian matrix is diagonal.
The eigenvectors of a hermitian matrix corresponding to different
eigenvalues are orthogonal.
Let λmin and λmax be the smallest and largest eigenvalues of a
hermitian matrix P . Then we have
λmin kxk22 ≤ x T Px ≤ λmax kxk22 for any x ∈ Cn .

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 78 / 82
 Positive Definite Matrix

A Hermitian matrix P ∈ Cn×n is positive definite (positive semidefinite) if

and only if any one of the following conditions holds:
All the eigenvlaues of P are positive (nonnegative).
All the leading principle minors of P are positive (all the principal
minors of P are nonnegative).
There exists a nonsingular matrix N (a singular matrix N) such that
P = N T N.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 79 / 82
 Cayley- Hamilton Theorem

Cayley- Hamilton Theorem

Given a matrix A ∈ Rn×n and the characteristics polynomial

p(λ) = det(λI − A) = λn + p1 λn−1 + · · · + pn−1 λ + pn

The C-H theorem states that the matrix A satisfies its own
characteristics polynomial, i.e.,

An + p1 An−1 + · · · + pn−1 A + pn I = 0

where I is the identify matirix with the the same dimension as A.

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 80 / 82
 Cayley- Hamilton Theorem
Application
A matrix polynomial of a matrix A of degree n, can always be
expressed as a polynomial of degree (n − 1) or less.
Example: Reduce the order of p(A)

p(A) = A4 + 3A3 + 2A2 + A + I

 
3 1
for the matrix A =
1 2
Characteristics polynomial 4(s) = |sI − A| = s 2 − 5s + 5.

p(s) s 4 + 3s 3 + 2s 2 + s + 1
=
4(s) s 2 − 5s + 5
146s − 184
= s 2 + 8s + 27 + 2
s − 5s + 5
2
p(s) = (s + 8s + 27)4(s) + 146s − 184
Dr. Manas Kumar Bera (Associate ProfessorDepartment
Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 81 / 82
 Cayley- Hamilton Theorem

Application
Let r (s) = 146s − 184 and from C-H theorem 4(A) = 0 which
implies,

p(A) = r (A)
⇒ p(A) = A4 + 3A3 + 2A2 + A + I
= 146A − 184

Dr. Manas Kumar Bera (Associate ProfessorDepartment

Review
of EENIT
of Linear
Rourkela)
Algebra February 11, 2025 82 / 82