Matrix Analysis: Lecture 4

Feng Wei
[email protected]

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 1 / 17

Basic Facts of Linear Spaces

1 Linear combination and linear representation

2 Linear (in-)dependence
3 Equivalence of collections of vectors
4 Maximal linearly independent subset
5 Rank of vector collections
6 Basis, dimension, coordinate transformation
7 The relationship of bases and the transition matrix
8 Subspaces, intersections and sums of subspaces, dimension formula

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 2 / 17

Linearly (In-)Dependent Collections

Example 1
In the vector space RR , verify that the collection of vectors

eλ1 x , eλ2 x , · · · , eλn x ,

is linearly independent, where λ1 , λ2 , · · · , λn are distinct real numbers.

Let us consider the following linear combination

k1 eλ1 x + k2 eλ2 x + · · · + kn eλn x = 0,

where k1 , k2 , · · · , kn ∈ R. It is enough for us to show that all

ki (i = 1, 2, · · · , n) are zero. We will use two different ways to complete our
proofs. The first way is to take x = 0, 1, 2, · · · , n − 1 into the above
equation in turn
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 3 / 17

Linearly (In-)Dependent Collections
and then we get the following system of linear equations
k1 + k2 + · · · + kn = 0
k1 e + k2 eλ2 + · · · + kn eλn = 0
λ1

k1 e2λ1 + k2 e2λ2 + · · · + kn e2λn = 0

..
.
k1 e(n−1)λ1 + k2 e(n−1)λ2 + · · · + kn e(n−1)λn = 0
The corresponding coefficient matrix is
 
1 1 ··· 1
 eλ1 e λ2 ··· eλn 
 
 2λ e2λ2 ··· e2λn 
A= e 1 
 .. .. .. 
 . . . 
e (n−1) λ 1 e(n−1) λ2 ··· e(n−1)λn
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 4 / 17

Linearly (In-)Dependent Collections

Note that λ1 , λ2 , · · · , λn are distinct real numbers. By invoking the

Vandermonde determinant we obtain

det A = ∏ (eλi − eλ j ) ̸= 0
1≤ j<i≤n

We claim that all ki (i = 1, 2, · · · , n) are zero. This shows that

eλ1 x , eλ2 x , · · · , eλn x are linearly independent.
Another way is to differentiate both sides of the equation with respect to x
continuously. One can get a system of linear equations

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 5 / 17

Linearly (In-)Dependent Collections

k1 eλ1 x + k2 eλ2 x + · · · + kn eλn x = 0

k1 λ1 eλ1 x + k2 λ2 eλ2 x + · · · + kn λn eλn x = 0
k1 λ12 eλ1 x + k2 λ22 eλ2 x + · · · + kn λn2 eλn x = 0
..
.
k1 λ1n−1 eλ1 x + k2 λ2n−1 eλ2 x + · · · + kn λnn−1 eλn x = 0
The coefficient matrix of the above system is
 
eλ1 x eλ2 x ··· eλn x
 λ1 eλ1 x λ e λ2 x · ·· λn eλn 
 2 
 λ 2 eλ1 x λ2 e
2 λ x ··· λn2 eλn x 
B= 1 2

 . . .. 
 .
. .
. . 
λ1n−1 eλ1 x λ2n−1 eλ2 x · · · λnn−1 eλn x
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 6 / 17

Linearly (In-)Dependent Collections

It is not difficult to verify that det B ̸= 0. Thus all ki (i = 1, 2, · · · , n) are

zero. We conclude that eλ1 x , eλ2 x , · · · , eλn x are linearly independent.
Example 2
Similarly, verify that the collection of vectors

xα1 , xα2 , · · · , xαn ,

is linearly independent as well, where α1 , α2 , · · · , αn are distinct real

numbers.
Hint: Consider the linear combination k1 xα1 + k2 xα2 + · · · + kn xαn = 0 and
take different values 1, k, k2 , · · · , kn−1 into this equation in turn. We
therefore a system of linear equations with respect the unknowns
k1 , k2 , · · · , kn . Observe the coefficient matrix of obtained system.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 7 / 17

Linearly (In-)Dependent Collections
Example 3
In RR , verify that the collection of vectors

1, cos x, cos 2x, cos 3x

is linearly independent.

However,
1, cos2 x, cos 2x
is linearly dependent.

Similarly,

sin x, cos x, sin2 x, cos2 x, · · · , sinn x, cosn x, n≥4

is linearly dependent too. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 8 / 17

Basis, Dimension and Coordinates

Example 4
In R2×2 , the following two collections of vectors
[ ] [ ] [ ] [ ]
0 1 1 0 1 1 1 1
α1 = , α2 = , α3 = , α4 = , (I)
1 1 1 1 0 1 1 0

and

[ ] [ ] [ ] [ ]
1 0 1 1 1 1 1 1
β1 = , β2 , β3 = , β4 = , (II)
0 0 0 0 1 0 1 1

are both bases of R2×2 .

What is the relationship between the bases (I) and (II) ? Find the
transition matrix from (I) to (II). . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 9 / 17

Basis, Dimension and Coordinates

Furthermore, if [ ]
1 2
A= ,
3 4
how can we determine its coordinate with respect to (I) and (II) ?
Let us see the standard basis of R2×2
[ ] [ ] [ ] [ ]
1 0 0 1 0 0 0 0
γ1 = , γ2 = , γ3 = , γ4 = .
0 0 0 0 1 0 0 1
Then we have
 
0 1 1 1
 1 0 1 1 
[α1 , α2 , α3 , α4 ] = [γ1 , γ2 , γ3 , γ4 ] 
 1

1 0 1 
1 1 1 0
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 10 / 17

Basis, Dimension and Coordinates

 
1 1 1 1
 0 1 1 1 
[β1 , β2 , β3 , β4 ] = [γ1 , γ2 , γ3 , γ4 ] 
 0

0 1 1 
0 0 0 1
By the above two relations we get
 −1  −1
0 1 1 1 1 1 1 1
 1 0 1 1   0 1 1 1 
[α1 , α2 , α3 , α4 ] 
 1
 = [β1 , β2 , β3 , β4 ]  
1 0 1   0 0 1 1 
1 1 1 0 0 0 0 1

Here P is the transition matrix from basis (I) to basis (II).

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 11 / 17

Basis, Dimension and Coordinates

This implies that

 −1  
0 1 1 1 1 1 1 1
 1 0 1 1   0 1 1 1 
[β1 , β2 , β3 , β4 ] = [α1 , α2 , α3 , α4 ] 
 1 1
  
0 1   0 0 1 1 
1 1 1 0 0 0 0 1
 2 
−3 − 13 0 13
 1 − 13 0 13 
= [α1 , α2 , α3 , α4 ] 
 1
3 
3
2
3 0 13 
1 2
3 3 1 13
= [α1 , α2 , α3 , α4 ]P

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 12 / 17

Basis, Dimension and Coordinates

The coordinate of the matrix A with respect to basis (I) is

[ ]T
7 4 1 2
X= , , ,−
3 3 3 3

Applying the coordinate transformation formula we can get the coordinate

of A with respect to another basis (II) is

Y = P−1 X = [−1, −1, −1, 4]T

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 13 / 17

Basis, Dimension and Coordinates

Example 5
In R[x]n , the following two collections of vectors

1, x, x2 , · · · , xn
and

1, x − a, (x − a)2 , · · · , (x − a)n (a ̸= 0)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 14 / 17

Basis, Dimension and Coordinates

both bases of R[x]n . It is easy to see that dim R[x]n = n + 1.

Question 1
For the vector space R[x], what is dim R[x] ?

Question 2
Consider the differences and connections between vector spaces and
collections of vectors; investigate the differences and connections between
basis and dimension.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 15 / 17

Subspaces

Let V be a linear space over a field F and W be a nonempty subset of V .

We say that W is a subspace of V If

kα + l β ∈ W,

for all k, l ∈ F and α , β ∈ W . For an arbitrary linear space V , it has two

trivial subspaces: {0} and itself.

Let us see three typical subspaces originated from various backgrounds.

1 Solution spaces of systems of homogeneous linear equations
2 Finitely generated subspaces
3 Eigen-subspaces of matrices/linear transformations

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 16 / 17

Solution Spaces of Systems of Homogeneous Linear
Equations
Let A ∈ Rm×n . Then
S = { X | AX = 0 }
is a subspace of Rn . S is called the solution space of AX = 0. Its
dimension is n − rank A.

The above-mentioned notion is not workable for a system of

non-homogeneous linear equations AX = b. In general,

T = { X | AX = b }

is not a subspace of Rn . Try to find the maximally linearly independent

subset of T . harder or challenging question.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

(Beijing Institute of Technology) Matrix Analysis: Lecture 4 17 / 17