Lecture 14

Dimension and Rank

Instructor: Ruoyu Sun

 Today’s Lecture: Outline

Today … Dimension, rank

1. Dimension

2. Rank

3. Nullity [if have >me]

Strang’s book: Sec 3.3, 3.4

 Today’s Lecture: Learning Goals

AAer this lecture, you should be able to

1. Explain why the dimension of a linear space is a well-defined

2. Explain why row-rank equals column-rank

3. Tell the rela>on of rank and nullity [if have >me]

Review: Basis
 From Lec 13: Use Inserting Trick to Solve a General System
Pivot columns Pivot variables
x1
Linear system 1 0 a 0 c x2
p1
0 1 b 0 d p
with RREF: x3 = 2 .
0 0 0 1 e p3
x4 RHS entry in non-
0 0 0 0 0 c1 pivot rows
x5
Non-pivot rows

Pivot columns RHS Free variables

Free columns

1 0 a 0 c ∣ p1 delete r=3 pivot col’s −a −c ∣ p1

0 1 b 0 d ∣ p2 Delete 1 non-pivot rows
−a −c ∣ p1 −b −d ∣ p2
Multiply by -1
0 0 0 1 e ∣ p3 [−F, p] = −b −d ∣ p2 [Z ∣ xp] = 1 0 ∣0
0 −e ∣ p3 0 −e ∣ p3
0 0 0 0 0 ∣ c1
0 1 ∣0

[0 1 ∣ 0]
Indices of free columns
1 0 ∣0
Insert (n-r)=2 rows to
decide which rows to insert
Free column:
[I | 0] =
become row 3 and 5
column 3 & 5

AX-bffAX-0.no
501 →

The complete solution is

x1 p1 −a −c
x2 p2 −b −d
α
[ β ] = 0 + α 1 + β 0 , ∀ α, β ∈ ℝ .
x3 = xp + Z
x4 p3 0 −e
x5 0 0 1
TT
 From Lec 13: Think: What Other Questions?

The complete solution is

M :
[圢 lgrowch.ge )
xp + N(A) = xp + C(M) = xp + α1v1 + … + vn−r .
-0

Question: Is there a “simpler” way to express C(M)?

e .

Thereisanexamplethotthedumnpaeofacertahmatnxca.be
snifed.tl 渄 啊
RetThndoesNOTme.cn
clmlonhsiplifedsinceMmgharepeeidstmcture.weneedext.ro
studiednext 。
1st be
 Review of Linear Independence and Basis

u1, …, un are linearly independent iff the following holds:

c1u1 + … + ckuk =8-oO-ss.net
0 only happens when c1 = … = ck = 0.
-_-

u1, …, un form a basis of V iff the following two hold:

i) they are linearly independent; [no redundant information; not→
-_-
too many]
→_→
ii) they span V. [no loss of information; not too few]

n
Eg1 {e1, …, en} is a basis of ℝ . (called “standard basis”)
Eg2 {Eij, i = 1,…, m; j = 1,…, n} is a basis of ℝm×n.
 Part I Dimension
Strang’s book Sec. 3.4
 NextR3.eg.la
InterestngQuestin7.Thereremfnitelymcybose.fr
ez.es/i ab si ,1ei, 2,eit yrsab n,lei, .tez,e.t zteyisab sn.Questim_e.
,

WhatisthenextihterestingQUETMOn.IQ/Whotsthesmplestlor smdlest ) basn ?

QIWhatsthebiggestbas.37Vot.fr
chosejgchoosecivotef.cl
Qli 100%
% choose % doe
" "
" "
2 :
8 。 ,
20 3 ,
Size of Bases ⽐ ⻔
上.my 们 们 们。
+ +

srnunv.A
Recall:
1

0
ℝ T
1

0
1
{[0], [1], [1] } is a basis of
a-A-o-n-t.v.ie
-1 :)
1
3
.
到 ⼆ 0

{e1, e2, e3} is also a basis of ℝ3. U_U =_=

(f)
x-magatilset.ge 。) 以… ⼀
(i)
⼆

Ay
,

Now we have two bases, each with 3 vectors.㴈 )

a taetas-authlbtdtslyEYln .nl
=

Can we have another basis with 2 vectors?

,

Can we have another basis with 4 vectors?

 Too many vectors cannot be a basis

1 -l m e c r l y m de p e nde nt i ui l s h oui m p os i bk .l t ? S h ow Ii
Claim: The set of four vectors {v1, v2, v3, v4} ⊆ ℝ3 is not a
basis of ℝ3.
-01 但
-
3
11 :
2

) 问 沿⻔

arehneagdependent.li
, ,

舆 showlivi.IE/R2arelmeerlydpendent
solutiontExerciselSMore-S.SN
[ 溺 ⻔ , , [ 当张
lmedgdepende.at ,

iii) + ⼈
问 ⼗ 多
闹⼆
0ha
non-zerosolul.tn ,

Axwhasnon-zerosolutim.lv
(iii)
满⻔ - 16
年间 -
1 6 - 孔阔
XF-MX3.la
X,
1
+
1.x 0
'
⼆

piw

x-pzX3.whatreal ymoH-ersmthepmfi -mn-ysoltwimti fevaol.inumn.la/ap6jdelhgmoreolumnsthonpiootsInns

+

'

仙 t
2 "
P '

3
Solut.int Exerc.se 2

cdumn.at
3 ( n= 3)

A ※
1

mostzpiwtslrEmzjfrowopennii.IT
:

-1国
2 rows.3columns-atleastonefeuoni.ba

x-mcnysolutions-columnsofzsmatrixarelme.org
dependent 。
Ge1iemnaet嗨
rn.cxal/r2elemseual.tlyG/7luEmPENDENT!
CommonPathtDeadEndAfewstudentstriedtgenerd.ie
thepooffExerciseltExercisezosfl owsilb.giab.ci
bthienbkblza-q.ci/Coansn-Butiderdifer tnhtceents .Ihfbggeti z=O,tshetnublcksmaiobk;IfCz-fGt=stoconpl.eu
acbz.cz 0

0
,

ImportantthngiTthespecificnumbers
theSHAPElmcnithusranmalters.NO ,
 Key Lemma
Lemma 14.1
Suppose m > n . Exercisezfnomtntoayn.lemmol4.lgeneral.us
Grdlggeneral.us
Corolkytanylmeorsp.ee 。

If {v1, …, vn} is a basis of a linear space V, then any m elements

vii.
u1, …, um are linearly dependent.
5th
notwntein nasamatnx.smcetimgnotbeave.ci
丛
n
Corollary: Any (n + 1) ℝ
vectors in are linearly dependent.
or

疑
然 !0min
,

enis-D.it
剾 ,
→

V:/ RYiorvi-ltx2.fltR.um-cmht-amnM-lgfzm.HR 0mn

"

Zmarelmeerlgdependent.lIZ.zziizmJENnisanidemotridi.li
SousetheresultfR.la ⼀

,
,

X.s.tsxntntxnZFO.VerifgyourselfiXU.tl/zUzt-tXnUn=O.
⺕
Exampleoflor espondingleiRhosabasntu.us
, v41 :
[ % 了 1 % 了 [% 了
, , ,
诎了 ,

以 =
[f i ] = V1.2 t V2 -3 tls.lt V40

Thenzi.fi/isthecorrespondngUctiunderbas.3lVnn,
1 6 ,
V3 V4
, ) 。

RetGwenabosisiui-nniofv.angelementutvcorrespnd.to iector "

Z ,
 Dimension = Size of Bases
4 (bases have same size)
Theorem 13.1 First “big”
If {u1, …, um} and {v1, …, vn} are bases of a linear Theorem
space V, then m = n .
s
in this course!

All bases have the same size!! We call it “dimension”!

prwfofThmM.ee
lusiglemmal4.DAssumemm.u.in
,
…

,
umarelmearlydependentlbylemmal4.in ,

NOTabasis.Contadictim.im
ns.miky.nem.Y-m-n
≤
。
 Dimension = Size of Bases
Theorem 14.1 (bases have same size) First “big”
If {u1, …, um} and {v1, …, vn} are bases of a linear Theorem
space V, then m = n . -
in this course!

All bases have the same size!! We call it “dimension”!

Definition 14.1 (dimension)
Suppose V is a linear space
If V has a basis 𝒰 with n elements, then we say the dimension of V
is n, denoted as dim(V)=n, or V is n-dimensional.

n-drmensiondspaelRnweusedthstermjbefrel .pro
Remark: If there is no finite set of elements that can span V, we say V

videamoregeneraldefirn nowl.io?,- -,I%97frmabas.x.

Is infinite dimensional.
Eg:
basis.

-
阎犫
# "

亚 0th
肶选
lme ?

# ofin 裨
fpomnhe.i-lmcng.cat
9ᵗʰ
"
)
。
 Dimension Measures “Size” of Space

We provide a perspective to understand dimension.

It comes from an old question:

Question: Is a plane “larger” than a line?
-

You may say: surely larger; but in what sense?

Option 1: # of points.
Doesaplanecontahmorepomts7NO-l-lmgpingfomR-R.2ex.su
nn
, 21 5
.

93871 ⾏ !刴
→

Option 2: Dimension.

Fundamentaleerspaaperpec.tn
difenencebetweenRandmllfoml.in
e)
 segn-3.io
Exercise

What is the dimension of the following linear spaces?

1 [
0
1
] 问
们 ⽐ Space of n × n diagonal ttttsnte-s.
。

tshesshstsststsehbhstosstrtfotohbhessFormabo. is
. 。 ,

triangular matrices

lsponlflm.ihdg.li.dinlVF3.lk
Space of n × n upper triangular matrices
1 % 了 [% ) 后 9 7 50 d.mu ) 3
Basfr 2×2 =
: .
.

, ,

6
1 31 8
的
1 3 3 ×

I 㸚 :
了 ,
dm =
.

T 以"

iEijijgfrmabasis.si?e0fba.
:
了

k-inlntn.dnlpnl-ntl.M.e-n-duklRlki.se
些 2.x ⽔ , ,
⼀

, 划 .
着
1 0 a 0 c
0 1 b 0 d
C
0 0 0 1 e
0 0 0 0 0
nextp.ge ,
SolutionloExerc.ie (5)

1: ⽐ ! ! 1
"
…
'

0 0 0 i e

t.VE V3 V4 V5

lasttnmeiArecolumnsdependentiYeoobserualim.la
jdtb
) ( b) 竺 红 0
-
⼆ V2 巧
1 (c)
.

ntkdmtteluty-T.lutdvzteq-gily-aeib.iyce.ua did.ie

qanlnnnn_i-t-g-spania.TT 㒣 ) 呼呼 :
"
 Quicker Way to Check Basis
Proposition 14.1
Consider a linear space V with dim(V) = n.
1) Any linearly independent set can be extended to a basis, by adding more
linearly independent elements if necessary.
2) Any spanning set in V can be reduced to a basis, by deleting some elements
if necessary.

⼀、
1
[1]
Example. v1 = 2 is not a basis of ℝ3 .

Weexpanditabasis.StepliFmdaithatc.cn
NOTbeexpressedbyti-checkfn.ij-siopdifnotbasis.cn
tinue

thotenNOTbeexpressedbyoii.ch
Etepnntndaū

akfi.i-l-sopfnotbosinontinue.no
Whenstoppedwegetaba.is ,
Expans.in?Theproces ofexpcndnganlm dep.set abasi kilEV
HOWMcngg.sn

↓
Whot.im?
亿州 ≤ V
ㄓ
mmustben.hu
↓

问 ,
V2 , EV

!
亿 , v2,
…
,
则 basisof V.
Processofshrinkingfnningsetspanningsetfu.tn
vmievfhechidgendentiy-ASTopwhensiop.wegetamin.mu
⼀

,
,

matonedement.tt
spcnningsettvii-u.IE V.

IV.
lili .im/EVIt'sabasis.fhech:dgendent?
⼀

,
,

kn.y-ASTopcutoneel.me
幽

nt

i
 Quicker Way to Check Basis

Proposition 14.2
Consider a linear space V with dim(V) = n.
If u1, …, un are linearly independent, then they form a basis of V.
If u1, …, un ∈ V can span V, then they form a basis of V.

Remark: For n vectors, only need to check either “span” or “lin. indep.”, not both.

1 2 2
[0] [0] [1]
E.g. 0 , 1 , 3 are linearly independent, thus a basis of ℝ3 .
"

noneedlodd 垒
ang3lmecrlgndgendentuectnrsfrmaba.is
了
In R ,
。
 Quicker Way to Check Basis

Proposition 14.2
Consider a linear space V with dim(V) = n.
If u1, …, un are linearly independent, then they form a basis of V.

l gc omb.i e
If u1, …, un ∈ V can span V, then they form a basis of V.

Remark: For n vectors, only need to check either “span” or “lin. indep.”, not both.

1 2 2
[0] [0] [1]
E.g. 0 , 1 , 3 are linearly independent, thus a basis of ℝ3 . ,

getwhat7.nvectrs-rl.es/tx=ohanonon-2ero-sohnn
Recall: Lec 13 Part II:
n × n matrix A is invertible iff the n columns are linearly dependent.
From Fact 14.1 and 14.2, this is equivalent to:
a) Its columns form a basis. x A 0 0 ntheuniquesoh.tt
=
-
1 =

b) Its columns span the whole space, i.e., Ax=b is solvable for any b.
眑
等vec.mg

basisnm_e.thuert.be
nvecmdg
 Invertibility Conditions Only

conditmshuolvenumberil-u.no
2

) 4 2 1 4 4 .4 ( 1
Let A ∈ ℝn×n.
The following statements are equivalent:
1. A is inver>ble
2. The linear system Ax = 0 has a unique solu>on x = 0 Equation solving
3. A is a product of elementary matrices
4. A has n pivots; or equivalently: rank(A) = n Rank

5. The columns of A span ℝn.

ㄧ
Span

6. The columns of A are linearly independent; Linear independence

-5.1507nsec
7. The columns of A form a basis; basi.tn columnsy Basis

ii.
8. dim(C(A)) = n..
Dimension

Most major concepts we learned

so far are related to invertibility.
General relation of dim and rank?
 Part II Rank
Strang’s book Sec. 3.4
 Relation of Dimension and Rank

Thm 14.2 says: rank(A) = o.0

n <==> dim(C(A)) = n.

Let’s recall what it means.

npiuots-EAx-ohosonlyIsolutiono-ncolumnsofAarehnea.ly
rank(A) = n <==>

mdepende.at
ncolumnsofAformabas.is
dmlc (A)) = n .
 Relation of Dimension and Rank

Thm 14.2 says: rank(A) = n <==> dim(C(A)) = n.

Let’s recall what it means.

rank(A) = n <==> n pivots [Def of rank]
<==> Ax=0 has unique solution 0 [row transf. does not
i.e. Ax=0 has no non-zero solution change solution]
<==> n columns are linearly independent [Def of lin. ind.]
<==> C(A) has dimension n [Fact 14.1; Def of dim]

⼀
Anything interesting to guess?
 Relation of Dimension and Rank
ˋ

Observa(on: Thm 14.2 says: rank(A) = n <==> dim(C(A)) = n.

Guess: rank(A) = dim(C(A)), for any A.

ofN-otsfofvectorsmbosisofc.CA )
"

井
Analysis: Need to consider the case of no unique solution.
-

" "
⟹ Ax=0 has non-zero solutions. 甇;嚟 第
笔
"

,
.net

cntioltrol.hn
thepnof 。

-
Action: So, let’s revisit the non-zero solutions of Ax=0.
ForRREFi.klRkdimkcmlpiuotcdumns-OC.to
1 oooool.tn#0fpiwtcolamns
NOTC01hcidenal.it
3 315 ⼆

eg q ronhlmd.hn KCRD
R =
0 0 0 1 e
# 尛 " 1点 "' "1

,
少 。

whataboutgeneralRREF7.cn
IF.li !
!!
"

※
lf-tgc-sponlta.nl )

piootcolumsfreecolumnsolumnsfFtspoultee-erjl.CI
TrgenerdRREF.thepi¢ votcol
) ,
thusd.hn
) umnsfrmabsofc.
ofpiuotolumn IR
(R = # ,

= # ofpiootstnkk) ,
 Revisiting Solution Set of Ax=0
Pivot columns Pivot variables
x1
1 0 a 0 c x2 0
Linear system: 0 1 b 0 d x3 = 0 .
0 0 0 1 e x4 0
0 0 0 0 0 x5 0

Free variables
Free columns
The complete solution is
x1 −a −c
x2 −b −d
x3 = α 1 + β 0 , ∀ α, β ∈ ℝ .
x4 0 −e
x5 0 1

Plugging in solutions: −c
−a 1 0 a 0 c 0
1 0 a 0 c −b 0 −d
0 1 b 0 d 0
0 1 b 0 d 1 = 0 0 0 0 1 e
0 =
0
0 0 0 1 e 0 0 −e
0 0 0 0 0 0 0 0 0 0 0 1 0
0

⟺ (−a)v1 + (−b)v2 + v3 = 0. ⟺ (−c)v1 + (−d)v2 + (−e)v4 + v5 = 0.

v1, v2, v3 can express v4, v5 .

⼀

They are linearly independent.

 Column Space Dim = Rank

ns.t so-s-stswilprovelemmel4.znext
Fact 1: In RREF, free columns can be expressed as LC of pivot columns.

Fact 2: dim(C(R)) = # of pivot columns = r.

What about dim(C(A))?

dim(C(A)) = r.

Guess: dim(C(A)) = dim(C(R)).

Lemma 14.2 Elementary row operations do not change dimension of column space.

。
 Elementary Row Operation does Not Change LC Relation

Pivot columns
Consider the matrix R̂ that becomes R after one step
−a
1 0 a 0 c −b 0 of elementary row operation.
0 1 b 0 d 1 = 0 1 0 a −β c−βe 1 0 a 0 c
0 0 0 1 e 0 0 βR3+R1 0 1 b 0 d
0 0 0 0 0 0 R̂ = 0 1 b 0 d ⟶
0 0 0 0 1 e 0 0 0 1 e
0 0 0 0 0 0 0 0 0 0
Free columns

⟺ (−c)v1 + (−d)v2 + (−e)v4 + v5 = 0. Check: (−c)v1 + (−d)v2 + (−e)v̂4 + v̂5 = 我

Claim 14.1: Elementary row operation does not change “linear combination” relation.
i.e. if α1 ⋅ (Column 1) + … + αn ⋅ (Column n) = 0 holds for M,
then it holds for the row-transformed matrix too.

emailiii)
-

M
Rigorous proof? u.cn ] [

7 xmtn_n-gthx.int i
.lt/t.

邀
t.IM 的
䪉
⼆
EM.fm

M.at?thenx.(col1ofNt--+xnldnofaiFO.
IxnkdifMt-txn-lalr_nofmw.net
Egf
ni.a-OM-EltEO-O.lt
哟
ˋ
keyideaofPnoflinecr ombinationofcolumnslghtmuy i.NO
iafectedbysrowoperationsclefmult.pl g)
 FinishhgProofofdmlclAD-r.Useaa.hn/4.l(r0woperaNndoaN0T dongeLClhneaanb.nax.in ) ,

lopwved.nl d.hn

KCRD-r.lt
(A) ) =

Ri-HHPH-HHHIRREFYifreeolu.ms !路

fponfre ceoeollumurnnsnsofpivoRtcofdAufmornmsoabasifA_psiwoftCcoMpilunomtcoduonsof
fRrpiotctl..tins
p

pafeecolumnsof A

mdependentEPNtchnsofRarel.n.mdependentidmlc.CAT
r
h
 Definition: Row Space
Definition 14.2 (row space)
a⊤(1)
Suppose A = … ∈ ℝm×n . is a linear space.
a⊤(m)
Then span({a(1), …, a(m)}) ⊆ ℝn is called the row space of A,
r
denoted as Row(A).

In words: A’s row space is the span of A’s row vectors.

Remark: Row(A) = C(A ⊤).

"

sfg
Eg: C(In) = _________
R
1 0
[2 3]
Eg: Row space of A = 4 3 is the set
⼀
三
{α1 [1,0] + α2 [4,3] + α3 [2,3] ∣ α1, α2 ∈ ℝ}.
⊤ ⊤ ⊤
侧 们 《煳 ,
 What is dim(Row(R))?

Consider the RREF R.

Pivot rows

Non-pivot rows
灃11 0 a 0 c
0 1 b 0 d
0 0 0 1 e
0 0 0 0 0
=
u⊤3
01×4
.
u⊤1
u⊤2

Observation 1: Span{rows} = span { pivot rows}.

Observation 2: Pivot rows are linearly independent.

Why?

。加
Together: dim(Row(R)) = # of pivots = r.
 Elementary Row Operations do Not Change Row Space

At-R.si?u,u)=spanlu,uy-Oj=-
Consider the matrix R̂ that becomes R after one step of elementary row operation.

w⊤1 βw⊤3 + w⊤1

w⊤2 βR3+R1 w⊤2
W= ⟶ = Ŵ .
w⊤3 w⊤3
w⊤4 w⊤4
Row(W) = span{w1, w2, w3, w4} span{βw3 + w1, w2, w3, w4}=Row(W).

Claim 14.2: Elementary row

operation does not change the
row space.

dim(Row(A)) = r.
dim(Row(R)) = # of pivots = r
 Rank Theorem

Definition 14.3 (row-rank and column rank)

Row-rank of A is defined as dim(Row(A)), denoted as rR(A).
Column-rank of A is defined as dim(C(A)), denoted as rC(A) .

o-RISomebo ksusedmlcADordimlRowlADasthedefhitm.fi
Theorem 14.3
Row-rank, column-rank and rank of a matrix are the same, i.e.,
rR(A) = rC(A) = rank(A).
Equivalent expression: dim(Row(A)) = dim(C(A)) = # of pivots.

iink
"

,
 Example

器
rR(A) = 3.

C(A)= 。

R=
rC(A) = 3.
3 pivots,

RetclAHCCM.checkyourself.Butdhk.CA
So rank(A) = 3.

Thus, rR(A) = rC(A) = rank(A). ) = d ⽐如

,

-
 Summary Today (write Your Own)

One sentence summary:

Detailed summary:
 Summary Today (of Instructor)

One sentence summary:

ottpeoeososhp.stizeestisrt s h.ch
Dimension, Rank and Nullity.
,

Detailed summary:
1. Dimension
—Dimension of linear space = size of any basis.

isthefundotnnofdefnithfdnens.tn
—Theorem 14.1: Any basis has the same dimension.
"

) ,

2. Rank
Column/row-rank = dim of column/row space
Rank theorem: row-rank = column-rank = rank (= # of pivots)

Bstssstttssssststttts
3. Nullity
—Nullity = dimension of null space.
—Nullity theorem: Nullity + rank = n.

Byproduct: Condi>ons for checking inver>bility by

dim, rank, nullity, linear independence.
䞕