Lecture 11

Linear Space and Solution Set

Instructor: Ruoyu Sun

 Today’s Lecture: Outline

Today … Linear Space and Solu2on Set

1. Mo2va2on: Solu2on set of rectangular system

2. Linear space

3. Null space and column space

Strang’s book: Sec 3.1, 3.2

 Today’s Lecture: Learning Goals

AAer this lecture, you should be able to

1. Verify linear space

2. Tell a few common linear spaces

3. Explain why the solu2on set of Ax=0 is a linear space

4. Write the column space of a matrix

Part 0 Review and
Future Roadmap
 Roadmap of Part 2 of Linear Systems (Lec 6-10)

Part. 2.1 Matrix-matrix multiplication

Lec 6: Matrix-vector multiplication
Preparation for Sec. 2.2
Lec 7: Matrix-matrix multiplication, with partition

Lec 8: Matrix-matrix multiplication, with row/column partion

Elementary operation as matrix multiplication

−1
Part. 2.2 Inverse and solution A b

Epg-Eib.sx.tn
⼆

Matrix representation of GE Test/Computation of inverse

Lec 9: LU decomposition Lec 10: Test/Compute inverse

☐ → …

⼝ → U →

AEEpEp-Ei.ir
☐ →_→ RREF
I
 Main Theorem of Lec 10

. ( 1 1 .. 1 1 .). .

Let A ∈ ℝn×n.

The following statements are equivalent:

1. A is inver*ble
2. The linear system Ax = 0 has a unique solu*on x = 0

quatimtestlunderstadingl-rlorAhcsnpivotsldgorithmte.to
3. A is a product of elementary matrices
4. The RREF of A is In
5. More …

We have proved the equivalence of 1,2,3,4.

 Roadmap of Linear Systems
Theme: Solving Math Tools What is Learned?
Linear Systems
Part 1 Gaussian Matrix A method to solve Numerical
elimination Row operations any system Methods
Lec 3,4,5

Part 2 Solutions of Matrix mulplication

LU Decomposition “Good” square system:
Square system
Inverse Solution A −1b Analytical
Lec 6,7,8,9,10 Expressions

Part 3 Solutions of
Vector space Rectangular system:
rectangular system
Solution xp + N(A)
Lec 11,12 ⼀

Solvable iff
Part 4 ————-
Lec 13, 14
Part I Motivation: Solution
of Rectangular System
 Square System: What’s New Compared to Primary School?

Primary school: -
{2x1 − 3x4 = 4. {7x1 − 5x4 = 4. {4x1 + 7x4 = − 9
x1 + 2x2 = 3, 3x1 + 2x2 = 1, x1 − 2x2 = 3,
…

solve 2*2 linear system? So easy!

-

Really? inimeonsfmdingasolut.in?N0!Itmeanfndingthe sohnnset

-

University:

thouts.lu?Needtfndsolutinsa!
Unknown unknowns

⼆
←

Unknown unknown
(for 5th grade students):
Do I know how to solve
all 2*2 linear systems?

—They thought yes;

—But actually no.
5th grade students don’t even know
What “solve” really means; it means
“finding solution set”
 Square System: Knowns

Unknown unknown Known known

(for 5th grade students): (After Lec 10):
2×2 Do I know how to solve Do I know how to solve
Linear system all 2*2 linear systems? all 2*2 linear systems?

—They thought yes; Yes.

—But actually no. Can enumerate.

Known known ?? ??
(After Lec 10): (After Lec 10):
m×n Do I know how to solve Do I know how to solve
Linear system all “good” square all rectangular linear
linear systems? systems?

Yes. ?
Theory guarantees.
From Lec 5: Rectangular Case 1: No Solution

Example 1: RREF is

Solution set: ☒
From Lec 5: Rectangular Case 2: Infinitely many solutions

Example 2: RREF is

⼆
Equivalent linear system:

Equivalent to:
- -
feei
t
- -

Solution set:
Rectangular Case 2: Infinitely many solutions
-

Example 3

Solution set:
t.tl
䰗 州 t
㤈
Rectangular Case 2: Infinitely many solutions

1 0 0 −1 4 ∣5
Example 3 0 1 0 3 1 ∣2
0 0 1 2 2 ∣1
0 0 0 0 0 ∣0

Equivalent linear system:

x1 − x4 + 4x5 = 5, x1 = x4 − 4x5 + 5,
x2 + 3x4 + x5 = 2,
x3 + 2x4 + 2x5 = 1,
⟹ 三
x2 = − 3x4 − x5 + 2,
x3 = − 2x4 − 2x5 + 1,
0 = 0. 0 = 0.

Solution set:

:蕋
维 㤈啊
-3 Ett 2

,
 Solution of Systems?

Unknown ??

-
Known known (After Lec 10):
(After Lec 10): Do I know how to solve all
Do I know how to solve rectangular linear systems?
all “good” square
linear systems? ?
I don’t know whether I know or not.
Yes.
Theory guarantees.
Q1: Have we enumerated all cases?

Identifgthnghayoudonitknow-Qt.at Q2: What is a proper way of expressing

ltgbeasoluninltnswenNoibuthowopo.tt?
(f) the solution set?

E.
Gnl 僽 )
ls.t /Rlbeasolulionset?iTheremgbemorequestionsthctgoudont
Need theory!

knowhoutcnswer.Ourtheorywil helponswe.ir
them
-0
Part II Linear Space
—Linear space
—Subspace
 Linear Space

Informally:
Linear space is a set:
i) that is equipped with addition and scalar multiplication;
ii) any linear combination of elements is in this space.

apple.taangennotwelldefned.IT
Non-ex.mil apple.orongey.sn
"

TrurnptBideitnotwelldefhed.itdd.im
rnmpBidenlthgl.sn
"
。

scdarmultiplicatioishouldbeuelldefnedir-xnvk.nolveshothddiwiadicdcrmuyicel.in
and
" "

category Set +
"

operation Linear combination +

property Closure under linear combination
 Def: Linear Space

Definition 11.1 (Linear space)

Suppose V is a set associated with two operations:
(i) Addition “+”: u + v ∈ V, ∀u ∈ V, v ∈ V .
(ii) Scalar multiplication: αu ∈ V, ∀α ∈ ℝ, u ∈ V . Idoedunderadditionlldosedundascdamulpl.in
V is called a linear space over ℝ if the 8

axiomsholdjotif_最
T.enegtedm-specidpwp.ws
First time in your study:
" Define a subject by “structure”!

" 的 啊 。

Remark: Linear space is also called “vector space”

(in textbook as well); we will use linear space more often.
𪆵
Ret 。

Rulesofoperaliectwaddihiiandicdatnnesvectaisatisf.es
miofenholdfcommmsetsiieg.ir
"

theserulesj-Threeipecidpnpenieioftenhddfrommaia.cl "
。
"

Uosureunderlmecrcombinatiapksiwil bethefocusoflodgsex.mn
-

,
 Example: Euclidean space
n
Eg 1: ℝ is a linear space, called n-dimensional Euclidean space.
-

Remark: Don’t take it for granted!

n
It can be verified by definition that ℝ is a linear space.

Verification:

—Vector addition, scalar multiplication satisfy the required axioms of linear space.

—Any linear combination of vectors is also a vector.

-

—Existence of zero element, inverse of addition, unit element:

0 + u = u; u + (−u) = 0; 1u = u .

dotproduct
What operation is NOT needed for verification?
 Example: Euclidean space
Geometrical interpretation:
n = 1 : R 1 is a line.
-.- 淼 融
下线
Each element corresponds to a point on the line.

露⾯
n = 2 : R 2 is a plane.
Each element corresponds to a point on the0
plane.
1
n = 3 : R 3 is a 3-dimensional space. -

Each element corresponds to a point in the。

㶑
遥感,
space.

空间

Remark: The name “space” comes

from the geometrical term “space”
(e.g. lines forms plane; planes form space);
But can be more than 3D.
 Example: matrix space
Eg 2: ℝm×n is a linear space, called a matrix space.
-

Remark: Don’t take it for granted!

It can be verified by definition.

UerigiRMMBalnfQHowmcngiparyoperatiarether.fr
Verification:

—Matrix addition, scalar multiplication satisfy the required axioms of linear space.

—Any linear combination of matrices is also a matrix of the same dimension.

—Existence of zero element, inverse of addition, unit element:

0 + A = A; A + (−A) = 0; 1 ⋅ A = A.

mahicuTThra.AB.aA.AB.Note.AT
What operation is NOT needed for verification?
Mare

Ntipcrtgoper-Mctixmultipl.ca
tim
 Anology :

Why
matrixmultipliationABnnotusedwhydoit-youneedanearphon.in
thedefnitionofinearsp.ci?forKornmgMAT204l?
畿器 ⾯ ⻔ ⼆⽇
whydontyounedanairplon.mx
fleorn.mg 丽
MAT 241 ?

exishhgthhgltablet.lgplareenough.x.lt
⽐ 11
Thnoperationdoesnotpreservedimens.in
,
,

AIBpreserresd.mension.Anotherreasm.ee
,

XAandAtBare noughloshowsomestructareofthemctrixset.io
wedontneedmore.tw
edontneedAIAaswell-thatanenotneededfrlmea.pe
Thereremgth.gs )
 s.egk-2.PH
Example: polynomial space
Eg 3: Set of polynomials with degree no more than k is a linear space.

fxtotaxtax2n.ao.a.ag.es.n-lfltFMMTPNni.fi
Verification:

fltnataxtaxtRfxf.dtdaflxnxatxaxtxax.tt
ˇ
fg
fltit axtaic-R.glti-bothxtbiep.hftgdefmdoshM-etb.lt
hixttnER.O_f.Iff.fi (f) = 0 ,
✓
 Non-Examples
In the following, assume the set is equipped with regular addition and scalar
multiplication of real numbers.

⼆
⻜
NonEg 1: {0,1,2,3,4,…} is not a linear space.10W R )
- 。

Why?
① nongnelemti-1424-eitiiin.tt
②
NOTdosedunderscdormul244
tiplici -z.ngetsthoutof
l
-_- ( ⻜)
NonEg 2: Set of non-negative real numbers ℝ+ is not a linear space.
,

Why? 化 : X2 0
1 .

?
① dosureunder.lt
"

:
Chakx ≥ 0 y ≥ 0
, ,
xty ≥ 0
Yes !
② Uosureunder
checkixER.xzo-xxzoNolcounter-exon.pk
"
戏 :

:
✗= -
3 , x=1 ,

Thus.NO Talineaspae ,
thenxxi-z.co ,
 ⼼有 甇噝点炎器
Exercise _
盥 ,

In the following, assume the set is equipped with standard addition and

eeser-r-r.gl
scalar multiplication.

Are the following linear spaces?

1-ohsu.tn 调 调
1) Set of n × n upper triangular matrices.

2) Set of n × n elementary matrices.

-_-
㘣 1
䬆 塷了 ,

5[ 凹北 凸
notdemeng-ss-NlnextnageiEM4.eesso.NO
1 2
3) ℝ ∪ℝ . ,

4) {[1,2], [3,4], [0,0]}

5.co?ctM4-M4iNo.Pickdementu=k).au- yEM4.Yes.pnkcnyel m
, 5 [ 1 2)
.

.
=
[

4) {x ∈ ℝ2 : x1 = 1} .
2
5){x ∈ ℝ : x2 = 2x1}. entl刲 ⼈
叫 筑)
EM5EM5IDethsnextpogeJP.tk
( ⾽ (I) 㑎品 ) , EM5.tn
,

E M .
3) RUR

✗
堢) = ?

2
1
Tdfedliht .ch?with estandordad itonThusN0Talmecrspace.
Matkb.xtlytlxty.buttisnotastandordaddil.im
型 In

,
RUR.Yes.puayd.net 崐)
5)
-
,
你 然 到 㙡)
wifg x = 2✗ t = 2⽕
,

soxutlxixz-zx.lv

✗
⼼;
叫荆 汇 ( 幻 EM 5 以 " "⼼
佻
, ,

Utv
( 某们
t.votu-u.EU:1
=

到
VUtl-UKO.EU
I. U_U , V u 1到
=
,

佳) ⼆
,
 Easier Way to Tell a Linear Space?

Is there an easier way to verify a linear space?

Key property: closed under linear combination

 腿⼦ 集
Subspace 驻 间
Definition 11.2 (subspace)
Suppose V is a linear space.
We say W is a subspace of V if two conditions hold:
i) W is a subset of V;
ii) W is a linear space.

In words: A subspace of V is a subset that is itself a linear space.

Proposition 11.1 (criteria of subspace)

Suppose V is a linear space. W is a subspace of V if:
i) W is a subset of V;
。

ii) W contains the zero element: 0 ∈ W; the real number 0.

NOT necessarily

iii) W is closed under addition: u + v ∈ W, ∀ u, v ∈ W .

iv) W is closed under scalar multiplication: αu ∈ W, ∀ u
ydodmder
∈ W, α ∈ ℝ .
l m eacomb. h n
In words: A subspace of V is a subset that is closed under linear combination.
 Subspace Example 1

Eg 4a (biggest subspace) V is a subspace of V.

-

Eg 4b {0} is a subspace of ℝn .

Verification:

closedunderscdcrmulnplicnion.x.Oi@ElGEg
underadd.tn ;
④嬎.dosd = ④
 线组成 ⾯
Subspace Examples ⽽组成 空间

Eg 5a {x ∈ ℝ2 : x2 = 2x1} is a subspace of ℝ2 .
-

{x ∈ ℝ2 : 3x1 + 5x2 = 0} is a subspace of ℝ2 .

15ns
Eg 5b
"
↑
Geometry: a line in the plane. x_x
。

>✗
,
1 ↓
notah.to
3K +5ㄨ20
mhp.ee

Eg 5c
→
{x ∈ ℝ3 : 3x1 + 5x2 + x3 = 0} is a subspace of ℝ3 .
Eg 5d {x ∈ ℝ3 : 3x1 = x3} is a subspace of ℝ3 . ˊ

※
_

Geometry: a ____________ in 3D space.

,_
_
_

-.- - _ -

萨 >

i
L !
 xtsxtxtfvisasubpceofP.NeedtutUEV.x.UEV.V-ER.lt
fierne.se {X : 3

VEV.ldosdunderdhnadscedmtphnn.IT
以

uerifyyoaneedtknowiOWhtistheekmentmVTIhighsc.hn
② Writetheexpressimsofutv.nu
IIXEV
pnkxEV.tn?Xt5X2tXiO,@ax-(x ,xXz,xX3).ChechiaXEi3.(x )t5lxX2
"

) + N) 0
1 3K 5✗2 灯
⼆ 1
- ⼗

lf
thusaXEU.YtyEV.PickxchyEV.hn?Xt5XztY=0,O.
o.0 = 0 ;

341 ⼗ 592 +4 了 ⼆ 0 . ③

npressionofxty-lxty.Xzty.hty.ch
echxtyEN-3lxtyit5lhtytlslf-PX.
如约tt 们 -1 [列 ,
+5

些 0 -
1 0 = 0 .

Thusxtyt V.
 An Important Subspace

{x ∈ ℝ3 : 3x1 + 5x2 + x3 = 0} is a subspace of ℝ3 .

Eg 5c
possingaIG.IM
Non-Eg 2 {x ∈ ℝ3 : 3x1 + 5x2 + x3 = 5} is NOT a subspace of ℝ3 .
○
04M ,
Ttpossing
Definition (homogeneous linear equation)
a1x1 + … + an xn = 0 is a homogeneous linear equation.
。

Remark: a1x1 + … + an xn = b where b ≠ 0 is NOT.

n
Fact: {x ∈ ℝn ∣
n
∑
i=1
ai xi = 0} is a linear space;
Proofisimilerlothelastp.ge
{x ∈ ℝn ∣ ai xi = b} where b ≠ 0 is not a linear space.
∑
i=1
 o
Part III Null Space and
Column Space
—Null space, or Solution space of Ax = 0
—Span and column space
 Taking intersection?
。
Revisit the examples.
Eg 5c {x ∈ ℝ3 : 3x1 + 5x2 + x3 = 0} is a subspace of ℝ3 .
Eg 5d {x ∈ ℝ3 : 3x1 = x3} is a subspace of ℝ3 .

What about the intersection of the two subspaces?

lhtersect.im
-_-
 Taking intersection?
Revisit the examples.
Eg 5c {x ∈ ℝ3 : 3x1 + 5x2 + x3 = 0} is a subspace of ℝ3 .

t.ie?Soluion-naoflneargstem.
Eg 5d {x ∈ ℝ3 : 3x1 = x3} is a subspace of ℝ3 .

What about the intersection of the two subspaces?

Expressed as {x ∈ ℝ3 : 3x1 + 5x2 + x3 = 0, 3x1 − x3 = 0}

Answer: Still a subspace.

Intuitively, the intersection of two planes is a line.

 Homogeneous Linear System
Definition 11.2 (homogeneous linear system)
A homogeneous linear system is
Ax = 0
where A ∈ ℝm×n are given and x ∈ ℝn×1 is the variable

In words: a homogeneous linear system is a linear system with RHS being 0.

 Homogeneous Linear System
Definition 11.2 (homogeneous linear system)
A homogeneous linear system is
Ax = 0
where A ∈ ℝm×n are given and x ∈ ℝn×1 is the variable

In words: a homogeneous linear system is a linear system with RHS being 0.

Theorem 11.1
The solution set of a homogeneous linear system Ax=0 is a linear space.
n
Remark: Also a subspace of ℝ .
thoThusthesol u ti n setofal n easystemonnotbel
Rl u l .t t
零
!
,

Definition 11.3 (null space)

The solution set of Ax=0 is called the null space of A, denoted as N(A).

⼀
 Proof
Proof of Thm 11.1:
Denote the solution set as W.

eoe-i .Closedunderscdormultipl.in:1/EWfXXEW
Need to verify:
(P1) 0 ∈ W .
(P2) W is closed under linear combination.

⼤ ,
袋 = 0 ✗肍 = 0 如此

closedunderaddins_XEW.ge w xtytw
慌 慌 与 肶,连
Two Ways to Generate Subspaces

Solution set of linear equation +

Taking intersection

⼀
==> null space

Next, we show another important mechanism

of generating a subspace: span (linear combination).
Gr.gr N) 侧 ,

← (i) 测-
✗
1 "侧 ,

Chwisal.nearspoco.Bydefinitinall meorcombmotirsar.in W 。
ut! 怕 㴊

W-ytplytrlglmnfsponlul.CI
W-R3.Puofimgv-ftpianbewnit enav.li )圳红哗) ,

sovEW.Thu.NEW.smaWER3.weha.eu⼆⼼

,
 Definition: Span
Definition 11.3 (span)
Suppose V is a linear space.
Suppose 𝒰 = {u1, u2, …, uk} is a subset of V.
The span of 𝒰 is defined as
span(𝒰) ≜ {a1u1 + … + ak uk ∣ a1, …, ak ∈ ℝ}),

In words: the span (of elements of a linear space) is the set of

all linear combinations of these elements

Eg: W={su + tv ∣ s, t ∈ ℝ} is the span of {u, v} .

Remark: For simplicity, we can also say W is the span of u, v .
 Definition: Spanning Set
Definition 11.4 (spanning set)
Suppose V is a linear space.
Suppose 𝒰 = {u1, u2, …, uk} is a subset of V.
If span(𝒰) = V, then we say 𝒰 is a spanning set of V, or 𝒰 spans V.

Eg: {e1, e2, …, en} is a spanning set of ℝn.

Eg: {u, v} is a spanning set of W={su + tv ∣ s, t ∈ ℝ}.

Exercise: Find a spanning set of the matrix space.

Remark: Spanning set is NOT unique. Cannot say “the spanning set”.
 Definition: Column Space

Definition 11.3 (column space)

Suppose A = [a1, …, an] ∈ ℝm×n . is a linear space.
Then span({a1, …, an}) is called the column space of A, denoted as C(A).

In words: A’s column space is the span of A’s column vectors.

Eg: C(In) = _________

1 0 1 0
[2 3] [2] [3]
Eg: Column space of A = 4 3 is the set α1 4 + α2 3 ∣ α1, α2 ∈ ℝ .

Have you seen this before?

 Column Space and Linear System
1 0 1 0
[2 3 ] [2] [3]
Eg: Column space of A = 4 3 is the set α1 4 + α2 3 ∣ α1, α2 ∈ ℝ ,

α1
or set 4α1 + 3α2 ∣ α1, α2 ∈ ℝ , or set {Aα ∣ α ∈ ℝ2} .
2α1 + 3α2

Matrix form:
b ∈ C(A) ⟺ ∃α1, α2, s.t. Aα = b ⟺ Ax = b has at least one solution x

Scalar form: α1 b1 α1 = b1
b ∈ C(A) ⟺ ∃α1, α2, s.t. 4α1 + 3α2 = b2 ⟺ 4α1 + 3α2 = b2 has at least one solution (α1, α2) .
2α1 + 3α2 b3 2α1 + 3α2 = b3

Column form:
b ∈ C(A) ⟺ ∃α1, α2, s.t. ___________ ⟺ ________ has at least one solution x
 Column space and solvability

Proposition 11.1
Ax=b has a solution iff b ∈ C(A) .
The first result in this class about solvability of linear system.
—more will come later.

It a direct outcome of definition:

—b= A x means b is a linear combination of columns of A
with coefficients x1, x2, …, xn .
—“Solvable” means such a linear combination exists.
—Now we consider set of all linear combinations of columns.
—Thus “solvable” iff b is in this set.

Perspective of looking at “space”

(“all possible”; global view).
 Solution Set

Q1: “What” is the solution set of Ax=b?

Partial answer: When b =0, it is a linear space N(A).

For general b?

Q2: How to express/compute the solution set?

Concluding Part
 Summary Today (write Your Own)

One sentence summary:

Detailed summary:
 Summary Today (of Instructor)

One sentence summary:

We study linear space, including null space, column space, span.

Detailed summary:
1. Linear Space
—Set + add, scalar mul2ply + closed under linear combina2on + 0,1, nega2ve element
—Euclidean space
—Matrix space; polynomial space

2. Subspace and span

—Subspace: subset that is closed under linear combina2on
—Span: set of linear combina2ons of v1, …, vn .

3. Null space and column space

—Null space N(A) is solu2on set of Ax = 0
—Column space C(A) consists of linear combina2ons of columns
—Ax = b solvable iff b in C(A)