Vectors basics

Vectors
is the representationof the collection ofnumbers

eachnumberXi iscalled a component orelementofthevector

Devote R theset of realvectors with n components
Unitvector in R the ith unitvectoristhe n vector e suchthat
all entries arezeroexcept theith itorex
ez is secondunitvectorink

product
Dotproduct A B AxButAyByt A B A IB cost
CrossProduct Ax
By tA'lBIsind
if ftp.gfz
A
IAyBz AzBy T lAxBz Be If AxBy AgBe I
uxvl.no v IVX W

equipping vectors in acertaindimension

Vectorspace is obtained by
andmultiplicationby a scalar
with theoperations ofaddition
and span
Subspaces

nonempty subsetVof a vector space is called a subspaceof X if

for anyscalar α andβ
my er ax By EV
and subspace must contain 0 vector originpoint
ex a line is Originpoint
R V is originpoint and linescrossoriginpoint
R originpoint and linescrossoriginpointandplanepassthro 0
 A linear combination of a set ofvectors S
em in a
vectorspace X is a vector oftheform 9 t αme where
Xp am
are givenscalar
vectors in S
Span S set of all possiblelinear combinations of
a subspaceofthevectorspace
span is
Given 2 subspace X or Y inRh the
directsum of X Y is
of the form Xty And
Y is the set of
vectors
Y itself is a subspace
Xand Y addtogether we get
proof Nonemptiness zerovectorfrom
zero vector in Y
Closureunderaddition
X
itupandv2 Y thenVituzmustalsobein Y
LetV1 Upty i V2 22 42 then
V2 1274702 lyntyal
is in i Yity is in Y Uatuis in Y
27 22 and V E X Ythen
closure under scalar mut if c is a scalar
c mustalsobe in Y
CA t Cy
Let Kty then CV
Cris in i cyis iny CV EXTY
Affineset is a translation of a subspace
A Kots
where S is a subspace Io is a point
Ex
 Independence
A set ofvectors 74 n in R is said to be independent if
there isno vectors can beexpressed as linearcombination oftheothers
OR α X 0 implies α 0
i
Forsquarematrix ifthedeterminant is 0then thevectorsare linearly
dependent if norzero independent
Bases
Subspaces of a vectorspaceX a basis of 5 is a set B of vectors
ofminimal ofelements such that span B 5
basis of it
271 In EX thesevectorsform a
1 Theyare linearlyindependent
2 theyspanthe vectorspace
Ex
f span B Stryingtoprove

4 1 I E
spar B S
EEL

f C1 C2 13 0
of5
linearly independence

ofelements vectors isthe

dimenson

theset of justthezerovectorhasdimensonzero
scalar product
2487 KYK
 17 2 y 2
length
Euclideanlength otx
fff.IT
4470 I D length 271
11241

Euclidean distance
Distance btw vectors x
y
d 17491 112 4112
11 4112 12 8 T Xty 11 112 114112 2
xty
2
1154511 11511 118

Pr 1 x̅ 8112 x̅ f x̅ 5
x̅ f J x̅ 87
1
11112 21 I 1151112
wehave x̅ If x̅ I 1151111811

11 x̅ 811 11515 211501111871 14112

115ty'll's111511 1181112
1151 8111 1175111 11811

y
1 41 Atyl
 Orthogonal

UPandw̅ when U V7 U V 0
nonzero vector I x are saidto be mutually orthogonal
if eachvectoris orthogonal to all othervectors
orthogonal vector are linearlyindependent
Mutually
5 741 2 is saidto be mutually orthogonal if i 5 1 d

Ix x 0 if I
to it i j
 If elements in5 are length 1 Sis orthornormal
A collection of orthonormal vectors forms an orthonormalbasis
of the span 5
Projection
J
Projection of a point onto a line p
114112 1 1
line L Xo span1h where p no
Pll
p arguing
sinceanypoint on L canbewritten as
Xotta 1101112 we needto find ascalar t
tu IP
Ht Il
thatminimize flt
14 11 tu Ip 20711
tu P 2011T Ita Ip 2011

t utu Zt at p 700 P to Tp Xo
t It at p 11 P 1011

1885142
E 2tuTP Kolt at P XD
WT P
flt min D when t

p Xo wtIP w
Cauchy Schwart inequality

1 x̅ Y 1411 1111511
1 3.811 11 11118ll x̅ Cy
proof
PIH I ty x̅ 12 10
 Ity x̅ lty x̅
0
Eft IgE EEl
PCH at be 70
ka Dfat 40
cs.
fi a

cx
Fa
C3 4ac7b
w
4111811 1141 12121.8712
1181111541 x̅ y't
when x̅ y ICHlyTHy
I x̅ I l cyt.gl 1Cl1yt.yY 1411Y1l
11cy1111811
115 ll 1181
Anglebtw vectors

w̅ I5
51 cosine rule
11511 11811 2115711511cosQ
1at5 11 112 11811 2115711511cosQ
ata at 5 5 w̅ 51.5 115
ar 2101.5 151 115112 11811 2115711511cosQ
 2m51 211811115710050

cost

Functions
Linear tune andaddition of
that preserve scaling
Linear fare are func
the input argument
R linear if andonlyif
A function f R 7 is
ER and α ER flax at 124
V74 ER that E flint flicz
Attinefunction linear const
expressedas
474 it 2 5
linear if 5 0
for any affinefunc
b 101
ai He b i1 n

Gradients

2
0414 1 4 7
affinefunction f R R f 2 ATCtb hassimplegradient
75174 a
 Norms and lp norms
Afine denoted 11.11 is a norm if
0 X EX and 11 1 0 if only if 2 0
and
11 11
11 xt y If 11 11 Hyll for any x y EX
andany KEX
T 11 α all 1211211 foranyscalar α

Up norm
11241
p 141 Kps

Ex by noun is the Euclidean length

absolutevalueslength
by norm is the sum of
Ux norm is the max abs value norm
to nor zero element
inequalities
generalize cachy Schwarz
common b iney
Ixtyl 11 112119112
Ixtyl 1174111141 a
general for p E 1
Ali
Ixtyl Hallp1411 where
14 1
 RMSvalue de mean variance and std

RMS
rmstu
f.beFE
l demeanvectori Y
x augfal1l x
1115 7
Var
varia 1117114

Std
Std174 Fork rms 51
 Orthonormal basis

A basis a is saidtobe orthogonal it Vitu D if i j

If 11will 2 1 we
say that thebasis is orthonormal
Othogonalization
Procedure to find an orthonormal basisof thespan ofgivenvectors

Givenvectors ay ok ER on orthogonalization procedure

computer vectors 41 Une Rnsuchthat
5 span a a span 91 9r

Projection on a line
consider the line
49 tq.it ER
whereER is given and normalized 119112 1
q
Theprojection of a givenpoint at R on theline is a vector
located on the line that is closestto a

Apro gta q
Ab
b bta
7442
a
f
proportion
Tai
b pro bona b pro bona
bta b btak
all
GS procedure
 74 24 0
122 0
24

x
x1 4

any can be represent

by Xp and
and and are independent
Dimension of S is 2
basis
en ee

e 0
24
1
1
14
 Matrices
A
L L
AER m row
n columns
if m
element vow I
Ai
coli kiln
Matricesvectors multiplication
Amnion an 12 n

mp's amazt tam flattens

mX X Xx1 Mx 1
1
Columns of
A is givenbyvectors a with it n

A Can ar
Ax
E ai

a
at a
I If
rows of A are therowvectors at i D mi

A
l
Left productt
12m41 2am
 2 Any w by xn Ai 2 1 I I n

Matrix matrix product

Amy Bak mxk

I I ID
AB
liii.IE
1AB ii AikBk
sothat B brim bp
if B
columsof are givenby bi with i 1 pi
then AB Alb bp Ab Abp

if A is givenby itsrows at
i 1 m then

AB
1 P I
AB BTAT
Matrices as linear
maps
canberepresented by amatrixA cRma
f R RM
Anylinear map
vector x E R to outputvectors ye R
mappinginput
y Ax
Attirenap
Axth
Ex f R R2
1 3 ER

lit
Azxy I
Range rank
Amen denote a is 1 no its i thecolumn sothat
A a an
for KE Rn Ax is known as therange of A denotedRIAI
y
RIA Ax KERY
Dimension of RIAI iscalled rankof A with rank A thetank
representsthenumber oflinearly dependent columnsofA

Null space
of A E R isthe set ofvectors inthe input
Nullspace
space that are mapped to 0
solutiontothesystemAx 0
NAI ER A x O

Range of f AN E IRM Xt IR
ER
Ax Xp a 122Act Know Ki
span ay an C A
column

row vectors of A C AT
dimension A dimension CAT rank A of
 independent column

ofindependent
row
RN CIA N A Rm
IAT NIA
dimCIA dimN
AT m
Matrixreduction method

A re autre lo
non zero row is rank 1 dim At 1
a basis

NIA Ax 0 Rx D
Yeduction
matrix
0 3 3 0
12 2
o 74
25 36

2differentparameter
dintA 2
bi
to

5 0,1 1

at
 I
Eh I rank
dimNAT 1

R 11 411
1 1 2
dim A 1
no
y
NIA
4
18T
C A NIATI IR
dim A dm At 0
1

ATB ATTBT
 Specialmatrices
Square rows col
Symmetric At A
PPT PTP I
orthonormal matrices
m orthornamed if its a colums or itsrows
R is
Squarematrix UE mutuallyorthogonal
and are
by Un E R are unit
norm
UTU 0 if i j 1 it i j
lengthsofvectors
Ottornomalmatrias Preservesanglesand
T
agg ER Ux Uy X U Uy sety
Lowrankmatrices
Rank one matrix A
a not
EEIIIE.la
with Uma btwn
scalar multiplesofeachother
all columns are
All rowsand
rank1 matrix isalso calleddyad
WERM VER
At Rm A wut for
The dyad acts on in putvector ICERh as follows

Ax with utc w
direction w in R output isalways
inthe same
Ax alwayspoints
scaled of w
 Trace
matrixA ER I trace A is the sum of its diagonal
square
elements

A ACH ay tacet Ann

Trace R R
A trace A
Trace ATB traceAt traceB
trace A trace A
trace A trace At
truceAB trace.BA
Determinant

A ERAN

A in 92 apr
4 922 Az
Any Anzi Ann

det A are 11 Ay 921 11 An

i and Ann
detA det AT
detCAB det BA det A det B
 det XA α det A
det A 0 AX 0 a set of solution
NYA dim NIA 70

1 A a

det A OF a 0
a 2 0 2 0

det A O a 0 a 0 ER
NA R
det A 0 A 2 0 has a uniquesolution in R

1
Matrice inverse

AER
Identity matrix
AA A A In
A
fetal
det A 1 A det A A det In 1
1 det A
det A 1
1
det A
 Flat
In
8
win

AB B A
A T
AT

214kt 73 1
 Matrix norms

AE Run
Axl P Aullp
Allp
11 max
240 1121p
p
11AM P NADP I AxllpKHAllp.HN

llxll HAlIpHBxllpfllAllplIBlp
11ABxllpHelp
11
214 AllPUBMP

HABHIP
my Alp HBIIP
4741p

11ABlip IAllp 11Blip

 bynorm
11A Aw aid
in

maximum of fault I 1102111 0311

maximum otabscolumn sum

horm

Allo mu
Ah EY.ME lait

max of abs row som

ly norm
AER ATA ER
values ofATA of A
eigen singlevalue
max

det ATA In O characteristic

ey of ATA
polynomialof X order n
t 4 0
in Cny

I att
ATA
 det ATA In

1
to
1 12.5
2 43 2
3 68.3
Amax ATA 68 3 6.3 8 3 HAIL

HAME TEXETT
Frobenius term
11AIF FracATA

3 68.3

trace At An
74 5
75
2