0% found this document useful (0 votes)
71 views585 pages

Introduction To Linear Algebra (5th)

Uploaded by

menlou3
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
71 views585 pages

Introduction To Linear Algebra (5th)

Uploaded by

menlou3
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 585

Table of Contents

1 Introduction to Vectors 1
1.1 Vectors and Linear Combinations. 2
1.2 Lengths and Dot Products. 11
1.3 Matrices ......... 22

2 Solving Linear Equations 31


2.1 Vectors and Linear Equations . 31
2.2 The Idea of Elimination ... 46
2.3 Elimination Using Matrices . 58
2.4 Rules for Matrix Operations 70
2.5 Inverse Matrices ....... 83
2.6 Elimination= Factorization: A= LU 97
2.7 Transposes and Permutations . . . . . . 109

3 Vector Spaces and Subspaces 123


3.1 Spaces of Vectors ....... . . . . . . . . . . . . 123
3.2 The Nullspace of A: Solving Ax = 0 and Rx = 0 135
3.3 The Complete Solution to Ax = b 150
3.4 Independence, Basis and Dimension 164
3.5 Dimensions of the Four Subspaces 181

4 Orthogonality 194
4.1 Orthogonality of the Four Subspaces 194
4.2 Projections . . . . . . . . . . . . . 206
4.3 Least Squares Approximations ... 219
4.4 Orthonormal Bases and Gram-Schmidt. 233

5 Determinants 247
5.1 The Properties of Determinants . 247
5.2 Permutations and Cofactors ... 258
5.3 Cramer's Rule, Inverses, and Volumes 273

iii
iv Table of Contents

6 Eigenvalues and Eigenvectors 288


6.1 Introduction to Eigenvalues.. 2 88
6.2 Diagonalizing a Matrix .... 30 4
6.3 Systems of Differential Equations 319
6.4 Symmetric Matrices........ 338
6.5 Positive Definite Matrices..... 350

7 The Singular Value Decomposition (SVD) 364


7.1 Image Processing by Linear Algebra .... 36 4
7.2 Bases and Matrices in the SVD ....... 371
7.3 Principal Component Analysis (PCA by the SVD). 382
7.4 The Geometry of the SVD ............. 39 2
8 Linear Transformations 401
8.1 The Idea of a Linear Transformation 40 1
8.2 The Matrix of a Linear Transformation. 411
8.3 The Search for a Good Basis ...... 42 1

9 Complex Vectors and Matrices 430


9.1 Complex Numbers ...... 431
9.2 Hermitian and Unitary Matrices 438
9.3 The Fast Fourier Transform. 445
10 Applications 452
10.1 Graphs and Networks ............. 452
10.2 Matrices in Engineering............ 46 2
10.3 Markov Matrices, Population, and Economics 474
10.4 Linear Programming ............. 483
10.5 Fourier Series: Linear Algebra for Functions. 49 0
10.6 Computer Graphics ........ 49 6
10.7 Linear Algebra for Cryptography....... 50 2
11 Numerical Linear Algebra 508
11.1 Gaussian Elimination in Practice 50 8
11.2 Norms and Condition Numbers. 518
11.3 Iterative Methods and Preconditioners 52 4

12 Linear Algebra in Probability & Statistics 535


12.1 Mean, Variance, and Probability ...... 535
12.2 Covariance Matrices and Joint Probabilities 546
12.3 Multivariate Gaussian and Weighted Least Squares 555

Matrix Factorizations 563


Index 565
Six Great Theorems/ Linear Algebra in a Nutshell 574
m+4m- UJ = m·
2
5
angle with v
less than 90 °

!
u. i = cos 0
, v+w
u- [-i l v- Ul w - [ n-
X1 [ - ~
0
l
+ X2 [ ~
-1
l ~l
+ X3 [
1
= [ :~ -
X3 -
X1
X2
l· (1)
X1 = b1
- x1+x2 =b2
-x2 +x3 = b3

b ~ m
gives x ~ m
Cx = 1
[ -1
01
0 -1
- 10
1
i[ l
X1
x2
X3
=
rn

V
1
right weights :r1, .'.C2, x3, can produce
weights A- 1 b.

1
0
~1 0
0 ~1
1 0 0l [ Y2
Y1 l = [ C1C2 l
[ 1 1 0 •
1 1 1 Y3 C3

- ·

1 1 ol o 0cl
[11 1 [c2
[ 321
74c 011 3
-- --
The column picture combines the column vectors on the left side to
produce the vector b on the right side.
''

3[ t ]+ [ -~ ] = [ 1~ ] ·
=
o [ ~ ] +o [ j]
+2 [ ; ] = [ i].
1
I = [ 0
0
ol o0
0 1
l
After ~ =l
~ =8
Before elimination 8y =8

2 3
X
X X
X
X
X
X
X
1
l X X

rn rn=
=

~]
~l m~ m.
P23
1 o ol
= [0 0 1 .
0 1 0
0
0 0
-4 1
3

1
E = [a
ol ol0 1 o ol
F= [ 0 1 0 .
b O l 0 C 1
0 0
1 0
2 1
3 3
1
[3
X
X
X]
X
4 X X
31
2
1
21 21
3 1
1 2
l
2 0
0 2
0 0
0 0
AB = [~ : :] [~ : :] = [ 0
00x 00x 00

Multiply Ax = b by A - 1 . Then x = A - 1 Ax = A - 1 b.
FE= [-t
20
0
1
-4
(6)
2

2
~ ~] ➔ [~
1 -~ -~J ➔ [~
3
A= [ 1 3
1
1 1
1
1 3
l 2
B= [ 1 2
1
1 1
1
1 3
l C = [ 11 11 11
1 1 3
l
6
[ 6 G] [ 6.· 6 ]
() 6 6

,-1 1 [ ()
C = :36 6
2 0
3 0
0 6
0 7
2 1
A= [ 1 2 1
ll
1 1 2

1
A= [ 2 1 3
o ol and
ll
A= [ 11 21 2 .
0 0 1 1 2 3
1 -11
0
-~i
1 -1
0 0 1
1
T = [ -l
-1
2
ol
-1 r- 1 = 3
[2
2
2
ll
1
0 -1 2 1 1 1

1
<Al'J' ~ r (uJ' j ( IJ,) 97
1 0
2 1
1 2
0 1
1 U12/d1 U13/d1
1 U23/d2

dn
2 1
A= [ O 4 2 .
ol
6 3 5

1 1
[ 112=
ol [e11 l [d gl
ef ~ -
121 mnl i

1 c
A= [ 2 4 1 .
ol
3 5 1
A= [~
a
~ ~
b c
~] .
c
a b c d
3by 3 case 2 -1 - O
[
The determinant (K)(4K- 1 ) = - 1 2 1
l [3 2 ll [4 4 l .
2 4 2
of this K is 4 0 -1 2 1 2 3 4
S [1 2]
= 2 5 = S T and D [l 0]
= O IO = D T .

[~ ;] = [~ ~]

u;] = [~ ~]
Ps2P21 = [
1 1 1]
P21Ps2 = [1 1 l
P = [0o1 010 10ol
-.'• ·:.
PS= [451 46 6]5
2
3

PS= [451 46 6]5


2
3 Q
o o
= [1 0 0
1]
0 1 0

S = LDLT = [ 41 o1 ol0 [1 -14 l [10 41 5l1 .


5 1 1 -8 0 0 1
A=
o o
[1 2
6]3 .
0 4 5
=

and S = [ -1
2 -1
2 -1
ol .
0 -1 2

and
1 b
S= [b d
cl
e .
C e j
o 1 1]
A= [ 1 0 1 and
1 2
A= [ 2 4 1 .
ol
2 3 4 1 1 1

o 1 2]
A= [ O 3 8 .
2 1 1
00

0 ~j
- 1
1 0
'•ft
Plane = =
1
[0
ll [X1]
11 1 x2 = [b1]
b2 and [1O 11 ll [X1]
1 X2 = [b1]
b2
0 0 1 X3 b3 0 0 0 X3 b3

and 1 1
[O O 1
ll [X1]
x2 = [b1]
b2 .
0 0 1 X3 b3
!]
1 becomes U = [~ ~
t t
1 3
R= [ 0 O
0 0
C ;t: 4

(b) B = [02 44 42] .


0 8 8
A= [o0 oO 3ol
2 4 6
A= [12 2 4]
4
B = c 2 ].
[1-0 2-c
1 42] [X
[011 330 O
1 6
X2
X3
ll = [ll
6
7
X4
A = [ ~
-2 -3
~] and b= [ ~~] ·
b3
S1 - [ -21
~ S2 - [ 2
: l
1 3
A= [ 3 8 2 .
2 4
l
0
0 6
0 62 ~] and 0 2
0 0 0 0

=
1
A= [ l
o1 ol0
1 1 1
Vj = mv, mv, m m
= = V4 =

Vj = h] H] [j]
v, = V3 = V4 = [-I] V5 = u] [j]
V6 =
A= [101 33l 212] and U = [100 30l 02]1 .
B - [c cl] .
cl C
5 0
0 1
5 1
:;;t

J .• ·-

J
and B-[J
A,

A= [0o0 001 ol01 and I +A = 1


[0
0
1
1
0
ol1 .
1
V W

w
I
C(A) _l_ N(AT)
dimension
all vectors
orthogonal to
the columns
dimm-r

3rows

row S\lace
33 Suppose I give you eight vectors r1, r2, n1, n2, c1, c2, l 1, l2 in R 4.
o o ol
Pi= [0 0 0
0 0 1
I
b
I
I

p =xa
and e = b- p = ( ig ' - !g ' - !9 ) .
=
2
column 8Pace

h1 :::::: 6

Pi :::::: 5 e b - 5 - Jr
best lin -
=
=
=
l
~ ""'[>-
/
/

'
,,"/
/
''
/
a ~ H] and b ~ U] and c ~ H] .
1
- 1

-1 -1
1
1
1 - 1 - 1-~]
1
= =

AT A= [99 189] = [11 10] [9 9] [10 11] .


v12
-v12
-1 -1]
-1 -1
1 -1 .
-1 1
A=
o o
[1 00
1] o
B = [1 0
1 1]1 C=
l
[1
1
1
ll
1 .
0 1 0 1 1 0 1 1 1

A=
l
[1
1 2 3
1
2
ll
2 A= [l 2 3]
2
3
2
3
3
3
.
<let [l
n+l
n

----=
2
1 21 1
[1 1 2 1
j
1 1 1 2
B = [14 42 3]4
5 6 7
B= [41 25 63] .
7 0 0
M= [-B0 A]I = [ABO A]
I
[-BI OJ .
I
0 1
0 0
1 0 0 0
0 1 1 0
Find x = A - 1b Check [ ~ !][_~ ]= [ ~ ]

detB2
x2 = -d-et_A_
----
L
Ya
X1 Y1
1 -
- X2 Y2
2
X3 Y3

(0,0)
I
I
1 4
A= [ 2 3 97
2 2 8
l 1 1
A= [ 1 1 21
1 1 1
l
Ci
1
(a) A= [ O 3
2 ol
O (b) A = [-
2 -1
l
ol
2 - 1 .
0 7 1 0 -1 2

1 1 4]
A= [l 2 2 C= [ .
6 -3

1 2 5
.' •

~.
·• I

. '\ -

. .
for >-1 = 0
- ::::- - .:;;.;:;;;; -
A = [01 21 ] and A- 1 = [-1112/2 10 ].
4 8 4
l
A= [ 31 62 3 and B = P APT = [682 431 43]1 .

P= [L0 0
.... Ax1 = x1

\ I
,\1
x - 1 Ax = A = [

[~~] [~i] -
A X =
] =XA.
>-n

[1 5] = [1 l][1 ][1-lJ= [1
0 6
k
0 1 6k O 1 0
6k -
6k
l]= Ak
.
1
-Yn-1
------- - - Yn
- Yn + l

Forward
(llF)
Yn+l = Yn + flt Zn
Zn+I = Zn - flt Yn
[ 1
becomes U n+l = - flt
flt][z:
1
Y, ]
= AUn · (12)
Backward
Yn+ l = Yn + .6.t Zn+ l
Zn+l = Zn - .6.t Yn+ l
--

--
db
[al [o1 0o0o0ol [alb
dt ~ = ~ ~ ~ ~ ~
d [ y]
dt y'
[ 0 1] [y'y] .
= -9 6
u = [t]
B
o
= [o -4]
0
. [ 1 -b;.t/2 ] [ Yn+l ] _ [ 1 6;.t/2 ] [ Yn ]
Trapezoidal 6;.t/ 2 1 Zn+l - - b;.t/2 1 Zn ·
A3 =
2
[ -1
0
-1
2
- 1
o
-1
2
l 1
...
9
B = [ 12

1
~1
-1
0
S
2
= [- 1
- 1 ol
2 - 1 T
2 -1
= [-1 2 - 1 .
bl
0 -1 2 b - 1 2

ol [- O11 - o11 o]01


O
-l O O - 1
X

x+y =X
J2
10] .
S4 = [ 101 101
2
S= [ 2 5 3 .
2 o]
0 3 8

5 = [
C
1
1 1]
C 1 T= [1 d2 3]
2 4 .
1 1 C 3 4 5

' '
t 3 ol
T= [ 3 t 4 .
0 4 t

S = 9 o1 ol S= [ 1 21 2ll .
1
[0 2
0 2 8 1 2 7
101 - •

10-1 < - - - - - ~_ _ _ _ ___,____ _ _


_ _ _ J L __ _ ____,
10-20 < - - - - - - ' - - - - - - ~ - - - - - " - - - - - '
0 10 20 30 40 0 10 20 30 40
AT A= [ 20 10 ]
10 5
~ [1]
Av2 - ~ [ -f ]
xTATAx xTSx
xTx xTx ·
2
(/1 + ··· + (JR2 1s
· (J/ .
99.10 0/ .
or 99.9.10 2
of <T1 + ••· + <T.,,.
2
'-.__/
V
'.
-~]
n J5 [~
= ~] . Then A = Q S.
o o0 l [2
1/3 0 3o o
0l = [1 o ol [i ~].
0 1 0 =
0 0 000 000
AA+p= ~~ A+ Ax+=

r
The input is v = (v1 , v2 , V3).
v,~~v) ➔
T(v3)

\ -~
,;. -), ~ w ) v,
lnp~t
basis
[vi v = [ 33 6]
2]
8

[ w 1 w 2 ] [ B ] -_ [v
l • [ 3
v 2 ] 1S 1 0
2 ] [ 11 23 ] -_ [ 3
3 6
8] .
0 1 0
Ac= [ 0 0 2
0 0 0

2
?
= o o
[0 0
0 0
1
0
0
l
0l
~
ll l
.\
1
~: = ,\
2

ll
.\
1
~:
2
= ,\ x
4
when A = 1.
J = [ 02 21 0]
1
0 0 2

J=
l O 01 00 00
0 1
0
J K= l
O 01 01 00
0 0
0
J
.' -. ,,
I,
=
iz l2 = 12 + 12 = 2

rcos0 + irsin0 = v'2 ( ~ ) + iv'2 ( ~ ) = 1 + i .


becomes
1
[ ~ - ~] [ ~] = [ ] .

[
1

e-Li/3
e -21ri/ 3
l
.
S = [. 0
i +1
1-
1
i]
2n
\8
~3] [~
w6
w9
1
l
1
i
i2
i3

1
w
w2
w3
[1
= 1
1
-1 -i] [1 1 i ] [1
1 i2 ] 1
F4 ~ i\ 1 1.

F 1024 = [ h12
I
D512] [Fs12 ]
512 - D 512 F,512
[o00 001 o01 ol01 [11 1i
1 i2
i12
i4
i13
i6
l
=
[11 1i
1 i2
i12
i4
i13
i6
l [>11 A2
A3
1 0 0 0 1 i3 i6 i9 1 i3 i6 i9
ol
0
0
-+ [-1
0
0
=
s

Xz a<==-- - - - ------ - - - - - - - - ""9 X3


Y3
=
[U1]
U2
U3
1[1
=-
16
1
l
l
0(.6.x) 2
C
=
A[-~] = .75 [-n .

Uo = [:~~] = [:~] + .18 [-~J.


A=[l0 .8.2 ] A= [·.82 10] A=[t
4
4
1
f
2
l
4
!]
4
1
2
·
481

1 0
0 1
0 0
0 0
The dot product of x ::: 0 and s =c - A '1' y ::: 0 gave x'I' s 2 0. This is xT AT y ~ xT c.
Algebra

(b)
Graphics

by
2 0 - 1 1 - 0 _ 1 -0 _ 2 0
[ 2 1 ] -1 =2 [ -2 2 ] = 2 [ -2 2 ] = [ 2 1 ] (mod 3) ·
A= [ 1l o
1 1
o
1 O
1
l B = [ 01 1l o
1 0
1
1
l 1
1
1
C= [ O O 1
0
1
0
l
=
-11 -1 1
1 - ~ .

- --

3 2
3 2
2 2
1 1
ForA- 1
2

ll
0 ------+ [1 1 ll
0 3 2
2 0 2 4
3
a32 =0 a32 = 2

.6
Q,,A = r -.8
0
Q,1Q21A =[
1 0 ll
A= [ 2 2 0 .
0 2 0
PKPT = [ DT
2I
'•
XT A Ax _ _ _ _ _ _ _ _ _ _ _ _ _ __
T - ------
- x'T"x
--·
...
F = l
. a

- - 2N.,,,..;.,
_ ....__ _ _ _ _ _ ___,__ --I-----+---+----+---+--

1 1) 4 1 4 6 4 1
(
2+ 2 = 16 + 16 + 16 + 16 + 16 = 1.
p = [ Pu
P21
P 12 ] =
P22
[ ¾ ¾
.!
4
.!
4
l
n
i i

i
- I
(x - m1) 2 (y - m2) 2
2o-f 2o-~
-I
X = b"
X = b2
X = b3

1][ 9 4 J[::]
I
(72
v-1 = [ 1
a12
C

You might also like