Shivkumar Kalyanaraman

Rensselaer Polytechnic Institute

1
: shiv rpi
Linear Algebra
A gentle introduction
Linear Algebra has become as basic and as applicable
as calculus, and fortunately it is easier.
--Gilbert Strang, MIT
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
2
: shiv rpi
What is a Vector ?
Think of a vector as a directed line
segment in N-dimensions! (has length
and direction)

Basic idea: convert geometry in higher
dimensions into algebra!
Once you define a nice basis along
each dimension: x-, y-, z-axis
Vector becomes a 1 x N matrix!
v = [a b c]
T
Geometry starts to become linear
algebra on vectors like v!
(
(
(

=
c
b
a
v

x
y
v
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
3
: shiv rpi
Vector Addition: A+B
) , ( ) , ( ) , (
2 2 1 1 2 1 2 1
y x y x y y x x + + = + = +w v
A
B
A
B
C
A+B = C
(use the head-to-tail method
to combine vectors)
A+B
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
4
: shiv rpi
Scalar Product: av
) , ( ) , (
2 1 2 1
ax ax x x a a = = v
v
av
Change only the length (scaling), but keep direction fixed.

Sneak peek: matrix operation (Av) can change length,
direction and also dimensionality!
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
5
: shiv rpi
Vectors: Dot Product
| |
T
d
A B A B a b c e ad be cf
f
(
(
= = = + +
(
(

2
T
A A A aa bb cc = = + +
) cos(u B A B A =
Think of the dot product as
a matrix multiplication
The magnitude is the dot
product of a vector with itself
The dot product is also related to the
angle between the two vectors
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
6
: shiv rpi
Inner (dot) Product: v.w or w
T
v
v
w
o
2 2 1 1 2 1 2 1
. ) , ).( , ( . y x y x y y x x w v + = =
The inner product is a SCALAR!
o cos || || || || ) , ).( , ( .
2 1 2 1
w v y y x x w v = =
w v w v = 0 .
If vectors v, w are columns, then dot product is w
T
v
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
7
: shiv rpi
Bases & Orthonormal Bases
Basis (or axes): frame of reference
vs
Basis: a space is totally defined by a set of vectors any point is a linear
combination of the basis

Ortho-Normal: orthogonal + normal

[Sneak peek:
Orthogonal: dot product is zero
Normal: magnitude is one ]
0
0
0
=
=
=
z y
z x
y x | |
| |
| |
T
T
T
z
y
x
1 0 0
0 1 0
0 0 1
=
=
=
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
8
: shiv rpi
What is a Matrix?
A matrix is a set of elements, organized into rows and
columns
(

d c
b a
rows
columns
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
9
: shiv rpi
Basic Matrix Operations
Addition, Subtraction, Multiplication: creating new matrices (or functions)
(

+ +
+ +
=
(

+
(

h d g c
f b e a
h g
f e
d c
b a
(

=
(

h d g c
f b e a
h g
f e
d c
b a
(

+ +
+ +
=
(

dh cf dg ce
bh af bg ae
h g
f e
d c
b a
Just add elements
Just subtract elements
Multiply each row
by each column
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
10
: shiv rpi
Matrix Times Matrix
N M L =
(
(
(

(
(
(

=
(
(
(

33 32 31
23 22 21
13 12 11
33 32 31
23 22 21
13 12 11
33 32 31
23 22 21
13 12 11
n n n
n n n
n n n
m m m
m m m
m m m
l l l
l l l
l l l
32 13 22 12 12 11 12
n m n m n m l + + =
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
11
: shiv rpi
Multiplication
Is AB = BA? Maybe, but maybe not!





Matrix multiplication AB: apply transformation B first, and
then again transform using A!
Heads up: multiplication is NOT commutative!

Note: If A and B both represent either pure rotation or
scaling they can be interchanged (i.e. AB = BA)

(

+
=
(

... ...
... bg ae
h g
f e
d c
b a
(

+
=
(

... ...
... fc ea
d c
b a
h g
f e
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
12
: shiv rpi
Matrix operating on vectors
Matrix is like a function that transforms the vectors on a plane
Matrix operating on a general point => transforms x- and y-components
System of linear equations: matrix is just the bunch of coeffs !



x = ax + by
y = cx + dy


(

(



=
(

(



'
'
y
x
d c
b a
(

(



y
x
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
13
: shiv rpi
Direction Vector Dot Matrix
c b a v
z y x
v v v + + =
'
0 0 0 1 1
x x x x x
y y y y y
z z z z z
x x x y x z x
y x y y y z y
z x z y z z z
a b c d v
a b c d v
a b c d v
v v a v b v c
v v a v b v c
v v a v b v c
( (
( (
( (
'
= =
( (
( (

'
= + +
'
= + +
'
= + +
v M v
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
14
: shiv rpi
Matrices: Scaling, Rotation, Identity
Pure scaling, no rotation => diagonal matrix (note: x-, y-axes could be scaled differently!)
Pure rotation, no stretching => orthogonal matrix O
Identity (do nothing) matrix = unit scaling, no rotation!
[cosu, sinu]
T
[1,0]
T
[0,1]
T
u
[-sinu, cosu]
T
cosu -sinu
sinu cosu

[1,0]
T
[0,1]
T
r
1
0
0 r
2


[r
1
,0]
T
[0,r
2
]
T
scaling
rotation
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
15
: shiv rpi
Scaling
P
P
r 0
0 r

a.k.a: dilation (r >1),
contraction (r <1)
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
16
: shiv rpi
Rotation
P
P
cosu -sinu
sinu cosu

Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
17
: shiv rpi
2D Translation
t
P
P
t P P + = + + = ) , ( '
y x
t y t x
P
x
y
tx
ty
P
t
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
18
: shiv rpi
Inverse of a Matrix
Identity matrix:
AI = A
Inverse exists only for square
matrices that are non-singular
Maps N-d space to another
N-d space bijectively
Some matrices have an
inverse, such that:
AA
-1
= I
Inversion is tricky:
(ABC)
-1
= C
-1
B
-1
A
-1

Derived from non-
commutativity property

(
(
(

=
1 0 0
0 1 0
0 0 1
I
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
19
: shiv rpi
Determinant of a Matrix
Used for inversion
If det(A) = 0, then A has no inverse
(

=
d c
b a
A bc ad A = ) det(
(

a c
b d
bc ad
A
1
1
http://www.euclideanspace.com/maths/algebra/matrix/functio
ns/inverse/threeD/index.htm
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
20
: shiv rpi
Projection: Using Inner Products (I)
p = a (a
T
x)
||a|| = a
T
a = 1
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
21
: shiv rpi
Homogeneous Coordinates
Represent coordinates as (x,y,h)
Actual coordinates drawn will be (x/h,y/h)
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
22
: shiv rpi
Homogeneous Coordinates
The transformation matrices become 3x3 matrices,
and we have a translation matrix!
1 0 t
x
0 1 t
y
0 0 1
=
x
y
1
x
y
1
New point Transformation Original point
Exercise: Try composite translation.
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
23
: shiv rpi
Homogeneous Transformations
1 0 0 0 1
1 1 0 0 0 1
3 3 3 3
2 2 2 2
1 1 1 1
3 3 3 3
2 2 2 2
1 1 1 1
+ + + =
+ + + =
'
+ + + =
'
+ + + =
'
(
(
(
(

(
(
(
(

=
(
(
(
(

'
'
'
=
'
z y x
z y x z
z y x y
z y x x
z
y
x
z
y
x
v v v
d v c v b v a v
d v c v b v a v
d v c v b v a v
v
v
v
d c b a
d c b a
d c b a
v
v
v
v M v
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
24
: shiv rpi
2
4
Order of Transformations
Note that matrix on the right is the first applied
Mathematically, the following are equivalent
p = ABCp = A(B(Cp))
Note many references use column matrices to
represent points. In terms of column matrices
p
T
= p
T
C
T
B
T
A
T


T R M
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
25
: shiv rpi
2
5
Rotation About a Fixed Point other than
the Origin
Move fixed point to origin
Rotate
Move fixed point back
M = T(p
f
) R(u) T(-p
f
)
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
26
: shiv rpi
Vectors: Cross Product
The cross product of vectors A and B is a vector C which is
perpendicular to A and B
The magnitude of C is proportional to the sin of the angle
between A and B
The direction of C follows the right hand rule if we are
working in a right-handed coordinate system
) sin(u B A B A =
B
A
AB
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
27
: shiv rpi
MAGNITUDE OF THE CROSS
PRODUCT
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
28
: shiv rpi
DIRECTION OF THE CROSS
PRODUCT
The right hand rule determines the direction of the
cross product
Shivkumar Kalyanaraman
Rensselaer Polytechnic Institute
29
: shiv rpi
For more details
Prof. Gilbert Strangs course videos:
http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-
2005/VideoLectures/index.htm

Esp. the lectures on eigenvalues/eigenvectors, singular value
decomposition & applications of both. (second half of course)

Online Linear Algebra Tutorials:
http://tutorial.math.lamar.edu/AllBrowsers/2318/2318.asp