Scribd
Upload
171 views17 pages

MATH 4A - Linear Algebra With Applications: Lecture 26: Orthogonal Projections

This document is the notes for a lecture on orthogonal projections. It begins by finishing a discussion on orthogonal sets and introducing orthogonal projections onto lines. It then defines orthogonal and orthonormal bases. It derives the formula for orthogonal projection of a vector onto a line and discusses projecting onto subspaces. It emphasizes that orthogonal projection provides the closest point in the subspace to the original vector.

Uploaded by

akshay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
171 views17 pages

MATH 4A - Linear Algebra With Applications: Lecture 26: Orthogonal Projections

This document is the notes for a lecture on orthogonal projections. It begins by finishing a discussion on orthogonal sets and introducing orthogonal projections onto lines. It then defines orthogonal and orthonormal bases. It derives the formula for orthogonal projection of a vector onto a line and discusses projecting onto subspaces. It emphasizes that orthogonal projection provides the closest point in the subspace to the original vector.

Uploaded by

akshay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

MATH 4A - Linear Algebra with Applications


Lecture 26: Orthogonal projections

3 June 2019

Reading: §6.3-6.4
Recommended problems from §6.3: 1-13 odd, 19, 21, 22, 23
Announcement: please fill out online evaluations!
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Lecture plan

1 Finishing up w/ orthogonal sets

2 Orthogonal projections onto lines

3 General orthogonal projections


Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Recall: life is better with orthogonal bases

If B = {b1 , b2 , . . . , bn } is an orthogonal basis of Rn , then there are


explicit formulas for writing v as a linear combination of B. More
generally, we have the following:
Theorem
Let {u1 , u2 , . . . , up } be an orthogonal basis for a subspace W of
Rn . For each y in W , the weights in the linear combination

y = c1 u1 + · · · + cp up

are given by
y · uj
cj =
uj · uj
for each j = 1, . . . , p.
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

iClicker
Recall from Friday that the set of vectors B consisting of
     
3 −1 −1/2
u1 = 1 u2 =  2  u3 =  −2 
1 1 7/2
is an orthogonal basis of R3 . What is the u2 coordinate of
 
1
v = 2

1
in the B-basis?
(a) uu22·u·v2
(b) v·u
v·v
2

(c) uv·u
2 ·u2
1

v·u2
(d) u2 ·u2
(e) uv·u
3 ·u3
2
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Orthonormal bases

There is one more thing we can impose on an orthogonal set to


make it even nicer: require each of the vectors to have unit length.

A set {u1 , . . . , up } is an orthonormal set if it is an orthogonal set


of unit vectors. If W is spanned by such a set, we call the set
{u1 , . . . , up } an orthonormal basis of W .
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Important properties of orthonormal sets

1 An m × n matrix U has orthonormal columns if and only if


UT U = I .
2 (Ux) · (Uy) = x · y for any two vectors x and y in Rn . In
particular:
(i) kUxk = kxk for any vector x in Rn .
(ii) (Ux) · (Uy) = 0 if and only if x · y = 0
Intuition: when m = n, we call a matrix U with orthonormal
columns an orthogonal matrix (although a better name would be
orthonormal matrix...). The transformation x 7→ Ux is a rotation
of Rn .
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Examples

The standard basis of Rn is an orthonormal basis:

InT In = In In = In .

The set consisting of the vectors


 √   √   √ 
3/√11 −1/√ 6 −1/√66
v1 = 1/√11 v2 =  2/√6  v3 = −4/√ 66
1/ 11 1/ 6 7/ 66

is orthonormal. It forms an orthonormal basis of R3 .


Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Motivation: component analysis

It can be very useful to “resolve” a vector into a sum of orthogonal


components, e.g. when trying to figure out how fast a box slides
down a ramp. See board.
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

The mathematical problem

Given a nonzero vector u in Rn , we want to decompose another


vector y into the sum of two vectors, one a multiple of u and the
other orthogonal to u. That is, we want to write

y = ŷ + z,

where ŷ = αu for some α and z is orthogonal to u.


Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Deriving the solution

“z is orthogonal to u” means z · u = 0. But

z = y − ŷ = y − αu,

so
(y − αu) · u = y · u − αu · u = 0,
and we conclude
y·u
. α=
u·u
(Note: we assumed u 6= 0, so we’re not dividing by 0 here.)
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

The answer: orthogonal projection onto a line


Given a nonzero vector u and another vector y, we call
y·u
ŷ = u
u·u
the orthogonal projection of y onto u, and
z = y − ŷ
the component of y orthogonal to u.

Remark: the formula for ŷ depends on u, but the answer is the


same for any nonzero scalar multiple of u:
y · (ku) k2 y · u
(ku) = 2 u = ŷ.
(ku) · (ku) k u·u
In other words, ŷ only depends on the line L spanned by u. For
this reason, we sometimes call ŷ the orthogonal projection of y
onto u, and use the equivalent notation projL y = ŷ.
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

iClicker 1
   
9 −1
Let u= and y = . What is the projection of y onto
−2 4
the line spanned by u?
 
85 9
(a) −17 −2
 
−17 −1
(b) 85 4
 
−17 9
(c) 85 −2
 
85 −1
(d) −17 4
 
68/85
(e)
306/85
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

iClicker 2
  
9 −1
Let u = and y = . What is the component of y
−2 4
orthogonal to u? (Hint: use the answer to the previous question.)
 
918/85
(a)
−204/85
 
−17 −1
(b) 85 4
 
−17 9
(c) 85 −2
 
85 −1
(d) −17 4
 
68/85
(e)
306/85
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Example: orthognal projection onto subspace

Let {u1 , . . . , u4 } be any orthogonal basis of R4 and let

y = c1 u1 + · · · + c4 u4 .

Consider the subspace W = Span{u2 , u4 }. Write y as a sum of a


vector z1 in W , and a vector z2 in W ⊥ .

See board.
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Orthogonal decomposition theorem

Theorem
Let W be a subspace of Rn . Then each y in Rn can be written
uniquely in the form
y = ŷ + z
where ŷ is in W and z is in W ⊥ . In fact, if {u1 , . . . , up } is any
orthogonal basis of W , then
y · u1 y · up
ŷ = u1 + · · · + up
u1 · u1 up · up

and z = y − ŷ.

We call ŷ the orthogonal projection of y onto W , and sometimes


write it projW y.
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Why we care

It requires a bit of work to prove, but here is why the previous


theorem is so important:
projW y is the point in W that is closest to y. (See Theorem 9 in
the textbook for a precise formulation.)

This fact means orthogonal projection shows up in many real world


applications, since experiments and data have errors.
Finishing up w/ orthogonal sets Orthogonal projections onto lines General orthogonal projections

Example

Let      
−2 −1 3
u1 =  0  u2 =  1  y = 2 .
1 2 1
Let’s find the orthogonal decomposition of y with respect to the
subspace W = Span{u1 , u2 }.

You might also like