Lecture 06
Principal Component Analysis
PCA
Contents
In this lecture we will look at:
PCA
Face recognition application
PCA 2
Face detection and recognition
Detection Recognition “Sally”
PCA 3
Applications of Face Recognition
Digital photography
PCA 4
Applications of Face Recognition
Digital photography
Surveillance
PCA 5
Consumer application: iPhoto 2009
http://www.apple.com/ilife/iphoto/
PCA 6
Starting idea of “eigenfaces”
1. Treat pixels as a vector
x
2. Recognize face by nearest neighbor
y1...y n
k argmin y k x
k
PCA 7
The space of all face images
• When viewed as vectors of
pixel values, face images
are extremely high-
dimensional
• 100x100 image = 10,000
dimensions
• Slow and lots of storage
• But very few 10,000-
dimensional vectors are
valid face images
• We want to effectively
model the subspace of face
images
PCA 8
Efficient Image Storage
PCA 9
Efficient Image Storage
PCA 10
Efficient Image Storage
PCA 11
Geometrical Interpretation
PCA 12
Geometrical Interpretation
PCA 13
Dimensionality Reduction
The set of faces is a “subspace” of the set
of images
• Suppose it is K dimensional
• We can find the best subspace using
PCA
• This is like fitting a “hyper-plane” to the
set of faces
- spanned by vectors u1, u2, ..., uK
Any face: x≈ +w1u1+w2u2+…+wkuk
PCA 14
Principal Component Analysis
A M x M pixel image of a face, represented as a vector
occupies a single point in M2-dimensional image space.
Images of faces being similar in overall configuration, will
not be randomly distributed in this huge image space.
Therefore, they can be described by a low dimensional
subspace.
Main idea of PCA for faces:
• To find vectors that best account for variation of face images in
entire image space.
• These vectors are called eigenvectors.
• Construct a face space and project the images into this face space
(eigenfaces).
PCA 15
Image Representation
Training set of N images of size
M*M are represented by vectors of
size d=M2
x1,x2,x3,…,xN
1
Example 2
1 2 3 3
3 1 2 3
1
4 5 1 33
2
4
5
1
91
PCA 16
Principal Component Analysis (PCA)
• Given: N data points x1, … ,xN in Rd
• We want to find a new set of features that are linear
combinations of original ones:
u(xi) = uT(xi – µ)
(µ: mean of data points)
• Choose unit vector u in Rd that captures the most data
variance
PCA 17
Principal Component Analysis
• Direction that maximizes the variance of the projected data:
N
Maximize
subject to ||u||=1
Projection of data point
N
1/N
Covariance matrix of data
The direction that maximizes the variance is the eigenvector
associated with the largest eigenvalue of Σ
PCA 18
Eigenfaces (PCA on face images)
1. Compute covariance matrix of face images
2. Compute the principal components (“eigenfaces”)
• K eigenvectors with largest eigenvalues
3. Represent all face images in the dataset as linear
combinations of eigenfaces
• Perform nearest neighbor on these coefficients
PCA 19
Eigenfaces example
Training
images
x1,…,xN
PCA 20
Eigenfaces example
Top eigenvectors: u1,…uk
Mean: μ
PCA 21
Representation and reconstruction
• Face x in “face space” coordinates:
PCA 22
Representation and reconstruction
• Face x in “face space” coordinates:
=
• Reconstruction:
= +
x^ = µ + w1u1+w2u2+w3u3+w4u4+ …
PCA 23
Recognition with eigenfaces
Process labeled training images
Find mean µ and covariance matrix Σ
Find k principal components (eigenvectors of Σ) u1,…uk
Project each training image xi onto subspace spanned by
principal components:
pi=(wi1,…,wik) = (u1T(xi – µ), … , ukT(xi – µ))
Given novel image x
Project onto subspace:
p=(w1,…,wk) = (u1T(x – µ), … , ukT(x – µ))
Optional: check reconstruction error x^ – x to determine
whether image is really a face
Classify as closest training face in k-dimensional
subspace
PCA 24
Recognition
The distance of p to each face class is defined by
Єk2 = ||p-pk||2; k = 1,…,N
A distance threshold Өc, is half the largest distance
between any two face images:
Өc = ½ maxj,k {||pj-pk||}; j,k = 1,…,N
PCA 25
Recognition
Find the distance Є between the original image x and
^
its reconstructed image from the eigenface space, x,
Є2 = || x – x^ ||2
Recognition process:
• IF Є≥Өc
then input image is not a face image;
• IF Є<Өc AND Єk≥Өc for all k
then input image contains an unknown face;
• IF Є<Өc AND Єk*=mink{ Єk} < Өc
then input image contains the face of individual k*
PCA 26
PCA
General dimensionality reduction technique
Preserves most of variance with a much more compact
representation
• Lower storage requirements (eigenvectors + a few
numbers per face)
• Faster matching
What are the problems for face recognition?
PCA 27
Limitations
Global appearance method: not robust to
misalignment, background variation
PCA 28
