See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/3970372

Face recognition using extended isomap

Conference Paper  in  Proceedings / ICIP ... International Conference on Image Processing · February 2002
DOI: 10.1109/ICIP.2002.1039901 · Source: IEEE Xplore

CITATIONS READS

62 202

1 author:

Ming-Hsuan Yang
University of California, Merced
108 PUBLICATIONS   18,191 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Ming-Hsuan Yang on 25 September 2022.

The user has requested enhancement of the downloaded file.

 FACE RECOGNITION USING EXTENDED ISOMAP

Ming-Hsuan Yang
Honda Fundamental Research Labs
800 California Street
Mountain View, CA

ABSTRQCT

The Isomap method has demonstrated promising results in

finding low dimensional manifolds from samples in the high
dimensional input space. While conventional subspace meth-
ods compute L1 or L2 metrics to represent distances be-
tween samples and apply Principal Component Analysis or
alike to induce linear manifolds, the Isomap method es-
timates geodesic distance between samples and then uses Fig. 1. A complex manifold that shows why Eucledian dis-
multidimensional scaling to induce a low dimensional man- tances may not be good metrics in pattern recognition.
ifold. Since the Isomap method is developed based on re-
construction principle, it may not be optimal from the clas-
sification viewpoint. In this paper, we present an extended Figure 1 shows one example where the data points of differ-
Isomap method that utilizes Fisher Linear Discriminant for ent classes are displayed in distinct shaded patches on the
pattern classification. Numerous experiments on image data left hand side and data points sampled from these classes
sets show that our extension is more effective than the orig- are shown on the right hand side. For any two points on the
inal Isomap method for pattern classification. Furthermore, manifold (circled data points on the right hand side), their
the extended Isomap method shows promising results com- Euclidean distance may not accurately reflect their intrin-
pared with best classification methods in the literature. sic similarity and consequently is not suitable for determin-
ing true embedding or pattern classification. This problem
1. MOTIVATION AND APPROACH can be remedied by using geodesic distance (i.e., distance
Subspace methods can be classified into two main categories: metrics along the manifolds) if one is able to compute or
either based on reconstruction (i.e., retaining maximum Sam- estimate such metrics. The Isomap method first constructs
ple variance) or classification principle (i.e., maximizing the a neighborhood graph that connects each point to all its k
distances between samples). Principal Component Analysis nearest neighbors, or to all the points within some fixed ra-
(PCA) and Multidimensional Scaling (MDS) have been ap- dius E in the input space. For each pair of points, the shortest
plied to numerous applications and have shown their ability path connecting them in the neighborhood graph is com-
to find low dimensional structures from high dimensional puted and is used as an estimate of the true geodesic dis-
samples (41. These unsupervised methods are effective in tance. These estimates are good approximations of the true
finding compact representations and useful for data interpo- geodesic distances if there are sufficient number of samples
lation and visualization. On the other hand, Fisher Linear (i.e., the patches of the samples are smooth). The classi-
Discriminant (FLD) has shown their success in pattern clas- cal multidimensional scaling method is then applied to find
sification where class labels are available [2] [4].In contrast a set of low dimensional points with similar painvise dis-
to PCA which finds a projection direction that retains max- tance. The Locally Linear Embedding (LLE) method cap-
imum variance, FLD finds a projection direction that maxi- tures local geometry of complex embedding manifold by a
mizes the distances between cluster centers. Consequently, set of linear coefficients that best approximates each data
FLD-based methods have been shown to perform well in point from its neighbors in the input space [6]. LLE then
classification problems such as face recognition [ 11. finds a set of low dimensional points where each can be lin-
Two dimensionality reduction methods have been re- early approximated by its neighbors with the same set of
cently proposed for learning complex embedding manifolds coefficients that was computed from the high dimensional
using local geometric metrics within a single global coor- data points in the input space while minimizing reconstruc-
dinate system [6] [lo]. The Isomap (or isometric feature tion cost. Although these two methods have demonstrated
'mapping) method argues that only the geodesic distance re- excellent results to find the embedding manifolds that best
.-fleets the true geometry of the underlying manifold [lo]. describe the data points with minimum reconstruction er-

0-7803-7622-6/02/$17.00 02002 IEEE I1 - 117 IEEE ICIP 2002

Authorized licensed use limited to: Univ of Calif Merced. Downloaded on September 25,2022 at 04:46:07 UTC from IEEE Xplore. Restrictions apply.
 Tor, they are suboptimal from the classification viewpoint. where p is mean of all samples f i , pi is mean of class Zi,
Furthermore, these two methods assume that the embedding Swi is the covariance of class Zi,and Ni is the number of
manifold is well sampled which may not be the case in some samples in class Zi. The optimal projection W F L Dis de-
classification problems such as face recognition since there termined by the matrix whose orthonormal columns maxi-
are typically only a few samples available for each person. mizes the ratio of the determinant of the between-class scat-
We propose a method that extends the Isomap method ter matrix of the projected samples to the determinant of the
with Fisher Linear Discriminant for classification. The crux within-class scatter matrix of the projected samples:
of this method is to estimate geodesic distance, similar to
what is done in Isomap, and use the painvise geodesic dis-
tance as feature vectors. We then apply FLD to find an opti-
mal projection direction to maximize the distances between where {wili = 1 , 2 , . . . ,m} is the set of generalized eigen-
cluster centers. Experimental results on two data sets show vectors of SB and S w ,corresponding to the m largest gen-
that the extended Isomap method consistently performs bet- eralized eigenvalues { Xili = 1 , 2 , . . . ,m}. The rank of SB
ter than the Isomap method, and performs better than or as is c - 1 or less because it is the sum of c matrices of rank
equally well as some best methods in the literature. one or less. Thus, there are at most c - 1 nonzero eigenval-
2. EXTENDED ISOMAP ues [4]. Finally, each data point xi is represented by a low
Consider a set of m samples {XI,. . . ,x m } and each Sam- dimensional feature vector computed by yi = WFLDf i.
ple belongs to one of the c class { 21, . . . ,Z c } ,the first step In practice, there may not be sufficient samples at one’s
in the extended Isomap method is, similar to the Isomap disposal so that the geodesic distances d~ (xi,xj) may not
method, to determine the neighbors of each sample xi on be good approximates. Consequently, the Isomap may not
the low dimensional manifold M based on some appropri- be able to find true structure from data points and not suit-
ate distance metrics dx (xi,xj) in the input space X ( e.g., able for classification purpose. In contrast, the extended
Eucledian distance that is often used in face recognition or Isomap method utilizes the distances between the scatter
tangent distance that has been shown to be effective in hand centers (i.e., poor approximates may be averaged out) and
digit recognition [9].). Two methods can be utilized to de- thus may perform well for classification problem in such
termine whether two data points are neighbors or not. The situations. While the Isomap method use classical MDS to
k-Isomap method uses a k nearest neighbor algorithm to de- find dimensions of the embedding manifolds, the dimen-
termine neighbors while the E-Isomap method includes all sionality of the subspace is determined by the number of
the points within some fixed radius E as neighbors. These class (i.e., c - 1)in the extended Isomap method.
neighborhood relationships are represented in a weighted To deal with the singularity problem of within-scatter
graph G in which &(xi, xj) = d x ( x i ,xj) if xi and xj are matrix SW that one often encounters in classification prob-
neighbors, and dx (xi,xj) = oc) otherwise. lem, we can add a multiple of the identity matrix to the
The next step is to estimate geodesic distance d~ (xi,xj) within-scatter matrix, i.e., SW + E I (where E is a small

between any pair of points on the manifold M . Since the number). This also makes the eigenvalue problem numeri-
embedding manifold is unknown, d ~ ( x ixj) , is approxi- cally more stable. See also [I] for a PCA-based method to
mated by the shortest path between xi and xj on G, which overcome the singularity problems.
is computed by the Floyd-Warshall algorithm [3]: 3. EXPERIMENTS
We test both the original and extended Isomap methods against
dc (xi,xj) = minidc (xi,xj 1, d c (xi,xk + dc(xk, xj)} LLE [6], Eigenface [ l l ] and Fisherface [l] methods using
The shortest paths between any two points are represented the publicly available AT&T [7] and Yale databases [I]. The
in a matrix D where Dij = &(xi, xj). face images in these databases have several unique charac-
The main difference between our extended Isomap and teristics. While the images in the AT&T database contain
the original method is that we represent each data point by a facial contours and vary in pose as well as scale, the face
feature vector of geodesic distance to any other points, and images in the Yale database have been cropped and aligned.
then apply Fisher Linear Discriminant on the feature vectors The face images in the AT&T database were taken under
to find an optimal projection direction for classification. In well controlled lighting conditions whereas the images in
other words, the feature vector of xi is an m- 1dimensional the Yale database were acquired under varying lighting con-
vector f i = [Dij]where j # i. The between-class and ditions. We use the first database as a baseline study and
within-class scatter matrices in Fisher Linear Discriminant then use the second one to evaluate face recognition meth-
are computed by: ods under varying lighting conditions.
The AT&T (formerly Olivetti) database contains 400
images of 40 subjects. To reduce computational complex-
ity, each face image is downsampled to 23 x 28 pixels. We

11- 118

Authorized licensed use limited to: Univ of Calif Merced. Downloaded on September 25,2022 at 04:46:07 UTC from IEEE Xplore. Restrictions apply.
 figure 2 shows 22 closely cropped images of two subjects
which include internal facial structures such as the eyebrow,
eyes, nose, mouth and chin, but do not contain facial con-
tours.
using the same leave-one-out strategy, we vary the num-
ber of principal components to achieve the lowest error rates
for eigenface and lle methods. For Fisherface and extended
Isomap methods, we project all samples onto a subspace
Fig. 2. Face images in the AT&T database (Top) and the
spanned by the c - 1 largest eigenvectors. The experimen-
Yale database (Bottom).
tal results are shown in Figure 4. Both implementations of
the extended Isomap method perform better than their coun-
represent each image by a raster scan vector of the inten- terparts in the Isomap method. Furthermore, the extended
sity values, and then normalize them to be zero-mean unit- 6-Isomap method performs almost as well as the Fisherface
variance vectors. Figure 2 shows images of two subjects. method (which is one of the best reported methods in the
In contrast to images of the Yale database, these images in- literature) though the original Isomap does not work well.
clude facial contours, and variation in pose as well as scale.
However, the lighting conditions remain relatively constant.
The experiments are performed using the "leave-one-
out" strategy (i.e., m fold cross validation): To classify an
image of a person, that image is removed from the training
set of ( m - 1) images and the projection matrix is com-
puted. All the m images in the training set are projected
to a reduced space and recognition is performed based on
a nearest neighbor classifier. The parameters, such as num- Fig. 4. Results with AT&T database.
ber of principal components in Eigenface and LLE methods,
are empirically determined to achieve the lowest error rate Figure 5 shows more performance comparisons between
by each method. For Fisherface and extended Isomap meth- Isomap and extended Isomap methods in both k nearest
ods, we project all the samples onto a subspace spanned by neighbor as well as E radius implementations. The extended
the c - 1 largest eigenvectors. The experimental results are Isomap method consistently outperforms Isomap method with
shown in Figure 3. Among all the methods, the extended both implementations in all the experiments.
Isomap method using E radius achieves the lowest error rate
and outperforms the Fisherface method by a significant mar-
I 501 1 I
gin. Notice also that two implementations of the extended
Isomap (one with k nearest neighbor (extended k-Isomap)
and the other with E radius (extended 6-Isomap) to deter-
mine neighboring data points) consistently perform better
than their counterparts in the Isomap method by a large mar-
gin.
1
I
10 2 0 10 4 0 $ 0 6 0 7 0 8 0 90 100
" k m r of Nclghbor
" " 2 "
"l""
-
" "
4
"
-

IR*dl"S,
d -
i
1
LO"
(a) Experiments of k-Isomap and (b) Experiments of e-lsomap and
extended k-Isomap methods on extended e-Isomap methods on
ORL database. ORL database.
6" - I 1 1

Fig. 3. Results with AT&T database.

The Yale database contains 165 images of 11 subjects

[c) Experiments of k-Isomap and (d) Experiments of e-Isomap and
that includes variation of facial expression and lighting. For extended k-Isomap methods on extended e-Isomap methods on
computational efficiency, each image has been downsam- Yale database. Yale database.
pled to 29 x 41 pixels. Similarly, each face image is repre- Fig. 5. Performance of Isomap and Extended Isomap meth-
sented by a centered vector of normalized intensity values. ods.

I1 - 119

Authorized licensed use limited to: Univ of Calif Merced. Downloaded on September 25,2022 at 04:46:07 UTC from IEEE Xplore. Restrictions apply.
 As one example to explain why extended Isomap per-
forms better than Isomap, Figure 6 shows training and test
samples of the Yale database projected onto the first two
eigenvectors extracted by both methods. The projected sam-
ples of different classes are smeared by the Isomap method
(Figure 6(a)) whereas the samples projected by the extended
Isomap method are separated well (Figure 6(b)).
4. CONCLUDING REMARKS
Our experiments suggest the following conclusions:
(a) Isomap method.
1. The extended Isomap method performs consistently
better than the Isomap method in classification (with
both IC nearest neighbor and E radius implementations)
by a significant margin.

2. Geodesic distance appears to be a better metric than

Eucledian distance for face recognition in all the ex-
periments.

3. The extended Isomap method performs better than

one of the best methods in the literature on the AT
&T database. When there exist sufficient number of (b) Extended Isomap method.

samples so that the shortest paths between any pair Fig. 6. Samples projected by both methods.
of data points are good approximates of geodesic dis-
tances, the extended Isomap method performs well in tion. IEEE Transactions on Pattern Analysis and Machine
classification. Intelligence, 19(7):711-720, 1997.
C. M. Bishop. Neural Networks for Pattern Recognition.
4. The extended Isomap method still performs well while Oxford University Press, 1995.
the Isomap method does not in the experiments with T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduc-
Yale database. One explanation is that insufficient tion to Algorithms. The MIT Press and McGraw-Hill Book
samples result in poor approximates of geodesic dis- Company, 1989.
tances. However, poor approximates may be aver- R. 0. Duda, P. E. Hart, and D. G. Stork. Pattern Classifica-
tion. Wiley-Interscience, New York, 2001.
aged out by the extended Isomap method and thus it
P. J. Phillips, H. Moon, S . A. Rizvi, and P. J. Rauss. The
performs better. FERET evaluation methodology for face-recognition algo-
5. Though the Isomap and LLE methods have demon- rithms. IEEE Transactionson Pattern Analysis and Machine
Intelligence, 22( 10):1090-1034, 2000.
strated excellent results in finding the embedding man- S. T. Roweis and L. K. Saul. Nonlinear dimensionality re-
ifolds that best describe the data points with mini- duction by locally linear embedding. Science, 290(5500),
mum reconstruction error, they are suboptimal from 2000.
the classification viewpoint. Furthermore, these two F. Samaria and S . Young. HMM based architecture for face
methods assume that the embedding manifold is well identification. Image and Usion Computing, 12(8):537-583,
sampled which may not be the case in face recogni- 1994.
tion since there are typically only a few samples avail- T. Sim. S. Baker, and M. Bsat. The CMU pose, illumina-
able for each person. tion, and expression (PIE) database of human faces. Techni-
cal Report CMU-RI-TR-01-02,Carnegie Mellon University,
Our future work will focus on efficient methods for esti- 2001.
P. Simard, Y. Le Cun, and J. Denker. Efficient pattern recog-
mating geodesic distance, and performance evaluation with
nition using a new transformation distance. In S. J. Hanson,
large and diverse data sets. We plan to compare the extended J. D. Cowan, and C. L. Giles. editors, Advances in Neural In-
Isomap method against other learning algorithms with UCI formationProcessing Systems,volume 5, pages 50-58. Mor-
machine learning data sets, as well as reported face recogni- gan Kaufmann, San Mateo, CA, 1993.
tion methods using FERET [5] and CMU PIE [S]databases. J. B. Tenebaum, V. de Silva,and J. C. Langford. A global ge-
ometric framework for nonlinear dimensionality reduction.
5. REFERENCES Science, 290(5500),2000.
M. Turk and A. Pentland. Eigenfaces for recognition. Jour-
[l] P. Belhumeur, J. Hespanha, and D. Kriegman. Eigenfaces vs. nal ofcognitive Neuroscience, 3(1):71-86, 1991.
Fisherfaces: Recognition using class specific linear projec-

I1 - 120

Authorized licensed use limited to: Univ of Calif Merced. Downloaded on September 25,2022 at 04:46:07 UTC from IEEE Xplore. Restrictions apply.
View publication stats