FACE DETECTION AND FACE RECOGNITION

A
MINOR PROJECT REPORT

Submitted by

DHRUV GOEL-44114802717
AKHILESH CHAUHAN-44514802717

BACHELOR OF TECHNOLOGY
IN
COMPUTER SCIENCE AND ENGINEERING

Under the Guidance

of
(Mrs. NEELAM SHARMA)
(Asst. Professor, CSE)

Department of Computer Science and Engineering

Maharaja Agrasen Institute of Technology,
PSP area, Sector – 22, Rohini, New Delhi – 110085
(Affiliated to Guru Gobind Singh Indraprastha University, New Delhi)
 ACKNOWLEDGEMENT

It gives me immense pleasure to express my deepest sense of gratitude and

sincere thanks to my respected guide Mrs. Neelam Sharma (Asst. Professor,
cse) MAIT Delhi, for their valuable guidance, encouragement and help for
completing this work. Their useful suggestions for this whole work and co-
operative behavior are sincerely acknowledged.

I also wish to express my indebtedness to my parents as well as my family

member whose blessings and support always helped me to face the
challenges ahead.
 ABSTRACT
The face is one of the easiest ways to distinguish the individual identity of each other. Face
recognition is a personal identification system that uses personal characteristics of a person to
identify the person's identity. Human face recognition procedure basically consists of two phases,
namely face detection, where this process takes place very rapidly in humans, except under
conditions where the object is located at a short distance away, the next is the introduction, which
recognize a face as individuals. Stage is then replicated and developed as a model for facial image
recognition (face recognition) is one of the much-studied biometrics technology and developed by
experts. There are two kinds of methods that are currently popular in developed face recognition
pattern namely, Eigenface method and Fisherface method. Facial image recognition Eigenface
method is based on the reduction of face-dimensional space using Principal Component Analysis
(PCA) for facial features. The main purpose of the use of PCA on face recognition using Eigen
faces was formed (face space) by finding the eigenvector corresponding to the largest eigenvalue of
the face image. The area of this project face detection system with face recognition is Image
processing. The software requirements for this project is matlab software.

Keywords: face detection, Eigen face, PCA, matlab

Extension: There are vast number of applications from this face detection project, this project can
be extended that the various parts in the face can be detect which are in various directions and
shapes.
Table of Contents

1 INTRODUCTION

1.1 Objectives 6
1.2 Parameters used 6
1.3 Face recognition 6
1.4 Face detection 7

2 LITERATURE SURVEY 9

2.1 feature base approach 9

2.2 low level analysis 10
2.3 motion 10
2.4 Feature analysis
12
2.5 Constellation method
13
2.6 Linear sub space method
14
2.7 Statistical approach
14

3 MATLAB

3.1 Introduction 15
3.2 MATLAB’s power of computational mathematics 15
3.3 features of MATLAB 16
3.4 Uses of MATLAB 16
3.5 Understanding MATLAB environment 17
3.6 Commands 18
3.7 M Files 19

4 FACE DETECTION

4.1 Detection in diff. algo.

20
4.2 Face detection process 23

5 FACE RECOGNITION

5.1 Algorithm selection

25
5.2 PCA 26
5.3 Eigen faces method

6 RESULTS
 6.1 Face detection in images 29
6.2 Real time face detection 30
6.3 Face recognition 30

7 CONCLUSION AND FUTURE SCOPE 31

8 REFERENCES 32

9 APPENDIX 34
 CHAPTER-1
INTRODUCTION
Face recognition is the task of identifying an already detected object as a known or unknown face.
Often the problem of face recognition is confused with the problem of face detectionFace
Recognition on the other hand is to decide if the "face" is someone known, or unknown, using for
this purpose a database of faces in order to validate this input face.

1.1 OBJECTIVE
This project will serve the following objectives: -

1. Face detection in images.

2. Real-time face detection.

3. Storing unknown image in image database using real time face detection.

4. Face recognition from a set of images in database.

1.2 PARAMETERS USED

For Face Detection following parameters are used:

1. grey scale intensity distribution on face

2. Haar like feature selection

3. eyes, nose, mouth selection for face detection

For Face Recognition following parameters are used:

1. 10 images each of 30 test subjects thus making a database of 300 images

2. subject’s eyes, nose, mouth

3. light intensity on face using gray scale intensity

4. average face area segmentation from subject’s image

1.3 FACE RECOGNITION:

DIFFERENT APPROACHES OF FACE RECOGNITION:

There are two predominant approaches to the face recognition problem: Geometric (feature based)
and photometric (view based). As researcher interest in face recognition continued, many different
algorithms were developed, three of which have been well studied in face recognition literature.

Recognition algorithms can be divided into two main approaches:

1. Geometric: Is based on geometrical relationship between facial landmarks, or in other

words the spatial configuration of facial features. That means that the main geometrical
features of the face such as the eyes, nose and mouth are first located and then faces are
classified on the basis of various geometrical distances and angles between features. (Figure
3)
2. Photometric stereo: Used to recover the shape of an object from a number of images taken
under different lighting conditions. The shape of the recovered object is defined by a
gradient map, which is made up of an array of surface normals (Zhao and Chellappa, 2006)
(Figure 2)

Popular recognition algorithms include:

1. Principal Component Analysis using Eigenfaces, (PCA)

2. Linear Discriminate Analysis,
3. Elastic Bunch Graph Matching using the Fisherface algorithm,

1.4 FACE DETECTION:

 Face detection involves separating image windows into two classes; one containing faces
(tarning the background (clutter). It is difficult because although commonalities exist between
faces, they can vary considerably in terms of age, skin colour and facial expression. The problem
is further complicated by differing lighting conditions, image qualities and geometries, as well as
the possibility of partial occlusion and disguise. An ideal face detector would therefore be able to
detect the presence of any face under any set of lighting conditions, upon any background. The
face detection task can be broken down into two steps. The first step is a classification task that
takes some arbitrary image as input and outputs a binary value of yes or no, indicating whether
there are any faces present in the image. The second step is the face localization task that aims to
take an image as input and output the location of any face or faces within that image as some
bounding box with (x, y, width, height).
The face detection system can be divided into the following steps:-

1. Pre-Processing: To reduce the variability in the faces, the images are processed before they are
fed into the network. All positive examples that is the face images are obtained by cropping
images with frontal faces to include only the front view. All the cropped images are then corrected
for lighting through standard algorithms.

2. Classification: Neural networks are implemented to classify the images as faces or nonfaces by
training on these examples. We use both our implementation of the neural network and the Matlab
neural network toolbox for this task. Different network configurations are experimented with to
optimize the results.

3. Localization: The trained neural network is then used to search for faces in an image and if
present localize them in a bounding box. Various Feature of Face on which the work has done on:-
Position Scale Orientation Illumination

Popular detection algorithms include:

1. Neural network based face detection

2. Image pyramid statistical method

 3. Voila and Jones method.

CHAPTER-2

LITERATURE SURVEY

Face detection is a computer technology that determines the location and size of human face
in arbitrary (digital) image. The facial features are detected and any other objects like trees,
buildings and bodies etc are ignored from the digital image. It can be regarded as a ‗specific‘ case
of object-class detection, where the task is finding the location and sizes of all objects in an image
that belong to a given class. Face detection, can be regarded as a more ‗general‘ case of face
localization. In face localization, the task is to find the locations and sizes of a known number of
faces (usually one). Basically there are two types of approaches to detect facial part in the given
image i.e. feature base and image base approach. Feature base approach tries to extract features of
the image and match it against the knowledge of the face features. While image base approach tries
to get best match between training and testing images.

Fig1 detection methods

2.1 FEATURE BASE APPROCH:

Active shape models focus on complex non-rigid features like actual physical and higher level
appearance of features Means that Active Shape Models (ASMs) are aimed at automatically
locating landmark points that define the shape of any statistically modelled object in an image.
. The training stage of an ASM involves the building of a statistical facial model from a training set
containing images with manually annotated landmarks. ASMs is classified into three groups i.e.
snakes, PDM, Deformable templates.

2.1.1 Snakes:
The first type uses a generic active contour called snakes, first introduced by Kass et al. in 1987
Snakes are used to identify head boundaries [8,9,10,11,12]. In order to achieve the task, a snake is
first initialized at the proximity around a head boundary. It then locks onto nearby edges and
subsequently assume the shape of the head. The evolution of a snake is achieved by minimizing an
energy function, Esnake (analogy with physical systems), denoted asEsnake = Einternal +
EExternal Where Einternal and EExternal are internal and external energy functions. Internal
energy is the part that depends on the intrinsic properties of the snake and defines its natural
evolution. The typical natural evolution in snakes is shrinking or expanding. The external energy
counteracts the internal energy and enables the contours to deviate from the natural evolution and
eventually assume the shape of nearby features—the head boundary at a state of equilibria

2.1.2 Deformable Templates:

Deformable templates were then introduced by Yuille et al. to take into account the a priori of
facial features and to better the performance of snakes. Locating a facial feature boundary is not an
easy task because the local evidence of facial edges is difficult to organize into a sensible global
entity using generic contours. The low brightness contrast around some of these features also
makes the edge detection process. Yuille et al. took the concept of snakes a step further by
incorporating global information of the eye to improve the reliability of the extraction process.

2.1.3 PDM (Point Distribution Model):

Independently of computerized image analysis, and before ASMs were developed, researchers
developed statistical models of shape. The idea is that once you represent shapes as vectors, you
can apply standard statistical methods to them just like any other multivariate object. These models
learn allowable constellations of shape points from training examples and use principal components
to build what is called a Point Distribution Model. These have been used in diverse ways, for
example for categorizing Iron Age broaches. Ideal Point Distribution Models can only deform in
ways that are characteristic of the object.

2.2 LOW LEVEL ANALYSIS:

Based on low level visual features like color, intensity, edges, motion etc. Skin Color Base Color is
a vital feature of human faces. Using skin-color as a feature for tracking a face has several
advantages. Color processing is much faster than processing other facial features. Under certain
lighting conditions, color is orientation invariant. This property makes motion estimation much
easier because only a translation model is needed for motion estimation. Tracking human faces
using color as a feature has several problems like the color representation of a face obtained by a
camera is influenced by many factors like ambient light, object movement, etc.

2.3) MOTION BASE:

When use of video sequence is available, motion information can be used to locate moving
objects. Moving silhouettes like face and body parts can be extracted by simply thresholding
accumulated frame differences. Besides face regions, facial features can be located by frame
differences .

2.3.1 Gray Scale Base:

Gray information within a face can also be treat as important features. Facial features such
as eyebrows, pupils, and lips appear generally darker than their surrounding facial regions. Various
recent feature extraction algorithms search for local gray minima within segmented facial regions.
In these algorithms, the input images are first enhanced by contrast-stretching and gray-scale
morphological routines to improve the quality of local dark patches and thereby make detection
easier. The extraction of dark patches is achieved by low-level gray-scale thresholding. Based
method and consist three levels. Yang and huang presented new approach i.e. faces gray scale
behavior in pyramid (mosaic) images. This system utilizes hierarchical Face location consist three
levels. Higher two level based on mosaic images at different resolution. In the lower level, edge
detection method is proposed. Moreover, this algorithm gives fine response in complex background
where size of the face is unknown.

2.3.2 Edge Base:

Face detection based on edges was introduced by Sakai et al. This work was based on analyzing
line drawings of the faces from photographs, aiming to locate facial features. Than later Craw et al.
proposed a hierarchical framework based on Sakai et al. ‘work to trace a human head outline. Then
after remarkable works were carried out by many researchers in this specific area. Method
suggested by Anila and Devarajan was very simple and fast. They proposed framework, which
consist three steps i.e. initially the images are enhanced by applying median filter for noise removal
and histogram equalization for contrast adjustment. In the second step the edge image is
constructed from the enhanced image by applying sobel operator. Then a novel edge tracking
algorithm is applied to extract the sub windows from the enhanced image based on edges. Further
they used Back propagation Neural Network (BPN) algorithm to classify the sub-window as either
face or non-face.

2.4 FEATURE ANALYSIS

These algorithms aim to find structural features that exist even when the pose, viewpoint, or
lighting conditions vary, and then use these to locate faces. These methods are designed mainly for
face localization

2.4.1 Feature Searching

Viola Jones Method:

Paul Viola and Michael Jones presented an approach for object detection, which minimizes
computation time while achieving high detection accuracy. Paul Viola and Michael Jones [39]
proposed a fast and robust method for face detection, which is 15 times quicker than any technique
at the time of release with 95% accuracy at around 17-fps.The technique, relies on the use of
simple Haar-like features that are evaluated quickly through the use of a new image representation.
Based on the concept of an ―Integral Image‖ it generates a large set of features and uses the
boosting algorithm AdaBoost to reduce the overcomplete set and the introduction of a degenerative
tree of the boosted classifiers provides for robust and fast interferences. The detector is applied in a
scanning fashion and used on gray-scale images, the scanned window that is applied can also be
scaled, as well as the features evaluated.

Gabor Feature Method:

Sharif et al proposed an Elastic Bunch Graph Map (EBGM) algorithmthat

successfullyimplements face detection using Gabor filters. The proposedsystem applies 40 different
Gabor filters on an image. As aresult of which 40 images with different angles and orientationare
received. Next, maximum intensity points in each filteredimage are calculated and mark them as
fiducial points. Thesystem reduces these points in accordance to distance betweenthem. The next
step is calculating the distances between thereduced points

using distance formula. At last, the distances arecompared with database. If match occurs, it means
that thefaces in the image are detected. Equation of Gabor filter [40] is shown below`

2.5 CONSTELLATION METHOD

All methods discussed so far are able to track faces but still some issue like locating faces
of various poses in complex background is truly difficult. To reduce this difficulty investigator
form a group of facial features in face-like constellations using more robust modelling approaches
such as statistical analysis. Various types of face constellations have been proposed by Burl et al. .
They establish use of statistical shape theory on the features detected from a multiscale Gaussian
derivative filter. Huang et al. also apply a Gaussian filter for pre-processing in a framework based
on image feature analysis.Image Base Approach.

2.5.1 Neural Network

Neural networks gaining much more attention in many pattern recognition problems, such as OCR,
object recognition, and autonomous robot driving. Since face detection can be treated as a two class
pattern recognition problem, various neural network algorithms have been proposed. The
advantage of using neural networks for face detection is the feasibility of training a system to
capture the complex class conditional density of face patterns. However, one demerit is that the
network architecture has to be extensively tuned (number of layers, number of nodes, learning
rates, etc.) to get exceptional performance. In early days most hierarchical neural network was
proposed by Agui et al. [43]. The first stage having twoparallel subnetworks in which the inputs are
filtered intensity valuesfrom an original image. The inputs to the second stagenetwork consist of
the outputs from the sub networks andextracted feature values. An output at thesecond stage shows
the presence of a face in the inputregion.Propp and Samal developed one of the earliest
neuralnetworks for face detection [44]. Their network consists offour layers with 1,024 input units,
256 units in the first hiddenlayer, eight units in the second hidden layer, and two outputunits.Feraud
and Bernier presented a detection method using auto associative neural networks [45], [46], [47].
The idea is based on [48] which shows an auto associative network with five layers is able to
perform a nonlinear principal component analysis. One auto associative network is used to detect
frontal-view faces and another one is used to detect faces turned up to 60 degrees to the left and
right of the frontal view. After that Lin et al. presented a face detection system using probabilistic
decision-based neural network (PDBNN) [49]. The architecture of PDBNN is similar to a radial
basis function (RBF) network with modified learning rules and probabilistic interpretation.

2.6 LINEAR SUB SPACE METHOD

Eigen faces Method:
An early example of employing Eigen vectors in face recognition was done by Kohonen in
which a simple neural network is demonstrated to perform face recognition for aligned and
normalized face images. Kirby and Sirovich suggested that images of faces can be linearly encoded
using a modest number of basis images. The idea is arguably proposed first by Pearson in 1901 and
then by HOTELLING in 1933 .Given a collection of n by m pixel training.
Images represented as a vector of size m X n, basis vectors spanning an optimal subspace
are determined such that the mean square error between the projection of the training images onto
this subspace and the original images is minimized. They call the set of optimal basis vectors Eigen
pictures since these are simply the Eigen vectors of the covariance matrix computed from the
vectorized face images in the training set. Experiments with a set of 100 images show that a face
image of 91 X 50 pixels can be effectively encoded using only50 Eigen pictures.

2.7 STATISTICAL APPROCH

Support Vector Machine (SVM):

SVMs were first introduced Osuna et al. for face detection. SVMs work as a new paradigm to train
polynomial function, neural networks, or radial basis function (RBF) classifiers.SVMs works on
induction principle, called structural risk minimization, which targets to minimize an upper bound
on the expected generalization error. An SVM classifier is a linear classifier where the separating
hyper plane is chosen to minimize the expected classification error of the unseen test patterns.In
Osunaet al. developed an efficient method to train an SVM for large scale problems,and applied it
to face detection. Based on two test sets of 10,000,000 test patterns of 19 X 19 pixels, their system
has slightly lower error rates and runs approximately30 times faster than the system by Sung and
Poggio . SVMs have also been used to detect faces and pedestrians in the wavelet domain.

CHAPTER-3

MATLAB
3.1 INTRODUCTION

The name MATLAB stands for MATrix LABoratory. MATLAB was written originally to
provide easy access to matrix software developed by the LINPACK (linear system package) and
EISPACK (Eigen system package) projects. MATLAB is a high-performance language for
technical computing. It integrates computation, visualization, and programming environment.
MATLAB has many advantages compared to conventional computer languages (e.g., C,
FORTRAN) for solving technical problems. MATLAB is an interactive system whose basic data
element is an array that does not require dimensioning. Specific applications are collected in
packages referred to as toolbox. There are tool boxes for signal processing, symbolic computation,
control theory, simulation, optimization, and several other fields of applied science and
engineering.

3.2 MATLAB's POWER OF COMPUTAIONAL MATHMATICS

MATLAB is used in every facet of computational mathematics. Following are some

commonly used mathematical calculations where it is used most commonly:
• Dealing with Matrices and Arrays

• 2-D and 3-D Plotting and graphics

• Linear Algebra

• Algebraic Equations

• Non-linear Functions

• Statistics

• Data Analysis

• Calculus and Differential Equations Numerical Calculations

 • Integration

• Transforms

• Curve Fitting

• Various other special functions

3.3 FEATURES OF MATLAB

Following are the basic features of MATLAB

It is a high-level language for numerical computation, visualization and application development.

• It also provides an interactive environment for iterative exploration, design and problem
solving.

• It provides vast library of mathematical functions for linear algebra, statistics, Fourier
analysis, filtering, optimization, numerical integration and solving ordinary differential
equations.

• It provides built-in graphics for visualizing data and tools for creating custom plots.

• MATLAB's programming interface gives development tools for improving code quality,
maintainability, and maximizing performance.

• It provides tools for building applications with custom graphical interfaces.

• It provides functions for integrating MATLAB based algorithms with external applications
and languages such as C, Java, .NET and Microsoft Excel.

3.4 USES OF MATLAB

MATLAB is widely used as a computational tool in science and engineering encompassing the
fields of physics, chemistry, math and all engineering streams. It is used in a range of applications
including:

• signal processing and Communications

• image and video Processing

• control systems

• test and measurement

 • computational finance

• computational biology

3.5 UNDERTANDING THE MATLAB ENVIRONMENT

MATLAB development IDE can be launched from the icon created on the desktop. The
main working window in MATLAB is called the desktop. When MATLAB is started, the desktop
appears in its default layout.

Fig1 MATLAB desktop environment

The desktop has the following panels:

Current Folder - This panel allows you to access the project folders and files.

Fig2 current folder

Command Window - This is the main area where commands can be entered at the command line.
It is indicated by the command prompt (>>).
 Fig3 command window

Workspace - The workspace shows all the variables created and/or imported from files.

Fig 4 workspace

Command History - This panel shows or rerun commands that are entered at the command line.

Fig 5 command history

3.6 COMMANDS
MATLAB is an interactive program for numerical computation and data visualization. You can
enter a command by typing it at the MATLAB prompt '>>' on the Command Window.
3.6.1 Commands for managing a session
MATLAB provides various commands for managing a session. The following table
provides all

Commands Purpose
Clc Clear command window
Clear Removes variables from memory
Exist Checks for existence of file or
variable.
Global Declare variables to be global.
Help Searches for help topics.
Look for Searches help entries for a
keyword.
Quit Stops MATLAB.
Who Lists current variable.
Whos Lists current variables (Long
Display).
Table 1 commands for managing a session

3.7 M FILES
MATLAB allows writing two kinds of program files:

Scripts:
script files are program files with .m extension. In these files, you write series of commands, which
you want to execute together. Scripts do not accept inputs and do not return any outputs. They
operate on data in the workspace.

Functions:
functions files are also program files with .m extension. Functions can accept inputs and return
outputs. Internal variables are local to the function.
 CHAPTER-4

FACE DETECTION

4.1 Different types of algorithms

There are three main approaches considered for implementing face detection. They are
‘Neural network based face detection’, ‘Image pyramid statistical method’ and ‘Voila &
Jones’. Each implementation is discussed below:

1. Neural network based face detection (paper by Henry A. Rowley, Shumeet

Baluja, and Takeo Kanade) – Faces are detected at multiple scales by calculating an
image pyramid. Each image in that pyramid is then scanned by using a fixed size
sub-window and its content is corrected. The correction is to make sure that the
lighting is non uniform. The histogram is also equalized. This corrected sub window
is then passed through many parallel neural networks. The networks decide if there
are any faces in the window. All of these multiple outputs are passed through an
AND gate to give the final result. As it’s an AND gate the number of false detection
is reduced.

2. Image pyramid statistical method (paper by Henry Schneiderman and Takeo

Kanade) – The basic mechanics of this algorithm is also to calculate an image
pyramid and scan a fixed size sub-window through each layer of this pyramid. The
content of the sub window is subjected to a wavelet analysis and histograms are
made for the different wavelet coefficients. The orientation of the object is
determined by differently trained parallel detectors. These detectors are trained so
that they are sensitive to the different orientations and one which received the
highest hit is considered as the best estimate of the actual orientation.
 3. Voila and Jones method (paper by Paul Viola and Michael Jones) – The main
focus of this method was to achieve a reasonably high face detection rate in the
smallest amount of time possible. The detector uses features which are calculated
using ‘Integral Image’. This is a new type of image representation which is
introduced in this method’s paper. Also a new type of learning algorithm was
introduced in this paper which inherits Adaptive Boosting (AdaBoost) algorithm
features. This learning algorithm is then applied which selects the critical visual
features and generates classifiers. Background or in other words the less interesting
regions are discarded by a cascade classifier.

4.1.1 Comparison of algorithms

Time taken by each method

8
7.2
7

6
5.0
Time (s)

3 Time taken

1
0.067
0
Neural network based Image pyramid Voila and Jones method
face detection statistical method

Figure 1 Plot showing time taken by each method. (Values are taken from their respective
papers)

The plot above shows the time taken for each method to process an image. The images
used to establish those benchmarks were approximately 320x240 pixels and were processed
on relatively similar processor.

In other algorithms the speed of detection wasn’t taken into account as such. Hence, we can
clearly see that ‘Voila and Jones’ method work significantly faster when compared with
others. It is also considered as one of the breakthroughs in the face detection industry.
 False detections 10 31 50 65 78 95 167
Detector
Rowley-Baluja-Kanade 83.2% 86.0% - - - 89.2% 90.1%
Schneiderman-Kanade - - - 94.4% - - -
Viola-Jones 76.1% 88.4% 91.4% 92.0% 92.1% 92.9% 93.9%

Table 1 Detection rate for various numbers of false positives on the MIT+CMU test set containing
130 images and 507 faces.

Table 1 shows the detection rate results of the different algorithms. From this we can
clearly say that ‘Voila and Jones’ method also provides reasonably high detection rate.

After analyzing each algorithm’s complexity, Voila & Jones seems a better choice. It
offers a high detection rate combined with a very low processing time which is what the
system needs.

4.1.2 Voila and Jones method theory

This method uses ‘Haar’ wavelets for feature extraction from the images. These wavelets
also allow feature evaluation.

A & B Edge features

C – Line features

D – Four rectangle Features

 Figure 2 Basic Haar wavelets

They are formed of one low interval and high interval or in other words are single
wavelength square waves. A square wave is a pair of one light and one dark adjacent
rectangles. The calculation of these wavelets is relatively easy as the white areas are just
subtracted from the black ones. Above Figure shows the four basic types of Haar wavelets
in 2D.

The feature extraction is made faster by integral image which is a special representation of
the image. A machine learning method, called ‘AdaBoost’ enables classifier training and
feature selection. All of the detected features are then combined efficiently by using a
cascaded classifier. This is shown in figure below

Figure 3: Cascaded classifier

4.2 FACE DETECTION PROCESS

It is process of identifying different parts of human faces like eyes, nose, mouth, etc… this process
can be achieved by using MATLAB code In this project the author will attempt to detect faces in
still images by using image invariants. To do this it would be useful to study the grey-scale
intensity distribution of an average human face. The following 'average human face' was
constructed from a sample of 30 frontal view human faces, of which 12 were from females and 18
from males. A suitably scaled color map has been used to highlight grey-scale intensity
differences.

scaled colormap scaled colormap (negative)

Figure 4 Average human face in grey-scale

The grey-scale differences, which are invariant across all the sample faces are strikingly apparent.
The eye-eyebrow area seem to always contain dark intensity (low) gray-levels while nose forehead
and cheeks contain bright intensity (high) grey-levels. After a great deal of experimentation, the
researcher found that the following areas of the human face were suitable for a face detection
system based on image invariants and a deformable template.

scaled colormap scaled colormap (negative)

Figure 5 Area chosen for face detection (indicated on average human face in gray scale)

The above facial area performs well as a basis for a face template, probably because of the clear
divisions of the bright intensity invariant area by the dark intensity invariant regions. Once this
pixel area is located by the face detection system, any particular area required can be segmented
based on the proportions of the average human face After studying the above images it was
subjectively decided by the author to use the following as a basis for dark intensity sensitive and
bright intensity sensitive templates. Once these are located in a subject's face, a pixel area 33.3%
(of the width of the square window) below this.

Figure 6: Basis for a bright intensity invariant sensitive template.

Now that a suitable dark and bright intensity invariant templates have been decided on, it is
necessary to find a way of using these to make 2 A-units for a perceptron, i.e. a computational
model is needed to assign neurons to the distributions displayed .

CHAPTER-5

FACE RECOGNITION

5.1 Algorithm selection

There are various available implementations of face recognition software. The Face
Recognition Technology (FERET) program has already analyzed most of those algorithms
and have published detailed performance results for each of these approaches.

The main goal of this program was to test each algorithm on the same data sets so a genuine
comparative results study can be formed. It is managed by the Defense Advanced Research
Projects Agency (DARPA) and the National Institute of Standards and Technology (NIST)
so that the best solution can be known and deployed.
 This Figure shows the FERET10
FAFB results. In this test each
algorithm was run with faces with
different facial expressions. All
other conditions were kept the
same.

Figure: FERET FAFB

This Figure shows the FERET

DUP2 results. In this test case
all of the algorithms were given
picture of faces taken with an
18 months time difference.

Figure: FERET DUP2

 Out of these algorithms the most promising one seems to be (PCA Euclidean) Eigenfaces
method for our system needs. From the results, this method provides a reasonable
detection rate when compared with other algorithms. It is based on the principal
component analysis (PCA). The main factor for selecting this method is that it’s not
complex, and is really fast which saves the processing time. It was developed by
Sirovich and Kirby11. Matthew Turk and Alex Pentland at MIT12 released a paper
regarding the use of this method for face classification.

5.2 PRINCIPAL COMPONENT ANALYSIS (PCA)

Principal Component Analysis (or Karhunen-Loeve expansion) is a suitable strategy for face
recognition because it identifies variability between human faces, which may not be immediately
obvious. Principal Component Analysis (hereafter PCA) does not attempt to categorize faces using
familiar geometrical differences, such as nose length or eyebrow width. Instead, a set of human
faces is analyzed using PCA to determine which 'variables' account for the variance of faces. In
face recognition, these variables are called eigen faces because when plotted they display an eerie
resemblance to human faces. Although PCA is used extensively in statistical analysis, the pattern
recognition community started to use PCA for classification only relatively recently. As described
by Johnson and Wichern (1992), 'principal component analysis is concerned with explaining the
variance- covariance structure through a few linear combinations of the original variables.' Perhaps
PCA's greatest strengths are in its ability for data reduction and interpretation. For example a
100x100 pixel area containing a face can be very accurately represented by just 40 eigen values.
Each eigen value describes the magnitude of each eigen face in each image. Furthermore, all
interpretation (i.e. recognition) operations can now be done using just the 40 eigen values to
represent a face instead of the manipulating the 10000 values contained in a 100x100 image. Not
only is this computationally less demanding but the fact that the recognition information of several
thousand.

5.3 Eigen faces method

This method is based upon Principal component analysis (PCA). An initial set of images of
faces are used to create a training set. The number of face shots of each person stored in the
database depends on how much processing time they will take. These faces are then broken
down into individual vectors. The magnitude of each vector represents the brightness of
individual sectors of the gray scale image. A covariance matrix is formed by normalizing
these vectors. After this eigenvectors are derived from this covariance matrix and a set of
eigenvectors of an image forms an eigenface. Eigenface helps in just focusing at the main
face features rather than the whole face data. In other words it enables us to find the
weight of each face.
When a new face image is acquired the weight of that face is calculated and then
subtracted from the each of the weights of other images in the database. Those
difference numbers represents how much different each image is from the original
image. The lower the number the closer is the match. This difference is also known as
the max Euclidean distance.

5.3.1 UNDERSTANDING EIGENFACES

Any grey scale face image I (x,y) consisting of a NxN array of intensity values may also be
2
consider as a vector of N . For example, a typical 100x100 image used in this thesis will have to be
transformed into a 10000-dimension vector!

Figure 3 a 7x7 face image transformed into a 49-dimension vector

This vector can also be regarded as a point in 10000 dimension space. Therefore, all the images of
subjects' whose faces are to be recognized can be regarded as points in 10000 dimension
space. Face recognition using these images is doomed to failure because all human face images are
quite similar to one another so all associated vectors are very close to each other in the 10000-
dimension space.

Figure 4
 Figure 5 Eigen faces

The transformation of a face from image space (I) to face space (f) involves just a simple
matrix multiplication. If the average face image is A and U contains the (previously calculated)
eigenfaces,
f = U * (I - A)
 CHAPTER-6

RESULTS

6.1 Face Detection In Images

Figure 1

Figure 2

6.2 Real Time Face Detection

 Figure 3

6.3 Face Recognition

Figure 4
 CHAPTER-7

CONCLUSION AND FUTURE SCOPE

6.1Conclusion

The approach presented here for face detection and recognition decreases the computation time
producing results with high accuracy. Viola Jones is used for detecting facial features. Not only in
video sequences, it has also been tested on live video using a webcam. Principal Component
Analysis (PCA) is used for recognizing images from a set of database using eigen values. Using the
system many security and surveillance systems can be developed and required object can be traced
down easily. In the coming days these algorithms can be used to detect a particular object rather
than faces.

6.2Future Scope

1 Future work is to work on the same domain but to track a particular face in a video sequence.
That is like avoiding all other faces except the face required.

2 Viola Jones Algorithm is only meant for frontal and upright movement of the faces. It doesn’t
work when it comes to any arbitrary movement and hence, doesn’t make sense. We would try
to train classifiers so that it is subtle to all sort of movements.

3 We would definitely try to find the displacement of the Eigen vectors using Taylor Series.
 REFERENCES

1. Adelson, E. H., and Bergen, J. R. (1986) The Extraction of Spatio-Temporal Energy in

2. Human and Machine Vision, Proceedings of Workshop on Motion: Representation and
3. Analysis (pp. 151-155) Charleston, SC; May 7-9
4. AAFPRS(1997). A newsletter from the American Academy of Facial Plastic and
Reconstructive Surgery. Third Quarter 1997, Vol. 11, No. 3. Page 3.
5. Baron, R. J. (1981). Mechanisms of human facial recognition. International Journal of Man
Machine Studies, 15:137-178
6. Beymer, D. and Poggio, T. (1995) Face Recognition From One Example View, A.I. Memo
No. 1536, C.B.C.L. Paper No. 121. MIT
7. Bichsel, M. (1991). Strategies of Robust Objects Recognition for Automatic Identification of
Human Faces. PhD thesis, , Eidgenossischen Technischen Hochschule, Zurich.
8. Brennan, S. E. (1982) The caricature generator. M.S. Thesis. MIT.
9. Brunelli, R. and Poggio, T. (1993), Face Recognition: Features versus Templates. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 15(10):1042-1052
10. Craw, I., Ellis, H., and Lishman, J.R. (1987). Automatic extraction of face features. Pattern
Recognition Letters, 5:183-187, February.
11. Deffenbacher K.A., Johanson J., and O'Toole A.J. (1998) Facial ageing, attractiveness, and
distinctiveness. Perception. 27(10):1233-1243
12. Dunteman, G.H. (1989) Principal Component Analysis. Sage Publications.
13. Frank, H. and Althoen, S. (1994). Statistics: Concepts and applications. Cambridge
University Press. p.110
14. Gauthier, I., Behrmann, M. and Tarr, M. (1999). Can face recognition really be
dissociated from object recognition? Journal of Cognitive Neuroscience, in press.
15. Goldstein, A.J., Harmon, L.D., and Lesk, A.B. (1971). Identification of human faces.
In Proc. IEEE, Vol. 59, page 748
16. de Haan, M., Johnson, M.H. and Maurer D. (1998) Recognition of individual faces and
average face prototypes by 1- and 3- month-old infants. Centre for Brain and Cognitive
17. Development, Department of Psychology, Birkbeck College.

18. Hadamard, J. (1923) Lectures on the Cauchy Problem in Linear Partial Differential
Equations, Yale University Press

19. Rowley, Baluja, and Kanade, “Neural Network-Based Face Detection,” PAMI, January ,
1998
20. Henry Schneiderman and Takeo Kanade, “A Statistical Method for 3D Object
Detection Applied to Faces and Cars Henry Schneiderman and Takeo Kanade”

21. Viola, P.; Jones, M.: Rapid, "Object Detection Using a Boosted Cascade of
Simple Features”, TR2004-043 May 2004
 APPENDIX

FACE DETECTION IN IMAGES:

%Matlab Program to detect face in an image

[filename,pathname]=uigetfile('*.*','Select the Input
Grayscale Image');

filewithpath=strcat(pathname,filename);

Img=imread(filewithpath);

faceDetector = vision.CascadeObjectDetector;

faceDetector.MergeThreshold = 5;

bboxes = faceDetector(Img);

if ~isempty(bboxes)
Imgf =
insertObjectAnnotation(Img,'rectangle',bboxes,'Face','LineWid
th',3);
imshow(Imgf)
title('Detected faces');
else
position=[0,0];
label='No face Detected';
Imgn =
insertText(Img,position,label,'FontSize',25,'BoxOpacity',1);
imshow(Imgn)
end

REAL TIME FACE DETECTION

%Program to detect face in video stream

cam = webcam;
videoFrame = snapshot(cam);
frameSize = size(videoFrame);
% Create the video player object.
videoPlayer = vision.VideoPlayer('Position',[100 100
[frameSize(2), frameSize(1)]]);
runLoop = true;

while runLoop
img = snapshot(cam);
 [croppedimage, bboxPoints]=myfacedetect(img);
bboxPolygon = reshape(bboxPoints', 1, []);
videoFrame = insertShape(img, 'Polygon', bboxPolygon,
'LineWidth', 4);
step(videoPlayer, videoFrame);
% check whether the video player window has been closed.
runLoop = isOpen(videoPlayer);
end

clear cam;
release(videoPlayer);

FACE RECOGNITION:

% Program for face recognition (Testing)

clc;
load pcadb;

%pcadb loads followings in workspace

%Image size [M=100, N=90]
%Mean Image [m]
%Reduced Eigen vectors transformation matrix [Ppca]
%Transformed dataset matrix[T]

[filename,pathname]=uigetfile('*.*','Select the Input

Grayscale Image');
filewithpath=strcat(pathname,filename);
img=imread(filewithpath);
imgo=img;

img=rgb2gray(img);
img=imresize(img,[M,N]);
img=double(reshape(img,[1,M*N]));

imgpca=(img-m)*Ppca; %Projecting query image to PCA space

distarray=zeros(n,1); %Initialize difference array

for i=1:n
distarray(i)=sum(abs(T(i,:)-imgpca)); %Finding L1
distance

end

[result,indx]=min(distarray); %Getting best match

resultimg=imread(sprintf('%d.jpg',indx)); %Getting best
matched image
%Plotting Images
subplot(121)
imshow(imgo);
title('Query Face');
subplot(122)
imshow(resultimg);
title('Recognized Face');

Explanation:

The code is meant to work on pictures only. The recognition function decides if there is any close
valid match from the database. In the scenario where the face is ‘unknown’ the function simply
returns not recognized. After this the function loops back to the beginning for a new frame.

Figure1 shows the flowchart of the

implemented code