CHAPTER 5

Detecting and Recognizing

Face
BY: MARY JANE PAGAY CIERVA
 AT THE END OF THIS CHAPTER,
WE ARE
EXPECTED TO:
• Use some of OpenCV's face detection and recognition functionality
• Have a good understanding of how face detection and face recognition
work and how to implement them in Python and OpenCV 4
• Look at Haar cascade classifiers, which analyze the contrast between
adjacent image regions
• We will integrate face tracking and rectangle manipulations into Cameo
 CONCEPTUALIZING HAAR
CASCADES
When we talk about classifying objects and
tracking their location.
Image detail tends to be unstable with
respect to variations in lighting, viewing
angle, viewing distance, camera shake, and
digital noise.
 CONCEPTUALIZING HAAR
CASCADES
Some means of abstracting image detail is
useful in producing stable classification and
tracking results.
The ABSTRACTIONS are called features,
which are said to be extracted from the
image data.
 CONCEPTUALIZING HAAR
CASCADES
There should be far fewer features than pixels,
though any pixel might influence multiple features.
A set of features is represented as a vector, and
the level of similarity between two images can be
evaluated based on some measure of the distance
between the images' corresponding feature
vectors.
 CONCEPTUALIZING HAAR
CASCADES
Haar-like features are one type of feature that is
often applied to real-time face detection.
Each Haar-like feature describes the pattern of
contrast among adjacent image regions..
Some features are distinctive in the sense that
they typically occur in a certain class of object
(such as a face) but not in other objects.
 CONCEPTUALIZING HAAR
CASCADES
Distinctive features can be organized into a
hierarchy, called a CASCADE, in which the
highest layers contain features of greatest
distinctiveness, enabling a classifier to quickly
reject subjects that lack these features.
Features may vary depending on the scale of the image and
the size of the neighborhood within which contrast is being
evaluated.
Window Size Scale-invariant
features may vary depending  the window size is kept
on the scale of the image and constant but images are
the size of the neighborhood rescaled a number of times
within which contrast is being
evaluated.
Image Pyramid
original image and the rescaled
images
 GETTING HAAR
CASCADE DATA
The data/haarcascades folder contains XML files that can be
loaded by an OpenCV class called cv2.CascadeClassifier. An
instance of this class interprets a given XML file as a Haar
cascade, which provides a detection model for a type of object
such as a face. cv2.CascadeClassifier can detect this type of
object in any image. As usual, we could obtain a still image from
a file, or we could obtain a series of frames from a video file or a
video camera.
 GETTING HAAR
CASCADE DATA
Once you find data/haarcascades, create a directory
elsewhere for your project; in this folder, create a subfolder
called cascades, and copy the following files from
data/haarcascades into cascades:
PERFORMING FACE DETECTION ON A
STILL IMAGE
Create the following basic script to perform face detection
 PERFORMING FACE DETECTION ON A
STILL IMAGE
Let's walk through the preceding code in small steps.
First, we use the obligatory cv2 import. Then, we
declare a face_cascade variable, which is a
CascadeClassifier object that loads a cascade for face
detection:
 PERFORMING FACE DETECTION ON A
STILL IMAGE
We then load our image file with cv2.imread and
convert it into grayscale because CascadeClassifier
expects grayscale images. The next step,
face_cascade.detectMultiScale, is where we perform
the actual face detection:
 PERFORMING FACE DETECTION ON A
STILL IMAGE
The parameters of detectMultiScale include
scaleFactor and minNeighbors. The
scaleFactor argument, which should be
greater than 1.0, determines the
downscaling ratio of the image at each
iteration of the face detection process.
 PERFORMING FACE DETECTION ON A
STILL IMAGE
The value returned from the detection operation is a list of tuples that
represent the face rectangles. OpenCV's cv2.rectangle function allows
us to draw rectangles at the specified coordinates. x and y represent
the left and top coordinates, while w and h represent the width and
height of the face rectangle. We draw blue rectangles around all of the
faces we find by looping through the faces variable, making sure we
use the original image for drawing, not the gray version:
PERFORMING FACE DETECTION ON A
STILL IMAGE
Lastly, we call cv2.imshow to display the resulting
processed image. To prevent the image window from
closing automatically, we insert a call to waitKey,
which returns when the user presses any key:
PERFORMING FACE DETECTION ON A
VIDEO
Create the following basic script to perform face detection
PERFORMING FACE DETECTION ON A
VIDEO
Let's break up the preceding sample into smaller, digestible chunks:
1. Import the cv2 module. After that, we initialize two CascadeClassifier
objects, one for faces and another for eyes:
PERFORMING FACE DETECTION ON A
VIDEO
2. As in most of our interactive scripts, we open a camera feed
and start iterating over frames. We continue until the user
presses any key. Whenever we successfully capture a frame, we
convert it into grayscale as our first step in processing it:
PERFORMING FACE DETECTION ON A
VIDEO
3. We detect faces with the detectMultiScale method of our face
detector. As we have previously done, we use the scaleFactor and
minNeighbors arguments. We also use the minSize argument to
specify a minimum size of a face, specifically 120x120. No
attempt will be made to detect faces smaller than this. Here is
the call to detectMultiScale:
PERFORMING FACE DETECTION ON A
VIDEO
4. We iterate over the rectangles of the detected faces. We draw
a blue border around each rectangle in the original color image.
Then, within the same rectangular region of the grayscale image,
we perform eye detection:
PERFORMING FACE DETECTION ON A
VIDEO
5. We loop through the resulting eye rectangles
and draw green outlines around them:

6. Finally, we show the resulting frame in the window:

PERFORMING FACE RECOGNATION
Our sample script uses a size of 200x200, but most
freely available datasets have smaller images than
this.
 LOADING THE TRAINING DATA FOR FACE
RECOGNITION
The same way. Earlier, in the Generating the data for face
recognition section, we generated training images and saved
them in folders that were organized according to people's
names or initials. The following folder structure could contain
sample face images of this book's authors, Joseph Howse (J. H.)
and Joe Minichino (J. M.):
../
data/
at/
jh/
jm/
 LOADING THE TRAINING DATA FOR FACE
RECOGNITION
Let's write a script that loads these images and labels them in a
way that OpenCV's face recognizers will understand. To work
with the file system and the data, we will use the Python
standard library's os module, as well as the cv2 and numpy
modules. Let's create a script that starts with the following
import statements:
 LOADING THE TRAINING DATA FOR FACE
RECOGNITION
Here is the function's implementation:
 PERFORMING FACE RECOGNITION WITH
EIGENFACES
Now that we have an array of training images and an array of
their labels, we can create and train a face recognizer with just
two more lines of code:
PERFORMING FACE RECOGNITION WITH
EIGENFACES
Done in previous scripts, we can use the following
line of code to initialize the face detector:
 PERFORMING FACE RECOGNITION WITH
EIGENFACES
The following code initializes the camera feed, iterates over
frames and performs face detection and recognition on each
frame:
 MODIFYING THE APPLICATION'S
LOOP
Let's open cameo.py so that we can walk through the overall
changes to the application:
1. Near the top of the file, we need to import our new modules, as
shown in bold in the following code block:
 MODIFYING THE APPLICATION'S
LOOP
2. Now, let's turn our attention to changes in the __init__method of our
CameoDepth class. Our updated application uses an instance of FaceTracker. As part
of its functionality, FaceTracker can draw rectangles around detected faces. Let's
give Cameo's user the option to enable or disable the drawing of face rectangles. We
will keep track of the currently selected option via a Boolean variable. The following
code block shows the necessary changes to initialize the FaceTracker object and the
Boolean variable:
 MODIFYING THE APPLICATION'S
LOOP
3. Depending on the currently selected option, we might tell FaceTracker to draw
rectangles around the faces. All of the relevant changes are shown in the following
code block:
MASKING A COPY OPERATION
We shall copy only those pixels in the source rectangle where the mask's value is not
zero. Other pixels shall retain their old values from the destination image.
We can add duplicate channels to mask using the repeat and reshape methods of
numpy.array. We perform the copy operation using the numpy.where.
MASKING A COPY OPERATION
We also need to define a swapRects function, which uses copyRect to perform a circular swap of a list
of rectangular regions. swapRects has a masks argument, which is a list of masks whose elements are
passed to the respective copyRect calls. If the value of the masks argument is None, we pass None to
every copyRect call. The following code shows the full implementation of swapRects: