INTERNATIONAL JOURNAL OF IMAGING SCIENCE AND ENGINEERING (IJISE)

76

Face Detection using Multi-Scale Morphological Segmentation

Nallaperumal Krishnan 1, Senior Member, IEEE, S. Ravi 2, Member, IEEE, Krishnaveni3. K, Member, IEEE, Justin Varghese3, Student Member, IEEE, S.Saudia3, Student Member, IEEE, R.K.Selvakumar4, Member, IEEE, A. Lenin Fred5, Member, IEEE
1

Centre for Information Technology and Engineering, Manonmaniam Sundaranar University, email:[email protected]
2

Department of Computer Science, Cardamom Planters Association College, Bodinayakanur, India Dept. of Computer Science, S.R.Naidu College, Sattur, India Dept of Software Engg., Kamaraj College of Engg. and Tech., Virudhunagar, India. Department of Information Technology, C. S. I Institute of Technology, Thovalai, India.
4 3

Abstract- A novel face detection technique is proposed in this paper, which uses multi-scale morphological segmentation. This technique works satisfactorily on gray scale face images containing bright and dark features of various scales even in an impulsecorrupted environment. The face detection algorithm involves four stages. In the first stage, using an Iterative Adaptive Switching Median Filter, which reduces the impact of impulse noise that cause over segmentation, preprocesses the face image. In the second stage, the multiple scales of bright and dark features of different objects are extracted by the respective opening and closing of the preprocessed image. The resultant image is binarized using an optimum threshold. In the third stage, valid segments of the bright top-hat and dark bottom-hat images are detected and the contours of these face images are combined to give the final skin segmented face image. In the last stage, the facial region is detected from the skin-segmented image using template matching technique. The scheme is implemented on a set of test face images and the performance of the algorithm is proved better both objectively and subjectively than the standard methods.

segmenting objects based on their shape as well as size a second attribute is incorporated to the structuring element, which is its scale. Morphological operations with such scalable SEs may be used in multi-scale image processing [1], [2] for segmentation. If the set representing the SE is convex, then dilation of the SE with itself (n-1) times produces the nth homothetic of the SE keeping its shape intact. A new face detection algorithm is proposed in this paper, which perfectly detects the face images corrupted with various quantum of impulse noise. The proposed algorithm exploits the multi-scale approach of the SE with suitable size to reduce the over-segmentation problem vivid in Watershed algorithm. The paper is organized in IV sections. Section II explains the various phases of proposed algorithm. Experimental and performance analysis is made in Section III. Conclusions are given in Section IV.
II. SEGMENTATION ALGORITHM

Index Terms-Face detection, Multi-scale morphology, Impulse noise, Median filters, Template matching. I. INTRODUCTION iven a single image, the goal of face detection is to identify all image regions, which contain a face regardless of its three-dimensional position, orientation, and lighting conditions. Such a problem is challenging because faces are non-rigid and have a high degree of variability in size, shape, color, and texture. To build fully automated face detection systems that analyze the information contained in face images, robust and efficient face detection algorithms are required. Mathematical morphology is a well-known technique used in image processing and computer vision for manipulating the features present in the image based on their shapes. This set theoretic shape oriented approach treats the image as a set and the kernel of operation known as structuring element (SE) as another set. Different standard morphological operations like dilation, erosion, opening and closing are basically set theoretic operations between these two sets. The shape and the size of the SE play an important role in detecting or extracting features of given shape and size from the image. Though the SE takes care of the shape of the features while processing the image, it cannot, however, treat objects of same shape but of different size equally. Thus, for
IJISE,GA,USA,ISSN:1934-9955,VOL.1,NO.3, JULY 2007

The gray level image consists of both bright and dark object features, which have a distribution with respect to size or scale. In this section, a new face detection algorithm is proposed which isolates the optimal contours of these bright and dark features, addresses the over and under segmentation problems and thereby extracts the sub features of the objects effectively even from a noisy environment. The face detection algorithm involves four stages. In the first stage, using an Iterative Adaptive Switching Median Filter, which reduces the impact of impulse that cause over segmentation preprocesses the image [4]. In the second stage the multiple scales of bright and dark features of different objects are extracted by the respective opening and closing of the preprocessed image. The resultant image is binarized using an optimum threshold. The process is repeated for multiple scales of the structuring element until all the features are extracted [3]. In the third stage, valid segments of the bright top-hat and dark bottom-hat images are detected and the contours of these images are combined to give the skin segmented face image [5]. In the final stage, the face is detected from the skin segmented face image using template matching techniques [7]. The scheme is outlined in Fig.1. A. Preprocessing A face detection algorithm often needs a preprocessing step like noise reduction to minimize the effects of undesired perturbations like impulse or impulse-like noise that may

NALLAPERUMAL KRISHNAN, S. RAVI, et.al.,: FACE DETECTION USING MULTI-SCALE MORPHOLOGICAL SEGMENTATION

77

cause over or under segmentation. In earlier papers an Iterative Adaptive Switching Median Filter (IASMF) algorithm have been proposed [4], which makes possible the perfect removal of impulse noises from images corrupted with higher or lower quantum of impulse noise in the most efficient manner while keeping the fine details of the image intact by leaving the uncorrupted pixels untouched. The proposed algorithm uses this filter in the pre-processing stage. The filtering technique involves two passes. An iterative detection pass, wherein image pixels that are corrupted by the impulse noise are determined and detected into a flag image and an adaptive correction pass for only those noisy pixels detected in pass one by a median value chosen from a set of uncorrupted pixels within a specified window. The resultant image is free from patchy effects and is very effective when images are corrupted with large percentage of impulse noises.
Input image Searching valid segments Preprocessing Multi-scale feature extraction and binarization

f: P(X) -> R (1) where P(X) set of all fuzzy subsets from the universe X. R is a set of real numbers and f satisfies the following axioms.
1. f (A) = 0 A is a crisp set and the degree of fuzziness is zero. 2. If A < B => f (A) <= f (B). i.e. A is sharper than B. 3. f(A) = maximum value A is maximally fuzzy.

To measure the fuzziness Lin et al. proposed three measures from which Gaussian measure of Fuzziness is used in our proposed method and is discussed here. Gaussian measure of fuzziness is based on the normal distribution of probability such that the Gaussian index of fuzziness of a fuzzy set, A is defined as
G(A(x)) f(A) = --------------|X| where G(A(x)) = A(x) c) / 2, if 0 < A < 1 0 ,if A = 0 or A = 1 (2) (3)

Skin segmented face image

Count number of holes

Let mean, c = 0.5, standard deviation, = 0.15, then (0.5) = 1 and (1) = 0.0039 0. The membership function can be defined in many forms. Using general bell curve, the membership function for gray level z in image, I at any spatial location (i, j), with an initial threshold, T is
(1 / | z mb (T)| 2b ) 1 + ----------------------- if z<=T D (1 / | z mb (T)| 2b ) 1 + ----------------------- if if z>T D (4)

YES
|Holes| >= 1 Template matching

1(z,T) =

No
Non-face Detected face region

Fig.1.The stages of multi-scale morphological face detection algorithm. B. Obtaining Fuzzy Optimal Threshold Value There exist several algorithms that involve determining the optimal threshold for segmenting gray scale images [2], [5]. A histogram threshold technique suffers from the problem of yielding contours with non uniform thickness as well as discontinuities due to difficulty in selecting optimum threshold [1], [5]. The optimal selection of threshold value is a difficult process due to the presence of noise, vagueness and ambiguity among the classes produced due to their overlapping in the histogram. In this paper, the optimal threshold value is obtained based on the Fuzzy Gaussian Measure. i) Fuzziness Measures: In fuzzy logic, the element belonging to a fuzzy set is represented by a membership function. Lin & Lee [3] identify two categories of uncertainty: vagueness and ambiguity. Vagueness is related to a measure of fuzziness. The measure of fuzziness (index of fuzziness) can be formalized in order to measure the fuzziness of the classes in the process of thresholding an image. Manuel and Oswaldo [5] has proposed an optimal threshold selection based on index of fuzziness. The measure of fuzziness is defined as
IJISE,GA,USA,ISSN:1934-9955,VOL.1,NO.3, JULY 2007

where D is the range of the gray levels in the image, I, mb and mo are the average of the gray levels of the pixels belonging to background and object respectively. The parameter b determines the sharpness of the fuzzy set. T zh(z) z =0 mb (T) = --------------(5) T h(z) z =0 z-1 zh(z)
z =T+1

mo(T) = --------------z-1 h(z)

z =T+1

(6)

Thus the index of fuzziness for gray level z in the image, I at any spatial location (i, j), with the threshold, T is given as, L-1 f(z,T) = h(z) G(1(z,T)) 0 <= T <= L-1 (7) z=0 where
G(1(z,T)) =

e A(x) c) / 2, if 0<A < 1

(8)

INTERNATIONAL JOURNAL OF IMAGING SCIENCE AND ENGINEERING (IJISE)

78
bot

, if A=0 or A =1

ii) Optimum Threshold Selection: The fuzziness function f(z, T) is calculated for all gray levels of the image, I and the optimal threshold, Topt is chosen as Topt = ArgMin f(z, T) such that Zmin T Zmax where Zmin, Zmax are the minimum and maximum gray level of the image respectively. The determination of the cost function, f for all pixels, is more computationally complex. It is only enough to determine the cost function of certain pixels. There are several possible numerical methods to determine gray level for which the cost function is minimized. A new algorithm has been proposed in this paper to select the most appropriate gray level for the determination of optimal threshold value. Algorithm:
Zmax -> Maximum gray level of the input Image. Zmin -> Minimum gray level of the input Image. Step 1: A = Zmin, B = Zmax, Find f (z, A) & f (z, B). Step 2: Let T = (A+B)/2. Find f (z, T), f (z, T-1), f (z , T+1) Step 3: If f (z, T-1) < f (x, T) then B = T-1; Go to Step 2. else If f (z, T+1) <f (z, T) then A = T+1; Go to Step 2. else Topt=T; go to step 4; Step 4 Segment the Image using Topt Exit.

Similarly the dark bottom-hat image, gk obtained as

at scale-k is
(11)

gkbot(i,j) = (g^k-1

kB) (i,j) -

g^k-1(i,j)

where (i, j) are the spatial co-ordinates of the image and g k-1 for the initial dark bottom hat is the preprocessed image, A. The dark bottom- hat image resulting due to closing at scale-k is binarized with the optimal threshold, Topt. The resulting binarized image, g of scale-k contains all possible dark features in the image. Only those features of the mth component, km of the n components in g that have sufficient contrast from its immediate background are to be identified using eqn. (3) and kept in the kth level of the dark feature tower. The binary images constructed using k and k will give rise to overlapping or encroachment in the segmented contours. To overcome this encroachment problem a pixel being identified simultaneously as a part of both the bright and dark scale is avoided using the following constraint for the same or different values of k.

k k =

(12)

This means, once a pixel is marked as a part of a dark feature it should remain in that part of the tower of all higher scales by assigning a MINVAL to that pixel and a MAXVAL to that pixel if it is the part of a bright feature of a tower of all higher scales. and
gkbot(i,j) = MINVAL, if(i,j) k & LC(km >= Topt g(i,j) otherwise (13)

An optimal threshold value Topt is obtained that gives minimal fuzziness and an optimized segmented output is visualized. C. Multi-scale Region Extraction In this stage, the information about potential bright and dark regions of the preprocessed image, A at different scales is extracted by the opening and closing of the image, A by various scales of the structuring element, B. The bright top-hat top image, gk at scale-k is obtained as

gktop(i,j) = MAXVAL, if(i,j) k & LC(km >= Topt g(i,j) otherwise

(14)

gktop(i,j) = g~k-1(i,j)

~ ( g k-1 o kB) (i, j)

(9) ~

where (i, j) are the spatial co-ordinates of the image and g k-1 for the initial bright top-hat is the preprocessed image, A. The bright top-hat image resulting due to opening at scale-k is binarized with the optimal threshold, Topt obtained. The resulting binarized image, g of scale-k contains all possible bright features in the image. Only those features of the component of g that have sufficient contrast from its m immediate background are to be identified. Suppose k is a th m component of n components in g, then local contrast, LC is measured as,
LC(km)=[max{g(i,j)|(i,j)km}]-[min{g(i,j)|(i,j)km}] (10)

where MINIVAL = 0, MAXVAL =255 and Topt is the optimum threshold value. Next higher homothetic scale of the SE, B is selected and the procedure is repeated until all the features are extracted. D. Selection of Valid Regions The details of the multi-scale dark and bright towers are used to construct the final segmented image. The objects to be included in the final segmented image are those objects, which are identical in adjacent scales of the corresponding bright and dark towers. The bright components are combined to form an image gbright and the dark components are combined to form an image gdark. The contours of these two images are combined to form the final skin segmented face image. E. Detection of face images In order to determine the frontal human face it is necessary to determine the number of regions in the image. A skin region is defined as a closed region in the image, if a region has one or more holes the region is assumed to be a face. i) Number of holes inside a region

If the contrast, LC is less than the specified threshold Topt , the corresponding component, m is discarded. Finally, the resultant binary image is kept in the kth level of the bright feature tower.

IJISE,GA,USA,ISSN:1934-9955,VOL.1,NO.3, JULY 2007

NALLAPERUMAL KRISHNAN, S. RAVI, et.al.,: FACE DETECTION USING MULTI-SCALE MORPHOLOGICAL SEGMENTATION

79

It is necessary to find which regions have at least one hole inside it. The regions without holes are ignored. Number of holes in a region can be calculated by using Euler number of the region For our case, the number of connected components is set to 1 since one skin region will be considered at a time. ii) Center of mass In order to study the region, its area and center of the region must be determined. The center of the area in binary image is the same as the center of the mass. iii) Orientation Most of the faces are vertically oriented. However, some of them have little inclination. If the template face is rotated in the right angle, there will be higher matching. The orientation of the axis of the elongation will determine the orientation of the region. In this axis the inertia will be the minimum. iv) Width and height of the region First, the regions with holes are filled out. This is to avoid problems when holes are encountered. Now it is necessary to determine the height and width by moving 4 pointers: one from the left, right, top and bottom of the image. If a pixel is found with a value other than 0, the pointer is stopped and this is the coordinate of a boundary. With the help of the 4 values, the height of the facial region is computed by subtracting the bottom and top values and the width is computed by subtracting the right and the left values. v) Region ratio The width and the height of the region are used to improve the decision process. The height to width ratio of the human faces is around 1. In order to have less misses however, it is found that a minimum good value is 0.8. Ratio values below 0.8 do not suggest a face since human faces are oriented vertically. The ratio should also have an upper limit. This is determined by analyzing the results in our experiments that a good upper limit should be around 1.6. While the above-mentioned information improves the classification, it can also be a drawback for cases such as the arms that are very long. If the skin region for the arms has holes near the top, this might yield into a false classification. vi) Template matching One of the most important characteristics of this method is that it uses a human face template to take the final decision of determining if a skin region represents a face. The template is chosen by averaging 16 frontal view faces of males and females wearing no glasses and having no facial hair. The template used is shown in figure 2. Template face (model) is used to verify the existence of faces in skin regions. The template face has to be positioned and rotated in the same coordinates as the skin regions image. The template face is resized according to the height and the width of the region of the region computed. The resized template face is rotated to theta, so that it is aligned in the same direction as the skin region. The center of the rotated template face is computed. Then the cross-correlation between the part of the image corresponding to the skin region and the template face is computed. The coordinates of the frontal face image are determined and a rectangle is drawn in the gray scale image.
IJISE,GA,USA,ISSN:1934-9955,VOL.1,NO.3, JULY 2007

Fig 2. Template face used to verify the existence of faces in skin regions III.EXPERIMENTAL RESULTS AND DISCUSSIONS The proposed face detection algorithm for gray scale images has been executed on a set of synthetic facial images corrupted by different levels of impulse noise for face image, which is used here for subjective analysis. The new face detection algorithm is providing all the sub-features of the objects from the face image even from the highly impulse corrupted environment by using an optimal threshold value determined by an improved measure. This threshold also specifies how distinct the segment could be relative to its immediate background. The proposed algorithm also avoids to a large extent the over and under segmentation problems that are vivid in the outputs of the standard algorithms used here for comparison. The size of the structuring element, SE is increased in the proposed algorithm, which relatively reduces the over-segmentation problem more prominent in the watershed algorithm. The range of the size of objects of interest determines the minimum and the maximum value of the diameter of the SE. The shape and size of the object are also preserved in the proposed algorithm by the multi-scale approach. The significant bright and dark regions are successfully separated by the proposed method. The space complexity is slightly higher and is mainly caused by multiscale opening and closing by reconstruction. In each scale these operations involve computation of the order of mN2, for an image of size N X N, m is a multiplier that depends on the scale factor. The time complexity may be reduced significantly by distributing the operations at different scales on parallel processors. This proposed algorithm detected 34 out of 36 face images resulting in 94.4 percent of successful face detection. IV CONCLUSION The new face detection algorithm proposed in this paper makes the perfect detection of face images even from a noisecorrupted environment. The algorithm segments the gray-level images of different shape and size in the most efficient manner than most other segmentation methods. The proposed algorithm exploits the multi-scale approach of the structuring element with suitable size to reduce the over-segmentation problem vivid in Watershed algorithm. It also uses an improved, less complex measure to determine the optimal threshold to extract the sub-features of the objects to reduce the under-segmentation problem more projected in histogram thresholding algorithm. The advantages of the proposed scheme are the shape maintaining, edge preserving and scalecalibrating features. The significant bright and dark regions

INTERNATIONAL JOURNAL OF IMAGING SCIENCE AND ENGINEERING (IJISE)

80

are successfully separated by contours produced in the proposed method. This proposed algorithm detected 34 out of 36 face images resulting in 94.4 percent of successful face detection.

Nallaperumal Krishnan received M.Sc. degree in Mathematics from Madurai Kamaraj University, Madurai, India in 1985, M.Tech degree in Computer and Information Sciences from Cochin University of Science and Technology, Kochi, India in 1988 and Ph.D. degree in Computer Science & Engineering from Manonmaniam Sundaranar University, Tirunelveli. Currently, he is heading Centre for Information Technology and Engineering of Manonmaniam Sundaranar University, Tirunelveli. His research interests include Signal and Image Processing, Remote Sensing, Visual Perception, Mathematical Morphology Fuzzy Logic and Pattern recognition. He has authored three books, edited 18 volumes and published 25 scientific papers in Journals. He is a Senior Member of the IEEE. Ravi Subban received B.E degree in Computer Science and Engineering from Bharathiar University, Coimbatore, India in 1989, M.Tech degree in Computer and Information Technology from Centre for Information Technology and Engineering, Manonmaniam Sundaranar University, Tirunelveli, in 2004. Currently, he is doing Ph. D in Computer Science and Engineering at Manonmaniam Sundaranar University Tirunelveli. He is working as a Selection Grade Lecturer in the Department of Computer Science, Cardamom Planters Association College, Bodinayakanur. His research interests include Image Processing, Face detection and Face recognition, Machine Vision and Pattern Recognition. He has published 2 scientific papers in IEEE International Conference. He is a Member of the IEEE.

Fig 3. Face detected results REFERENCES

[1] Susanta Mukhopadhyay and Bhabatosh Chanda. Multiscale Morphological Segmentation of Gray Scale Image IEEE Transactions on Image Processing, Vol.12, No. 5, May 2003. [2] P.Salembier and J.Serra. Morphological Multiscale Image Segmentation in proc. Visual Communication and Image Processing, Boston, MA.1992,pp. [3] Huang.K. and Wang M.J. Image thresholding by minimizing the measure of fuzziness. Pattern recognition. Vol.28, pp.41-51, 1995 [4] Krishnan Nallaperumal, Justin Varghese, et.al Iterative Adaptive Switching Median filter, IEEE Conf. on Industrial Electronics and Applications ICIEA 2006, P0604 [5]. Krishnan Nallaperumal, Krishnaveni.K, Ravi Subban, et al, Fuzzy Optimal Threshold Multiscal orphological Segmentation of Digital Images, IEEE Conf. On Wireless and Optical Communication Networks WOCN 2006 [6]. M.-H. Yang, D. Kriegman, and N. Ahuja, Detecting Faces in Images: A Survey, IEEE Trans. Pattem Analysis and Machine Intelligence, vo1.24, no.I , Jan. 2002 [7] Krishnan Nallaperumal, Ravi Subban, et al, Human Face Detection in Color Images Using Skin Color and Template Matching Models for Multimedia on the Web, IEEE Conf. On Wireless and Optical Communication Networks WOCN 2006.

IJISE,GA,USA,ISSN:1934-9955,VOL.1,NO.3, JULY 2007