Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

Facial Expression Recognition System Based on SVM and HOG

Techniques

Safa Rajaa [email protected]

Faculty of Information Technology,
Department of Information Technology
Industrial Innovation and Robotics Center
University of Tabuk, Tabuk, Kingdom of Saudi Arabia

Rafika Harrabi [email protected]

Faculty of Information Technology,
Department of Information Technology
Industrial Innovation and Robotics Center
University of Tabuk, Tabuk, Kingdom of Saudi Arabia

Slim Ben Chaabane [email protected]

Faculty of Information Technology,
Department of Information Technology
Industrial Innovation and Robotics Center
University of Tabuk, Tabuk, Kingdom of Saudi Arabia

Abstract

Facial expression is one of the most commonly used nonverbal means by humans to transmit
internal emotional states and, therefore, it plays a fundamental role in interpersonal interactions.
Although there is a wide range of possible facial expressions, psychologists have identified six
fundamental ones (happiness, sadness, surprise, anger, fear and disgust) that are universally
recognized. Automatic facial expression recognition (FER) is a topic of growing interest mainly
due to the rapid spread of assistive technology applications, as human–robot interaction, where a
robust emotional awareness is a key point to best accomplish the assistive task. The proposed
work aims to design a robust facial expression recognition system (FER). FER system can be
divided into three modules, namely facial registration, feature extraction and classification. The
objective of this work is the recognition of facial expressions based the Histogram of Oriented
Gradients (HOG) and support vector machine (SVM) algorithm.

Firstly, a comprehensive study on the application of histogram of oriented gradients HOG

descriptor in the FER problem is presented, highlighting as this powerful technique could be
effectively exploited for this purpose. Then, a multi SVM is then trained to perform the facial
expression classification.

The proposed technique is applied to two public datasets, such as the JAFFE dataset and an
extended Cohn-Kanade (CK+) dataset. Facial expression recognition from the proposed method
are validated and the True Success Rate (TSR) for the test data available is evaluated, and then
a comparative study versus existing techniques is presented. Face Recognition using HOG and
SVM are better compared to existing state of the art methods.

Keywords: Component, Facial Expressions, Histogram of Oriented Gradients (HOG), Support

Vector Machine (SVM), Facial Expression Recognition, HOG Features, Facial Component
Detection.

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 14
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php
Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

1. INTRODUCTION
Although humans are experts in facial recognition, it is not yet understood how to perform this
recognition with high accuracy [1] [2] [3]. For many years, psychologists and neuroscientists have
been looking at whether facial recognition is done holistically or by analyzing local features, i.e.
facial recognition is done by looking at the face as a whole or by looking at local facial features
independently to identify people.

The study of the face and its characteristics is an active search area and very increased in the
past few decades. Many efforts have been made and many are still being made to develop a
robust and accurate system in the field of information security and the security in general. It is
mainly used in the fields of human-computer interaction, psychological behavior analysis, image
understanding, e-learning, etc. Successfully recognized expressions can be used in many sectors
of our lives for the betterment of everyday experience. It can be used for security purposes,
robots can be given this ability to enhance their performance, automated machines can be
articulated with expression analyzing feature, face expressions can be analyzed to determine the
persons authorized to enter places that have security levels to allow or prohibit a person from
doing crucial tasks and in many other sectors, this facial expression recognition (FER) can be
used. As the world is heading towards automation, so the ultimate goal of the research is to
recognize expressions flawlessly and spontaneously using machines, as humans can do in high
accuracy. Usually, seven basic facial expressions are considered while dealing with FER
problems and they are neutral, angry, fearful, happy, sad, surprise, disgust [5] [6] [7].

However, the proposed research took the form of a new research but on an existing research
subject. In order to satisfy the objectives of this work, a quantitative research was held, is that it
offers a complete description and analysis of a face recognition research subject [8] [9].

In this paper, we introduce an effective appearance-based method to handle the facial expression
recognition problem. Given a face image, the system detects the face first and then, extracts the
facial components from the face image. After that, the Histogram Oriented Gradient (HOG) [10]
[11] is extracted to encode these facial components and concatenate them into a single feature
vector. Once the feature vectors are determined for each image to be classified, the SVM rule
and decision are applied to obtain the final classification.

Section 2 introduces the proposed method for face recognition. The experimental results are
discussed in Section 3, and the conclusion is given in Section 4.

2. THE PROPOSED METHOD

The approach facial expression recognition can be seen as a three-stage process: face detection,
feature extraction, and expression classification. In recent years, the extraction methods of
expression feature mainly include Global feature extraction and Local Feature extraction [12] [13].

In order to extract facial expression features better and improve facial expression recognition
rate, this paper proposes a face expression classification based on method Global Features
extraction geometric features.

In the case of facial expression recognition [14] [15] [16], the determination of features is a crucial
step of recognition [17] [18]. Whether the extraction of expression features is effective or not will
directly affect the expression classification process [19] [20].

In this work, facial expression recognition is a combined effort of the feature extraction process
and classifier. The HOG [18] are applied on the face images to determine the feature vectors and
the classifier SVM is employed for the classification. The seven expressions used in our facial
expression recognition system are: angry, sad, surprise, happy, fear, disgust, neutral [19].

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 15
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php
Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

2.1 Preprocessing
First, the input image is converted to grayscale if it is a color image. Otherwise, this step is
ignored. Gray scaling is a method of transforming an RGB image into a grayscale image using
the following equation:

(1)

Hence the input images are transformed to gray level images, the facial images are then
transformed into a fixed sizes based on resampling method, where the image will be added by
new pixels within the resized pixel mathematically and the color and new pixel will be predicted to
be added within the pixels colors nearby.

2.2 Features Extraction Approach

Feature extraction [21] [22] [23] is a process that identifies important features or attributes of the
data. Feature extraction is an important task in facial expression recognition. In our application,
the Histogram of Oriented Gradient (HOG) is applied for extracting features of various facial
expressions [24] [25].

Basically, HOG is a feature descriptor used for object detection and feature that represents shape
change by excluding all other unnecessary information. Prominent areas are extracted from a
face image by compute centered horizontal and vertical gradient with no smoothing and compute
gradient orientation and magnitude.

In short, the gradient intensities of an image can reveal some useful local information that can
lead to recognition of the image and then Show confusion and diagram to HOG.

In HOG (see Figure 1), a feature descriptor tries to capture the important information in an image
and keeps all the not so important information behind the scenes. Then we can use the useful
information from the feature descriptor for image recognition and object detection.

Therefore, we propose a HOG Feature Descriptor of the general form:

o Computing the edge and Gradients.

o Spatial / Orientation Binning
o Block Normalization.
o Get the HOG Feature Vector.

FIGURE 1: Histogram of Oriented Gradient (HOG).

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 16
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php
Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

2.3 Classification Approach

Support vector machines (SVMs) are a set of supervised learning methods used
for classification, regression and outlier's detection. However, it is mostly used in classification
problems.

In this paper, a novel learning method, Support Vector Machine (SVM), is applied on the features
extracted from the input images, which have seven classes. SVM, a powerful machine method
developed from statistical learning and has made significant achievement in some field.
Introduced in the early 90's, they led to an explosion of interest in machine learning. In our
experiment, HOG is used to obtain the support vectors, which are critical for classification. The
proposed method can be described the flowchart given in Fig. 2.

FIGURE 2: Flowchart of the Proposed Method.

3. RESULTS AND DISCUSSION

In this section, several simulation results on the facial expression recognition, which illustrate the
ideas presented in the previous section, are given. In addition, the performance analysis of the
proposed algorithm is discussed. The face databases: JAFFE dataset and an extended Cohn-
Kanade (CK+) are considered for performance analysis. The results are compared with the
existing techniques.

Traditionally, the most important problem that a researcher faces in an SVM algorithm the
potential overlap of multiple classes, so the best solution was to use machine-learning technology
to train the classifier on multi classes in the training group.

Recently, simpler and more rapid functions of multi classes SVM have been developed in training
set followed by the test group in which classifier predicts the required output (labels).

The most obvious finding to emerge from the analysis is that results were impressive in the
classification phase and there are several possible explanations for this finding.

In our application, we can noticed that the more the database was large and the content of the
images in it was many, the better the results would be and the accuracy was very high, so the
database accuracy rate JAFFE was less than Cohn-Kanade and the reason is due to the number
of images in the two databases.

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 17
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php
Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

Regarding the accuracy, Tables 1, Table 2 and Table 3 list the accuracy of the different methods
for the data set used in the experiment.

The Accuracy is determined as follows:

Accuracy=(NCP / TNPM)x100 (2)

where:

NCP is the number of genuine persons recognized correctly and TNPM is the total number of
persons inside the database.

Table 1 and Table 2 illustrate the accuracy and the response Time of the proposed method
applied to the JAFFE dataset and an extended Cohn-Kanade (CK+) dataset, respectively.

JAFFE CK+

Accuracy 95.23 99.23

TABLE 1: Table Accuracy of Data Base.

JAFFE CK+

Response Time 0.51094 0.592032

TABLE 2: Table Response of Data Base.

Although the proposed method is accurate in terms of predicting the label of a test image,
compared to previous studies, the results of our system showed that accuracy is high, and this
means that our system supports previous studies by using an SVM and HOG algorithm to
recognize facial expressions.

Also, let us compare the performance of our proposed algorithm to those in other published
reports that have recently been applied to facial expression recognition. These include Bakchy et
al. 2017, Islam et al. 2018, and Mehta al. 2018. The results are illustrated in Table 3.

Study Technique Dataset Accuracy

JAFFE 95.23
Our System 2020 System Based on SVM and HOG
CK+ 99.23

JAFFE 94.89
2017 2D Gabor Filter and MultiSVM
CK+ 99.57

2018 LBP and HOG Mediu-S-DB 97.82

Viola Jones, Gabor filter and JAFFE 94.42

2018
SVM CK+ 99.59

HOG and LBP Features and JAFFE 95.76

2018
Multiclass SVM CK+ 99.15

TABLE 3: The Accuracy of Applied Techniques.

As seen on Table 3, the proposed method HOG with SVM for classification still gets the best
performance on the JAFFE dataset and an extended Cohn-Kanade (CK+) dataset.

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 18
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php
Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

4. CONCLUSION
In this paper, we have proposed a new method for facial expression recognition based on HOG
and SVM techniques. In the first phase, The HOG is applied to the entire database to generate a
final set of features. Then, SVM technique is employed for classification task.

The most obvious finding to emerge from this study is that a work HOG and SVM emerge as
reliable predictors of facial recognition, where the relevance of SVM algorithm is clearly supported
by the current findings for recognition systems as a model high security.

Therefore, the findings of this investigation complement those of earlier studies and supports it.
These results provide more support and power for HOG and SVM in facial recognition systems.

Despite these promising results, there is still abundant room for further progress in determining
facial recognition.

We obtained highest recognition rate as 99.23% with HOG–SVM method. Considering weighted
means of recognition rates, the proposed Facial expression recognition method gave better
results than some existing approaches.

The results obtained demonstrated the significant improved performance in Facial expression
recognition. The proposed method can be useful for facial expression recognition.

5. REFERENCES
[1] Eleyan, A. and H. Demirel. "Face recognition system based on PCA and
feedforward neural networks". International Work-Conference on Artificial Neural Networks,
1: 935–942, 2005.

[2] Faruqe, M.O. and M.A.M. Hasan. "Face recognition using PCA and SVM". Third
International Conference on Anti-counterfeiting, Security, and Identification in
Communication, 1: 97–101, 2009.

[3] M. M. Sani, K. A. Ishak and S.A. Samad. "Evaluation of face recognition system using
Support Vector Machine". IEEE Student Conference on Research and Development
(SCOReD), 1: 139–141, 2009.

[4] . "Face recognition using eigenface approach". Serbian Journal

of electrical engineering, 9: 121–130, 2012.

[5] Ç rı çı F Öz . "A face recognition system based on eigenfaces method".

Procedia Technology, 1: 18–23, 2012.

[6] Zarbakhsh, P. and H. Demirel. "Fuzzy SVM for 3D facial expression classification using
sequential forward feature selection". International Conference on Computational
Intelligence and Communication Networks (CICN), 3: 131–134, 2007.

[7] Sino, H. W. and I. S. Areni (2019). "Face Recognition of Low-Resolution Video Using Gabor
Filter Adaptive Histogram Equalization". International Conference of Artificial Intelli- gence
and Information Technology (ICAIIT), 1: 417–421, 2019.

[8] Mehta V., Khandelwal S. and A. K. Kumawat. "A Survey on Face Recognition Algorithm". In:
International Conference on Trends in Electronics and Informatics (ICOEI), 1: 1– 3, 2018.

[9] Meena, D. and R. Sharan. "An approach to face detection and recognition". International
Conference on Recent Advances and Innovations in Engineering (ICRAIE), 1: 1– 6, 2016.

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 19
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php
Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

[10] Mady, H. and S. M. Hilles. "Face recognition and detection using Random forest and
combination of LBP and HOG features" In: International Conference on Smart Computing
and Electronic Enterprise (ICSCEE), 2: 1–7, 2018.

[11] Liu S., Chen X. Fan D. Chen X. Meng F. Huang Q. (2016). "3D smiling facial expression
recognition based on SVM". IEEE International Conference on Mechatronics and
Automation, 2: 1661–1666, 2016.

[12] Y. Kumar and S. Sharma. "A systematic survey of facial expression recognition techniques"
In: international conference on computing methodologies and communication (ICCMC), pp.
1074–1079, 2017.

[13] Y. Kortli, M. Jridi, A. Falou and M. Atri. "A novel face detection approach using local binary
pattern histogram and support vector machine". International Conference on Advanced
Systems and Electric Technologies (ICASET), 3: 28–33, 2018.

[14] N. Kauser and J. Sharma. "Automatic facial expression recognition: a survey based on fea-
ture extraction and classification techniques". International Conference on ICT in Busi- ness
Industry & Government (ICTBIG), 3: 1–4, 2016.

[15] B. Islam, F. Mahmud and A. Hossain. "High Performance Facial Expression Recognition
System Using Facial Region Segmentation, Fusion of HOG & LBP Features and Multi- class
SVM". International Conference on Electrical and Computer Engineering (ICECE), 3: 42–45,
2018.

[16] B. S. Doiphode and S. D. Sapkal. "classifying Facial Expression Using Support Vector
Machine Based on Bidirectional Local Binary Pattern Histogram Feature Descriptor".
Second International Conference on Intelligent Computing and Control Systems (ICICCS),
5: 1890–1895, 2018.

[17] K. Dang and S. Sharma. "Review and comparison of face detection algorithms". In: In-
ternational Conference on Cloud Computing, Data Science & Engineering-Confluence, 5:
629– 633, 2017.

[18] N. Dalal and B. Triggs. "Histograms of oriented gradients for human detection". IEEE
computer society co f r mpu r s p r r g (CVPR’05), 1: 886–
893, 2005.

[19] S. C. Bakchy, M. J. Ferdous, A. H. Sathi, K. C. Ray F. Imran and M. M Ali. "Facial

expression recognition based on support vector machine using Gabor wavelet filter".
International Conference on Electrical & Electronic Engineering (ICEEE), 4: 1–4, 2017.

[20] B. Adil, K. M. Nadjib and L. Yacine (2019). "A novel approach for facial expression
recognition". International Conference on Networking and Advanced Systems (ICNAS), 2:
1–5, 2019.

[21] Khan Hamayun. "Feature Fusion and Classifier Ensemble Technique for Robust Face
Recognition". International Journal of Image Processing (IJIP), 11: 1-15, 2017.

[22] CH. Hima Bindu, K. Manjunathachari. "An Efficient Face Recognition Using Multi-Kernel
Based Scale Invariant Feature Transform". International Journal of Image Processing (IJIP),
12: 39-47, 2018.

[23] A. Neyaz, A. Kumar, S. Krishnan, J. Placker and Q. Liu. "MSB based Face Recognition
Using Compression and Dual Matching Techniques". International Journal of Image
Processing (IJIP), 13: 62-75, 2019.

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 20
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php
Safa Rajaa, Rafika Harrabi & Slim Ben Chaabane

[24] K. Abdelraouf ElDahshan, E. K. Elsayed, A. Aboshosha and E. Ali Ebeid. "Feature selection
for face authentication systems: Feature Space Reductionism and QPSO.". Int. J.
Biometrics, 11: 328--341, 2019.

[25] K. A. El Dahshan, E. K. Elsayed, A. Aboshoha and E. A. Ebeid . "Recognition of Facial

Emotions relying on Deep Belief Networks and Quantum Particle Swarm Optimization".
International Journal of Intelligent Engineering and Systems, 13: 90-101, 2020.

International Journal of Image Processing (IJIP), Volume (15) : Issue (2) : 2021 21
ISSN: 1985-2304, https://www.cscjournals.org/journals/IJIP/description.php