EMOTION DETECTION USING INTEGRATION OF DEEP

LEARNING AND SVM

Rupali J.Dhabarde1, Dr. D.V. Kodavade2, Dr. Sheetal Zalte3, Aditya Konnur4
Department of Technology, Shivaji University, Kolhapur1,4 , DKTE society’s Textile & Engineering
Institute Ichalkaranji2, Department of Computer Science, Shivaji University, Kolhapur.4
1
[email protected] , [email protected]
, [email protected], [email protected]

Abstract— Emotion detection from facial combines the powerful feature extraction
expressions is a challenging yet crucial task in capabilities of the RESNET50 CNN with the
various fields including human-computer discriminative classification ability of the
interaction, healthcare, and psychology. In Support Vector Machine (SVM) classifier for
this paper, we propose a novel approach for emotion detection.
emotion detection by integrating the
RESNET50 deep convolutional neural II. LITERATURE REVIEW
network (CNN) architecture with a Support Deep learning algorithms offer a
Vector Machine (SVM) classifier. The promising avenue for exploring the automated
RESNET50 CNN is employed for feature extraction of complex data representations,
extraction from facial images, capturing particularly at higher levels of abstraction. These
hierarchical representations, while SVM is algorithms employ a layered and hierarchical
utilized for effective classification. architecture that enables the learning and
Experimental results on benchmark datasets representation of data. In this architecture,
demonstrate the effectiveness of the proposed higher-level features are defined based on the
approach in accurately recognizing emotions lower-level features, promoting a progressive
from facial expressions, outperforming abstraction of information [4].
existing methods in terms of classification By leveraging this layered approach,
accuracy and robustness. Deep Learning algorithms excel at capturing
Index Terms—Emotion detection, intricate patterns and structures within data. The
RESNET50, Support Vector Machine (SVM), initial layers of the network learn simple and
deep learning, machine learning. local features, while subsequent layers build
upon these representations to extract more
I. INTRODUCTION sophisticated and abstract features. This
Emotion detection from facial expressions plays hierarchical arrangement enables the system to
a vital role in numerous applications such as automatically uncover increasingly complex and
human-computer interaction, affective meaningful representations of the input data [5].
computing, and mental health diagnosis. With The ability of Deep Learning algorithms
the advancements in deep learning techniques, to learn and represent data in this hierarchical
convolutional neural networks (CNNs) have manner has proven highly effective in various
shown remarkable performance in various domains. Applications such as computer vision,
computer vision tasks, including facial natural language processing, speech recognition,
expression recognition. However, traditional and many others have witnessed significant
CNNs often require large amounts of labeled advancements due to the capacity of Deep
data for training and may suffer from overfitting. Learning models to extract abstract and
In this paper, we propose a hybrid approach that discriminative features from raw data [6].

ISSN (PRINT): 2393-8374, (ONLINE): 2394-0697, VOLUME-11, ISSUE-2, 2024

18
 INTERNATIONAL JOURNAL OF CURRENT ENGINEERING AND SCIENTIFIC RESEARCH (IJCESR)
The layered and hierarchical nature of examples that cover various emotions. The
Deep Learning algorithms enables them to dataset contains two directories, one for thermal
handle the complexities of real-world data, images and other for normal/visual images of
leading to improved performance in tasks such which we have used the thermal directory. The
as classification, regression, and generation. By datasets consist of 2538 labeled thermal facial
automatically learning high-level abstractions images corresponding to different emotional
from raw input, these algorithms offer a expressions such as happiness, sadness, anger,
powerful tool for solving complex problems and fear, surprise, and disgust. We compare the
advancing our understanding of data in a more performance of our method with accuracy,
interpretable and generalizable manner [7]. precision, recall, and F1-score.
Deep Learning algorithms offer
significant advantages when it comes to B. Implementation Details
handling extensive volumes of unsupervised
data, as they typically adopt a greedy layer-wise
approach to learning data representations [8].

III. METHODOLOGY

(a)

Fig. 1 Proposed Methodology

The proposed research work is mainly

focused on development of an improved deep
learning algorithm for processing of thermal
images which will overcome drawbacks of
previous approaches. It should actually works
by image acquisition from the dataset. And then
preprocessed to remove unwanted information,
then feature extraction will be carried out by for
capturing the patterns of each emotions. Then by (b)
using RESNET50 algorithm for processing
thermal image data in combination with any
classification algorithm will be applied. At last
trained the modified deep learning model on
preprocessed thermal image datasets, with
cross-validation to assess model performance on
validation datasets to detect different emotions.

IV. EXPERIMENTAL SETUP

A. Dataset
We evaluate the proposed approach on dataset
for facial expression recognition, the dataset
used in this research is taken from Kaggle (Chia
theo nguoi_KTFEV2-7 emotions) which is (c)
carefully curated to include a diverse range of

ISSN (PRINT): 2393-8374, (ONLINE): 2394-0697, VOLUME-11, ISSUE-2, 2024

19
 INTERNATIONAL JOURNAL OF CURRENT ENGINEERING AND SCIENTIFIC RESEARCH (IJCESR)
These evaluation metrics provide the details of
model's performance for each emotion category
by using an integrated approach. High precision
values indicate accurate predictions, high recall
values signify the model's ability to capture
instances of a particular emotion, and the F1
score provides an overall assessment of the
model's performance by considering both
precision and recall.
B. Accuracy
(d)
Fig. 2 prediction performance of model
(a) Anger (b) Happy (c) Fear (d) Surprise
Fig. 2 shows the prediction performance of the
Resnet50_SVM model through the predictions
of sample images. Fig. 2 (a) shows the sample
image of anger which has been predicted
correctly as anger. Fig. 2 (b) shows the sample
image of happy which has been predicted
correctly as happy. Fig. 2 (c) shows the sample
image of fear which has been predicted correctly Fig. 4 Accuracy of the proposed model to
as fear. Fig. 2 (d) shows the sample image of detect different emotions
surprise which has been predicted incorrectly as Fig. 4 is the graphical representation of the
fear. percentage of accuracy of the model to detect
V. RESULTS AND DISCUSSION and classify different emotions.
Experimental results demonstrate that the
proposed approach achieves superior VI. CONCLUSION
performance compared to existing methods in In this paper, we proposed an approach for
terms of accuracy and robustness. An integrated emotion detection by integrating the RESNET50
model RESNET50 for feature extraction and CNN architecture with a Support Vector
SVM for classification effectively captures Machine (SVM) classifier. The experimental
discriminative features from facial images and results using this approach have demonstrated
accurately recognizes emotions. This section promising performance i.e. 87.23% training
presents the results of the experiments conducted accuracy and 82% test accuracy. Our findings
using the proposed RESNET50-SVM highlight the potential of an integrating deep
framework. It includes performance evaluation learning and traditional machine learning
metrics such as accuracy, precision, recall, and techniques for detecting emotions.
F1-score.
A. Accuracy, Precision, Recall and F1 Score
REFERENCES
[1] “Recognition of facial expressions of
1.5 RESNET50_SVM Infrared images for lie detection with the use
of Support Vector Machine” Book on
1
Synergistic Interaction of Big Data with
0.5 Cloud Computing for Industry 4.0 CRC
Press, Taylor and Francis Publisher, USA
0 [2] K. Reese, Y. Zheng, and A. Elmaghraby, "A
comparison of face detection algorithms in
visible and thermal spectrums," in Intl Conf.
Precision recall F1 Score on Advances in Computer Science and
Fig. 3 Precision, Recall and F1-Score of an Application, 2012.
integrated Model

ISSN (PRINT): 2393-8374, (ONLINE): 2394-0697, VOLUME-11, ISSUE-2, 2024

20
 INTERNATIONAL JOURNAL OF CURRENT ENGINEERING AND SCIENTIFIC RESEARCH (IJCESR)
[3] Bengio Y, LeCun Y: Scaling learning
algorithms towards, AI.In Large Scale
Kernel Machines Edited by: Bottou L,
Chapelle O, DeCoste D, Weston J. MIT
Press, Cambridge, MA; 2007, 321–360.
[4] Bengio Y, Courville A, Vincent P:
Representation learning: A review and new
perspectives. Pattern Analysis and Machine
Intelligence, IEEE Transactions on
2013,35(8):1798–1828.
doi:10.1109/TPAMI.2013.50
doi:10.1109/TPAMI.2013.50
10.1109/TPAMI.2013.50.
[5] Arel I, Rose DC, Karnowski TP: Deep
machine learning-a new frontier in artificial
intelligence research [research frontier].
IEEE Comput Intell 2010, 5: 13–18.
10.1109/MCI.2010.938364.
[6] Hinton GE, Osindero S, Teh Y-W: A fast
learning algorithm for deep belief nets.
Neural Comput 2006,18(7):1527–1554.
10.1162/neco.2006.18.7.1527
[7] Bengio Y, Lamblin P, Popovici D,
Larochelle H2007. Greedy layer-wise
training of deep networks, Vol. 19. Bengio
Y, Lamblin P, Popovici D, Larochelle
H2007. Greedy layer-wise training of deep
networks, Vol. 19.
[8] A. Mostafa, M. I. Khalil, and H. Abbas,
"Emotion recognition by facial features
using recurrent neural networks," in IEEE
International Conference on Computer
Engineering and Systems, pp. 417-422,
2008.
[9] A. Prabhakaran, J. Nair, and S. Sarath,
"Thermal facial expression recognition using
modified resnet152," in Advances in
Computing and Network Communications,
pp. 389-396, 2021.
[10] “Hybrid Machine Learning Model for
Lie-Detection” IEEE 8th International
Conference on Convergence of Technology
(I2CT) India, 07th-09th April 2023.
[11]“Amalgamation of SVM And RestNet50 for
Deception Detection” International
Conference on Futuristic Trends in
Information Technology (ICFTIT-2023) on
29thApril, 2023.

ISSN (PRINT): 2393-8374, (ONLINE): 2394-0697, VOLUME-11, ISSUE-2, 2024

21