19CS71C-RESEARCH PAPER AND PATENT REVIEW
Title: Emotion Detection Using Deep Learning
Reg No : 2112033
Name : G. Abishek
Domain: Deep Learning
Problem Statement: The challenge of accurately recognizing and interpreting human facial
emotions from various angles and perspectives poses significant difficulties in the fields of medical
research and computer science. Traditional methods in facial emotion recognition (FER) rely on
feature extraction and feature selection using Convolutional Neural Networks (CNNs). However,
these methods often struggle with images that are not captured from a frontal view, limiting their
effectiveness in real-world applications. This limitation is particularly critical for enhancing
patient care and supporting children with speech impairments, where accurate emotion detection
can provide essential psychological insights.The proposed research aims to develop an advanced
facial emotion recognition model utilizing a Bi-Directional Long Short-Term Memory (Bi-LSTM)
network. This model seeks to overcome the limitations of conventional CNN-based approaches by
accurately recognizing facial emotions from images captured at various angles. The primary
objective is to create a robust system that can identify and interpret a wide range of emotional
states, enhancing its application in medical and social contexts.
No. of Papers reviewed: 2
Paper 1:
Title : Deep Learning based Emotion detection in an Online Class
Authors: Prabhjot Kaur, Aditya Kapoor, Yash Solanki, Piyush Singh, Devansh Sehgal
Publisher: IEEE
Date of Publication: May 2022
Proposed work: This paper proposes a model for detecting students' emotions during online
classes using deep learning techniques. The model leverages a Convolutional Neural Network
(CNN) trained with the publicly available "Jonathan Oheix" dataset from Kaggle, which consists
of images classified into seven emotional states (happy, angry, disgust, fear, sad, neutral, surprise).
The model is designed to work in real-time to identify and categorize student emotions, thus
enabling educators to adjust their teaching strategies dynamically. The goal is to enhance the
effectiveness of online education by understanding students' emotional states..
Paper 2:
Title: Pre-trained Deep Learning Models for Facial Emotions Recognition
Authors: Atanas Atanassov, Dimitar Pilev
Publisher:IEEE
Date of Publication: 2020
Proposed work: This study conducts a comparative analysis of various pre-trained deep
learning models, such as VGG16, VGG19, DeepFace, and others, for offline facial
emotion recognition. The goal is to select lightweight models suitable for recognizing
facial emotions during students' semester studies. The paper evaluates these models
based on their accuracy and computational efficiency to determine the best-suited model
for analyzing students' emotions during learning sessions. This evaluation aims to
personalize and adapt lecture material based on students' emotional responses.
Comparative Analysis:
Content Paper 1 Paper 2
Objective Real-time emotion detection in online Evaluate and select pre-trained deep
classes to improve teaching learning models for offline facial emotion
effectiveness. recognition.
Dataset Used "Jonathan Oheix" dataset from Various datasets like FER-2013, CK+,
Kaggle, containing images labeled etc., with different characteristics and
with seven emotional states. labeled emotions.
Technique used The study uses SVM for classification Hybrid approach combining decision
and regression, ANN for modeling trees, neural networks, and ensemble
complex relationships, and TLBO for methods.
feature selection and optimizing ANN
architecture.
Modules 1. Data Preprocessing: Data 1. Model Selection: Choose pre-trained
description extraction, cleaning, and formatting. models like VGG16, VGG19, and
2. Image Augmentation: Generate DeepFace.
more data using techniques like 2. Dataset Analysis: Evaluate datasets
rotation, padding. for model training and validation.
3. Feature Extraction: Numerical 3. Performance Evaluation: Measure
value combination to reduce data for model accuracy on different datasets.
computation. 4. Model Optimization: Fine-tune
4. Model Training: Train using models based on accuracy and
CNN architecture with layers performance metrics.
optimized using ReLU activation
function.
Performance 1. Accuracy: Measures the model's 1. Accuracy: Evaluate test accuracy for
Measure correct emotion classifications. different models.
2. Validation Accuracy: Assess 2. Confusion Matrix: Analyze the
model performance over 11 epochs misclassification rates for emotions.
with early stopping.
Future work 1.Enhancing Model Complexity: 1. Further Model Testing: Validate
Investigate additional hybrid models models on additional datasets.
or advanced optimization techniques 2. Explore Lightweight Models:
to further improve prediction Identify models with lower
accuracy. computational costs for broader
2.Expanding Data Sources: application.
Incorporate more diverse data sets or 3. Improve Model Accuracy: Refine
additional features to enrich the model models to enhance emotion recognition
and increase generalizability. performance.
3.Ethical Considerations: Develop
frameworks to address ethical issues
related to data privacy and bias in
predictive analytics.
4.Real-World Application: Test the
developed models in different
educational contexts to validate their
practical effectiveness and
applicability.
Conclusion:
• Paper 1 is focused on developing a model specifically for real-time emotion detection in
an online education context, with the goal of improving the learning experience.
• Paper 2 provides a broader evaluation of pre-trained models for offline use, aiming to find
the most effective solution for recognizing facial emotions to personalize learning.
Both papers contribute valuable insights to the field of emotion detection, with Paper 1
emphasizing real-time application in online learning and Paper 2 focusing on model evaluation
and optimization for offline use.
Project Guide Course Instructor
