Affective Analysis in Machine Learning Using AMIGOS With Gaussian Expectation-Maximization Model
Affective Analysis in Machine Learning Using AMIGOS With Gaussian Expectation-Maximization Model
Corresponding Author:
Bakkialakshmi Vaithialingam Sudalaiyadumperumal
Department of Computing Technologies, SRM Institute of Science and Technology
SRM Nagar, Kattankulathur, Chengalpattu District, Tamil Nadu 603203, India
Email: [email protected]
1. INTRODUCTION
The field of sociology has a rich tradition in studying emotions, particularly in the context of devising
viral questions and self-assessment assessments aimed at detecting and understanding human emotional
responses. Human personality predictions with human emotions in real-time scenarios impact the research
industry more. Emotions may be conscious or unconscious; human behavior directly impacts behavior.
Affective computing, influenced by psychological factors, is a difficult field of study with different ideological
paths. Audio signals can accurately depict the emotional impact because pitch variations define the emotional
element. Numerous cross-modal emotion embedding systems employ audio and video correlations inside the
ensemble learning framework to ascertain the genuine emotion elicited by the individual [1].
Long-term mental diseases like depression and anxiety start mildly and gradually affect people's
emotions. Affect sensing is a large and challenging field of study. It is unavoidable to mention the term
"emotion contagion" when explaining the effect scenario. Emotional contagion is a social contagion in which
one person's emotions and behaviors spread to another [2]. The emotional reflection from one person to another
in a specific circumstance may occur. Actions caused emotions in some cases. When inconvenient, people can
act differently based on how they respond to specific conditions [3].
With the development of artificial intelligence technology, it is now possible to do significant
interactive analysis to understand people's emotions from various perspectives. Standardized datasets are
accessible for research purposes, and the study uses diverse publicly available data. The determination of
emotional affects is achieved by the analysis of speech cues. Pitch and tone alterations serve as evident
indications of emotional transitions or notifications of mood. Neuro-fuzzy logic-based resilience evaluations
are employed to differentiate speech patterns that modify emotional impact [3]. The most obvious expression
of psychological impacts is found on one's face. Expressions are a universally important concept. The mood
can be read from the person's expressions.
The identification of concealed emotions through facial expressions can be challenging under some
circumstances. Another method of expression formulation is to use virtual face vectors and landmark
extractions [4]. By manipulating current neural network challenges, machine learning technology can enhance
algorithms. Using linear discriminant analysis (LDA) models, the extensive collection of feature vectors
acquired from subject analysis data is investigated. Emotion analysis outcomes improve with LDA as trials
increase [5]. The algorithm is proposed for multi-label learning to aid in investigating emotional effects in
several modalities. Diverse modality scenarios allow the emotional effect components to be investigated from
several perspectives regarding the primary polarity of happy and sad emotions [6].
The genuine depiction of the brain's responsiveness to an input is emotion. It expresses the innate
sense present in the predicament. Based on dimensionality, there are two categories of emotion models: 2D is
meant to be two-dimensional models, and 3D is meant to be three-dimensional models. The valance and arousal
dimensions are where the 2D model's most potent emotions can be located. On the other hand, the 3D model
holds valid emotions like valence, arousal dominance, and so forth [7]. The existing research, its drawbacks,
and proposed future research are all summarised in section 2. Studies are underway in the background to
advance section 3. The model selection and design constraint analysis are explained in section 4. The strategy,
data collection, and proposed method are all described in depth in section 5. Challenges faced with the work
and future challenges are addressed in section 6, describing the outcomes and follow-up conversations as
conclusions in section 7.
2. BACKGROUND STUDY
Li et al. [8] multiple polarities concerning emotions are detected using a multi-step enabled deep
emotion detection framework. Deep neural networks (DNN) are used to extract movies and physiological
information from publicly available databases (DNN). Pattern comparison is conducted to analyse and evaluate
the training and testing properties.
Hoang et al. [9] studied detection posture, face, and detection, which is evaluated with a mainstream
multi-task cascaded neural network model using a virtual semantic module. The extraction of the reasoning
stream is accomplished by the utilisation of a multi-level perceptron (MLP). The utilisation of the EXOTIC
dataset, which incorporates simulated heat stream patterns, enhances the efficacy of the detection technique.
Islam et al. [10] stated that emotion detection is a method for identifying and extending a person's
emotional state. Upon the basis of in-depth and surface-level learning, the detection and evaluation of irrational
emotions is put into practice. The coupling of electrocardiogram (ECG) and electroencephalogram (EEG) data
is utilised to showcase the interconnectedness in the context of emotion identification. The PRISMA technique
is employed for comprehensive analysis, encompassing the processes of identification, screening, and
eligibility assessment.
Albraikan et al. [11] present study showcases a system that utilises the MAHNOB dataset and the K-
nearest neighbour method for the purpose of analysing emotion through the application of weighted multi-
dimensional discrete wavelet transform (DWT). Following a series of training rounds, a meta classifier utilises
a combination of video clips depicting various emotions to determine the ultimate emotional affect. The present
study used MWDWT simulations to identify and delineate nine distinct emotional states. The user's text does
not contain any information to rewrite in an academic manner [11].
Qayyum et al. [12] explained a method of emotion recognition via an Android application is offered
due to the prevalence of mobile use. Convolutional neural networks (CNN) and recurrent neural networks
(RNN) are combined to generate a powerful model for emotion detection. CNN and RNN have both accuracy
rates of 65% and 41%, respectively. The mentioned recommendation mechanism is used for fresh content.
Bakkialakshmi and Sudalaimuthu [13] in a self-supervised learning system, unlabeled data are
converted to bias weights based on iterative learning and used return updates. The hardware sensors collected
with diverse temporal features assess an ECG-based emotion identification system. The maximum accuracy
achieved using the typical emotion datasets of AMIGOS, SWELL, WESAD, and DREAMER was 97%.
Int J Reconfigurable & Embedded Syst, Vol. 13, No. 1, March 2024: 201-209
Int J Reconfigurable & Embedded Syst ISSN: 2089-4864 203
3. METHOD
3.1. Data collection
The AMIGOS data collection serves as a widely used resource for conducting research on emotional
personality and mood. It has data pertaining to both individuals and groups, which have been annotated
externally and characterised by their personality profiles. Neurophysiological recordings from the subject
during the exam include ECG, EEG, and galvanic skin response (GSR) signals [19]. The volunteers are shown
short and long video experimental movies during the test. Forty volunteers saw 16 successful movies that
elicited feelings such as valence, arousal dominance, familiarity, and like in the brain. The viewers experience
a range of fundamental emotions when watching the videos, encompassing neutrality, happiness, sadness,
surprise, fear, anger, and contempt. The determination of mood assessment should be made by considering the
available information and evaluating the patients' kinematics. AMIGOS is a well-documented dataset that has
passed a self-assessment exam. The AMIGOS dataset considers ECG, EEG, and GSR signals in the proposed
system.
Affective analysis in machine learning using AMIGOS with Gaussian … (Balamurugan Kaliappan)
204 ISSN: 2089-4864
1 𝑥−𝜇
1
(𝑥) = 𝑒−2( 𝜆
)
(1)
𝜆√2𝜋
Where,
−𝛼<𝑥<𝛼,
𝜆→ 𝑉𝑎𝑟𝑖𝑒𝑛𝑐𝑒,
𝜇→ 𝑀𝑒𝑎𝑛 𝑜𝑓 𝑡ℎ𝑒 𝑝𝑜𝑝𝑢𝑙𝑎𝑛𝑎𝑡𝑖𝑜n
4. SYSTEM ARCHITECTURE
4.1. Design architecture
Figure 1 demonstrates the system architecture and analysis of the proposed GEM model, built on the
normal probability distribution. The input data include preprocessed physiological signal records, including
ECG, EEG, and GSR [22]. The emotional effect that causes the variances makes these patterns distinctive.
According to the stated test record, the impact point is prohibited. Only the correlated points may be obtained
using the normal distribution of random data of overall physiological information.
Int J Reconfigurable & Embedded Syst, Vol. 13, No. 1, March 2024: 201-209
Int J Reconfigurable & Embedded Syst ISSN: 2089-4864 205
The entire procedure simplifies extracting unique points from a vast collection. The system can learn
and execute fresh data searches iteratively thanks to the discrepancy between the expected value and the highest
value found. The technique can complete the provided test data in no more than 500 million seconds. The
working model for the first maximization value achieved is formed during the training procedure. Table 1
shows for the specified test sample, the number of iterations ranged from 0.00 after the ninth iteration to
3,548.8753 at the beginning of the random selection of the starting location. It evaluates the statistical data
points about the mean [26].
The Figure 4 shows the graphical representation of population search for a particular test sample over
numerous iterations. The visualization effect is shown in the visualization iterations vs. error rate (e), mean (u)
estimation analysis. Table 2 shows about the obtained emotion label, the overall parameter measurement with
parameter measurements such as are determined.
Figure 5 demonstrates the classification of emotional impact for the given test samples. Anger,
contempt, disgust, happiness, and normal are the four categories into which the test data is divided in the
proposed model. The statistical measurements variance (λ), standard deviation (SD), error rate (e), and mean
(u) aid in classifying the various variables. Table 3 shows the comparison table of existing implementation on
emotional affect detection with proposed GEM algorithm performance and analysis.
Figure 4. Visualization iterations vs. error rate (e) and mean (u) estimation
Int J Reconfigurable & Embedded Syst, Vol. 13, No. 1, March 2024: 201-209
Int J Reconfigurable & Embedded Syst ISSN: 2089-4864 207
14
Varience(λ), Standard
12
10
Deviation(∑)
8
6
4
2
0
0 5 10 15 20 25 30
Classified Emotions
Lamda (Varience) Sigma 1(SD)
6. CHALLENGES
The presented work's key problem is dealing with large amounts of data and the processing delay
required for training and testing. The GEM model uses probabilistic distribution and similarity mapping to
determine the relative convergence of grouped data. To scale the data before processing, the system model
should focus on improving the preprocessing stage and feature extraction procedures.
7. CONCLUSION
The GEM algorithm is used to assess emotion identification. The AMIGOS dataset is taken into
consideration for analysis. Analyses are done on physiological signals like the ECG, EEG, and GSR. The
proposed research project focuses on in-depth investigation and the creation of a simple model for emotion
Affective analysis in machine learning using AMIGOS with Gaussian … (Balamurugan Kaliappan)
208 ISSN: 2089-4864
analysis that results in shorter latency. Participants are randomly selected and assessed using data covariance
analysis for ECG, EEG, and GSR along with Emo-GEM and GEM models based on data regression. There are
more distinct correlation points produced to determine emotions the stronger the processing depth-wise
convergence, which leads to data size equality. With 0% error to the maximum iterations, the suggested model
yields a latency of around 438 million second for overall processing. The statistical measurements for detecting
emotions such as anger, contempt, disgust, happiness, and normal are emphasized as mean=0.60, and SD=0.80.
To uncover the detailed variances, the system must also be categorized using a deep learning model.
REFERENCES
[1] J. Han, Z. Zhang, Z. Ren, and B. Schuller, “EmoBed: strengthening monomodal emotion recognition via training with crossmodal
emotion embeddings,” IEEE Transactions on Affective Computing, vol. 12, no. 3, pp. 553–564, Jul. 2021, doi:
10.1109/TAFFC.2019.2928297.
[2] J. Arroyo-Palacios and M. Slater, “Dancing with Physio: a mobile game with physiologically aware virtual humans,” IEEE
Transactions on Affective Computing, vol. 7, no. 4, pp. 326–336, Oct. 2016, doi: 10.1109/TAFFC.2015.2472013.
[3] K. Soltani and R. N. Ainon, “Speech emotion detection based on neural networks,” in 2007 9th International Symposium on Signal
Processing and Its Applications, Feb. 2007, pp. 1–3, doi: 10.1109/ISSPA.2007.4555476.
[4] G. Yang, J. S. Y. Ortoneda, and J. Saniie, “Emotion recognition using deep neural network with vectorized facial features,” in 2018
IEEE International Conference on Electro/Information Technology (EIT), May 2018, pp. 0318–0322, doi:
10.1109/EIT.2018.8500080.
[5] E. Kroupi, J.-M. Vesin, and T. Ebrahimi, “Subject-independent odor pleasantness classification using brain and peripheral signals,”
IEEE Transactions on Affective Computing, vol. 7, no. 4, pp. 422–434, Oct. 2016, doi: 10.1109/TAFFC.2015.2496310.
[6] V. S. Bakkialakshmi and T. Sudalaimuthu, “A survey on affective computing for psychological emotion recognition,” in 2021 5th
International Conference on Electrical, Electronics, Communication, Computer Technologies and Optimization Techniques
(ICEECCOT), Dec. 2021, pp. 480–486, doi: 10.1109/ICEECCOT52851.2021.9707947.
[7] X. Zhang, W. Li, H. Ying, F. Li, S. Tang, and S. Lu, “Emotion detection in online social networks: a multilabel learning approach,”
IEEE Internet of Things Journal, vol. 7, no. 9, pp. 8133–8143, Sep. 2020, doi: 10.1109/JIOT.2020.3004376.
[8] M. Li, L. Xie, Z. Lv, J. Li, and Z. Wang, “Multistep deep system for multimodal emotion detection with invalid data in the internet
of things,” IEEE Access, vol. 8, pp. 187208–187221, 2020, doi: 10.1109/ACCESS.2020.3029288.
[9] M.-H. Hoang, S.-H. Kim, H.-J. Yang, and G.-S. Lee, “Context-aware emotion recognition based on visual relationship detection,”
IEEE Access, vol. 9, pp. 90465–90474, 2021, doi: 10.1109/ACCESS.2021.3091169.
[10] M. R. Islam et al., “Emotion recognition from EEG signal focusing on deep learning and shallow learning techniques,” IEEE Access,
vol. 9, pp. 94601–94624, 2021, doi: 10.1109/ACCESS.2021.3091487.
[11] A. Albraikan, D. P. Tobon, and A. El Saddik, “Toward user-independent emotion recognition using physiological signals,” IEEE
Sensors Journal, vol. 19, no. 19, pp. 8402–8412, Oct. 2019, doi: 10.1109/JSEN.2018.2867221.
[12] R. Qayyum et al., “Android based emotion detection using convolutions neural networks,” in 2021 International Conference on
Computational Intelligence and Knowledge Economy (ICCIKE), 2021, pp. 360–365, doi: 10.1109/ICCIKE51210.2021.9410768.
[13] V. S. Bakkialakshmi and T. Sudalaimuthu, “Emo-Net artificial neural network: a robust affective computing prediction system for
emotional psychology using AMIGOS,” Indian Journal of Computer Science and Engineering, vol. 13, no. 4, pp. 1040–1055, Aug.
2022, doi: 10.21817/indjcse/2022/v13i4/221304034.
[14] P. Sarkar and A. Etemad, “Self-supervised ECG representation learning for emotion recognition,” IEEE Transactions on Affective
Computing, vol. 13, no. 3, pp. 1541–1554, Jul. 2022, doi: 10.1109/TAFFC.2020.3014842.
[15] L. Song and W. Luo, “Self-supervised learning of visual odometry,” in 2020 International Conference on Information Science,
Parallel and Distributed Systems (ISPDS), Aug. 2020, pp. 5–9, doi: 10.1109/ISPDS51347.2020.00009.
[16] V. S. Bakkialakshm and T. Sudalaimuthu, “Dynamic cat-boost enabled keystroke analysis for user stress level detection,” in 2022
International Conference on Computational Intelligence and Sustainable Engineering Solutions (CISES), May 2022, pp. 556–560,
doi: 10.1109/CISES54857.2022.9844331.
[17] H. Ranganathan, S. Chakraborty, and S. Panchanathan, “Multimodal emotion recognition using deep learning architectures,” in
2016 IEEE Winter Conference on Applications of Computer Vision (WACV), Mar. 2016, pp. 1–9, doi:
10.1109/WACV.2016.7477679.
[18] S. Mishra, S. Kumar, S. Gautam, and J. Kour, “Real time expression detection of multiple faces using deep learning,” in 2021
International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), Mar. 2021, pp. 537–
542, doi: 10.1109/ICACITE51222.2021.9404561.
[19] H. Wan, H. Wang, B. Scotney, and J. Liu, “A novel gaussian mixture model for classification,” in 2019 IEEE International
Conference on Systems, Man and Cybernetics (SMC), Oct. 2019, pp. 3298–3303, doi: 10.1109/SMC.2019.8914215.
[20] P. Barros, N. Churamani, E. Lakomkin, H. Siqueira, A. Sutherland, and S. Wermter, “The OMG-emotion behavior dataset,” in 2018
International Joint Conference on Neural Networks (IJCNN), Jul. 2018, pp. 1–7, doi: 10.1109/IJCNN.2018.8489099.
[21] V. S. Bakkialakshmi, T. Sudalaimuthu, and B. Umamaheswari, “Emo-Spots: detection and analysis of emotional attributes through
bio-inspired facial landmarks,” in International Conference on IoT, Intelligent Computing and Security. Lecture Notes in Electrical
Engineering, Singapore: Springer, 2023, pp. 103–115, doi: 10.1007/978-981-19-8136-4_9.
[22] W. M. B. Henia and Z. Lachiri, “Emotion classification in arousal-valence dimension using discrete affective keywords tagging,”
in 2017 International Conference on Engineering and MIS (ICEMIS), May 2017, pp. 1–6, doi: 10.1109/ICEMIS.2017.8272991.
[23] V. S. Bakkialakshmi, T. Sudalaimuthu, and S. Winkler, “Effective prediction system for affective computing on emotional
psychology with artificial neural network,” EasyChair Preprint. 2022.
[24] Y. Baveye, C. Chamaret, E. Dellandrea, and L. Chen, “Affective video content analysis: a multidisciplinary insight,” IEEE
Transactions on Affective Computing, vol. 9, no. 4, pp. 396–409, Oct. 2018, doi: 10.1109/TAFFC.2017.2661284.
[25] D. Nikolova, P. Georgieva, P. Petkova, and A. Manolova, “ECG-based emotion recognition: overview of methods and applications,”
in ANNA 2018 - Advances in Neural Networks and Applications 2018, 2018, pp. 118–122.
[26] K. Maehara and K. Fujinami, “Psychological effects on positional relationships between a person and a human-following robot,”
in 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), Aug.
2018, pp. 242–243, doi: 10.1109/RTCSA.2018.00041.
Int J Reconfigurable & Embedded Syst, Vol. 13, No. 1, March 2024: 201-209
Int J Reconfigurable & Embedded Syst ISSN: 2089-4864 209
BIOGRAPHIES OF AUTHORS
Affective analysis in machine learning using AMIGOS with Gaussian … (Balamurugan Kaliappan)