Electroencephalogram (EEG) Based

Emotion Recognition by Distinguishing

EEG Segments Having High Affective
Content From Appropriate Channels
at Scalp Level
Thesis submitted to
Indian Institute of Information Technology Guwahati
For award of the degree
of

Doctor of Philosophy
by

Parthana Sarma
Roll Number: 1703012

Under the supervision of

Dr. Shovan Barma

Department of Electronics and Communication Engineering

Indian Institute of Information Technology Guwahati
Assam-781015, India

©2021 Parthana Sarma. All rights reserved.

March 2022
 5

To my parents with love and respect ...

 Declaration

I certify that

a. The work contained in this thesis is original and has been done by me
under the guidance of my supervisor.
b. The work has not been submitted to any other Institute for any degree
or diploma.
c. I have followed the guidelines provided by the Institute in preparing the
thesis.
d. I have conformed to the norms and guidelines given in the Ethical Code
of Conduct of the Institute.
e. Whenever I have used materials (data, theoretical analysis, figures, and
text) from other sources, I have given due credit to them by citing them
in the text of the thesis and provided their details in the references.

Parthana Sarma
 Certificate

This is to certify that the thesis entitled “Electroencephalogram (EEG) Based

Emotion Recognition by Distinguishing EEG Segments Having High Af-
fective Content From Appropriate Channels at Scalp Level” submitted by
Ms. Parthana Sarma to Indian Institute of Information Technology Guwahati,
is a record of bona fide research work carried under my supervision and I consider
it worthy of consideration for the award of the degree of Doctor of Philosophy of
the Institute.

Dr. Shovan Barma

Associate Professor,
Department of Electronics and Communication Engineering,
Indian Institute of Information Technology Guwahati,
Assam-781015, India.
 Acknowledgments

At the very outset, I would like to express my genuine appreciation to my su-

pervisor, Dr. Shovan Barma, for his immense patience, valuable suggestion and
inspiring instruction to my Ph.D. study in this institute. His profound knowledge,
insightful comments as well as practical advice helped me a lot in writing this
thesis and published papers. He has been a true guide and a friend all along this
journey, helping me to “think” and explaining to me what actually research entails
and how it can be employed to improve people’s life.
Secondly, I would like to show my special thanks to the distinguished professors
at our institute including members of my Doctoral Committee, Dr. Rusha Patra,
Dr. Surajit Panja and Dr. Suranjana Barua. The experience and knowledge I
gained from them make me feel enthusiastic to complete this thesis.
The journey to my Ph.D. is about doing scientific research individually. However,
my work would be difficult without help from my colleagues including Sumona
Biswas and Anakhi Hazarika. Without their constant mental support and help
I would not have been able to complete my work. I also thank all the past
and current Ph.D. members of IIITG, including Anand, Ankita, Nikumani, Anir-
ban, Shounak, Sultan, Shekhar, Jitumoni, Richa, Dimple, Kalyani, Mridul, Arjun,
Mehbub, Indrajit, Arijit, Sourav, Rima, Bondita, Nirmali and so on. It was a
wonderful experience to have studied and worked alongside them.
I would like to express my heartfelt gratitude to the Government of India funded
scheme under Technical Education Quality Improvement Programme (TEQIP).
Without their financial support, I could not have realized my objectives in the
academic field.
Lastly and most significantly, I want to extend my gratitude to my family. I am
thankful to my parents (Dr. Biraj Sarma and Mrs. Santana Sarma), elder sister
(Dr. Upasana Sarma), husband (Mr. Gitartha Dev Sarma, Assam Police Service),
brother-in-law (Mr. Manash Kalita) and grandparents, for their understanding,
support and care. Thank you for always being my pillars of support and for pro-
viding me with the much needed mental strength and spiritual guidance through
this time. Particularly, I want to express my deepest gratitude to my husband, for
his continual encouragement, selfless giving and endless love for my life. I am also
thankful to my in laws (Mr. Hitesh Dev Sarma (IAS), Mrs. Sandeepa Sarma and
Ms. Dikshita Sarma) for their understanding of my work and their constant love
and assistance.
Finally, I am grateful towards God whose love and blessings have aided me to
realize my dreams.

Parthana Sarma
List of Figures

1.1 Discrete emotion measurement space. . . . . . . . . . . . . . . . . . 3

1.2 Dimensional emotion measurement space. . . . . . . . . . . . . . . . 4
1.3 Visual stimuli presentation in EEG based emotion analysis experi-
ment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Conventional EEG based emotion recognition system. . . . . . . . . 10

2.1 System overview of proposed method. . . . . . . . . . . . . . . . . . 36

2.2 Classification accuracy for different segment lengths for PSD using
k -NN (a) SEED (b) DEAP. . . . . . . . . . . . . . . . . . . . . . . 42
2.3 Classification accuracy for different segment lengths for CWT (a)
SEED (b) DEAP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.4 Prominent channels for PSD (a) Distinct space (SEED) and (b)
Dimensional space (DEAP). . . . . . . . . . . . . . . . . . . . . . . 44
2.5 Prominent channels for CWT (a) Distinct space (SEED) and (b)
Dimensional space (DEAP). . . . . . . . . . . . . . . . . . . . . . . 45
2.6 Entropy variation for (a) Distinct space (SEED) and (b) Dimen-
sional space (DEAP). . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.1 System overview of proposed method. . . . . . . . . . . . . . . . . . 53

4.1 System overview of proposed method. . . . . . . . . . . . . . . . . . 70

4.2 Significant channels in emotion recognition. . . . . . . . . . . . . . . 71
4.3 Classification performance achieved by five EEG sub-bands for dif-
ferent values of EEG segment length (L). . . . . . . . . . . . . . . . 80
vi List of Figures

4.4 Classification performance achieved by five EEG sub-bands for dif-

ferent number of channel sets, for β and γ at L5 , α at L7 , δ and θ
at L5 –L7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.5 Prominent channels for five sub-bands during displaying their high-
est classification performance. . . . . . . . . . . . . . . . . . . . . . 82
4.6 Activation timing for positive, neutral and negative emotions. . . . 83
4.7 Classification performances of individual subject. . . . . . . . . . . 87
4.8 Classification performances of individual subject. . . . . . . . . . . 88
List of Tables

1.1 Emotions and corresponding brain regions. . . . . . . . . . . . . . . 8

2.1 Steps for appropriate Segment Selection . . . . . . . . . . . . . . . 30

2.2 Different Segment Length (L) . . . . . . . . . . . . . . . . . . . . . 39
2.3 Results employing PSD as feature for SEED . . . . . . . . . . . . . 41
2.4 Results employing PSD as feature for DEAP . . . . . . . . . . . . . 41

3.1 Results for different sub-bands for distinct emotion (SEED) using
PSD feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2 Results for different sub-bands for distinct emotion (SEED) using
CWT feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3 Results for different sub-bands for Valence (DEAP) using PSD feature 59
3.4 Results for different sub-bands for Valence (DEAP) using CWT
feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.5 Results for different sub-bands for Arousal (DEAP) using PSD feature 61
3.6 Results for different sub-bands for Arousal (DEAP) using CWT
feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.7 Brain region activation for EEG sub-bands . . . . . . . . . . . . . . 62
3.8 Comparative Study with Related Works . . . . . . . . . . . . . . . 63

4.1 Steps for Implementing Proposed Method . . . . . . . . . . . . . . 77

4.2 Different Segment Lengths (L) . . . . . . . . . . . . . . . . . . . . . 79
4.3 Performance Considering Discarded EEG Segments . . . . . . . . . 81
4.4 Classification Performance for Different PLV Values . . . . . . . . . 84
4.5 Selection of Brain Regions for Different PLV Values . . . . . . . . . 85
viii List of Tables

4.6 Performance Considering Discarded Channels . . . . . . . . . . . . 86

4.7 Results of LOSO and LOTO experiments . . . . . . . . . . . . . . . 86
4.8 A Summary of Results . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.9 Comparative Study with Related Works . . . . . . . . . . . . . . . 92
List of Symbols

δ Delta sub-band of EEG with frequency range 0.5-4Hz

θ Theta sub-band of EEG with frequency range 5-7Hz
α Alpha sub-band of EEG with frequency range 8-15Hz
β Beta sub-band of EEG with frequency range 16-31Hz
γ Gamma sub-band of EEG with frequency range 32-41Hz
H EEG data matrix for different channels and trials
L EEG segment length
hL Normalized EEG data
LS EEG segment length for SEED dataset
LD EEG segment length for DEAP dataset
X Correlation matrix for hL
λ Eigenvalues of X
λmax Maximum eigenvalue from λ
Pd Wishart distribution
λh Highest Wishart eigenvalue
λl Lowest Wishart eigenvalue
P (f ) Periodogram of EEG segments
N Number of sections to calculate power spectral density
V Normalization factor
b Interval between each N sections
ψ(t) Morlet wavelet
τ Time shift factor
c Scaling factor
C(c, τ ) Continous wavelet transform
x List of Symbols and Operations

Nt Number of decision trees in random forest classifier

η Accuracy
TP True positive
TN True negative
FP False positive
FN False negative
EL Entropy of EEG segment of length L
σE Standard deviation of entropies
µE Mean of entropies
fL Lower frequency cut-off of band pass filter
fH Higher frequency cut-off of band pass filter
g(i) Analytical representation of EEG signals
i Number of channels
q(i) EEG data from channel i
Ai (t) Instantaneous amplitude
ϕ(t) Instantaneous phase
w List of channel pairs
r Target channel
QL EEG data from different channels
C Total correlation strength
List of Abbreviations

EEG Electroencephalogram
ANN Artificial Neural Network
BPF Band Pass Filter
BPF Band Pass Filter
BVP Blood Volume Pressure
COV Covarience
CWT Continuous Wavelet Transform
DASM Differential Asymmetry
DFT Discrete Fourier Transform
DWT Discrete Wavelet Transform
ECG Electrocardiography
EEG Electroencephalogram
EMD Empirical Mode Decomposition
EMG Electromyogram
FD Fractal Dimension
fMRI Functional Magnetic Resonance Imaging
GSP Graph Signal Processing
GSR Galvanized Skin conductance
HA High Arousal
HMM Hidden Markov Model
HOC Higher Order Crossings
HRV Heart Rate Variability
HV High Valance
IMF Intrinsic Mode Functions
xii List of Abbreviations

k -NN k -Nearest Neighbor

LA Low Arousal
LDA Linear Discriminant Analysis
LOSO leave-one-session-out
LOTO leave-one-trial-out
LV Low Valance
MEG Magnetoencephalogram
MIF Multivariate iIterative Filtering
MIMF Multivariate Intrinsic Mode Functions
MLP Multilayer Perceptron
MPED Multi-Modal Physiological Emotion Database
MVA Majority Voting Algorithm
PLV Phase Locking Value
PSD Power Spectral Density
RASM Rational Asymmetry
RBF Radial Basis Function
RF Random Forest
RMT Random Matrix Theory
RVM Relevance Vector Machine
SDA Stepwise Discriminant Analysis
SFS Sequential Forward Search
SNN Spiking Neural Network
SVM Support Vector Machine
V-A Valance-Aarousal