Deep RNN Learning for EEG based Functional Brain

State Inference
Amogh Chaudhari
Suprava Patnaik Lalita Moharkar MBBS, M G M Inst. of
Electronics & Tele. Com. Engg. Electronics & Tele. Com. Engg. Health Sciences
Xavier Institute of Engineering Xavier Institute of Engineering Navi Mumbai,
Mahim, Mumbai, India Mahim, Mumbai, India [email protected]
[email protected] [email protected]

Abstract—Use of deep learning for sequence dependent task in the mentioned prototype is one or two seconds after
classification has shown considerable improvement in recent observing the cue on the monitor event related
past. This paper is on task identification from the associated desynchronization (ERD) and event-related synchronization
electroencephalograph (EEG) signal. This paper deals with (ERS) takes place in a particular fashion at some pre-identified
extraction of EEG sub-band features by using wavelet transform locations of scalp. The goal is to identify the pattern and
followed through classification by means of recurrent neural characterize the changes in the brain state through monitoring
network (RNN) and deep learning. The methodology is based on
of ERD/ERS profile.
comparison of correlation among various EEG spectral
components by using a holdup RNN architecture and winner-
takes-all logistic regression approach to identify the associated II. PROCESSING OF EEG SIGNAL
task. The bottleneck of current EEG processing approaches is
signal power getting biased due to noisy trials which deteriorates A. Neurodynamics and EEG
the analysis in the manifold space. This study uses a similarity or The Electroencephalogram (EEG) is a representative signal
coherence analysis between sub-band spectrum of individual trial containing information about the condition of the brain. The
which invalidates the effect of noise. To assess the proposed shape of the brain waves, correlation among neuronal
method, a dataset, containing 5 subjects performing four mental activations, pattern along many neurons, etc. exhibit
tasks, is utilized. The results show that in the multiclass scenario information about the states of the brain. However, the human
the proposed algorithm performs at par with state-of-art results,
observer cannot directly monitor these subtle details. Besides,
whereas in two class scenario, the proposed algorithm
outperforms most of the reported results.
since bio -signals are highly subjective, the symptoms may
appear at random in the time scale. Therefore, the EEG signal
Keywords—EEG; recurrent neural network; wavelet transform, parameters, extracted and analyzed using computers, are
phase coherence highly useful in brainwave based diagnostics. Chaotic
measures like correlation dimension (CD), Largest Lyapunov
I. INTRODUCTION Exponent (LLE), Hurst Exponent (HE) and entropy are some
commonly used features to characterize the brain wave.
Scalp EEG recording system is non-invasive, relatively
cheap and easily wearable. Identification of mental task from This paper aims to investigate the topography pattern of
EEG signal can be very useful for designing of brain computer EEG alpha sub-band cross coherence and the possibility to
interfaced systems which can be employed to help the patients identify the underneath task. Cross coherence corresponds to
with amyotrophic lateral sclerosis to better communicate with coherence between corresponding locations of left and right
their surroundings. EEG waveforms are generally classified hemisphere. Topographic distribution of complex alpha
according to their frequency, amplitude, and location on the generators is essentially important for the modulation of
scalp at which they are recorded. Frequency (Hertz, Hz) is a resting, wake and mindfulness. Our analysis is based on
key characteristic used to define brain state and highlight constructing trajectories of this under for different activity
about presence of abnormality. The standard spectral states in the wavelet coefficient space. Based on many
components of EEG signal previous results, specifically relating to profiles of DWT
amplitude spectra of the analyzed signals, we have applied
The main challenge of EEG signal classification is its off-line band pass filtering (4th order Butterworth filter) on all
performance in terms of accuracy and robustness. Since the EEG EEG signals in the range 6–14 Hz. The wider alpha range is
signals are highly sensitive to additive noise, the main concern of chosen to incorporate small differences in cross hemispheric
this study was designing an efficient algorithm to reduce the alpha frequency that exist between neurons of central and
effect of noisy trials during neural network training. The occipital locations [1].
paradigm considered in this study is cue-based synchronous mode
training and testing. Detecting the task from an ongoing EEG
acquisition, which is known as asynchronous or self-paced mode
is much more challenging. A mental task

978-1-5386-3852-1/17/$31.00 ©2017 IEEE

Authorized licensed use limited to: KIIT University. Downloaded on July 17,2023 at 04:22:40 UTC from IEEE Xplore. Restrictions apply.
B. Database whitening. ICA is a kind of blind source separation technique
The data used in this study was taken from publicly that extracts statistically independent sources called
available on BCI website [4] .The said EEG was made by independent components (ICs) from a set of recorded signals.
Keirn and Aunon (Colorado State University). Electrodes
were placed at O1, O2, P3, P4, C3 and C4 according to the
D. DWT for EEG timefrequency analysis:
10–20 System. Measurements were made with reference to
electrically linked mastoids, A1 and A2. The electrodes were EEG signal has a limited band power that can be divided to
connected through a bank of amplifiers, whose band-pass several unequal bands power. For example: delta (0–3 Hz),
analog filters were set at 0.1–100 Hz. The data were sampled theta (4–7 Hz), alpha (8–13 Hz), beta (14–20 Hz) and gamma.
at 250 Hz. Eye blinks were detected by means of a separate Analysis of this segmented bands are considered differently
channel of data recorded from two electrodes placed above from one study to another. EEG signal is a noisy signal in time
and beneath the subject's left eye. Eye blink artifact is noticed domain but it is a significant signal in frequency domain,
in the dataset. However for the present study artifacts common therefore, some studies [24–35] have demonstrated the
to all acquisition and not more than 50mv are expected not to efficiency of EEG power bands features in discriminating
contaminate our result, since our goal is a system for mental tasks. Time–frequency transformation of each channel
recognizing patterns based on difference in correlation pattern. signal was calculated by using Discrete Wavelet Transform
EEG signals were recorded for 10 s during each task, and each (DWT). On the basis of the former study [2], 5 -level of
task was repeated for ten sessions. Seven subjects, 21–48 decomposition with Daubechies filter in order 4 (db4) has
years old, participated in the study involving a total of five been used to obtain time-frequency approximation coefficients
distinct tasks. for the EEG signals. Table.1 is the represents the frequency
distribution of the DWT-based sub-bands of the EEG signals
The tasks to be inferred or activities undertaken by the sampled at 250Hz. Fig.1 shows power spectral density (PSD)
subjects are: 1) Baseline Task (Base): The subject was asked for various EEG sub-bands for sample letter composing and
to completely relax. This task was used as a control and as a rotation visualization trials.
baseline measure of the electro-encephalogram (EEG). 2)
Mathematical Multiplication Task (M): The subject was TABLE I: Frequencies corresponding to different levels of
asked to do metal multiplication. The problem was not trivial decomposition for Daubechies five level wavelet
such that answer was not deceptive till the 10s acquisition decomposition for EEG with sampling frequency of 256 Hz
time gets over. 3) Geometric Figure Rotation (R): The Decomposed Frequency EEG sub-bands
subject was instructed to visualize the object being rotated Signal In Hz
along an axis. 4) Mental Letter Composing (L): The subject D1 60-120 High Gamma
was instructed to mentally compose a letter to a friend without D2 30-60 Gamma
vocalizing. 5) Visual Counting (C): The subject was asked to 15-30 Beta
D3
visualize numbers being written on the board sequentially,
D4 7.5-15 Apha
with the previous number being erased before the next number
D5 3.75-7.5 Theta
was written.
A5 0-3.75 Delta
The tasks or functional brain states are based on
involvement of different depths of logical thinking and
reasoning. As per the fundamentals of neuroscience left
hemisphere of human brain performs tasks that have to do
with logic, such as in science and mathematics. On the other
hand, the right hemisphere performs tasks that have to deal
with creativity and the arts. Considering the above tasks the
correlation between signals captured from corresponding left
and right hemispheres electrodes are likely to fluctuate
depending on underlying activities. For example involvement
of neuronal activities for visualizing while counting is likely to
have equal activations from either side of the brain with high Fig. 1: PSD plot of sub-band decomposed EEG signal from
correlation. On the other hand, nontrivial mental computation, ‘P3’ position. (a)-Letter composing and (c)-Visualize rotation.
ideally, would involve and stress left hemisphere more than
the right hemisphere. Envelope analysis is a very efficient method for detecting
C. Pre-Processing for Artifact removal the periodic impacts of EEG signal. In this method, the
envelope spectrum is created by using the common method of
The EEG signal is often distorted by the different
Hilbert transform (HT), which is proven to produce clearer
artifacts, such as eye blinks, eye movement, and vital and smoother curves. The widespread use of HT can be
muscular response of the body. These may result poor attributed to several factors, some of which include its
accuracy of classification. In this study Independent excellent facility and practicability. The formulation of HT
Component Analysis (ICA) is used through centering done by based envelope of EEG spectrum can be given as:
mean subtraction followed by linear transformation for

Authorized licensed use limited to: KIIT University. Downloaded on July 17,2023 at 04:22:40 UTC from IEEE Xplore. Restrictions apply.
 ෱ signals being non-stationary in nature; cross-correlation is to
S (t ) = x (t ) * 1 = ³ x(˱ ) d˱ …….(1) be estimated on relatively short-time frame, in which signals
˭ t ෥෱ t ෥ ˱ can be assumed quasi-stationary.

where S(t) is the envelope spectrum of x(t). The phase synchrony of x(t ) and y (t ) is computed as

The HT is conducted on the decomposed DWT sub-bands of N ª º

1
EEG signal, and then sub-band PSD envelopes are obtained. Rxy = N ¦ei ¬˳ x ( t j ) ෥ ˳ y ( t j )¼ ………..(3)
k =1
E. Coherrence Analysis
Synchronization between neuronal populations plays an Where ˳x ( t j ) and (tj)
denote the phase variable of
˳y
important role in information transmission between brain areas. x(t ) and y(t ) , N is the number of samples within the specified
The neuronal potential variation, which is the cause behind non-
stationary behavior of EEG signal is largely related to event time frame IJ. For an arbitrary signal x(t ) phase variable is
related synchronization (ERS) and de-synchronization (ERD). computed as:
Event related cognitive state reflects degree of coherence or
synchronism among the active neurons. Therefore coherence x ( t)
analysis may provide valuable information as a tool to detect ˳ ( t ) = arctan x (t ) ,………….(4)
differences in the dynamic behavior and different degrees of
determinism required for various tasks. Critical parameters for
Where the Hilbert transform x ( t ) is
estimating the EEG coherence are embedding a specific sub-band
signal and time frame IJ while computing coherence. Having
hypothesized that the topographic distribution of complexity of 1 ෱ x (t' )
p.v.
alpha generators is essentially important for the modulation of
alertness the aim of our investigation is based on analysis of x(t)= ˭ ෥෱ ³ x (t ) dt ' , p.v. is Cauchy’s principal value.
alpha-sub-band coherence. …………..(5)
The auto -correlation function, defined as A(t) = œ V(t')V(IJ
+t')dt', gives information on the correlations in time present in III. RECURRENT NEURAL NETWORK
the signal. The auto-correlations for the EEG signal at the Among the various training based approaches available,
different channels show that it is an oscillating function with a neural networks have been recognized as the most successful
decaying envelope. This indicates that the signal has important classifiers for categorizing dynamic and chaotic
correlations but they decay with time which implies that there manifestations. Generally, neural networks can be classified
is loss of information in time. Selection of appropriate signal into two categories: Feed Forward Neural Network (FNN) and
is very crucial for EEG dynamic analysis. Considering the fact Recurrent Neural Network (RNN). RNNs contain feedback
that the activities recorded in this dataset are expected to connections in addition to feedforward connections. In this
involve simultaneous variations in degree of visual study, we utilized a notable Elman RNN as a classifier for
concentration and logical analysis, the coherence between categorizing EEG based neuronal activation. The Elman
degree of difference among O1 ~ C1 and O2 ~ C2 are network consists of an input layer, a hidden layer, a context
considered and are used to infer the underlying activities. layer and an output layer. The neurons of output layer compete
Occipital (Oi) and central (Ci) electrodes respectively are against each other, the neuron with maximum output
likely to replicate the degree of perceptual and logical magnitude wins the race and is set to “1” with remaining
responses and the required assimilation for various tasks. neurons set to “0”. Unlike FNN, RNN uses “cortex units”,
The information about the dynamics of brain from signals which are feedback connections to hidden layer neurons, as
measured at two different positions can be extracted by internal memory to capture information about what happened
analyzing short time properties. Given two time series x(t ) in all the previous time steps of a given input sequence and
learn the underlying dynamics. An Elman network with more
and y (t ) , the correlation is defined as than one hidden layers resembles deep learning and uses
multiple sets of “context units” at deeper layers.
E [ xy]
Cxy = ……………(2) A. Network Architecture
ª 2º ª 2º
In this study we have used Elman’s RNN with 4-hidden
E y layers and bottle neck architecture for classifying the EEG
E¬x ¼ ¬ ¼ signals.
For stationary stochastic processes the cross-correlation Elman RNN is defined as follows:
function Cxy is used as a measure of the mutual relation of the 1) The first layer is the input layer and the neurons of this
band-power modulations as a function of their delay IJ, where layer convey the input features to the first hidden layer. In the
present study EEG coherence coefficiants are treated as the
E [.] is the expected value operator. Event related EEG input features for the input layer.

Authorized licensed use limited to: KIIT University. Downloaded on July 17,2023 at 04:22:40 UTC from IEEE Xplore. Restrictions apply.
 2) Hidden layer neurons receive activations
simuntaneously from the input or previous hidden layer
neurons and from the cortex layer neurons. Fig.2 is the
shematic of Elman’s RNN. Neurons in hidden layers perform
log-sigmoid processing of the net input activations. The
processing of the jth neuron is defined as:
Fig.3 Deep Learning RNN Architecture
p q

net kj = ¦ w kji uik + ¦ d kjc hik + bj IV. PROPOSED BRAIN STATE INFERENCE METHOD
i =1 c =i
……………….(6)
1
O kj = Fig.4 is the block diagram of the proposed brain state
1 + exp ( ෥net j )
k
inference method. The goal is to identify the function, which is
where, wki is the link weight between the jth neuron of current nothing but the task carried out by a subject, from acquiring
j
and analyzing his EEG signal. DWT is used to extract the
layer and ith neuron of the previous hidden layer or the input alpha sub-band coefficients of EEG signals. Proposed feature
layer in case of first hidden layer k is the iteration or epoch analysis method is based on coherence between alpha sub-
k bands. In order to account for both perceptual and logical
number depending on online or batch learning, d jc is the
insinuation differential spectral power from central and
k
context input connection strength and b j is the bias occipital electrodes of left and right hemispheres are
considered for coherence computation. Collective dynamics of
associated with the neuron. neurons is learned by averaging coherence over consecutive
B. Bottleneck Architecture 50 samples. We used a sliding window approach with non-
A bottleneck layer is a layer that contains fewer nodes overlapping samples and used Hilbert transform to obtain
compared to the previous layers. It is commonly used to obtain phase coherence.
a representation of the input with reduced dimensionality. Phase coherence sequence is given as the input to a
Bottleneck architectures are generally used for image Recurrent Neural Network. A 5-layer network with 53-400 -
compression and encoding applications. Conceptually 50-200-20-T neurons is trained to classify the tasks, where ‘T’
bottleneck layer neurons are expected to convert the large corresponds to the number of tasks. RNN is trained to classify
dimensional features into lower variance and less overlapped considering two, three or four different tasks at a time.
clusters of sub-space features. This leads to make fuller or Network training is done for 100 epochs. The database
more efficient use of available space and better generalization considered for this study has 65 instances for four activities.
of the neural network. While preserving the coherence space 25 instances are deliberated for testing and remaining 40 are
layout, the network has at the same time removed much of the considered for training.
variation unimportant for its task of predicting phoneme
outputs from input features, which is somewhat similar to
human perception. Bottleneck features are also likely to be
less distorted. Fig.3 is the representation of the RNN
architecture used for this study.

Fig.4-Block diagram for the proposed RNN and EEG

based brain functionality inference method

Output layer neurons compete with each other and only the
neuron with the highest activation stays active while all other
neurons are marked neutral. Thus the network uses nonlinear
Fig.2 Topology of Elman’s RNN
inhibition to pick out the largest. This is similar to
computational model of brain cortex, particularly for processes
based on distributed activation based decision making.
Decision is taken in favour of the class with maximum output.

Authorized licensed use limited to: KIIT University. Downloaded on July 17,2023 at 04:22:40 UTC from IEEE Xplore. Restrictions apply.
 Class ( c ) = arg max{ neti }
……….(7)
i TABLE III CONFUSION MATRIX (3-TASKS)

V. RESULTS AND PERFORMANCE METRIC Name of the tasks

LETTER MATH ROTATION
A validation of the classification rate of the proposed EEG
based brain state inference and task identification is presented LETTER 19 4 2
in this section. Generally, the evaluation of a classification MATH 2 21 2
problem is done based on entries of a matrix called as
ROTATION 2 1 22
confusion matrix. For a two class problem the diagonal
elements of the matrix correspond to number of entries ACCURACY=82%
correctly classified and are called true-positive (TP) and true Name of the tasks
negative (TN), whereas the off diagonal elements ROTATION COUNTING MATH
corresponding to wrong classification are called false negative ROTATION 21 2 2
(FN) and false positive (FP). So, the accuracy can be 1
COUNTING 3 21
measured as:
MATH 3 3 19
TN + TP
ACCURACY=81%
accuracy = TN + TP + FP + FN ……(8)
TABLE IV CONFUSION MATRIX (4-TASKS)
The dataset used for this study has four classes, namely
math, counting, rotation, and letter writing. Rest is considered Name of the tasks
as a reference state and between trials of each of these LETTER MATH ROTATION COUNTING
activities subjects were asked to seat in rest state and then start LETTER 20 4 0 1
the activity. Tables-II records the performance for
combinations of two classes. Tables-III and IV are the metrics MATH 2 20 1 2
for combinations of three and four tasks respectively. Fig 4 ROTATION 3 3 18 1
shows the graphs for mean square error with respect to the
COUNTING 2 2 2 19
number of iterations for different combinations.
ACCURACY=81%

TABLE II CONFUSION MATRIX (2-TASKS)

VI. CONCLUSION AND FUTURE SCOPE
Name of the tasks
In this research, a RNN model is trained to identify the
MATH LETTER
phase coherence patterns of EEG alpha-bands. Difference
MATH 23 2 between EEG signals from central and occipital (C1-O1 & C2-
LETTER 3 22 O2) locations is considered to compute phase coherence
patterns for various activities. Average accuracy attained is
ACCURACY=90% around 90% for two tasks, 82% for three tasks and 77% for all
Name of the tasks the four tasks. The results are far better than the classification
ROTATION LETTER rates reported in [6, 7 & 8]. However these results are from
ROTATION 22 3
different paradigm and do not bear a straight forward
comparisons. In future studies, we intend to improve RNN
LETTER 3 22 architecture and deal with other database and EEG
ACCURACY=90% classification problems.
Name of the tasks
COUNTING LETTER
COUNTING 23 2

LETTER 3 22

ACCURACY=90%

Authorized licensed use limited to: KIIT University. Downloaded on July 17,2023 at 04:22:40 UTC from IEEE Xplore. Restrictions apply.
 (a)Math vs Letter

(d) Letter, Math, Rotation & Counting

Fig. 4 MSE vs Epochs: (a) two classes, (b & c) three classes, (d) four classes

REFERENCES
[1] Aleksandar Kalauzi, Aleksandra Vuckovic, Tijana Bojiü, “Topographic
distribution of EEG alpha attractor correlation dimension,” International
Journal of Psychophysiology, Volume 95, Issue 3, March 2015, Pages
278-291
[2] H. Adeli, Z. Zhou, N. Dadmehr, “Analysis of EEG records in an
epileptic patient using wavelet transform,” J. Neurosci. Method., 123 (1)
(2003), pp. 69-87
[3] P. Nguyen, M. Kang, J.M. Kim, B. Ahn, J. Ha, B. Choi, “Robust
(b) Letter, Math & Rotation condition monitoring of rolling element bearings using de-noising and
envelope analysis with signal decomposition techniques,” Expert Syst.
Appl., 42 (22) (2015), pp. 9024-9032
[4] http://www.cs.colostate.edu/eeg/main/data/1989_Keirn_and_Aunon
[5] Noshadi S., “Selection of an efficient feature space for EEG-based
mental task discrimination.[MSc thesis] Yazd,” Iran: University of
Yazd; 2010
[6] Yu-Liang Hsu, Ya-Ting Yang, Jeen-Shing Wang, Chung-Yao Hsu,”
Automatic sleep stage recurrent neural classifier using energy features of
EEG signals,” Neurocomputing, 2013, vol.104, pp. 105-114.
[7] Fatemeh A., Reza B.,Benjamin B,”Weighted spatial based geometric
scheme as an efficient algorithm for analyzing single-trial EEGs to
improve cue based BCI classification,” Neural Networks, 2017, Vol. 92,
pp-69-76.
[8] Abdulhamit Subasia, Ergun Ercelebi, “Classification of EEG signals
using neural network and logistic regression,” Computer Methods and
Programs in Biomedicine ,2005, Vol. 78, pp. 87—99

(c) Rotation, Counting, Math

Authorized licensed use limited to: KIIT University. Downloaded on July 17,2023 at 04:22:40 UTC from IEEE Xplore. Restrictions apply.