Biomedical Signal Processing and Control 92 (2024) 106081

Contents lists available at ScienceDirect

Biomedical Signal Processing and Control

journal homepage: www.elsevier.com/locate/bspc

NF-EEG: A generalized CNN model for multi class EEG motor imagery
classification without signal preprocessing for brain computer interfaces
Emre Arı a, *, Ertuğrul Taçgın b
a
Department of Mechanical Engineering, Faculty of Engineering, Dicle University, Diyarbakır, Turkey
b
Department of Mechanical Engineering, Faculty of Engineering, Doğuş University, Istanbul, Turkey

A R T I C L E I N F O A B S T R A C T

Keywords: Objective: Brain Computer Interface (BCI) systems have been developed to identify and classify brain signals and
Brain computer interface (BCI) integrate them into a control system. Even though many different methods and models have been developed for
Deep learning the brain signals classification, the majority of these studies have emerged as specialized models. In addition,
EEG motor imagery
preprocessing and signal preprocessing methods which are largely based on human knowledge and experience
Classification
Input reshaping
have been used extensively for classification models. These methods degrade the performance of real-time BCI
Data augmentation systems and require great time and effort to design and implement the right method. Approach: In order to
eliminate these disadvantages, we developed a generalized and robust CNN model called as No-Filter EEG (NF-
EEG) to classify multi class motor imagery brain signals with raw data and without applying any signal pre­
processing methods. In an attempt to increase the speed and success of this developed model, input reshaping has
been made and various data augmentation methods have been applied to the data. Main results: Compared to
many other state-of-the-art models, NF-EEG outperformed leading state-of-the-art models in two most used
motor imagery datasets and achieved 93.56% in the two-class BCI-IV-2A dataset and 88.40% in the two-class
BCI-IV-2B dataset and 81.05% accuracy in the classification of four-class BCI-IV-2A dataset. Significance: This
proposed method has emerged as a generalized model without signal preprocessing and it greatly reduces the
time and effort required for preparation for classification, prevents human-induced errors on the data, presents
very effective input reshaping, and also increases the classification accuracy.

1. Introduction BCI signals are generally collected from electroencephalography

(EEG), magnetoencephalography (MEG), electrocorticograms (ECoG),
Brain Computer Interface (BCI) is the name given to systems that positron emission tomography (PET), functional magnetic resonance
allow communication and interaction between a device and the brain. imaging (fMRI), local field potentials and action potentials, and near-
People with neurological disorders, bedridden, paralyzed, and those infrared spectral imaging (NIRS). One of the most used source from
who cannot perform their motor functions, can control a device through these methods is EEG, because it is non-invasive, low cost, easy to apply
their brains using these systems. In addition, it can be used in the fields and has high temporal resolution [9]. EEG signals are generally defined
of utility tools and entertainment for who do not have any health as a curse of dimensionality problem because of non-stationery signals
problems, and it can be used for commercial purposes for different in­ and they have channel correlation, multi-channel recording paradigm,
dustries [1–5]. These BCI systems collect and classify the brain signals of noise and artifacts and also fast, reliable and promising systems are
people and turn them into commands. In this way, people who experi­ systems that can accurately and quickly detect hidden variables in these
ence limb loss or cannot control their limbs, can independently control physiological signals [10]. EEG-Based BCI systems are applied in many
their artificial or existing limbs with these systems. In addition, these different areas and there are applications in neuromarketing [4],
systems have recently started to be used for post-stroke rehabilitation epileptic seizure detection [11], BCI-based wheelchairs [12,13], upper
processes by moving the limbs via brain signals, not by moving the or lower limb assistive robots [3,5], neuro entertainment [14] and some
muscles of the person [6–8]. Generalized steps of a Brain Computer similar systems. BCI applications have started to find a place for them­
Interface shown in Fig. 1. selves in the game and entertainment industry as well. Tetris [15],

* Corresponding author.
E-mail address: [email protected] (E. Arı).

https://doi.org/10.1016/j.bspc.2024.106081
Received 29 May 2022; Received in revised form 28 December 2023; Accepted 29 January 2024
Available online 8 February 2024
1746-8094/© 2024 Elsevier Ltd. All rights reserved.
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 1. Generalized Steps of a Brain Computer Interface.

Fig. 2. Traditional Steps of EEG Motor Imagery Classification.

World of Warcraft [16] and BrainArena [17] are some of them. EEG- low classification success and inability to generalize the models. In many
based BCI systems have different applications such as P300 evoked po­ studies, wavelets [20,23] or short-time Fourier transform (STFT)
tentials, visual evoked potentials (VEP), steady-state visual evoked po­ [21,24] have been used for time–frequency feature extraction of EEG
tentials (SSVEP), motion onset visual evoked potentials (moVEP), motor signals. In [24], researchers created 2D images from EEG signals using
imagery (MI) and so on [18,19]. Motor imagery classification is a very short-time Fourier transform and classified these images by training
popular BCI paradigm using the EEG signals. A person can generate EEG them using CNN and stacked autoencoders (SAE) where CNN was used
signals in brain by imagining making any movement (such as hand, arm, to extract the features and SAE was used to do the classification. In [25],
foot movements) without physically moving, and these EEG signals can the authors used dynamic frequency feature selection algorithm and
be collected through electrodes and classified automatically with this wavelet packet decomposition (WPD) for feature selection. In [30],
method. There are 5 traditional steps in EEG motor imagery classifica­ authors converted EEG signals into 2D images using continuous wavelet
tion: Signal acquisition, signal preprocessing, feature extraction, feature transform (CWT) and used CNN to train and classify these images. In
selection and classification as shown in Fig. 2. [26], the authors made a choice of time period with desynchronization /
Signal preprocessing methods are used extensively before and during synchronization (ERD / ERS) and used WPD. In [31], deep learning al­
feature extraction processes and most of the methods used at this stage gorithms used for the classification of signals such as Electroencepha­
are designed manually and are based on human experience and logram (EEG), Electrocardiogram (EKG), Electrooculogram (EOG) and
knowledge [20–22] and it is highly likely to cause unwanted losses on Electromyogram (EMG). Another important feature extraction from EEG
the data with signal preprocessing and feature extraction. In the feature signal processing methods is common spatial pattern (CSP) for space
extraction of EEG signals, it is aimed to extract the information in the domain [1,26,27]. The performance of the CSP operation varies
time, frequency and space domains, and with this information, neural depending on the frequency band to be processed. To solve this problem,
networks are trained and classifications are made. In addition, for the authors developed a filter bank common spatial model (FBCSP) [28].
classification of EEG signals, the signals were converted into 2D images FBCSP creates different frequency bands and applies the CSP process to
with various preprocessing methods, and these images were trained and these frequencies and uses an automatic feature selection algorithm
classified with deep learning algorithms. In these studies, mostly a among these bands. This process increases the classification accuracy in
specific time range or features in the specific frequency range are general. FBCSP uses a specific time range and ignores the most effective
extracted to detect EEG features. In order to convert EEG signals to 2D time ranges for various subjects and therefore cannot use time domain
images, the image of the signal at a fixed time or frequency was information efficiently enough. Refs. [29] and [30] indicate the best
extracted, or the average of the images obtained at different times or time choice for FBCSP, but the increase in classification accuracy is not a
frequencies was transformed. EEG signals may contain important in­ notable increase. In [32], various EEG paradigms are reviewed and
formation at different times or frequencies, however, these time and possible DNN algorithms for each paradigm are discussed. In [10], it was
frequency regions may vary from person to person and these applied stated that the neural network architecture used for the classification
methods may cause to missing important information for classification, process, the input data structure and the frequency range used during

2
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 3. Proposed Steps of EEG Motor Imagery Classification.

the analysis are important in the classification of MI EEG signals with learning. In [53], authors developed a multi-branch CNN model for the
deep learning methods. Limited human knowledge and experience may classification of motor imagery signals, which trains the EEG signals in
limit the classification accuracy of these applied methods and systems parallel in 3 different CNN models and collects the results from these
[33,34]. However, to design a feature manually requires a lot of time- CNNs. The authors of [46] attempted to classify motor imagery signals
consuming experimentation and observations, and eventually these by using a deep belief network (DBN). In addition, some studies classi­
features need to be tested and their performance have to be measured for fied motor imagery signals by using spiking neural network (SNN)
many different scenarios. In order to solve this problem and to obtain the [47,48]. While performing the learning process, deep neural networks
needed features, the researchers were able to automatically perform train millions or even tens of millions of learnable parameters, and they
feature extraction and classification steps using neural networks and do a lot of matrix multiplication for this training [54–56] causing a very
they achieved remarkable results with these structures [35–40]. heavy computational cost. GPUs and TPUs are used to perform these
Recently, the use of deep learning techniques in the analysis of large calculations more efficiently and faster. Neural network structures have
data sets has produced promising results and high classification success gone beyond human knowledge and experience which make them a
[41]. Deep learning methods gave much more successful and stable re­ limit and have achieved higher classification accuracies. This method
sults in extracting valuable features and in depth information from big also reduces time and effort for feature design because neural networks
data compared to traditional neural networks. Deep learning techniques automatically design the appropriate features needed for successful
have also eliminated most of the disadvantages such as feature extrac­ classification. Existing CNN-based classification methods have been
tion, preprocessing, model changing according to the subject, and loss of designed and implemented as single and multiple CNN dimensions [50].
information during training, which rely on human knowledge and Since EEG signals have very low amplitude and high noise, various
experience and cause time and effort [42]. Deep learning methods are signal processing methods have been applied to the signals before
very effective for image, video, speech and pattern recognition and very classification. These signal processing methods can act in a way that
good results have been obtained. Apart from these data types, it has reduces the size and complexity of the signal, but also loses the
started to be applied to time series data types and very good results have distinctive features within the existing signal. In addition, these classi­
been obtained in this field. Deep learning models, which have also been fication methods may encounter classification success problems when
used in the EEG brain signals classification, have achieved promising data to train are insufficient. In the light of all this information, one of
results. The most popular one among deep learning methods is the CNN the biggest difficulties in MI EEG based BCI systems is the lack of large
method, due to its great achievements in various fields. In addition to enough datasets and the inability to train deep learning models deeply
CNN, researchers have also used different neural network structures. In enough. Accessible MI EEG datasets contain a limited number of subjects
[75], authors reviewed the deep learning techniques for motor imagery and a limited number of trials. However, various solutions have been
EEG classification. The authors developed a CNN by applying techniques investigated by researchers to deal with this problem. Collecting EEG
such as the exponential linear unit activation function and batch motor imagery signals is a very difficult and laborious process. The same
normalization and aimed to achieve higher accuracy than FBCSP [43]. experimental setup and eeg sensor placements need to be set up in the
In [44], the authors proposed a multilayer perceptron (MLP) CNN, same way for each subject over and over again, the subjects must not
aiming to achieve a high classification accuracy by separately processing make any physical movements, they must be trained before the exper­
static energy and dynamic energy. In [45], they proposed a CNN with iment, and they must maintain their concentration throughout the
good accuracy rates applicable to various BCI paradigms. In [51], re­ experiment. Even blinking one’s eyes or a strong heartbeat can distort
searchers used CNN to extract high-level features from motor imagery the EEG signals. For all these reasons, there is very little data collected
signals. In [18], authors applied the Choi-Williams distribution (CWD) and accessible. Researchers have used and developed various data
transform, which reflects the energy distribution in both time and fre­ augmentation methods to reproduce existing data to overcome this
quency domains, to obtain 2D images from EEG signals. In [38], they problem. Another problem is the preprocessing of EEG data, which is
applied fast Fourier transform (FFT) and wavelet package decomposi­ very sensitive, noisy and difficult to analyze [59,60]. In order to find
tion (WPD) to EEG signals and compared them. Then they trained the meaningful properties and in-depth information contained in EEG sig­
information they obtained in the frequency domain using deep belief nals with time series data type, researchers generally used certain fre­
network (DBF) and restricted Boltzman machines (RBM). In the classi­ quency bands and time intervals and applied various preprocessing
fication of EEG motor imagery signals, CNN has achieved better classi­ processes to the signals. In addition, many preprocessing methods have
fication success than other methods [35,37,40,49,50], but there are been applied to convert the signals to 2D images. It has been stated in
some problems to be solved for these methods. Without using any of various studies that different frequency bands contain important infor­
these signal preprocessing methods and focusing on neural network mation. However, the preprocessing methods applied to the signals can
modeling, we have proposed a neural network based motor imagery cause the loss of important information in such sensitive data and create
classification steps as shown in Fig. 3. a limit for the classification success. All these mentioned problems
In [52], researchers developed a CNN model for training motor im­ prevent the creation of a generalized classifier and cause the creation of
agery signals by transferring VGG-16 network parameters via transfer customized classifiers that vary from person to person and cause low

3
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 4. BCI Competition IV – 2A Signal Acquisition Session.

Fig. 5. BCI Competition IV – 2B Signal Acquisition Session

classifier accuracies. Considering all the above mentioned in this study, the people in this dataset, a trial was started with the fixation cross
NF-EEG, a new generalized CNN classifier model has been developed displayed on a computer screen at time t = 0. Then, an arrow pointing to
that increases the accuracy by input reshaping and data augmentation the left or right is displayed, indicating that the right-hand or left-hand
methods without any signal processing. movement should be imagined at the time t = 3 s, and this arrow
remained on the screen for 1.25 s. The person started to imagine the
2. Datasets and input shape relevant hand movement at t = 4 s, and this process took approximately
four seconds. Finally, there was a break of at least 1.5 s. These trial
In this study, it is aimed to compare the proposed method with other stages are shown in Fig. 5.
studies and classifiers by using Graz BCI Competition IV-2A and Graz
BCI Competition IV-2B datasets [57], which are the most used datasets 2.2. Input shape
for EEG motor imagery classification. While developing their method­
ologies, researchers make great efforts to find the right input shape In CNN model designs, one of the most important variables to ach­
structures in which they can achieve the best and stable classification ieve the best classification success from the available data is the input
accuracy [10]. In this study, a 3D input shape structure is proposed to shape structure [10]. Researchers have tried many different input shape
develop a faster model and achieve a better classification accuracy. structures in EEG motor imagery classification studies. In this study, a
3D input shape was designed to achieve better classification success and
2.1. Datasets a faster training process. Rather than scanning the timestamp rows and
eeg channel columns through 1D or 2D kernels to find temporal and
As mentioned above, Graz BCI-IV-2A and Graz BCI-IV-2B datasets spatial relationships on the 2D axis, our goal in doing this conversion is
were used to test the NF-EEG model. The BCI Competition IV 2A dataset to add a single eeg channel column to the timestamps rows and add the
contains data collected from 9 healthy individuals with a sampling rate remaining channels as different columns to the third dimension to
of 250 Hz. These data consist of four classes: left hand, right hand, both obtain a new structure and perform a deeper temporal feature scan.
feet and tongue. Motor imagery data were recorded with two different When we compare this method with 2D (height × width) input shape, a
sessions, training and evaluation, for each person. Each session contains very high rate of classification success and performance increase has
288 trials and the dataset contains 5184 attempts in total. Persons sitting been achieved as shown in following sections. Details of this method will
in front of a computer screen were first shown a fixation cross at time t = be shown in section 3.2, Table 2, and Fig. 7 and classification perfor­
0 s, and a warning tone was rung at the same time. Two seconds later, at mance comparison will be explained in section 4.
t = 2 s, the left, right, upper or lower (left hand, right hand, foot and
tongue classes respectively) parts were shown on the screen via an arrow 3. Methods
and remained on the screen for 1.25 s. In this way, people were asked to
produce the desired motor imagery signals until t = 6 s and the fixation Within the framework of the methods and problems mentioned in the
cross disappeared from the screen. After this process, a short break was previous sections, we developed the NF-EEG model, which is a gener­
taken. These trial stages are shown in Fig. 4. alized CNN model that increases the classification success without
In the BCI Competition IV 2B dataset, there are data collected from 9 applying any signal preprocessing to the raw data. Input reshaping has
healthy individuals, different from those in the 2A dataset, with a been done to make this model faster and more efficient, as explained in
sampling rate of 250 Hz. Data were collected with 5 sessions for each the input shape section above. In addition, we applied two different data
subject, 120 trials for the first two sessions, and 160 trials for the last augmentation methods to generate new data from the original data and
three sessions, with a total of 6520 attempts. This dataset consists of 2 enlarge the dataset in order to increase the classification success. The
classes as right-hand and left-hand movements. While collecting data for details of the model architecture, input reshaping and data augmetation

4
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 6. NF-EEG Structure.

5
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 7. Input Shape Transformation.

Fig. 8. NF-EEG Hyperparameters and Grid Search Lists.

methods are explained below. used together with the mentioned layers. Convolution and average
pooling layers were used to extract relationships and features, and to
perform classification fully connected layers were used. CNN structures
3.1. NF-EEG architecture contain relatively fewer parameters than other neural networks and are
less prone to overfitting. The first priority during model design is to
NF-EEG was developed as a CNN model that performs automatic develop a model that extracts data features well, generalizable, robust,
feature extraction using raw eeg data. The training process of the model has low standard deviation, suitable for real time BCI systems, has high
is carried out without developing any filters for the signals or using classification success, and has appropriate hyperparameters. Previous
existing filters. In order to extract the distinctive and in-depth infor­ studies generally used a filter bank consisting of frequency ranges from 4
mation in the signals, this process is carried out automatically within the to 7 Hz, from 8 to 13 Hz, and from 13 to 32 Hz and also from 8 to 32 to
model without developing any feature extraction method. CNN struc­ obtain infromation in beta and mu bands [58–60] and also theta band
tures contain layers such as convolution, pooling and fully connected helps to identify difference between motor imagery tasks [61,62]. In
layers and in this study, depthwise and seperable convolution layers are

6
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

many studies, researchers used different model parameters for different column data refers to data from EEG channels. For each trial, a matrix of
subjects and trials and couldn’t generalize the models to work in a wide L × Nc structure was obtained, where L representing the timestamp and
area according to these parameters, or they trained many models with Nc the number of eeg channels. Where Nt representing the number of
different kernel sizes and created hybrid models. In [50], authors stated trials, a matrix of 1 × L × Nc structure was obtained for a trial, and Nt ×
that different kernel size values gave better results in different trials L × Nc structure was obtained for all trials.
from person to person and even for the same person, and kernel size In many studies, 2D matrix structures in the form of L × Nc or Nc × L
value could not be generalized, and they trained more than one model have been used to extract EEG features. In this study, by adding another
with different kernel size values. dimension to the 2D data for the NF-EEG model, the data was converted
NF-EEG aims to create a generalized model for all individuals and into a 3D matrix structure. This process has been done with the aim of
trials without using or developing any filter bank or signal preprocessing reducing the kernel size, the total number of parameters, complexity of
methods, rather than using signal preprocessing methods, training calculations rather than using 2D kernels, collecting each sensor data on
multiple models, using different parameters for different models and a single axis, scanning temporal information on a channel basis, and
subjects, and focusing on to develop models from person to person or speeding up the classification process. The matrix in the form of L × Nc
from trial to trial. Besides, for the hyperparameter adjustment of this in the trial example given above was transformed into L × Nc × 1 by
developed model, it is aimed to increase the success of the model by adding a third axis, and finally, the EEG channels were transformed into
using grid search in a wide variety of parameters. Detailed structure and L × 1 × Nc by moving the each EEG channel to the channel section in the
hyperparameter lists of the developed NF-EEG model are given in Fig. 6 deep learning library used. This transformation is shown in Fig. 7.
and Fig. 8.
In the proposed model, first the EEG data converted to L × 1 × Nc 3.3. Model generalization
form is entered into the system. Input shape, where B to represent batch
size, is a 4D tensor in the form of B × L × 1 × Nc. Then, the data group as Generalization is the ability of an artificial neural network to be
much as the batch size is entered into the batch normalization block to sensitive to new and previously unseen data from the same distribution
be normalized among themselves as mini batches. after it has been trained with a particular data [73] and also the ability to
The normalized data first enters Block 1 and passes through the obtain evaluation results close to the classification success achieved
layers in this block to extract the temporal features. While extracting during training. Neural networks are required to be fast, robust and have
these features, convolutional, normalization, depthwise convolutional high classification success during implementation. In order to develop
layers were used. 1D kernels are used along the vertical axis to extract models with these capabilities, generalization methods are applied and
time, frequency and space features between CNN layers. Then the elu it is aimed that the model approaches test successes close to the training
activation function is used, and then the data is average pooled at a rate success. For this purpose, layers and parameters such as dropout,
of 2 × 1 and sent to Block 2 with a dropout rate of 0.5. The data coming normalization, regulators, constraints are used and hyperparameter fine
to Block 2 is again subjected to the elu activation process by passing tuning is performed. Situations such as overfitting and underfitting may
through the convolution, separable convolution and normalization occur that may hinder the generalizability of a model. Overfitting is a
layers to extract the deep temporal features, and then it enters the situation where a model results from memorizing the data exactly,
dropout process at a rate of 0.5 by applying 2 × 1 average pooling and is rather than finding the relationships between the data used for training.
sent to Block 3. In this case, although the model achieves very high training accuracy
In Block 3 and Block 4, it was aimed to extract in-depth temporal during training, it achieves very low success in validation and test sets.
features within the signals and to establish temporal and spatial re­ Underfitting, on the other hand, occurs when a model cannot find the
lationships better by passing through the same layers in Block 1 and relationships between the data used during training, and in this case, the
Block 2, respectively, but with different parameters. The data in the form training accuracy values are very low and the model needs to be
of L × 1 × Nc coming out of Block 4 enters the flatten layer for classi­ developed before starting the testing process.
fication and is converted to a 1D feature vector. The feature vector is In this study, kernel constraint and kernel regulizer parameters were
sent to the dense layer for classification, where it performs classification. used, together with the above-mentioned dropout and normalization
After the classification process, classification success is achieved by layers, in order to find the relationships between multidimensional,
testing for each subject and also all data collectively by evaluation. complex and noisy EEG data and to obtain a generalized model. These
In order to determine the model parameters, the classification ac­ layers and parameters have been optimized with fine tuning and grid
curacies were examined by testing many different combinations from a search, overfitting of the model has been prevented and generalizability
wide range of options and selections were made from the model grid has been increased. Optimization processes for generalization and used
search list according to trained model and tested classification results. layers and parameters are explained in section 3.1 and section 4.
Model parameters and grid search lists are shown in Fig. 6 and Fig. 8.
3.4. Data augmentation
3.2. Input reshaping
The ability of neural networks to achieve a good classification suc­
In conventional methods for extracting spatio-temporal features, cess largely depends on the size of the training data. As the size of the
EEG data is commonly presented in 2D formats with dimensions (N, T), dataset increases, the desired features are found in the dataset in many
where N represents the number of electrodes and T represents the different scenarios and forms, thus making it easier for neural networks
number of sampling points [76]. Models with these data representations to establish relationships within the data. Conversely, if the dataset is
extract temporal and spatial features distinctly and subsequently small or insufficient, classification success is usually low.
[77,78]. And models can be unstable due to noise and interference in the The datasets used for classification of EEG motor imagery signals are
input and in their research [79], authors reshaped 2D data into 3D generally few in number and these datasets contain a limited number of
representation and derived spatial features from actual electrode loca­ trials. Strict rules such as the subject’s going through a training and
tions and reduced the interference and noise. In [80], researchers preparation stage for the collection of EEG signals, the person’s limb and
showed that 3D EEG model had superior performance than their 2D EEG muscle movements were prohibited, and person’s requirement to
model. maintain a constant state of focusing while imagining a movement are
In our process, each trial data was taken with time on the horizontal various obstacles to the creation of these datasets in large sizes. In order
axis and EEG channel data on the vertical axis. Therefore, for each trial, to overcome this problem, researchers obtained new training data both
dataset converted to matices where row data refers to timestamps and by making various transformations on the original data and adding

7
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Table 1
1 (x− μ)2
Number of Samples for Training and Test Sets. p(x) = √̅̅̅̅̅̅̅̅̅̅e− 2σ 2 (1)
2πσ2
Number of Samples

Dataset Classes Training Validation Test Set Where σ the standard deviation and μ is the mean. The square of the
Set Set (Unseen standard deviation, σ 2 , is called the variance.
(%80) (%20) Data)

2 Class BCI-IV-2A Left Hand, 930 233 1183 3.4.2. Scaling

Right Hand
Scaling or scale is the name given to the process that changes the
2 Class BCI-IV-2A Left Hand, 6513 1628 1183
w/Augmentation Right Hand
magnitude of the data by multiplying the data with a random scalar.
2 Class BCI-IV-2B Left Hand, 2421 605 2241 Random scalar is obtained by using STD and mean through Gaussian
Right Hand distribution in the scaling process and scaled data is obtained by
2 Class BCI-IV-2B Left Hand, 16946 4236 2241 multiplying these values with the data. Random scalar was obtained by
w/Augmentation Right Hand
using all data in each dataset with different STD values and the values
4 Class BCI-IV-2A Left Hand, 1862 466 2368
Right Hand, were multiplied with the signals and then these signals added to the
Foot, Tongue dataset. STD values of 0.03, 0.07 and 0.1 and mean value of 1 were
4 Class BCI-IV-2A Left Hand, 13037 3259 2368 applied.
w/Augmentation Right Hand,
Foot, Tongue
4. Experimental results

In this study, python were used as development language and to

Table 2 develop deep learning algorithms tensorflow and keras libraries were
Epoch Time Comparison of Models Before and After Input Reshaping. used. 12 core 3.80 GHz processor, 32 GB ram as memory and GeForce
Dataset BCI-IV-2A BCI-IV-2B RTX 2080 Super as GPU were used. As mentioned before, there are
Input Shape L × Nc × 1 L × 1 × Nc L × Nc × 1 L × 1 × Nc separate training and test sets in the datasets. All the training processes
Epoch Time 43 s 4s 18 s 7s were done only by using training sets, and then the trained model was
tested with the unseen test sets. Training sets were separated as 80%
training and 20% validation sets, and the model was evaluated using a 5-
various noises to the original data [52–56].
fold cross validation approach. The performance of the model was
In this study, in order to improve the classification accuracy, we
evaluated using the cross-subject approach. Total number of samples for
augmented the dataset by applying jittering and scaling operations
training, validation and test sets are shown in Table 1.
which explained in [63]. Details of the jittering and scaling operations
In order to prevent overfitting, the validation loss value was used as a
and variables are explained below.
variable and early stopping was applied and sparse categorical cross
entropy was used as loss function. If the validation loss value did not
3.4.1. Jittering
decrease during the repetitive 100 epochs, the model with the best
Jittering or jitter, is a way of simulating additive sensor noise. In
validation loss value was selected among the last 100 epochs, and
order to apply jittering to the signals, Gaussian distribution was applied
evaluation was made on the test set with this model. Evaluation process
with two parameters, standard deviation (STD) and mean. Gaussian
was implemented separately for each subject’s data to measure each
noise was obtained by using all data in each dataset with different STD
subject’s accuracy. Many different model combinations including
values and added to the signals and then these signals added to dataset.
different layer numbers and different layer types were examined to
STD values of 0.03, 0.07 and 0.1 and mean values of 0 were applied.
determine the model architecture, and many different values were
The probability density for the Gaussian distribution is
tested for many hyperparameters by applying grid search to determine

Fig. 9. Classification Performance of Different Input Shapes for BCI-IV-2A.

8
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 10. Classification Performance of Different Input Shapes for BCI-IV-2B.

Fig. 11. Classification Performance of Different Data Augmentation Methods for BCI-IV-2A.

Fig. 12. Classification Performance of Different Data Augmentation Methods for BCI-IV-2B.

9
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 13. Training and Validation Accuracies and Losses.

the model hyperparameters, and the hyperparameters that gave the best for the models they trained and classified on the same datasets. The
classification success were selected from this hyperparameter pool. Grid highly cited and recommended EEG-Net [35], ShallowConvNet [40] and
search lists and selected hyperparameters are shown in Fig. 6 and Fig. 8. DeepConvNet [40] EEG classification models were also adapted to
In this section, total of three main data groups as two-class (left hand, existing datasets and retrained with reference to the model structures
right hand) BCI Competition IV 2A data, two-class (left hand, right hand) and parameters specified in their papers for a fair comparison and these
BCI Competition IV 2B data and four-class (left hand, right hand, both classification results were used for comparison. Considering the infor­
feet and tongue) BCI Competition IV 2A data were trained and classified mation mentioned in the previous sections, NF-EEG, which was devel­
separately. The developed NF-EEG model and state-of-the-art CNN- oped using raw data without any signal preprocessing, achieved the best
based models and other deep learning and machine learning methods classification accuracy comparing to all state-of-the-art models. Classi­
were compared. In addition, the results before data augmentation were fication accuracy comparison tables and graphs are explained in detail
compared with the results after data augmentation, and the results in the following sections.
before the input reshaping with the results after the input reshaping
were also compared. Also different data augmentation methods were
compared with each other. The classification results of the compared 4.1. Performance of the input reshaping
methods were taken as the values stated in the researchers’ own papers
Input reshaping operations have provided great and significant

10
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 14. Training and Validation Accuracies with and without Data Augmentation.

Table 3
Classification Accuracy Comparison of State-of-the-art Methods for 2-class BCI-IV-2A Dataset.
Method S1 S2 S3 S4 S5 S6 S7 S8 S9 AVG

DFFS [25] 63.69 61.97 91.09 61.72 63.41 66.11 59.57 62.84 84.46 68.32
WTRCSP [64] 88.89 54.86 96.53 70.14 65.97 61.81 81.25 95.83 90.97 79.40
ALCSD [65] 90.28 54.17 93.75 64.58 57.64 65.28 65.20 90.97 85.42 73.84
TLCSD [65] 90.28 57.64 95.14 65.97 61.11 65.28 61.11 91.67 86.11 74.92
EMD-MI [66] 66.71 63.90 77.80 63.20 72.20 70.10 64.60 76.40 77.1 70.20
SS-MEMDBF [67] 91.49 60.56 94.16 76.72 58.52 68.52 78.57 97.01 93.85 79.93
EEG-Net [35] 72.92 69.90 74.99 78.72 93.33 82.63 91.29 72.90 72.77 78.75
DeepNet [40] 68.67 68.20 71.34 73.55 93.33 72.44 79.57 67.16 77.38 75.12
ShallowNet [40] 69.38 69.61 77.18 80.45 91.11 76.22 90.29 76.13 78.92 78.84
HS-CNN [50] 90.07 80.28 97.08 89.66 97.04 87.04 92.14 98.51 92.31 91.57
NF-EEG 91.49 89.44 95.62 93.10 94.81 88.89 97.14 99.25 92.31 93.56a
a
indicates p < 0.05 in Wilcoxon signed-rank test.

Table 4
Classification Accuracy Comparison of State-of-the-art Methods for 2-class BCI-IV-2B Dataset.
Method S1 S2 S3 S4 S5 S6 S7 S8 S9 AVG

BSP BCI [68] 77.00 65.00 60.5 96.00 82.00 84.50 75.00 90.50 87.50 80.00
FBCSP [69] 70.00 60.00 61.00 97.50 92.80 81.00 77.50 92.50 87.20 80.00
HCRF [27] 80.00 66.00 53.00 98.50 93.50 89.00 81.50 94.00 90.50 83.00
DFFS [25] 73.20 67.50 63.00 97.40 95.50 86.70 84.70 95.90 92.60 84.10
WPD + SE-Isomap [26] 84.58 66.25 62.92 95.83 89.17 97.92 82.08 86.25 97.08 84.68
FDBN [38] 81.00 65.00 66.00 98.00 93.00 88.00 82.00 94.00 91.00 84.00
CNN-SAE [24] 76.00 65.80 75.30 95.30 83.00 79.50 74.50 75.30 73.30 77.60
RSMM [70] 72.50 56.43 55.63 97.19 88.44 78.75 77.50 91.88 83.44 77.97
ALCSD [65] 71.90 50.00 52.80 93.40 54.40 73.10 62.20 77.80 75.00 67.90
TLCSD [65] 70.30 50.60 52.80 93.80 63.80 74.10 61.90 83.10 77.20 69.70
EMD-MI [66] 62.80 67.10 98.70 88.40 96.30 75.30 72.20 87.80 85.30 81.60
EEG-Net [35] 71.49 61.22 76.96 93.49 97.07 72.11 92.67 83.04 73.47 80.72
DeepNet [40] 64.47 60.00 73.04 74.59 89.38 71.31 83.62 90.87 59.18 74.16
ShallowNet [40] 66.67 65.31 73.91 80.13 95.24 67.33 90.95 89.13 61.22 76.89
HS-CNN [50] 80.50 70.60 85.60 94.60 98.30 86.60 89.60 95.60 87.40 87.60
NF-EEG 82.46 75.47 78.70 96.74 98.53 86.85 91.61 95.04 90.24 88.40a
a
indicates p < 0.05 in Wilcoxon signed-rank test.

improvements in classification accuracy. Among the first three subjects Also, as mentioned earlier, this input reshaping process has reduced
with the highest results and highest increase in BCI-IV-2A dataset, the the number of complex computations for the layers to wander along the
classification accuracy increase rate was 62.83% for subject 2, 54.64% 2D data axes of the kernels and considerably shortened the time required
for subject 8, 51.78% for subject 1 and 44.67% on average. BCI-IV-2A for each epoch. A shortening of 2.57 times for the 3-channel 2B dataset
input reshaping results are shown in Fig. 9. For the BCI-IV-2B dataset, and 10.75 times for the 22-channel 2A dataset has occurred. As ex­
the classification accuracy rate increased by 36.97% for subject 2, pected, as the number of channels increases, the gain in the 22-channel
35.08% for subject 3, 23.95% for subject 5 and an average of 17.01%. dataset was higher than in the 3-channel dataset, as the kernels would
BCI-IV-2B input reshaping results are shown in Fig. 10. have to scan a larger area. With this striking change, it is clearly seen

11
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Table 5
Classification Accuracy Comparison of State-of-the-art Methods for 4-class BCI-IV-2A Dataset.
Method S1 S2 S3 S4 S5 S6 S7 S8 S9 AVG

DFFS [25] 63.69 61.97 91.09 61.72 63.41 66.11 59.57 62.84 84.46 68.32
EMD-MI [66] 66.70 63.90 77.80 63.20 72.20 70.10 64.60 76.40 77.10 70.20
EA-CSP-LDA [71] 87.50 56.25 98.61 73.61 50.00 64.58 68.75 89.58 72.93 73.53
EEG-Net [35] 77.73 63.07 86.52 73.61 68.58 61.60 79.17 67.94 56.03 70.69
DeepNet [40] 74.75 62.24 82.29 72.98 70.94 61.74 82.98 66.25 51.59 69.74
ShallowNet [40] 75.82 60.12 85.59 69.04 68.41 54.30 80.09 68.10 59.17 69.36
WTRCSP [64] 88.89 51.39 96.53 70.14 54.86 71.53 81.25 93.75 93.75 78.01
TLCSD [65] 90.28 54.17 93.75 64.58 57.64 65.28 65.20 90.97 85.42 73.84
C2CM [72] 87.50 65.28 90.28 66.67 62.50 45.49 89.58 83.33 79.51 74.46
FBCSP-SVM [28] 85.31 64.51 90.00 64.02 73.66 52.72 92.10 88.62 72.10 75.89
FBCNet [74] 85.76 61.07 94.51 68.84 82.54 58.71 93.08 86.21 80.54 79.03
MBEEGSE [81] 89.14 69.73 95.27 81.42 80.00 63.25 94.06 89.57 83.35 82.87
CNN-BiLSTM [82] 89.23 72.53 97.23 76.28 82.42 69.15 94.76 86.14 86.10 82.84
TCNet-Fusion [83] 90.74 70.67 95.23 76.75 82.24 68.83 94.22 88.92 85.98 83.73
NF-EEG 83.63 68.90 90.11 82.89 75.72 73.95 89.53 86.72 78.03 81.05a
a
indicates p < 0.05 in Wilcoxon signed-rank test.

Table 6
Subject-Based Average Accuracy Increase Rates of NF-EEG according to Compared Models.
2 Class BCI-IV-2A 2 Class BCI IV-2B 4 Class BCI-IV-2A

Average NF-EEG IR* Average NF-EEG IR* Average NF-EEG IR*

Subject 1 79.24 91.49 15.46 73.49 82.46 12.20 82.36 83.63 1.54
Subject 2 64.11 89.44 39.51 62.45 75.47 20.84 62.64 68.90 10.00
Subject 3 86.91 95.62 10.03 67.41 78.70 16.75 90.05 90.11 0.01
Subject 4 72.47 93.10 28.47 92.94 96.74 4.09 70.20 82.89 18.07
Subject 5 75.37 94.81 25.80 87.46 98.53 12.66 69.24 75.72 9.35
Subject 6 71.54 88.89 24.25 80.35 86.85 8.09 62.39 73.95 18.54
Subject 7 76.36 97.14 27.21 79.19 91.61 15.68 81.39 89.53 10.01
Subject 8 82.94 99.25 19.66 88.51 95.04 7.38 81.33 86.72 6.63
Subject 9 83.93 92.31 9.99 81.43 90.24 10.82 76.29 78.03 2.28
Average 77.09 93.56 21.37 79.33 88.40 11.44 75.18 81.05 7.81
*
Increase Rate.

that the time to be gained during the training of models that has multi- 3.46 among all models compared. BCI-IV-2A 2-class performance com­
channel and multiple epochs is quite high. The times taken for an epoch parison results are shown in Table 3.
are shown in Table 2.
4.4. BCI Competiton IV 2B – 2 class performance comparison
4.2. Performance of the data augmentation methods
In this section, the proposed method is compared with the methods
As a result of the jittering and scaling processes applied to the developed with the 2-class BCI-IV-2B dataset. NF-EEG achieved the best
datasets, among the first three subjects with the highest results and average classification accuracy results among all models compared.
highest increase in BCI-IV-2A dataset, the classification accuracy in­ Among the first three subjects with the highest results and highest in­
crease rate was 15.39% for subject 9, 7.86% for subject 6, 7.63% for crease in 2-class BCI-IV-2B dataset, the classification accuracy increase
subject 2, and 4.42% on average. BCI-IV-2A data augmentation results rate was 6.90% for subject 2, 3.25% for subject 9, 2.43% for subject 1
are shown in Fig. 11. and 1.00% on average. This model has a standard deviation as 8.13. BCI-
For the BCI-IV-2B dataset, the classification accuracy rate increased IV-2B 2-class performance comparison results are shown in Table 4.
by 12.43% for subject 3, 5.29% for subject 9, 4.30% for subject 6 and an
average of 2.59%. BCI-IV-2B data augmentation results are shown in 4.5. BCI Competiton IV 2A – 4 class performance comparison
Fig. 12.
Training and validation accuracies and losses were recorded during In this section, the proposed method is compared with the methods
training of datasets. These records include information according to developed with the 4-class BCI-IV-2A dataset. NF-EEG achieved one of
number of epochs. These records include information before and after the best average classification accuracy results among all models
data augmentation and their graphs are shown in Fig. 13 and Fig. 14 compared. Among the first three subjects with the highest results and
respectively. highest increase in 4-class BCI-IV-2A dataset, the classification accuracy
increase rate was 12.61% for subject 4, 5.55% for subject 2, 3.38% for
4.3. BCI Competiton IV 2A – 2 class performance comparison subject 6 and 2.56% on average. This model has a standard deviation as
7.35. BCI-IV-2A 4-class performance comparison results are shown in
In this section, the proposed method is compared with the methods Table 5.
developed with the 2-class BCI-IV-2A dataset. NF-EEG achieved the best
average classification accuracy results among all models compared. 4.6. Statistical informations of NF-EEG models
Among the first three subjects with the highest results and highest in­
crease in 2-class BCI-IV-2A dataset, the classification accuracy increase In this section, statistical data of the developed NF-EEG model were
rate was 11.41% for subject 2, 5.43% for subject 7, 3.80% for subject 4 calculated and these data were compared with previous studies. In this
and 2.20% on average. This model has a lowest standard deviation as context, the average accuracy values of the compared models for each

12
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

Fig. 15. Confusion Matrices.

Along with all these information, the true positive, false positive,
Table 7 false negative, and true negative rates and amounts of the NF-EEG model
Statistical Values of NF-EEG Models. for each dataset and each class are shown with confusion matrices in
Dataset Precision Recall F1 Score Kappa STD* Fig. 15. In addition, the average precision, recall, F1 score, kappa and
standard deviation values of the proposed model are given in Table 7.
2 Class BCI-IV-2A 0.938 0.937 0.937 0.873 3.461
2 Class BCI-IV-2B 0.886 0.884 0.884 0.768 8.131
4 Class BCI-IV-2A 0.813 0.812 0.812 0.749 7.346 5. Conclusion
*
STD (standard deviation) values are for accuracy.
Almost all of the existing EEG based motor imagery classification
methods have applied a wide variety of signal preprocessing methods to
dataset were calculated according to the subjects. These values were
a greater or lesser extent and have developed their models in this way. At
compared with the accuracy values of the proposed method and the
the same time, various data augmentation methods have been developed
average accuracy increase rates (IR) were calculated. Although it has
to enlarge the limited datasets. However, signal preprocessing methods
different ranges in each dataset, it has subject-based accuracy increase
are largely based on human knowledge and experience, and the devel­
rates ranging from 0.01% to 39.51% for all datasets. These values are
opment and implementation of these methods causes a great loss of
shown in Table 6.
labor and time. In addition, inappropriate or incorrectly applied signal

13
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

preprocessing methods greatly affect and reduce the classification suc­ [5] Y. He, D. Eguren, J. Azorín, R. Grossman, T. Luu, J. Contreras-Vidal,
Brain–machine interfaces for controlling lower-limb powered robotic systems,
cess. However, since there may be unknown relations and patterns in the
J. Neural Eng. 15 (2018) 021004, https://doi.org/10.1088/1741-2552/aaa8c0.
dataset, it is not possible to perform preprocessing operations to detect [6] H. Pan, W. Mi, X. Lei, J. Deng, A closed-loop brain–machine interface framework
these patterns without having this pattern information. design for motor rehabilitation, Biomed. Signal Process. Control 58 (2020) 101877,
For the reasons mentioned above, we developed the NF-EEG model, https://doi.org/10.1016/j.bspc.2020.101877.
[7] N. Johnson, J. Carey, B. Edelman, A. Doud, A. Grande, K. Lakshminarayan, et al.,
which trains and classifies directly with raw data without applying any Combined rTMS and virtual reality brain–computer interface training for motor
signal preprocessing, and with this model, we achieved higher classifi­ recovery after stroke, J. Neural Eng. 15 (2018) 016009, https://doi.org/10.1088/
cation success than state-of-the-art models with or without signal pre­ 1741-2552/aa8ce3.
[8] R. Riener, The Cybathlon promotes the development of assistive technology for
processing. In addition, we increased the classification success with people with physical disabilities, J. Neuroeng. Rehabil. 13 (2016), https://doi.org/
input reshaping and data augmentation methods. Along with all this, we 10.1186/s12984-016-0157-2.
have reduced the amount of effort and time required to achieve a [9] B. He, B. Baxter, B. Edelman, C. Cline, W. Ye, Noninvasive brain-computer
interfaces based on sensorimotor rhythms, Proc. IEEE 103 (2015) 907–925,
notable classification result by automating the EEG signal classification https://doi.org/10.1109/jproc.2015.2407272.
processes, which are very difficult to analyze and to extract features, and [10] A. Al-Saegh, S. Dawwd, J. Abdul-Jabbar, Deep learning for motor imagery EEG-
also require a lot of effort and time. based classification: A review, Biomed. Signal Process. Control 63 (2021) 102172,
https://doi.org/10.1016/j.bspc.2020.102172.
With this study, we outperformed many state-of-the-art models by [11] K. Abualsaud, M. Mahmuddin, M. Saleh, A. Mohamed, Ensemble classifier for
achieving a classification success of 93.56% in the two-class BCI Com­ epileptic seizure detection for imperfect EEG Data, Scientific World Journal 2015
petiton IV 2A dataset, a classification success of 88.40% in the two-class (2015) 1–15, https://doi.org/10.1155/2015/945689.
[12] C. Tsui, J. Gan, H. Hu, A Self-Paced Motor Imagery Based Brain-Computer Interface
BCI Competiton IV 2B dataset, and a classification success of 81.05% in
for Robotic Wheelchair Control, Clinical EEG And Neuroscience. 42 (2011) 225-
the four-class BCI Competiton IV 2A dataset. We have shown that high 229. doi:10.1177/155005941104200407.
classification successes can be achieved with the right model architec­ [13] D.W. Ng, Y. Soh, S. Goh, Development of an autonomous BCI wheelchair 2014,
ture, appropriate hyperparameter selection and efficient input shape IEEE Symp. on Comput Intell. Brain Comput. Interfaces. (2014).
[14] J. Heo, G. Yoon, EEG studies on physical discomforts induced by virtual reality
structure by using raw data without applying signal preprocessing gaming, J. Electr. Eng. Amp Technol. 15 (2020) 1323–1329, https://doi.org/
methods. 10.1007/s42835-020-00373-1.
NF-EEG is also emerging as an option for the development and [15] Pires G, Torres M, Casaleiro N, Nunes U and Castelo-Branco M 2011 Playing Tetris
with non-invasive BCI 2011 IEEE 1st Int. Conf. Serious Games Appl. Heal. SeGAH
implementation of real-time BCI systems. It is generalizable, robust, and 2011.
fast with high classification accuracies, low standard deviation rates, [16] B. van de Laar, H. Gurkok, D. Plass-Oude Bos, M. Poel, A. Nijholt, Experiencing BCI
and fast evaluation times which needed for these systems. control in a popular computer game, IEEE Trans. Comput. Intell. AI Games 5
(2013) 176–184, https://doi.org/10.1109/tciaig.2013.2253778.
[17] L. Bonnet, F. Lotte, A. Lecuyer, Two Brains, One Game: Design and Evaluation of a
CRediT authorship contribution statement Multiuser BCI Video Game Based on Motor Imagery, IEEE Transactions On
Computational Intelligence And AI In Games. 5 (2013) 185-198. doi:10.1109/
tciaig.2012.2237173.
Emre Arı: Conceptualization, Formal analysis, Investigation, Meth­ [18] R. Alazrai, M. Abuhijleh, H. Alwanni, M. Daoud, A deep learning framework for
odology, Software, Visualization, Writing – original draft, Writing – decoding motor imagery tasks of the same hand using EEG signals, IEEE Access 7
review & editing. Ertuğrul Taçgın: Conceptualization, Methodology, (2019) 109612–109627, https://doi.org/10.1109/access.2019.2934018.
[19] X. Zhu, P. Li, C. Li, D. Yao, R. Zhang, P. Xu, Separated channel convolutional neural
Supervision, Writing – original draft, Writing – review & editing.
network to realize the training free motor imagery BCI systems, Biomed. Signal
Process. Control 49 (2019) 396–403, https://doi.org/10.1016/j.bspc.2018.12.027.
Declaration of competing interest [20] A. Procházka, J. Kukal, O. Vyšata, Wavelet transform use for feature extraction and
EEG signal segments classification, 2008 3rd Int Symp. Commun. Control. Signal
Process. ISCCSP 2008 (2008) 719–722.
The authors declare that they have no known competing financial [21] A. Zabidi, W. Mansor, Y.K. Lee, C.W.N.F. Che Wan Fadzal, Short-time Fourier
interests or personal relationships that could have appeared to influence Transform analysis of EEG signal generated during imagined writing, Proc. 2012
the work reported in this paper. Int Conf. Syst. Eng. Technol. ICSET 2012 (2012) 12–15.
[22] N. Bagh, M. Reddy, Hilbert transform-based event-related patterns for motor
imagery brain computer interface, Biomed. Signal Process. Control 62 (2020)
Data availability 102020, https://doi.org/10.1016/j.bspc.2020.102020.
[23] B. Edelman, B. Baxter, B. He, EEG source imaging enhances the decoding of
complex right-hand motor imagery tasks, IEEE Trans. Biomed. Eng. 63 (2016)
Publicly available BCI Competition IV 2A and BCI Competition IV 2B 4–14, https://doi.org/10.1109/tbme.2015.2467312.
datasets were used in this research and these can be accessed via https:// [24] Y. Tabar, U. Halici, A novel deep learning approach for classification of EEG motor
www.bbci.de/competition/iv/ imagery signals, J. Neural Eng. 14 (2016) 016003, https://doi.org/10.1088/1741-
2560/14/1/016003.
[25] J. Luo, Z. Feng, J. Zhang, N. Lu, Dynamic frequency feature selection based
Acknowledgment approach for classification of motor imageries, Comput. Biol. Med. 75 (2016)
45–53, https://doi.org/10.1016/j.compbiomed.2016.03.004.
[26] M. Li, W. Zhu, H. Liu, J. Yang, Adaptive feature extraction of motor imagery EEG
This research did not receive any specific grant from funding
with optimal wavelet packets and SE-isomap, Appl. Sci. 7 (2017) 390, https://doi.
agencies in the public, commercial, or not-for-profit sectors. org/10.3390/app7040390.
[27] J. Saa, M. Çetin, A latent discriminative model-based approach for classification of
References imaginary motor tasks from EEG data, J. Neural Eng. 9 (2012) 026020, https://doi.
org/10.1088/1741-2560/9/2/026020.
[28] K.K. Ang, Z.Y. Chin, H. Zhang, C. Guan, Filter Bank Common Spatial Pattern
[1] J. Meng, S. Zhang, A. Bekyo, J. Olsoe, B. Baxter, B. He, Noninvasive (FBCSP) in brain-computer interface Proc, Int. Jt. Conf. Neural Networks (2008)
electroencephalogram based control of a robotic arm for reach and grasp tasks, Sci. 2390–2397.
Rep. 6 (2016), https://doi.org/10.1038/srep38565. [29] K. Ang, Z. Chin, H. Zhang, C. Guan, Mutual information-based selection of optimal
[2] Z. Liu, J. Shore, M. Wang, F. Yuan, A. Buss, X. Zhao, A systematic review on hybrid spatial–temporal patterns for single-trial EEG-based BCIs, Pattern Recogn. 45
EEG/fNIRS in brain-computer interface, Biomed. Signal Process. Control 68 (2021) (2012) 2137–2144, https://doi.org/10.1016/j.patcog.2011.04.018.
102595. [30] H. Lee, Y. Choi, Application of continuous wavelet transform and convolutional
[3] N. Bhagat, A. Venkatakrishnan, B. Abibullaev, E. Artz, N. Yozbatiran, A. Blank, et neural network in decoding motor imagery brain-computer interface, Entropy 21
al., Design and optimization of an EEG-based brain machine interface (BMI) to an (2019) 1199, https://doi.org/10.3390/e21121199.
upper-limb exoskeleton for stroke survivors, Front. Neurosci. 10 (2016), https:// [31] O. Faust, Y. Hagiwara, T. Hong, O. Lih, U. Acharya, Deep learning for healthcare
doi.org/10.3389/fnins.2016.00122. applications based on physiological signals: A review, Comput. Methods Programs
[4] M. Aldayel, M. Ykhlef, A. Al-Nafjan, Deep learning for EEG-based preference Biomed. 161 (2018) 1–13, https://doi.org/10.1016/j.cmpb.2018.04.005.
classification in neuromarketing, Appl. Sci. 10 (2020) 1525, https://doi.org/ [32] A. Craik, Y. He, J. Contreras-Vidal, Deep learning for electroencephalogram (EEG)
10.3390/app10041525. classification tasks: a review, J. Neural Eng. 16 (2019) 031001, https://doi.org/
10.1088/1741-2552/ab0ab5.

14
E. Arı and E. Taçgın Biomedical Signal Processing and Control 92 (2024) 106081

[33] Zhang X, Yao L, Huang C, Sheng Q Z and Wang X 2017 Intent Recognition in Smart [58] D.J. McFarland, L.A. Miner, T.M. Vaughan, J.R. Wolpaw, Mu and beta rhythm
Living Through Deep Recurrent Neural Networks Lect. Notes Comput. Sci. topographies during motor imagery and actual movements, Brain Topogr. 12
(including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) 10635 LNCS (2000) 177–186.
748–58. [59] R. Djemal, A. Bazyed, K. Belwafi, S. Gannouni, W. Kaaniche, Three-class EEG-based
[34] Z. Zhang, J. Sun, T. Chen, A new dynamically convergent differential neural motor imagery classification using phase-space reconstruction technique, Brain Sci.
network for brain signal recognition, Biomed. Signal Process. Control 71 (2022) 6 (2016) 36, https://doi.org/10.3390/brainsci6030036.
103130. [60] S. Shahid, R. Sinha, G. Prasad, Mu and beta rhythm modulations in motor imagery
[35] V. Lawhern, A. Solon, N. Waytowich, S. Gordon, C. Hung, B. Lance, EEGNet: a related post-stroke EEG: a study under BCI framework for post-stroke
compact convolutional neural network for EEG-based brain–computer interfaces, rehabilitation, BMC Neurosci. 11 (2010), https://doi.org/10.1186/1471-2202-11-
J. Neural Eng. 15 (2018) 056013, https://doi.org/10.1088/1741-2552/aace8c. s1-p127.
[36] R. Sharma, M. Kim, A. Gupta, Motor imagery classification in brain-machine [61] E. Weber, M. Doppelmayr, Kinesthetic motor imagery training modulates frontal
interface with machine learning algorithms: Classical approach to multi-layer midline theta during imagination of a dart throw, Int. J. Psychophysiol. 110 (2016)
perceptron model, Biomed. Signal Process. Control 71 (2022) 103101, https://doi. 137–145, https://doi.org/10.1016/j.ijpsycho.2016.11.002.
org/10.1016/j.bspc.2021.103101. [62] Y. Liu, L. Lin, C. Chou, Y. Chang, Y. Hsiao, W. Hsu, Analysis of
[37] S.R. Carvalho, I.C. Filho, R.D.O. De, A.C. Siravenha, S.C. De, H.G. Debarba, electroencephalography event-related desynchronisation and synchronisation
B. Gomes, R. Boulic, A deep learning approach for classification of reaching targets induced by lower-limb stepping motor imagery, J. Med. Biol. Eng. 39 (2018)
from EEG images, Proc. - 30th Conf Graph. Patterns Images, SIBGRAPI 2017 (2017) 54–69, https://doi.org/10.1007/s40846-018-0379-9.
178–184. [63] Um T T, Pfister F M J, Pichler D, Endo S, Lang M, Hirche S, Fietzek U and Kulic D
[38] N. Lu, T. Li, X. Ren, H. Miao, A deep learning scheme for motor imagery 2017 Data augmentation of wearable sensor data for Parkinson’s disease
classification based on restricted boltzmann machines, IEEE Trans. Neural Syst. monitoring using convolutional neural networks ICMI 2017 - Proc. 19th ACM Int.
Rehabil. Eng. 25 (2017) 566–576, https://doi.org/10.1109/tnsre.2016.2601240. Conf. Multimodal Interact. 2017-Janua 216–20.
[39] Bashivan P, Rish I, Yeasin M and Codella N 2016 Learning representations from [64] F. Lotte, C. Guan, Regularizing common spatial patterns to improve BCI designs:
EEG with deep recurrent-convolutional neural networks 4th Int. Conf. Learn. unified theory and new algorithms, IEEE Trans. Biomed. Eng. 58 (2011) 355–362,
Represent. ICLR 2016 - Conf. Track Proc. 1–15. https://doi.org/10.1109/tbme.2010.2082539.
[40] R. Schirrmeister, J. Springenberg, L. Fiederer, M. Glasstetter, K. Eggensperger, [65] H. Raza, H. Cecotti, Y. Li, G. Prasad, Adaptive learning with covariate shift-
M. Tangermann, et al., Deep learning with convolutional neural networks for EEG detection for motor imagery-based brain–computer interface, Soft. Comput. 20
decoding and visualization, Hum. Brain Mapp. 38 (2017) 5391–5420, https://doi. (2015) 3085–3096, https://doi.org/10.1007/s00500-015-1937-5.
org/10.1002/hbm.23730. [66] Gaur P, Pachori R B, Wang H and Prasad G 2015 An empirical mode decomposition
[41] N. Padfield, J. Zabalza, H. Zhao, V. Masero, J. Ren, EEG-based brain-computer based filtering method for classification of motor-imagery EEG signals for
interfaces using motor-imagery: techniques and challenges, Sensors 19 (2019) enhancing brain-computer interface Proc. Int. Jt. Conf. Neural Networks 2015-
1423, https://doi.org/10.3390/s19061423. Septe.
[42] G. Litjens, T. Kooi, B. Bejnordi, A. Setio, F. Ciompi, M. Ghafoorian, et al., A survey [67] P. Gaur, R. Pachori, H. Wang, G. Prasad, A multi-class EEG-based BCI classification
on deep learning in medical image analysis, Med. Image Anal. 42 (2017) 60–88, using multivariate empirical mode decomposition based filtering and Riemannian
https://doi.org/10.1016/j.media.2017.07.005. geometry, Expert Syst. Appl. 95 (2018) 201–211, https://doi.org/10.1016/j.
[43] Schirrmeister R, Gemein L, Eggensperger K, Hutter F and Ball T 2017 Deep learning eswa.2017.11.007.
with convolutional neural networks for decoding and visualization of EEG [68] S. Shahid, G. Prasad, Bispectrum-based feature extraction technique for devising a
pathology 2017 IEEE Signal Process. Med. Biol. Symp. SPMB 2017 - Proc. 2018- practical brain–computer interface, J. Neural Eng. 8 (2011) 025014, https://doi.
Janua 1–7. org/10.1088/1741-2560/8/2/025014. J. Neural Eng. 8.
[44] R. Ron-Angevin, F. Velasco-Álvarez, Á. Fernández-Rodríguez, A. Díaz-Estrella, [69] K. Ang, Z. Chin, C. Wang, C. Guan, H. Zhang, Filter bank common spatial pattern
M. Blanca-Mena, F. Vizcaíno-Martín, Brain-Computer Interface application: algorithm on BCI competition IV datasets 2a and 2b, Front. Neurosci. 6 (2012),
auditory serial interface to control a two-class motor-imagery-based wheelchair, https://doi.org/10.3389/fnins.2012.00039.
J. Neuroeng. Rehabil. 14 (2017), https://doi.org/10.1186/s12984-017-0261-y. [70] Q. Zheng, F. Zhu, P. Heng, Robust support matrix machine for single trial EEG
[45] H. Yu, H. Lu, S. Wang, K. Xia, Y. Jiang, P. Qian, A general common spatial patterns classification, IEEE Trans. Neural Syst. Rehabil. Eng. 26 (2018) 551–562, https://
for EEG analysis with applications to vigilance detection, IEEE Access 7 (2019) doi.org/10.1109/tnsre.2018.2794534.
111102–111114, https://doi.org/10.1109/access.2019.2934519. [71] H. He, D. Wu, Transfer learning for brain-computer interfaces: A Euclidean space
[46] B. Rim, N. Sung, S. Min, M. Hong, Deep learning in physiological signal data: A data alignment approach, IEEE Trans. Biomed. Eng. 67 (2020) 399–410, https://
survey, Sensors 20 (2020) 969, https://doi.org/10.3390/s20040969. doi.org/10.1109/tbme.2019.2913914.
[47] Tayeb Z, Ercelik E and Conradt J 2017 Decoding of motor imagery movements [72] S. Sakhavi, C. Guan, S. Yan, Learning temporal information for brain-computer
from EEG signals using SpiNNaker neuromorphic hardware Int. IEEE/EMBS Conf. interface using convolutional neural networks, IEEE Trans. Neural Networks Learn.
Neural Eng. NER 263–6. Syst. 29 (2018) 5619–5629, https://doi.org/10.1109/tnnls.2018.2789927.
[48] A.N. Niranjani, M. Sivachitra, Motor imagery signal classification using spiking [73] B. Swiderski, S. Osowski, G. Gwardys, J. Kurek, M. Slowinska, I. Lugowska,
neural network, Proc. Int. Conf. Intell. Sustain. Syst. ICISS 2017 (2018) 901–904. Random CNN structure: tool to increase generalization ability in deep learning,
[49] D. Zhang, L. Yao, X. Zhang, S. Wang, W. Chen, R. Boots, Cascade and parallel EURASIP J Image Video Process. 2022 (2022), https://doi.org/10.1186/s13640-
convolutional recurrent neural networks on EEG-based intention recognition for 022-00580-y.
brain computer interface, 32nd AAAI Conf. Artif. Intell. AAAI 2018 (2018) [74] Mane, R., Chew, E., Chua, K., Ang, K. K., Robinson, N., Vinod, A. P., ... & Guan, C.
1703–1710. (2021). FBCNet: A multi-view convolutional neural network for brain-computer
[50] G. Dai, J. Zhou, J. Huang, N. Wang, HS-CNN: a CNN with hybrid convolution scale interface. arXiv preprint arXiv:2104.01233.
for EEG motor imagery classification, J. Neural Eng. 17 (2020) 016025, https:// [75] H. Altaheri, et al., Deep learning techniques for classification of
doi.org/10.1088/1741-2552/ab405f. electroencephalogram (EEG) motor imagery (MI) signals: a review, Neural
[51] Y. Li, X. Zhang, B. Zhang, M. Lei, W. Cui, Y. Guo, A channel-projection mixed-scale Comput. Applic. (2021).
convolutional neural network for motor imagery EEG decoding, IEEE Trans. Neural [76] D. Borra, S. Fantozzi, E. Magosso, Interpretable and lightweight convolutional
Syst. Rehabil. Eng. 27 (2019) 1170–1180, https://doi.org/10.1109/ neural network for EEG decoding: Application to movement execution and
tnsre.2019.2915621. imagination, Neural Netw. 129 (2020) 55–74.
[52] G. Xu, X. Shen, S. Chen, Y. Zong, C. Zhang, H. Yue, et al., A deep transfer [77] H. Zeng, Z. Wu, J. Zhang, C. Yang, H. Zhang, G. Dai, W. Kong, EEG emotion
convolutional neural network framework for EEG signal classification, IEEE Access classification using an improved SincNet-based deep learning model, Brain Sci. 9
7 (2019) 112767–112776, https://doi.org/10.1109/access.2019.2930958. (11) (2019).
[53] X. Zhao, H. Zhang, G. Zhu, F. You, S. Kuang, L. Sun, A multi-branch 3D [78] D. Zhao, F. Tang, B. Si, X. Feng, Learning joint space–time–frequency features for
convolutional neural network for EEG-based motor imagery classification, IEEE EEG decoding on small labeled data, Neural Netw. 114 (2019) 67–77.
Trans. Neural Syst. Rehabil. Eng. 27 (2019) 2164–2177, https://doi.org/10.1109/ [79] D. Zhang, L. Yao, K. Chen, S. Wang, X. Chang, Y. Liu, Making sense of spatio-
tnsre.2019.2938295. temporal preserving representations for EEG-based human intention recognition,
[54] I. Majidov, T. Whangbo, Efficient classification of motor imagery IEEE Trans. Cybern. 50 (2020) 3033–3044.
electroencephalography signals using deep learning methods, Sensors 19 (2019) [80] E. Arı, E. Taçgın, Input shape effect on classification performance of raw EEG
1736, https://doi.org/10.3390/s19071736. motor imagery signals with convolutional neural networks for use in
[55] S. Khan, M. Hayat, F. Porikli, Regularization of deep neural networks with spectral brain–Computer interfaces, Brain Sci. 13 (2) (2023).
dropout, Neural Netw. 110 (2019) 82–90, https://doi.org/10.1016/j. [81] G.A. Altuwaijri, et al., A multi-branch convolutional neural network with squeeze-
neunet.2018.09.009. and-excitation attention blocks for eeg-based motor imagery signals classification,
[56] Nurvitadhi E, Venkatesh G, Sim J, Marr D, Huang R, Ong J G H, Liew Y T, Srivatsan Diagnostics 12 (4) (2022) 995.
K, Moss D, Subhaschandra S and Boudoukh G 2017 Can FPGAs beat GPUs in [82] S.U. Amin, et al., Attention-inception and long-short-term memory-based
accelerating next-generation deep neural networks? FPGA 2017 - Proc. 2017 ACM/ electroencephalography classification for motor imagery tasks in rehabilitation,
SIGDA Int. Symp. Field-Programmable Gate Arrays 5–14. IEEE Trans. Ind. Inf. 18 (8) (2021) 5412–5421.
[57] Brunner C, Leeb R, Müller-Putz G, Schlögl A and Pfurtscheller G BCI Competition [83] Y.K. Musallam, et al., Electroencephalography-based motor imagery classification
2008—Graz data sets 2A and 2B (Graz: Institute for Knowledge Discovery) (http:// using temporal convolutional network fusion, Biomed. Signal Process. Control 69
bbci.de/ competition/iv/). (2021) 102826.

15