2023 IEEE International Conference on Bioinformatics and Biomedicine

MEGNet: A MEG-Based Deep Learning Model for

Cognitive and Motor Imagery Classification
Minerva Sarma∗ , Charles Bond∗ , Sanjeev Nara† and Haider Raza∗
∗ School of Computer Science and Electronics Engineering, University of Essex, Colchester, United Kingdom.
2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) | 979-8-3503-3748-8/23/$31.00 ©2023 IEEE | DOI: 10.1109/BIBM58861.2023.10385695

† Department of Mathematics and Computer Science, Physics, Geography, Mathematics Institute,

Justus-Liebig-Universität Gießen, Gießen, Germany.

Abstract—Decoding complex patterns associated with task- distinction exists between cognitive and motor imagery (MI).
specific activities embedded within magnetoencephalography Cognitive imagery (CI) encapsulates the mental processes
(MEG) signals is pivotal for understanding brain functions and involved in visualizing scenarios, concepts, or objects, while
developing applications such as brain-computer interfacing. It
is widely recognized that machine learning algorithms rely on MI pertains to the mental simulation of movement without
feature extraction before undertaking decoding tasks. In this actual physical execution. Exploration of CI and MI is sig-
work, we introduce MEGNet, aiming to enhance the single-trial nificant because it provides insights into various aspects of
decoding framework of a compact deep neural network inspired human cognition and behaviour. Studies have shown that MI
by EEGNet, a model widely utilized in electroencephalography involves the generation, maintenance, manipulation, and tem-
(EEG) studies. MEGNet accepts raw MEG signals, evoked
responses and frequency spectrum as input. For validation, the poral sequencing of motor images [5]. Additionally, cognitive
MEG dataset containing motor and cognitive imagery tasks and psychological measures have been found to impact the
was used for classification. We performed pair-wise decoding of performance of MI brain-computer interfaces (MI-BCIs), with
cognitive and motor tasks. Classification accuracy was evaluated factors such as vividness of visual imagery, personality traits,
using metric scores and benchmarked against ShallowConvNet and motivation playing a role [6]. CI and MI classification,
and DeepConvNet. Our findings demonstrate that MEGNet
can successfully decode between cognitive and mental imagery within the context of this study, pertains to the discernment
tasks. This MEGNet model surpasses existing feature extraction and categorization of distinct mental states and intentions
techniques, exhibiting consistent and stable mean accuracy of through brain activity patterns. Remarkably, researchers have
64.76% ± 3% across tasks and subjects. All codes are available made a significant stride in categorically classifying visual
at our GitHub repository: https://github.com/Charliebond125/ and CI from MEG signal data [7], [8]. A large number
MEGNet.git.
Index Terms—Magnetoencephalography, Convolutional Neural
of existing research studies have predominantly focused on
Network, Deep Learning, Cognitive and Motor Imagery. decoding visual and motor intentions, often bypassing the
complex interplay between cognitive processes and motor
responses. This noticeable gap in research highlights the need
I. I NTRODUCTION
for a comprehensive investigation into the neural correlates of
In recent years, the convergence of neuroscience and arti- cognitive and MI, and how these can be accurately classified
ficial intelligence has been a driving force behind the explo- using deep learning approaches.
ration of innovative techniques for understanding human brain Traditional methods for CI and MI classification have pri-
activity. MEG harnesses high-resolution spatiotemporal data marily relied on hand-engineered features extracted from neu-
to decode real-time neural dynamics non-invasively [1], facil- rophysiological signals. However, the intricate and dynamic
itating advanced deep learning methodologies. Spatiotemporal nature of brain activity patterns poses challenges for conven-
elements refer to the combination of spatial and temporal in- tional feature engineering approaches. Many approaches were
formation in a dataset. In the context of MEG data, spatiotem- presented to classify brain activities by segmenting MEG sig-
poral elements represent the patterns of brain activity over nal data into epochs and statistical features [9], [10]. Although
both space and time. MEG data provides information about the the majority of research still relies on the use of hand-crafted
timing and location of brain activity. The capacity to decode features, many recent studies have explored the potential of
MEG signals on a trial-by-trial basis has enormous potential in deep learning approaches. Particularly convolutional neural
a variety of fields, including cognitive neuroscience [1], brain- networks (CNNs) and recurrent neural networks (RNNs), offer
computer interfaces (BCIs) [2], and clinical applications [3]. a promising solution. These architectures excel in automatic
The study of visual perception in neuroscience is intriguing, feature learning from raw data, thereby bypassing the need
particularly due to the common neural foundation that has for explicit feature extraction which refers to transforming
been identified between visual perception and mental imagery raw data into a set of meaningful and representative features
processes. It has been reported that both the perception of that can be used as input for a machine learning model.
visual stimuli and the ability to generate mental images This process typically involves selecting or creating a subset
activate similar regions of the brain and cognitive processes of relevant features from the original data, which can be
[4]. However, despite this shared neural foundation, a crucial time-consuming and require domain expertise. The success of

2571
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.
979-8-3503-3748-8/23/$31.00 ©2023 IEEE
machine learning and deep learning algorithms in a variety of use of MaxShield™ . Each participant underwent two recording
domains has inspired researchers to apply similar techniques to sessions on different days. Each session consists of two data
neuroimaging data processing including MEG and EEG signal runs due to session breaks. For better handling of the data,
classification. the authors have merged the sessions. The dataset includes
Craik and team [11] provided a detailed assessment of 1,134 minutes of MEG recordings and a total of 6,800 imaging
several deep learning algorithms used in EEG classification trials. By containing many MEG recordings and imaging trials,
tasks, providing insights into potential modifications for MEG this dataset provides a valuable resource for investigating
data interpretation. EEGNet, a small CNN particularly built and developing brain-computer interface systems based on
for EEG-based BCIs, was introduced by [12], and captures motor and cognitive imagery. The single trial classification
important characteristics from EEG data effectively, making was performed by [10] using a linear classifier i.e., a Support
it an appealing choice for adaption into MEGNet for single- Vector Machine (SVM) classifier to estimate accuracies for the
trial classification. In this study, we intend to suggest a fresh six binary tasks i.e. hand versus feet (H-F), hand versus word
approach: converting EEGNet into MEGNet. We hypothe- generation (H-W), hand versus subtraction (H-S), feet versus
sise that MEGNet’s compact design, which was inspired by word generation (F-W), feet versus subtraction (F-S), and word
EEGNet’s success in EEG-based BCIs, will efficiently capture generation versus subtraction (W-S). The SVM classifier was
significant spatiotemporal elements from MEG data, boosting trained using the feature set of Session 1 data and evaluated
single-trial classification accuracy. We used publicly accessi- on the feature set of Session 2 data. This study also compares
ble MEG datasets which include a wide range of cognitive the findings with the previous results presented by the authors
activities and motor imaging paradigms. These tasks include of the dataset. The block diagram of the proposed system is
hand imagery, feet imagery, subtraction imagery, and word illustrated in Fig. 1.
generation imagery. The MEG signals underwent advanced
artefact removal techniques and filtering processes. In addition, B. Data Pre-processing
we will compare MEGNet’s performance to that of other MEG data was processed offline using the MNE Python [14]
state-of-the-art existing models such as ShallowConvNet and library. Bad channel detection, correction, jump artifacts, and
DeepConvNet [13]. head movements were corrected for, by the implementation
The rest of the paper is organized as follows: Materials and of Signal Space Projection (SSS) and Maxwell Filtering. In
methods are presented, including dataset, data pre-processing the MNE Python Library, SSS and Maxwell Filtering are
and model training in Section 2, the performance analysis and performed using one function to deal with environmental
results are described in Section 3, the study is discussed in noise and artefacts. We implemented the inbuilt spatiotemporal
Section 4, and the conclusions are summarized in Section 5. Signal Space Separation method (tSSS) [15] which is activated
by passing in a time value as an argument to the function. By
II. M ATERIALS AND M ETHODS incorporating the use of the Shannon-Nyquist Theorem, taking
A. Dataset half the sampling frequency and dividing this by the duration
to create evenly spaced temporal windows and passing this
This study uses the magnetoencephalography (MEG)
as an argument into the required function. Following this, the
dataset, specifically designed for motor and cognitive imagery-
MEG data was then down-sampled to 500Hz. A Signal Space
based brain-computer interface (BCI) applications[10]. It con-
Projection method was used to remove and suppress the effects
sists of MEG signals recorded during four mental imagery
of eye blinks (EOG) and Electrocardiogram (ECG) artefacts
tasks using a typical BCI imagery paradigm. The four tasks
from MEG Data. Notch filtering was performed using a power-
were:
line frequency of 50Hz, accounting for the 3r d harmonic. Data
1) Hand imagery- Participants were asked to imagine open- was then bandpass filtered, using a double forward-backwards
ing and closing their right hand. pass using second-order sectioning, and filtered between 1-
2) Feet imagery- Participants were asked to imagine mov- 40Hz. The data was baseline corrected in a window of -200ms
ing their toes up and down. to 0 ms, which was then epoched into segments using the time
3) Subtraction imagery- Participants were asked to subtract window of 2000ms at the window start, concluding at 6000ms
7 from a given number and imagine the result. for the window end, from the trial onset. It emerged that one
4) Word generation imagery- Participants were asked to of the subject’s channels did not align between session 1 and
imagine generating words starting with a given letter. session 2, as a result, this had to be discarded.
The dataset used in this study involved the recruitment
of 20 healthy participants. Three subjects are not included C. Model Architecture and Configuration
due to noisy data present within the recordings The final 1) MEGNet: The study utilized the MEGNet architecture,
dataset (N= 17, 14 Males (82.35%), 3 Females (17.64%), inspired by the compact EEGNet model [12]. The MEGNet
with a mean age of 28. The minimum age is 22, with the architecture is illustrated in Fig 2. EEGNet’s effectiveness
highest age being 40) Data was acquired using an Elekta has been demonstrated across four BCI paradigms, including
Neuromag™ system [10], recorded with 306-channels (102 P300 visual-evoked potentials, ERN, MRCP, and SMR [12].
magnetometers and 204 planar gradiometers), including the MEGNet integrates a conventional 2D convolutional layer,

2572
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.
Fig. 1: MEG data analysis and deep learning classification implementation, with sample data from one participant as it goes
through the classification pipeline. A. Loading of raw MEG signals of both sessions. B. Pre-processing pipeline of raw MEG
signal data. C. Data Preparation of the pre-processed MEG data to meet the input requirements of the models. D. Model
Training pipeline to find best parameters. E. Storing the best parameters along with accuracy score

depth-wise convolution, and a separable convolution— the TABLE I: MEGNet modal Summary, where F1 = number
latter combining depth-wise followed by point-wise convo- of temporal filters, D = depth multiplier (number of spatial
lution. This separation reduces parameters, minimizing over- filters), F2 = number of pointwise filters, and N = number of
fitting risks. The model’s adaptability allows for the extraction classes, respectively.
of spatial and temporal EEG features, a capability supported Layer (Type) Output Shape Param #
by [16], [17], and [18]. The tailored model for MEG data, InputLayer (None, Channels, Samples, F1) 0
detailed in Table I, was crucial for precise event classification Conv2D (None, Channels, Samples, F1) 64
BatchNormalization (None, Channels, Samples, F1) 32
from pre-processed MEG data. Hyper-parameters like kernel DepthwiseConv2D (None, 1, Samples, F1 * D) 3264
length and dropout rate were fine-tuned for optimal results, BatchNormalization (None, 1, Samples, F1 * D) 64
with the model compiled using categorical cross-entropy loss, Activation (None, 1, Samples, F1 * D) 0
AveragePooling2D (None, 1, DownSampled, F1 * D) 0
the Adam optimizer, and an accuracy metric. Dropout (None, 1, DownSampled, F1 * D) 0
SeparableConv2D (None, 1, DownSampled, F2) 512
2) Shallow Convolutional Network and Deep Convolutional BatchNormalization (None, 1, DownSampled, F2) 64
Network: The Shallow and Deep Convolutional Network Activation (None, 1, DownSampled, F2) 0
serves as a foundational architecture for EEG and MEG event AveragePooling2D (None, 1, DownSampled // N, F2) 0
Dropout (None, 1, DownSampled // N, F2) 0
classification. Shallow Convolutional Networks (SCNN) and Flatten (None, F2) 0
Deep Convolutional Networks (DCNN) have been used in Dense (None, Classes) 1986
EEG signal classification. SCNN has been proposed for motor Activation (None, Classes) 0
imagery (MI) classification, achieving an accuracy of 68.77%
on the BCI Competition IV-2a dataset [16]. On the other hand,
DCNN has been used for alcoholism classification, achieving
an average accuracy of 98% on the UCI-ML EEG dataset [19]. accomplished through the utilization of the source code found
in [12] with few modified parameters.
The models underwent a rigorous configuration and train-
ing process to achieve reliable performance. The meticulous D. Model Compilation and Training
configuration process involved the tuning of hyperparameters,
including filter size, stride length, and activation functions, to The pre-processed magnetoencephalography (MEG) data
optimize feature extraction. The model’s compilation encom- underwent further refinement to meet the input requirements
passed categorical cross-entropy loss, a stochastic gradient de- of the MEGNet, Deep and Shallow ConvNets. To ensure that
scent optimizer, and accuracy as the primary evaluation metric. the model focuses on relevant spatiotemporal information,
The implementation of both Deep and Shallow ConvNets was reducing computational complexity but still capturing the

2573
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.
Fig. 2: The visualization of the MEGNet architecture, taken in its entirety, is a comprehensive representation of the convolutional
kernel connectivity between inputs and outputs. The network initiates with a temporal convolution which is the first part of
Block 1, to acquire knowledge of frequency filters. It then proceeds to utilize a depthwise convolution, located in the second
part of Block 1, which is connected to each feature map individually, to learn frequency-specific spatial filters. The separable
convolution (Block 2), is a combination of a depthwise convolution that learns a temporal summary for each feature map
individually, and a pointwise convolution that learns how to optimally mix the feature maps together.

distinct neural activity, gradiometer channels were specifically and 5% testing subsets. Within the training subset, a further
chosen to capture the changes in the magnetic field gradient. subdivision was made to create a validation set.
To address the potential issue of class imbalance when
In consideration of the research objective, distinct event
creating the splits, a stratification sampling[20] technique is
pairs were defined as the foundation for binary classification
used to ensure that each class or category is represented
tasks. Subsequently, the dataset corresponding to each event-
proportionally in the sample. It involves dividing the dataset
pair was extracted from the pre-processed datasets. Min-max
into subgroups based on the classes and then sampling from
scaling was utilized for data normalization, which mapped the
each stratum in a way that maintains the original class
values to the range of -1 to 1. This normalization process
distribution, thereby reducing bias in the model assessment.
was implemented to prevent variations in data magnitudes
This approach facilitates a more accurate representation of the
from impacting the model’s performance. The data was also
model’s performance.
reshaped to conform to the input format of the models,
The EEGNet model used the filter size of the first convo-
which includes dimensions denoting trials, channels, and time
lutional block as half of the sampling frequency rate. In this
samples. Moreover, an additional dimension representing the
study, a grid search implementation was employed with vari-
number of MEG electrodes (channels) was included to ensure
ous combinations of kernel length and dropout. By attaining
compatibility with the MEGNet, Deep and Shallow ConvNets
the maximum training epochs or through the implementation
architecture.
of the early stopping strategy, the optimal weights of the
For the intra-subject learning, the epochs from both ses- network were recorded. The utilization of model checkpoints
sions were combined to create a comprehensive dataset for was to preserve the superior performing model as determined
each subject. This merging process aimed to maximize data by the validation accuracy, while concurrently achieving an
utilization as the amount of available data is limited in our equilibrium between bias and variance. This feat was accom-
case; we have 200 trials in each session and capture a broader plished through a comprehensive evaluation process through
range of variability. For each subject, the combined epochs assiduous experimentation and scrupulous analysis.
were initially split in the ratio of approximately 95% training Several experiments were conducted to compare the neural

2574
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.
networks from the EEGNet family (Shallow ConvNet, Deep TABLE II: The classification accuracy (%) for Hand vs Feet
ConvNet and E/MEGNet) [21][8][12],[22]. For experimental (H-F)
subjects who partook in various experiments on different days, Models
the information was managed as though distinct subjects had SVM (FB1) MEGNet ShallowConvNet DeepConvNet
Freq. band 8-12 Hz 1-40 Hz 1-40 Hz 1-40 Hz
taken part instead, which is referred to as an independent Sub 1 58 70 50 60
days configuration. We aimed to rank the neural networks; Sub 3 74 70 80 70
therefore, to evaluate the model‘s capacity to classify various Sub 4 50 70 60 80
Sub 6 47 60 50 70
event pairs, the accuracy score is calculated from the model‘s Sub 7 51 50 70 60
ability to correctly classify binary pair-wise configuration. Sub 9 51 50 80 50
Sub 11 47 50 50 50
Sub 12 56 60 80 70
E. Significance investigation of data Sub 13 49 70 80 50
Sub 14 50 50 70 50
The significance of the data variance was investigated. Two Sub 15 80 70 90 50
Sub 16 57 60 70 60
subjects were randomly selected to employ the t-test and Sub 17 55 60 40 50
compute the p-value to scrutinize the divergence between Sub 18 53 70 70 60
them. The resulting t-statistic value of -0.023 and the asso- Sub 19 54 90 50 80
Sub 20 64 50 60 50
ciated p-value of 0.982 provide insight into our assessment. Mean 56 62.5 65.625 60
The t-statistic measures the level of differentiation between Std 9.37 11.25 14.59 10.95
groups, and in this study, a value near zero indicates limited
distinction. On the other hand, the p-value represents the
probability of observing such a discrepancy by chance. In TABLE III: The classification accuracy (%) for Hand vs
this case, the high p-value suggests that the observed variance Subtraction (H-S)
among subjects falls within the range of random fluctuations, Models
indicating a lack of statistically significant differentiation. SVM (FB1) MEGNet ShallowConvNet DeepConvNet
Freq. band 8-12 Hz 1-40 Hz 1-40 Hz 1-40 Hz
Sub 1 53 60 80 50
III. R ESULTS Sub 3 95 80 90 80
Sub 4 50 70 40 70
In pursuit of understanding and harnessing the capabilities Sub 6 50 60 80 70
of MEGNet, we embarked on a comprehensive comparison of Sub 7 62 70 60 50
Sub 9 83 50 60 50
three distinct networks MEGNet (derived from EEGNet), Shal- Sub 11 91 80 90 60
lowConvNet, and DeepConvNet. To comprehend the strengths Sub 12 51 50 80 50
Sub 13 86 60 80 50
and limitations of these networks in the context of event Sub 14 50 50 70 60
prediction using MEG data. Our investigation delved into the Sub 15 56 50 70 50
nuanced differences between EEGNet (MEGNet), Shallow- Sub 16 57 70 80 50
Sub 17 71 50 60 60
ConvNet, and DeepConvNet. These networks, each with its Sub 18 88 80 90 60
unique configuration, were put to the test to gauge their predic- Sub 19 56 80 70 60
Sub 20 90 60 100 50
tive prowess. The architecture variations spanned from MEG- Mean 68.06 63.75 75 57.5
Net‘s specialized depth-wise and separable convolutions to the Std 17.57 12.0 15.05 9.30
simpler yet potent design of ShallowConvNet and the more
complex layers of DeepConvNet. To unravel the networks’ true
potential, we cautiously evaluated their performance across a
set of event pairs. These pairs, encompassing diverse cognitive of the brain‘s responses to different cognitive tasks, shedding
tasks, provided a robust and varied ground for assessment. light on the intricacies that define our cognitive experiences.
The selection of event pairs and their corresponding functions Tables 2-5 present the accuracy score of the SVM classifier
added a layer of specificity to the evaluation, ensuring a reported by [10] along with the performance of three distinct
comprehensive exploration of the networks’ capabilities. neural network architectures: MEGNet, ShallowConvNet, and
The experiment involved careful tuning of hyperparameters DeepConvNet, across six pair-wise binary classification tasks
to optimize the model‘s performance. Through meticulous involving motor and cognitive imagery-based brain-computer
experimentation with different values of kernel length and interfaces. Our main focus lies in understanding how each
dropout rate, we discerned the configurations that led to the model performs in terms of mean accuracy and the variabil-
highest accuracy score. This iterative process underscored ity of these accuracies across the tasks. When it comes to
the importance of hyperparameter tuning in fine-tuning the distinguishing between hand and feet movements given in
model‘s predictive capabilities. As we turned our attention Table II, we observed that the MEGNet model exhibited a
to individual event pairs, we observed nuanced variations superior mean accuracy of 62.%, surpassing the traditional
in the model‘s performance. All the neural network models SVM classifier‘s mean accuracy of 56%. This indicates the
showcased varying degrees of accuracy scores across different potential of deep learning in decoding intricate cognitive
event pairs. These results provided a deeper understanding tasks. ShallowConvNet hovered around 65.62%, and Deep-

2575
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.
TABLE IV: The classification accuracy (%) for Hand vs Word TABLE VI: The classification accuracy (%) for Feet vs Sub-
(H-W) traction (F-S)
Models Models
SVM (FB1) MEGNet ShallowConvNet DeepConvNet SVM (FB1) MEGNet ShallowConvNet DeepConvNet
Freq. band 8-12 Hz 1-40 Hz 1-40 Hz 1-40 Hz Freq. band 8-12 Hz 1-40 Hz 1-40 Hz 1-40 Hz
Sub 1 53 70 60 50 Sub 1 51 80 80 70
Sub 3 94 60 80 60 Sub 3 69 50 90 60
Sub 4 50 70 80 70 Sub 4 50 70 60 70
Sub 6 54 90 60 80 Sub 6 57 80 80 50
Sub 7 69 70 60 50 Sub 7 66 80 60 50
Sub 9 67 60 80 70 Sub 9 75 80 80 80
Sub 11 86 50 80 60 Sub 11 85 50 70 50
Sub 12 56 60 100 50 Sub 12 59 60 60 70
Sub 13 90 50 100 50 Sub 13 87 60 60 50
Sub 14 62 50 50 60 Sub 14 55 50 80 50
Sub 15 65 80 80 50 Sub 15 75 50 100 50
Sub 16 62 70 70 70 Sub 16 70 50 80 60
Sub 17 57 60 60 50 Sub 17 70 70 70 100
Sub 18 88 50 60 50 Sub 18 64 50 70 50
Sub 19 61 60 60 60 Sub 19 63 80 60 80
Sub 20 91 80 90 50 Sub 20 91 70 90 80
Mean 69.06 64.37 73.125 58.12 Mean 67.93 65.62 74.37 63.75
Std 15.35 12.09 15.37 9.81 Std 12.42 12.63 12.63 15.43

TABLE V: The classification accuracy (%) for Feet vs Word TABLE VII: The classification accuracy (%) for Subtraction
(F-W) vs Word (S-W)
Models Models
SVM (FB1) MEGNet ShallowConvNet DeepConvNet SVM (FB1) MEGNet ShallowConvNet DeepConvNet
Freq. band 8-12 Hz 1-40 Hz 1-40 Hz 1-40 Hz Freq. band 8-12 Hz 1-40 Hz 1-40 Hz 1-40 Hz
Sub 1 61 60 60 70 Sub 1 51 70 70 50
Sub 3 70 80 90 70 Sub 3 74 70 80 50
Sub 4 50 70 90 50 Sub 4 49 60 70 50
Sub 6 58 80 80 90 Sub 6 53 80 80 80
Sub 7 87 100 80 70 Sub 7 58 70 50 70
Sub 9 50 60 90 50 Sub 9 70 60 60 50
Sub 11 83 50 70 50 Sub 11 90 80 70 50
Sub 12 68 50 70 50 Sub 12 63 70 50 50
Sub 13 91 50 70 60 Sub 13 57 70 50 60
Sub 14 54 70 50 50 Sub 14 65 50 60 30
Sub 15 69 70 80 50 Sub 15 59 60 60 50
Sub 16 62 60 70 70 Sub 16 72 80 80 70
Sub 17 55 80 80 80 Sub 17 47 80 70 60
Sub 18 45 50 60 60 Sub 18 60 50 50 80
Sub 19 57 60 80 70 Sub 19 62 80 50 60
Sub 20 87 80 90 50 Sub 20 77 50 50 50
Mean 65.43 66.87 75.62 61.87 Mean 62.93 67.5 62.5 56.87
Std 14.66 14.47 12.09 12.76 Std 11.43 11.25 11.83 13.02

ConvNet stood at about 60%. This translates to MEGNet and around 74.34%, and DeepConvNet reached an average of
ShallowConvNet performing fairly well, whereas DeepCon- about 63.75%. MEGNet displayed consistent accuracy results
vNet lagged slightly behind. What‘s interesting is that while and ShallowConvNet held its ground, while DeepConvNet
ShallowConvNet scored higher on average, MEGNet showed showed a bit more variation in its performance. While distin-
more consistent accuracy results across different scenarios. guishing hand movements from imagined words given in Table
In distinguishing hand movements from mental subtractions IV, MEGNet achieved an average accuracy of around 64.37%,
given in Table III, MEGNet exhibited an average accuracy ShallowConvNet peaked at roughly 73.12%, and DeepCon-
of 63.75%, outperforming the SVM classifier with a mean vNet settled at around 58.12%. ShallowConvNet claimed the
accuracy of 56%, ShallowConvNet settled at around 75%, and highest average accuracy, yet MEGNet and DeepConvNet
DeepConvNet emerged as the leader with roughly 57.5%. This seemed to trade some accuracy for consistency. For foot
time, ShallowConvNet took the lead with its higher average movements from imagined words given in Table V, MEGNet
accuracy and a moderate level of stability. ShallowConvNet showcased an average accuracy of 66.87%, ShallowConvNet
remained steady, but MEGNet demonstrated consistency in its rose to around 75.62%, and DeepConvNet lagged a bit at
performance. In the case of foot movements versus mental sub- 61.87%. ShallowConvNet shone brightly in terms of both
tractions given in Table VI, MEGNet maintained an average accuracy and stability, while MEGNet maintained a compet-
accuracy of approximately 65.62%, ShallowConvNet stayed itive edge. Lastly, in the case of mental subtractions versus

2576
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.
imagined words given in Table VII, MEGNet stayed around variability of all the models, highlighting its sensitivity to task
67.5%, ShallowConvNet hovered at 62.5%, and DeepConvNet complexity and architecture parameters. DeepConvNet, as its
scored around 56.87%. MEGNet demonstrated stable perfor- name suggests, adopts a more complex architecture with a
mance, ShallowConvNet retained its accuracy with moderate greater number of layers. This suggests that DeepConvNet
fluctuations, and DeepConvNet showcased a wider range of requires more data to be trained effectively than MEGNet.
results. When comparing the current research results with the These observations emphasize the importance of a balanced
previous work conducted by the dataset author [10], where consideration between model performance and stability in the
a Support Vector Machine (SVM) classifier was employed, context of brain-computer interface applications.
notable differences and advancements become apparent. In the
previous study, the SVM classifier achieved accuracy levels V. C ONCLUSIONS
that ranged from approximately 50% to 95% across various In the present study, we conducted a comprehensive analysis
cognitive event pairs. Interestingly, the current deep learning of three deep neural network models, namely MEGNet, Shal-
models, including MEGNet, ShallowConvNet, and DeepCon- lowConvNet, and DeepConvNet on MEG signal classification.
vNet, exhibited competitive or even improved accuracy in The comparison was conducted on a dataset containing motor
many cases, indicating the potential for neural networks to and cognitive imagery tasks. We performed pair-wise decoding
outperform traditional feature extraction methods and machine of cognitive and motor tasks. Through an extensive evaluation
learning classifiers in capturing intricate patterns within MEG of accuracy and stability, we provide valuable insights into
data. the strengths and limitations of each model. Our findings
IV. D ISCUSSION underscore the significance of the trade-off between accuracy
and stability in model selection for these applications. The
Previous studies [23][10] have reported significant classifier MEGNet model shows consistent performance and generalizes
performance at the individual level, which was reliant on dra- well across paradigms compared to ConvNet. It also performs
matic feature extraction and used multivariate pattern analysis comparably to the reference algorithms even with limited
to decode MEG signals. Conversely [7] used a linear discrim- training data across all tested paradigms. ShallowConvNet
inant analysis (LDA) classifier with 5-fold cross-validation excels in terms of accuracy, although its performance varies
to classify and evaluate the brain response to visual stimuli. with task intricacies. DeepConvNet, while occasionally com-
In their research [24] used EEGNet to categorize objects, petitive, exhibits more significant variability. In our study,
specifically faces, tools, animals, and scenes captured from MEGNet exhibited an average accuracy of 64.37% (± 3%),
MEG data with very high levels of accuracy in both binary positioning it slightly below the performance of other es-
and multi-class classification settings. The prevailing challenge tablished models like ShallowConvNet. Despite this, it is
however lies in the decoding of motor tasks from cognitive essential to underscore the unique effectiveness and advantages
imagery using the high spatial and temporal precision of MEG, offered by MEGNet. The model’s capabilities may extend
leading towards identifiable areas responsible for each related beyond a singular accuracy metric, showcasing strengths in
task; an objective that is addressed in our current study. specific scenarios or applications. Moreover, it is imperative to
The consistency of our results with the previous litera- acknowledge the limitations inherent in MEGNet. Recognizing
ture [10] proved the capabilities of deep learning models to these limitations provides valuable insights and perspectives
perform vast feature extraction and decoding of motor tasks for future research and enhancement. By presenting a nuanced
from cognitive imagery in MEG signals without the use of understanding of both the strengths and limitations, we aim to
additional algorithms such as the Common Spatial Pattern foster a comprehensive evaluation of MEGNet’s applicability
(CSP) [10] or Independent Component Analysis (ICA) [25]. in diverse contexts. This study serves as a stepping stone for
The study reveals that the classification performance of the further exploration and refinement of MEGNet’s potential in
EEGNet-inspired MEGNet model and ShallowConvNet were neuroimaging applications.
similar across the subjects achieving high accuracy levels,
especially in tasks that involve distinguishing between motor ACKNOWLEDGEMENT
and cognitive imagery. This propensity for accuracy could be
attributed to its shallow structure that enables efficient extrac- HR was supported by the Economic and Social Research
tion of key features. However, ShallowConvNet also exhibits Council (ESRC) funded Business and Local Government Data
more variability, which suggests that its performance might Research Centre under Grant ES/S007156/1. SN acknowl-
be sensitive to specific task characteristics. ShallowConvNet edges the support from “The Adaptive Mind” funded by
efficiently captures prominent features from MEG signals but the Excellence Program of the Hessian Ministry of Higher
may struggle with capturing nuanced patterns present in more Education, Science, Research and Art.
complex tasks. The MEGNet architecture allows for more
R EFERENCES
comprehensive pattern extraction, capturing both spatial and
temporal characteristics of MEG signals, making it a suitable [1] J. Gross, “Magnetoencephalography in cognitive neuro-
choice for various MEG tasks. In contrast, DeepConvNet science: a primer,” Neuron, vol. 104, no. 2, p. 189–204,
displayed poorer performance showcasing also the highest 2019.

2577
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.
 [2] J. Mellinger, G. Schalk, C. Braun, H. Preissl, W. Rosen- [15] S. Taulu and J. Simola, “Spatiotemporal signal space
stiel, N. Birbaumer, and A. Kübler, “An meg-based separation method for rejecting nearby interference in
brain–computer interface (bci),” Neuroimage, vol. 36, meg measurements,” Physics in Medicine and Biology,
no. 3, pp. 581–593, 2007. vol. 51, p. 1759–1768, 04 2006.
[3] S. Braeutigam, “Magnetoencephalography: fundamentals [16] Y. Han, B. Wang, J. Luo, L. Li, and X. Li, “A classi-
and established and emerging clinical applications in ra- fication method for EEG motor imagery signals based
diology,” International Scholarly Research Notices, vol. on parallel convolutional neural network,” Biomedical
2013, 2013. Signal Processing and Control, vol. 71, p. 103190, 2022.
[4] N. Dijkstra, S. E. Bosch, and M. A. van Gerven, “Shared [17] H. Raza, A. Chowdhury, S. Bhattacharyya, and
neural mechanisms of visual perception and imagery,” S. Samothrakis, “Single-trial EEG classification with
Trends in cognitive sciences, vol. 23, no. 5, pp. 423–434, EEGNet and neural structured learning for improving bci
2019. performance,” in 2020 International Joint Conference on
[5] S. N. Kraeutner, S. N. Eppler, A. Stratas, and S. G. Neural Networks (IJCNN). IEEE, 2020, pp. 1–8.
Boe, “Generate, maintain, manipulate? exploring the [18] M. Riyad, M. Khalil, and A. Adib, “Mi-EEGNET: A
multidimensional nature of motor imagery,” Psychology novel convolutional neural network for motor imagery
of Sport and Exercise, vol. 48, p. 101673, 2020. classification,” Journal of Neuroscience Methods, vol.
[6] N. Leeuwis, A. Paas, and M. Alimardani, “Vividness 353, p. 109037, 2021.
of visual imagery and personality impact motor-imagery [19] T. Wen, Y. Du, T. Pan, C. Huang, Z. Zhang et al., “A
brain computer interfaces,” Frontiers in Human Neuro- deep learning-based classification method for different
science, vol. 15, p. 634748, 2021. frequency EEG data,” Computational and Mathematical
[7] S. Nara, H. Raza, M. Carreiras, and N. Molinaro, Methods in Medicine, vol. 2021, 2021.
“Decoding numeracy and literacy in the human brain: [20] T. Amr, Hands-On Machine Learning with scikit-learn
insights from meg and mvpa,” Scientific Reports, vol. 13, and Scientific Python Toolkits: A practical guide to im-
no. 1, p. 10979, 2023. plementing supervised and unsupervised machine learn-
[8] I. A. Abdellaoui, J. G. Fernandez, C. Sahinli, and ing algorithms in Python. Packt Publishing Ltd, 2020.
S. Mehrkanoon, “Deep brain state classification of meg [21] R. T. Schirrmeister, J. T. Springenberg, D. Josef,
data,” arXiv preprint arXiv:2007.00897, 2020. M. Glasstetter, K. Eggensperger, M. Tangermann, F. Hut-
[9] T. N. Alotaiby, S. R. Alrshoud, S. A. Alshebeili, and ter, W. Burgard, and T. Ball, “Deep learning with
M. H. Alhumaid, “Automatic epileptic tendency screen- convolutional neural networks for EEG decoding and
ing using statistical features of meg data and svm,” in visualization,” Human Brain Mapping, vol. 38, pp. 5391–
2019 International Conference on Electrical and Com- 5420, 08 2017.
puting Technologies and Applications (ICECTA). IEEE, [22] D. Rathee, H. Cecotti, and G. Prasad, “Single-trial effec-
2019, pp. 1–3. tive brain connectivity patterns enhance discriminability
[10] D. Rathee, H. Raza, S. Roy, and G. Prasad, “A mag- of mental imagery tasks,” Journal of Neural Engineering,
netoencephalography dataset for motor and cognitive vol. 14, p. 056005, 08 2017.
imagery-based brain-computer interface,” Scientific Data, [23] I. Simanova, M. Van Gerven, R. Oostenveld, and P. Ha-
vol. 8, no. 1, p. 120, 2021. goort, “Identifying object categories from event-related
[11] A. Craik, Y. He, and J. L. Contreras-Vidal, “Deep learn- EEG: toward decoding of conceptual representations,”
ing for electroencephalogram (eeg) classification tasks: a PloS one, vol. 5, no. 12, p. e14465, 2010.
review,” Journal of neural engineering, vol. 16, no. 3, p. [24] R. Shi, Y. Zhao, Z. Cao, C. Liu, Y. Kang, and J. Zhang,
031001, 2019. “Categorizing objects from meg signals using EEGNet,”
[12] V. J. Lawhern, A. J. Solon, N. R. Waytowich, S. M. Cognitive Neurodynamics, pp. 1–13, 2021.
Gordon, C. P. Hung, and B. J. Lance, “EEGNet: a [25] A. H. Treacher, P. Garg, E. Davenport, R. Godwin,
compact convolutional neural network for EEG-based A. Proskovec, L. G. Bezerra, G. Murugesan, B. Wagner,
brain–computer interfaces,” Journal of neural engineer- C. T. Whitlow, J. D. Stitzel et al., “Megnet: automatic
ing, vol. 15, no. 5, p. 056013, 2018. ica-based artifact removal for meg using spatiotemporal
[13] R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, convolutional neural networks,” NeuroImage, vol. 241, p.
M. Glasstetter, K. Eggensperger, M. Tangermann, F. Hut- 118402, 2021.
ter, W. Burgard, and T. Ball, “Deep learning with
convolutional neural networks for EEG decoding and
visualization,” Human brain mapping, vol. 38, no. 11,
pp. 5391–5420, 2017.
[14] A. Gramfort, M. Luessi, E. B. Larson, D. A. Enge-
mann, D. Strohmeier, C. Brodbeck, L. Parkkonen, and
M. Hämäläinen, “Mne software for processing MEG and
EEG data,” vol. 86, pp. 446–460, 02 2014.

2578
Authorized licensed use limited to: Universita' Politecnica delle Marche. Downloaded on December 10,2024 at 15:59:36 UTC from IEEE Xplore. Restrictions apply.