Systematic Review

A Systematic Review of Machine Learning Models in Mental

Health Analysis Based on Multi-Channel Multi-Modal
Biometric Signals
Jolly Ehiabhi * and Haifeng Wang *

Industrial and Systems Engineering, Mississippi State University, Starkville, MS 39762, USA
* Correspondence: [email protected] (J.E.); [email protected] (H.W.)

Abstract: With the increase in biosensors and data collection devices in the healthcare industry,
artificial intelligence and machine learning have attracted much attention in recent years. In this study,
we offered a comprehensive review of the current trends and the state-of-the-art in mental health
analysis as well as the application of machine-learning techniques for analyzing multi-variate/multi-
channel multi-modal biometric signals.This study reviewed the predominant mental-health-related
biosensors, including polysomnography (PSG), electroencephalogram (EEG), electro-oculogram
(EOG), electromyogram (EMG), and electrocardiogram (ECG). We also described the processes used
for data acquisition, data-cleaning, feature extraction, machine-learning modeling, and performance
evaluation. This review showed that support-vector-machine and deep-learning techniques have
been well studied, to date.After reviewing over 200 papers, we also discussed the current challenges
and opportunities in this field.

Keywords: system review; multi-channel multi-modal biometric signals; machine learning; mental
health

1. Introduction
Citation: Ehiabhi, J.; Wang, H. A
Systematic Review of Machine It is a bitter pill to swallow: At least one-in-five adults suffers from at least one form
Learning Models in Mental Health of mental health issue or disorder.These health conditions involve changes in emotions,
Analysis Based on Multi-Channel thinking, behavior, or a combination of these [1], such as attention-deficit/hyperactivity
Multi-Modal Biometric Signals. disorder (ADHD), sleep apnea disorder, and depression [2–6]. Mental health issues affect
Biomedinformatics 2023, 3, 193–219. well-being, impairing relationships and cognitive activities and causing body responses
https://doi.org/10.3390/ that may place individuals at risk.
biomedinformatics3010014 A significant amount of research has leveraged the application of machine learning
Academic Editor: Jörn Lötsch
(ML) techniques for extracting, detecting, and classifying mental health biomarkers in
sensor datasets [7–12]. These biosensor data are usually multi-channel, and even multi-
Received: 17 January 2023 modal, time series [13]. In the medical field, two types of signals are commonly collected
Revised: 22 February 2023 for diagnosis, which includes bio-electric and non-bio-electric signals. These signals typ-
Accepted: 22 February 2023 ically require expert evaluation to make a valid diagnosis [14]. With the assistance of
Published: 1 March 2023 ML techniques, there is the potential to increase the efficiency of mental health diagnosis
and even the prognoses of mental disorders at an early stage, given the widely moni-
tored signals through wearable devices in recent years. Biological signals can be collected
Copyright: © 2023 by the authors.
through different modalities. For example, in this paper, we reviewed the application of ML
Licensee MDPI, Basel, Switzerland. techniques for electroencephalograms (EEGs), which records signals from the brain [15];
This article is an open access article electro-oculograms (EOGs), which record the movement signals of the eyes [16]; electromyo-
distributed under the terms and grams (EMGs), which record signals from muscle activities during sleep stages [17–19]; and
conditions of the Creative Commons electrocardiograms (ECGs), which record signals from the heart via a heart-rate monitor.
Attribution (CC BY) license (https:// Non-bio-electric signals include body temperature, respiration, and blood pressure. Despite
creativecommons.org/licenses/by/ there being many biological signals for diagnosing mental diseases, this work concentrated
4.0/).

Biomedinformatics 2023, 3, 193–219. https://doi.org/10.3390/biomedinformatics3010014 https://www.mdpi.com/journal/biomedinformatics

Biomedinformatics 2023, 3 194

on bio-electrical signals and ML techniques that have been used to promote the diagnosis
of mental health issues [20].
There are various biological signal types [21]: bio-electrical signals, bio-acoustic signals,
bio-mechanical signals, bio-chemical signals, and body temperature. Bio-electrical signals
occur in the body of cells, and they originate from the electric activities occurring in the
body. These signals have been used for diagnosing various diseases using ML techniques,
which is a subset of artificial intelligence (AI) methodologies. In this work, we reviewed
the trends and the state-of-the-art of these ML techniques for mental health diagnosis, and
we investigated the methods by which these signals are used to increase the efficiency of
diagnosing mental health diseases.These bio-electric signals are collected through electrodes
and specialty devices. In sleep medicine, large datasets have been generated with these
devices to assist in characterizing and quantifying sleep and sleep-related disorders [22].
Polysomnography (PSG) data [23] have been the most commonly used test for the diagnosis
of obstructive sleep apnea syndrome (OSAS) and other related ailments. PSG procedures
have been conducted primarily overnight in a sleep laboratory. To effectively diagnose
sleep disorders, PSG records have been used, collected, and scored by experts [24–27]. PSG
records are data extracted from brain-wave recordings, oxygen levels, heart rate monitors,
breathing rates, as well as leg and eye movements of patients.EEG, EOG, EMG, and ECG
signals as well as sleep videos have also been integrated into PSGs [17,23–25,28]. It has
been estimated by World Health Organization (WHO) that nearly one-third of the world
population suffers from sleep disorders [29]. PSG analysis has been defined as the gold
standard for detecting sleep disorders and other mental health diseases [30]. PSG records
are multi-signal channels. For sleep studies and scoring, an expert is often required to
manually examine PSG records. Therefore, the results are at risk of human error, and it is
time-consuming and expensive to carry out [31].
Collecting PSG data can be very expensive and uncomfortable for the patient; therefore,
it is vital to ensure an accurate diagnosis based on this test’s results [32–35].The traditional
PSG process requires the measuring of EEG, EOG, EMG, and ECG signals [28]. A significant
amount of research has employed deep-learning approaches to model the spatio-temporal
aspects of PSG data [24]. Later in this paper, we reviewed the advantages of ML in a study of
mental health diseases. Since 1970, there have been improvements in the automatic scoring
of PSG records, in accordance to Rechtschaffen and Kales (RK) sleep research [33,36] based
on the American Academy of Sleep Medicine (AASM) rules [37]. The visual interpretations
of the PSG signals of patients have been a widely accepted approach for analyzing sleep
stages and mental-health-related diseases [38]. In many countries, PSG technology and
experts in sleep study have been limited, however, so there is an urgent need to achieve
automated PSG data analysis with the help of AI techniques [39].
Sleep recordings require the measurement of brain activity (EEG), eye movement
(EOG), and muscle activity (EMG) to accurately identify specific sleep stages. EEGs have
been intensely researched by many scholars. EEG signals are classified by employing a
common spatial pattern (CSP) and differential entropy (DE) characteristics to the delta,
theta, alpha, beta, and gamma frequency bands [40,41]. The diagnosis of mental health
and sleep disorders can be tedious and requires significant time investment and expertise
to obtain a reliable and accurate diagnosis. In many cases, patients have been subjected
to prolonged interviews to improve the diagnostic accuracy of the health personnel or
expert [41]. With an EEG system, some limitations have been overcome, and the process of
feature extraction, classification, and prediction for the diagnosis of mental health diseases
based on PSG datasets could potentially be automated using ML techniques.
Among other signal types, EEGs have been a focus of much study for mental health
diagnosis by many researchers. However, there are significantly fewer articles on other
biological signals, such as EMG, EOG, and ECG. The study in [42] presented a compre-
hensive survey of ECG signals, and the authors concluded that a significant amount of
studies will be published on ECG in the near future. The study in [43] showed that HRV
analysis was a viable method for feature extraction from ECG signals. The researchers
Biomedinformatics 2023, 3 195

in [44] proposed and evaluated an automated analysis of single-lead ECG analyses using
human recognition patterns.EMGs have different statistical and spectral properties from the
other signals [25]. EMG signals have been used as a bio-signal for hand-and-wrist-gesture
recognition [45]. PSG data provide comprehensive information for sleep studies and sleep
disorder diagnosis.
It has been estimated that at least 2–4 percent of adults and 1–3 percent of children
suffer from sleep-related ailments [31]. There are many classifications that have been used
for determining sleep stages. The application of ML and AI has assisted scientists and
health professionals in recent times to improve the accuracy of sleep-stage classification
and mental health diseases [46]. Combrisson et al. [47] implemented several algorithms for
the automatic detection of sleep features and embedded them within a software platform,
which they referred to as “detection” panels.
PSG records have been broken into 30s epochs, which were then classified as dif-
ferent sleep stages by experts [48], based on the AASM and Rechtschaffen and Kales
sleep classification recommendations [29,48,49]. Sleep has been classified into periods of
rapid eye movement (REM) and non-rapid eye movement (NREM), including Stage W
(wakefulness), Stage N1 (NREM 1), Stage N2 (NREM 2), Stage N3/N4 (NREM 3), and
Stage R (REM) [22,24,25,32,50]. Many studies have identified EEG signals as a more effec-
tive bio-electric signal for sleep classification [51] and for the diagnosis of other mental
health diseases, such as depression and ADHD [35,52]. Figure 1 [29,38] shows a schematic
flowchart of sleep-stage classification with PSG signals from bio-electric signals, specifi-
cally EEG. We classified sleep as light sleep and deep sleep, and the wavelength shown in
Figure 1 is a typical wave pattern for EGG signals [53]. Each sleep stage can be distinguished
based on the wavelengths.

Figure 1. Schematic diagram of the sleep stages according to EEG signals. The annotation was created
according to RK standards and AASM recommendations for classifying sleep stages.

In this work, we reviewed the application of ML techniques on multi-modal and

multi-channel PSG datasets. This work aimed to provide researchers with information
on the current trends related to the application of ML for bio-electrical signals.The rest of
this paper is structured as follows: First, a background section with a subsection details
the method of article selection. Secondly, a section illustrates the methods of applying
ML techniques on multi-modal and multi-channel PSG datasets. This section has multiple
subsections, including data acquisition, data preparation, feature extraction, balancing
datasets, ML techniques, and performance evaluation. Thirdly, a summary and discus-
Biomedinformatics 2023, 3 196

sion section is presented. Finally, this study is concluded, and some recommendations
are discussed.

2. Background and Prior Work

2.1. Literature Search Process
For this review, we carried out keyword searches on specific literature databases.
The keywords used for this literature search were based on the goal of this study. The
keywords used for this search were as follows: (“EEG” OR “ECG” OR “EOG” OR “EMG”
OR “PSG”) AND “Machine Learning” AND “Mental Health”. This search was carried out
on commonly used databases, such as Science Direct, IEEE Xplore, MDPI, and PubMed.
The search criteria used for obtaining literature for this work are summarized, as follows:
• Publications had to be released in 2017 or later.
• To deepen the understanding of the research questions, we also added 25 articles pub-
lished between the years 2000 and 2016. This range was selected based on references
from similar research.
• Articles had to have at least one or more of the keywords.
• Articles had to be published in recognized literature databases/websites.
• All selected papers had to be written in English.
• All papers were either studies, surveys, or reviews of the application of ML on PSG
data and the classification of mental health issues using ML.
Table 1 shows the digital-database advanced search strings used to collect articles
for this review. Using the following search criteria (“EEG” OR “ECG” OR “EOG” OR
“EMG” OR “PSG”) AND “Machine Learning” AND “Mental Health”, which consisted of
commonly used boolean operators [1,54], i.e., AND (must be included in the search) as well
as OR (may or may not be included in the search), 1074 article were identified, and they
were further screened for inclusion in this work.

Table 1. A summary of search criteria and results from the different digital databases.

Digital Database Search String Used Total Articles Collected

(“EEG” OR “ECG” OR “EOG”
OR “EMG” OR “PSG”) AND
IEEE Xplore Access 41
“Machine Learning” AND
“Mental Health”
(“EEG” OR “ECG” OR “EOG”
OR “EMG” OR “PSG”) AND
Science Direct 944
“Machine Learning” AND
“Mental Health”
(“EEG” OR “ECG” OR “EOG”
OR “EMG” OR “PSG”) AND
MDPI 26
“Machine Learning” AND
“Mental Health”
(“EEG” OR “ECG” OR “EOG”
OR “EMG” OR “PSG”) AND
PubMed 76
“Machine Learning” AND
“Mental Health”

Figure 2 shows The Preferred Reporting Items for Systematic Reviews and Meta-
Analyses (PRISMA) for article search and collections for our review. Using PRISMA 2020
statements and guidelines, a checklist was used to structure this systematic review and
avoid biases during article selection. PRISMA was designed in 2009 [55] to address poor
or weak reporting of systematic reviews, and it also assisted in structuring a review in
order to provide useful value for the readers of systematic reviews. This study was also
registered in INPLASY, which is an international platform of registered systematic review
Biomedinformatics 2023, 3 197

and meta-analysis protocols. Many existing review articles studied in this systematic review
also used the PRISMA statement and guidelines [32,54,56–60]. This work leveraged prior
research from different authors to perform a detailed review of the applications of ML on
multi-channel and multi-modal PSG data.

Figure 2. PRISMA Analysis of articles searched: Flow diagram for study collection and reviews,
which included searching databases .

In this work, we included 218 papers that met our criteria and were related to our
research questions in terms of material presentation, methods, and results, as shown in
the PRISMA flow diagram in Figure 2. Records were screened and reviewed against the
quality of work conducted in the studies and its relevance to our research goal.

2.2. Word-Cloud Overview

Word clouds and bar graphs were created based on the titles and abstracts of all
the articles reviewed [61]. Figure 3 shows a word cloud and bar graph of all the titles
of the articles reviewed for this work. The most frequent words, as shown by the word
cloud, were learning, EEG, sleep, machine, detection, and anomaly, among others, and
Figure 4 shows a word cloud and the 30 most frequent words used in the abstracts from
all the articles reviewed in this paper. The most frequent words used in the abstracts were
similar to those used in the full-text articles, but with more weight on words such as data,
model, and so on.This showed that the collected articles aligned with the main goal of this
study. Furthermore, most of the sleep studies were related to the use of EEG signals and
considered the problem as a time-series anomaly detection problem.
Biomedinformatics 2023, 3 198

Figure 3. Word cloud and bar graph of the titles of the articles used in this review.

Figure 4. Word cloud and bar graph of all the abstracts of the articles used in this review.

2.3. Literature Distribution by Publication Year

Figure 5 is a summary of the number of articles reviewed in this paper, grouped by
publication year, with 55 and 43 articles published in 2022 and 2021, respectively. We
focused on recent research that had been carried out by researchers in this field.

Figure 5. Reviewed paper distribution by year of publication.

Biomedinformatics 2023, 3 199

2.4. Methods
In this section, Figure 6 shows a process flow of deploying ML on PSG datasets. We
considered the details of each individual block, as shown in Figure 6. Each block shows
how the PSG data were prepared [62], extracted, processed, and classified for disease
diagnosis, step by step. A similar approach was also used by Sekkal et al. [63].

Figure 6. Overview of the process flow or steps of data preparation, feature extraction, ML model-
ing, and classification. Note the nomenclature section for abbreviations in the above process flow
image. (a) Data Preparation [64]. (b) Feature Extraction [65]. (c) Machine-Learning Modeling [66].
(d) Validation.

2.4.1. Data Preparation

The methodologies of processing PSG signal-processing have been divided into four
stages [67]: data acquisition, pre-processing, feature extraction, and classification. PSG
data have become the most prevalent records used for studies that apply ML for mental
health diagnosis and classification, and there were many open-access datasets that had
been generated by previous research. In this review, we summarized some of the available
datasets and the papers they used. We identified three predominant resources that provided
open access to PSG data.
1. An official website (https://sleepdata.org/; accessed on 21 February 2023) managed
by The National Sleep Research Resource, provides open access for researchers and
those interested in sleep studies, with large collections of physiological signals and
clinical data elements. These datasets were collected from structured research cohorts
and clinical trials. The Nationwide Children’s Hospital (NCH) Sleep Data Bank
consisted of three folders: Sleep Data, Health Data, and the Sleep Data with annotated
PSG data recordings [68].
2. The Montreal Archive of Sleep Studies (MASS) (http://ceams-carsm.ca/en/MASS/;
accessed on 21 February 2023) is an open-access and collaborative database of laboratory-
based PSG recordings. It also comprised a cohort of subsets [25,69].
3. PhysioNet (https://physionet.org/; accessed on 21 February 2023) is an open-access
physiologic-signal data source that is managed by members of the MIT Laboratory for
Computational Physiology [68,70,71].
Table 2 summarizes the commonly used open-access PSG datasets. Khosla et al. and
Engemann et al. [72–74] have more detailed lists of PSG/EEG datasets.
Biomedinformatics 2023, 3 200

Table 2. A summary of open-access PSG datasets. We presented basic information used by previous
research in this field of study.

Dataset Description Source

Sampling frequency = 173.61 Hz; No. of persons = 5
Epileptic seizure developed by University of (healthy) + 5 (Unhealthy); Total duration of
[75]
Bonn, Germany a segment = 23.6 s .; No. of trails/channels in a
class = 100; Data size in this study = 200 × 4097
Sampling frequency = 250 Hz; No. of persons = 14
Schizophrenia EEG dataset collected by the Institute of (healthy) + 14 (Unhealthy); Epoch size = 60 s × 250 Hz
[75]
Psychiatry and Neurology in Warsaw, Poland = 15,000; Data size in this study = [19 × (14 + 14)] ×
15,000 = 532 × 15,000
Sampling frequency = 100 Hz; No. of persons = 8;
Sleep-EDF (S-EDF) (Scored by 1 sleep expert) [29,38,76]
Epoch length = 30 s; Data size = 15,139
Sleep-EDF (Expanded) (SE-EDF) (Scored by 1 sleep Sampling frequency = 100Hz; No. of persons = 20
[29,38,76]
expert) Epoch length = 30 s; Data size =40,100
Sampling frequency = 128 Hz; No. of persons= 45
Laboratory for Neurophysiology and NeuroComputer
(schizophrenic) + 39 (Normal); Data size = 16 × 7680; [77]
Interfaces of M. V. Lomonosov Moscow State University
Matrix with 1344 instances
The Epilepsy Ecosystem dataset Sampling frequency = 400 Hz; No. of persons = 3 [78]
The CHB-MIT dataset Sampling frequency = 256 Hz; No. of persons = 23 [78]
The BCI competition-II Dataset-III Sampling frequency = 128 Hz [79,80]
Sampling frequency for ECG = 125 Hz in SHHS1 while
Test Set of SHHS1 Test Set of SHHS2 ECG for SHHS2 = 250 Hz; No. of persons= 5793 for [24,81,82]
SHHS1 and 2651 for SHHS2
Sampling frequency = 256 Hz for ECG; No. of
MESA by National Sleep Research Resource [81,83]
persons = 2056
The SLPDB database Sampling frequency = 250 Hz; No. of persons = 16 [81]
Sampling frequency = 128 Hz; No. of persons = 57 men
Apnea-ECG dataset [82,84]
+ 13 women); Epoch length = 60 s; Segments = 17,045
Sampling frequency = 250 Hz; No. of persons = 16;
The MIT-BIH polysomnography dataset [85]
Epoch length = 30 s
The Massachusetts General Hospital (MGH Dataset)
Sampling frequency = 200 Hz; Epoch length = 30 s [24]
Sleep Laboratory
Sampling frequency = 128 Hz for EEG and 256 Hz for
DREAMER dataset [76,82,86,87]
ECG No. of persons = 23
Haaglanden Medisch Centrum Sleep Center Sampling frequency = 256 Hz; No. of persons = 85 male
[82]
Database (HMC) + 66 female
Sampling frequency = 200 Hz; No. of
Sleep Telemetry Study (Telemetry) [82]
persons = 22 subjects (male and female)
Sampling frequency = 100 Hz; No. of
ISRUC-SLEEP dataset (ISRUC) [82]
persons = 100 subjects (55 male and 45 female)
National Institute of Mental Health of the Czech
Sampling frequency = 250 Hz; No. of persons = 18 [23]
Republic (NIMH-CZ).
Sampling frequency = 16,000 Hz; No. of
DAIC-WOZ depression dataset [5]
persons = 189 Subjects (54 % male and 46 % female )
Sampling frequency = 256 Hz; No. of
Montreal Archive of Sleep Studies (MASS) [69,88]
persons = 97 male + 103 female
Sampling frequency = 256 Hz; No. of
Department of Epileptology at Bonn University [89]
persons = 23 subjects
Biomedinformatics 2023, 3 201

2.4.2. Data Acquisition

For a typical sleep study, at least a one-night PSG test was required per subject for data
collection [37,90–92]. With the assistance of expert interviews, the data were determined.
All raw PSG datasets were stored in the European data format (EDF). Devices/sensors
were always attached to the body of the subject overnight to collect data [41,42,93]. The
key characteristics of a good dataset were the following: It had an appropriate sampling
frequency in Hz with multiple channels and a reference electrode. The source of the
dataset was also necessary to ensure researchers could reference the datasets in their works.
The collected datasets had to be large in size and heterogeneous in nature. Most EGG
dataset used 19 or 16 channels [67,94], i.e., (F3, F4, T3, T4, C3, C4, P3, P4, FZ, CZ, PZ, Fp1,
Fp2, F7, F8, T5, T6), and (O1, O2) [67,74]. Notations including F, T, C, P, and O denoted
frontal, temporal, central, parietal, and occipital, respectively, which were used to identify
the brain lobes and placements of the electrodes on the scalp surface [14,21,54]. Figure 7
shows the various locations of electrode placement for electroencephalogram (EEG) data
collection using a 10–20 scalp electrode system [14,95] for capturing EEG signals. The scalp
electrodes were used to record brain activities via EEG signals. This was recorded by an
electroencephalograph. In clinical practice, a standard ECG signal was obtained using 10
electrodes (4 limb and 6 thorax electrodes) [21,96]. All these bio-signals included noise
that originated from the patient’s body and the environment. Moreover, noise caused
distortions in the time and frequency of the signals. A filtering process is typically required
to eliminate these noises, which is known as pre-processing [97]. Most existing sleep studies
considered EEG signals the predominant PSG signals for the diagnosis. These are brain
wave signals collected at a sampling frequency.

Figure 7. EEG electrode placement locations [96].

2.4.3. Pre-Processing of Signals

For feature extraction, all acquired signal needed to be pre-processed by setting up
a frequency threshold. It is common practice to set the EEG signal at 100 (µν) . Signals
above this threshold are then considered to be noise [95,98]. Figure 8 shows the step for pre-
processing to eliminate various noises from EEG signals. There are many techniques used
for pre-processing. For instance, the ResNet-50 model [99] was adapted to automatically
extract EEG features and reduce the manual steps required for pre-processing data.
Biomedinformatics 2023, 3 202

Figure 8. Flowchart of EEG signal pre-processing.

Categorizing EEG waveform is typically according to five frequency bands namely,

Delta (δ), Theta (θ ), Alpha (α), Beta ( β), and Gamma (γ) [34,74,79,95,100–105]. These
bands include informative detail frequency for signal classification based on the waveform,
such as sleep stage disorder classification [28,31]. Table 3 shows the various frequency and
amplitude as recommended by the American Academy of Sleep Medicine (AASM).

Table 3. EEG waveform frequency band.

Bands Frequency (Hz) Amplitude (µν) Activities

Delta (δ) 0–4.5 20–100 Deep sleep
Theta (θ ) 4–8 10 Light sleep
Alpha (α) 8–13 2–100 Calm or relaxed
Beta ( β) 15–22 5–10 Alert
Gamma (γ) >30 - Hyperactive

2.5. Feature Extraction

After filtering and signal pre-processing, informative features needed to be
extracted [43,80,106,107]. This was one of the critical steps for the application of ML
models for bio-electric signals, and the appropriate design of this step has improved model
performance [108,109]. Different ML models have been used for PSG data analysis [28,31].
Most used for PSG datasets required robust feature extraction that was sufficiently corre-
lated. These features have been extracted based on uni-variate (measures taken on each
channel separately), multi-variate (measures taken on two or more channels) [110], and
multi-modal (measures taken from multiple modalities, such as EEG, ECG, EMG, and
EOG) approaches.There are four predominant features commonly extracted from PSG
data: (1) time-domain, (2) frequency-domain scale, (3) time–frequency domain, and (4)
non-linear [39,111–113]. EEG has typically been analyzed for the frequency domain, while
EOG and EMG have been analyzed for the time domain [36]. Power spectrum density
(PSD) has been a common approach for feature extraction from EEG signals [114–117]. The
most popular method of estimating PSD was based on the measure of the signal’s power
from the device against the device’s time–frequency [34,47,71,118]. PSD was calculated
using Welch and Fourier transformations [114,119–123]. Other widely researched feature
Biomedinformatics 2023, 3 203

extraction methods for PSG signals have included continuous wavelet transform (CWT)
coefficients, autoregressive (AR) coefficients [124,125], and Hjorth parameters [126]. Zhang
et al. [112,127] and Galvao et al. [87] discussed additional feature extraction methods.
Table 4 shows a summary of feature extraction methods for PSG data.

Table 4. A summary of feature extraction methods for PSG data.

Feature Extraction Techniques Signal Type Reference

Adaptive auto-regressive (AAR) EEG-Motor-Imagery [79]
Adaptive auto-regressive
EEG-Motor-Imagery [124]
Fuzzy discernibility matrix (first adaptation)
Random asynchronous particle swarm optimization Eye Movement EEG [128]
Least angle regression + the direct leave-one-out error
Motor-Imagery [129]
estimation by the PRESS statistic
Principal component analysis + decision-tree-based feature
Motor-Imagery [129–131]
ranking (C4.5)
Wavelet packet decomposition + approximation entropy +
Motor-Imagery, Emotional Recognition [132,133]
one-dimensional real-valued particle-swarm optimization
Common spatial model (CSP) Motor-Imagery [134]
Discrete wavelet decomposition (DWT) in five frequency bands,
Motor-Imagery, Emotional Recognition [21,76,135]
combined with wavelet entropy
Differential entropy (DE) Motor-Imagery [136]

2.6. Balancing Datasets

It has been established that PSG data in their raw state are not balanced, as a normal
sleep pattern contains more non-REM sleep than REM sleep, as well as more light sleep than
deep sleep [137]. The imbalance makes it difficult for an ML model to be trained effectively.
Zhou et al. [138] studied different dataset-balancing approaches. Efe et al. [139] proposed a
hybrid neural network architecture using focal-loss and discrete-cosine-transform methods
to solve the training data imbalance. Utomo et al. [133] proposed a model based on ECG
signal to address imbalanced learning challenges. Over-sampling and under-sampling
have been two common strategies, but each has critical weaknesses. For instance, the
straightforward and simple way to handle class imbalance has been to increase the minority
class, i.e., over-sampling, but this approach disrupts the data architecture [140].

2.7. Machine-Learning Modeling

ML leverages the framework of mathematical modeling to classify, predict trends,
and detect anomalies in specified time series [4,30,141,142]. In healthcare, ML has been
used for the feature extraction and classification of disease in many studies [57,143,144].
There have been a plethora of ML approaches used for PSG data classification and per-
formance improvement. The growth of research concerning AI and ML approaches as
well as for the analysis of PSG datasets has shown an upward trend [145,146]. When
applying ML techniques to any dataset, statistical and machine-learning models have
been the two most common models applied [66,147], and this has been further bro-
ken down into sub-categories, including supervised, semi-supervised, and unsupervised
learning [8,59,148,149]. PSG signals have been treated as multi-variate, multi-modal time
series [150]. Lu et al. [7,40] concluded that deep learning was the most-used ML approach
for feature extraction. A significant amount of research and other literature has explored
both statistical and deep-learning approaches on PSG datasets [151], in which support
vector machine was also widely studied in shallow ML approaches and CNNs for deep
learning. Sarkar et al. [152] studied the suitability of recurrent neural networks (RNN)
with long short-term memory (LSTM), support-vector-machine (SVM) [151,153,154], and
Biomedinformatics 2023, 3 204

logistic-regression (LR) models [155,156] to monitor depressive symptoms via EGGand

found that under supervised learning, SVM and LR outperformed the others. In recent
years, different ML models [40,46] have been used to classify and diagnose mental health,
imagery, emotions, behaviors, etc. With the recent increase in the use of ML in the health-
care domain, some of these models have been intensely studied, while others have not
been sufficiently explored. Thamaraimanalan et al. [157] proposed a radial basis function
network (RBFN), which is a variation of artificial neural network (ANN) models. The main
aim of the RBFN model was to solve problems faster and more accurately. The authors
of [158] studied explainable artificial intelligence (XAI), which assisted the final users in
obtaining a reasonable explanation as to underlying fundamentals of the AI model.There
have been a plethora of models proposed in various studies. Below are some of the popular
models noted in the literature selected for this review.
K-Nearest Neighbor (KNN): KNN has been shown to provide high accuracy for
EEG-based emotion classifications [44,86,159,160]. It is a supervised learning method that
was first developed in 1951. KNN has commonly been used for both classification and
regression [161]. KNN is considered to be one of the simplest ML models. It promotes
the concept of the “majority carries the day”. An object is classified based on the plurality
vote of its neighbors [44]. There is a decrease in the classification speed as the number of
variables increases. KNN algorithms are peculiar because of their sensitivity to the actual
data structure.
Support Vector Machine (SVM): SVM is a kernel-based learning method, a supervised
ML algorithm, which has also been commonly adopted for regression problems [154,159,162].
It has been widely studied and used for the classification of PSG datasets [161,163–165].
In many studies, SVM has resulted in a higher accuracy score than its unsupervised
counterparts [161]. Similar to KNN, it is efficient in analyzing data for classification and
regression. In contrast to KNN, however, SVM is a fast and reliable algorithm, and it also
performs well with a limited amount of samples for analysis.
Logistic Regression (LR): LR uses a logistic function on the dependent variable [166].
Subani et al. [165] used LR to model the relationship between a reduced set of features and
the corresponding treatment outcomes based on captured datasets that had been processed
and feature-extracted [163–165].For feature interpretation, LR model coefficients have been
noted as indicators. Unknown records are easily classified, and it is easy to implement and
interpret. When the datasets were linearly structured, LR was very effective [166], because
LR assumes linearity among independent variables [166].
Extreme Learning Machine (ELM): ELM uses a single layer of feedforward neuron
networks (SLFN) and chooses the input weights randomly [43,125,133,134,167,168]. ELM is
a simplified form of an artificial neural network (ANN). ELM was invented in 2006. It differs
from the other neural network model as it does not need gradient-based backpropagation
to be trained. It is not as accurate as other neural network models. Kadam et al. [125]
studied a different type of ELM, called hierarchical ELM, which extended the basic ELM to
multiple layers. Hierarchical ELM was implemented as a supervised learning method.
Multi-Layer Perceptron (MLP): MLP is classified as a feedforward ANN [152,169,170]
that has input, output, and hidden layers in its architecture [137]. It is credited as the
algorithm that forms the base of a complex neural network. MLP classifies data that are not
linearly separable. For difficult or complex datasets, MLP can be customized with a robust
architecture to solve regression and classification tasks. In many applications, MLP has
been shown to be sensitive to feature-scaling due to the option of its activation functions.
Long Short-Term Memory (LSTM): LSTM has been considered by many researchers
as an effective and scalable model for several learning problems related to time-series
data [7,8,13,35,82,105,119,169,171–174]. Using LSTM on PSG data has also resulted in
much success. LSTM has been firmly established as a state-of-the-art approach in sequence
modeling [175,176]. In addition, LSTM has been credited with advanced results in sequence-
processing tasks [131,142,177–183]. The study in [175] presented a more robust model called
a transformer, which was the first sequence-transduction model entirely based on attention
Biomedinformatics 2023, 3 205

and replaced recurrent layers [175,176,184].To the best of our knowledge, there have been
few studies that have applied this model on a PSG dataset [185].
Convolutional Neural Network (CNN): A CNN is typically composed of two types
of layers, where the convolutional layer is followed by a max-pooling layer [169,186–188].
CNNs are more commonly used for image recognition and feature extraction. Using a CNN
alone has produced relatively low forecasting accuracy for time-series data; therefore, a
CNN–LSTM hybrid model has been widely studied on PSG datasets [7,66,142,169,189–196].
For EEG-based analysis, it provided high accuracy and contained a non-linear domain due
to its random and chaotic properties [192,194].
Spiking Neural Network (SNN): SNN is often referred to as the third generation of
ANN [197]. It is a relatively rare approach used to model spatio-temporal brain data (STBD),
and EEG is a well-known non-invasive type of STBD [198]. It has the ability to learn from
changes in temporal data. SNN was inspired by information processing in biology [199].
Despite the increase in the research using SNN, SNN performance has been reported as
relatively low, as compared to other ML counterparts [199]. This limitation was found
in major benchmark datasets. However, because of SNN’s ability to measure biological
spikes without further transformation issues, it has attracted the interest of AI researchers.
The training time of SNN has been an impediment, due to the fact that SNN uses a more
complex method, as compared to other CNN approaches [199].
Table 5 provides a summary of the studies using machine learning for the classification
and prediction of PSG data.

Table 5. A summary of machine learning-based studies for classification and prediction of PSG data.

Model Application Data Used Accuracy Year Ref.

EEG abnormalities of micro-states in Privately sourced dataset from a
LR 66.70% 2018 [156]
temporal lobe epilepsy (TLE) tertiary institute
emotions.csv available on the
Mental depression from EEG dataset 96.60% 2022 [152]
Kaggle website
Emotion Recognition DREAMER (discrete emotion recognition) 94.49% 2021 [86]
KNN EGG, (stress and emotion classification 97.00% 2022 [200]
Obstructive sleep apnea (OSA), ECG and
PhysioNet Sleep Apnea Database 95.08% 2017 [71]
SPO2 signals
SVM EEG image data and emotion classification SEED dataset 56.00% 2022 [99]
Obstructive sleep apnea (OSA), ECG and
PhysioNet Sleep Apnea Database 96.64% 2017 [71]
SPO2 signals
EEG sleep quality Sleep-EDF Database 91.40% 2019 [28]
Imaging and EEG data for ADHD ADHD-200 dataset 97.60% 2022 [151]
Human recognition EEG EMOTIV INSIGHT dataset 94.44% 2016 [201]
mental stress detection using EEG signal mental arithmetic tasks database 97.26% 2022 [56]
EEG-dimensionality reduction Dataset III of BCI competition II 81.40% 2017 [124]
motor imagery EEG signal The BCI competition-II Dataset-III 78.57% 2019 [79]
Identification of chronic alcohol users from
NIMHANS- ECG dataset 87.50% 2017 [43]
ECG signals
Sleep quality measurement Sleep-EDF Database 93.50% 2019 [28]
emotions.csv available on the
Mental depression from EEG dataset 95.89% 2022 [152]
Kaggle website
EEG dataset from NNCI M. V. Lomonosov
Detection of schizophrenia from EEG data 53.50% 2022 [77]
Moscow State University
Biomedinformatics 2023, 3 206

Table 5. Cont.

Model Application Data Used Accuracy Year Ref.

ResNet-50 EEG image data and emotion classification SEED Dataset 85.11% 2022 [99]
EEG-sleep stage using multi-scale
CNN Sleep-EDF Database 96.70% 2022 [29]
dual-attention
emotions.csv available on the
Mental depression from EEG dataset 49.82% 2022 [152]
Kaggle website
Multiple EGG dataset was used for
Automatic sleep scoring 74.17% 2021 [82]
this work
Emotion recognition DREAMER (discrete emotion recognition) 99.90% 2021 [86]
Identification of chronic alcohol users from
ELM NIMHANS- ECG Dataset 94.64% 2017 [43]
ECG signals
emotions.csv available on the
MLP Mental depression from EEG dataset 76.43% 2022 [152]
Kaggle website
emotions.csv available on the
RNN Mental depression from EEG dataset 93.90% 2022 [152]
Kaggle website
RNN with emotions.csv available on the
Mental depression from EEG dataset 97.65% 2022 [152]
LSTM Kaggle website
EEG dataset from NNCI M. V. Lomonosov
Detection of schizophrenia from EEG Data 98.00% 2022 [77]
Moscow State University
MASS Dataset- EEG, EOG, EMG, ECG,
Insomnia detection 79.20% 2021 [88]
and respiratory signals
BCI project for EEG signal and frontal
Depression using EEG 99.66% 2021 [202]
facial data
CNN– Multiple EGG dataset was used for
Automatic sleep scoring 80.17% 2021 [82]
LSTM this work
Sleep apnea Apnea-ECG dataset 97.21% 2022 [84]

2.8. Performance Evaluation

In this section, we discuss model performance measures, which quantified the ef-
fectiveness of a model for classifying or predicting new cases or disease conditions after
being trained, validated, and tested using the available dataset. The results are described
based on different aspects of performance [193,195,203,204]. Most of the papers in this re-
view measured accuracy, precision, sensitivity (recall), specificity, F1-scores, and confusion
matrices [13,54,97,205–210].
1. Sensitivity: It is also known as recall. This measures the ratio of the number of samples
correctly predicted to the total samples in the class. Sensitivity can be calculated based
on true positive (TP) and false negative (FN) parameters [31,208,211]. Equation (1)
shows a mathematical representation of the sensitivity computation.

True Positive
Sensitivity(Sx ) = (1)
True Positive + False Negative

2. Accuracy: This is the fraction of samples that were correctly classified. Accuracy can
be expressed as the ratio of the summation of true-positive (TP) and true-negative (TN)
parameters to the total sample size, which includes true positive (TP), false positive
(FP), false negative (FN), and true negative (TN) [31,137,208,211]. Equation (2) shows
a mathematical representation of accuracy.

True Positive + True Negative

Accuracy( Acc ) =
True Positive + False Positive + True Negative + False Negative
(2)
Biomedinformatics 2023, 3 207

3. Precision: It is the ratio of the samples correctly predicted to the total predicted
positive samples. Equation (3) shows a mathematical representation of the precision
computation.
True Positive
Precision( Ps ) = (3)
True Positive + False Positive
4. Specificity: It measures how many healthy (negative) samples were identified as
healthy (negative) samples by a model. Equation (4) shows a mathematical represen-
tation of the specificity computation.

True Negative
Specificity(Se ) = (4)
True Negative + False Positive

5. F1-score: It is a function of precision and sensitivity (recall). It is represented as

the harmonic mean of sensitivity and precision. Equation (5) shows a mathematical
representation of F1-score computation. F1-scores range from 0 to 1, with 1 being
a perfect precision sensitivity (recall) and 0 being the lowest precision sensitivity.
Equation (5) shows a mathematical representation of the F1-score computation.

2Ps Sx
F1-score( F1 ) = (5)
( Ps + Sx )

The above are the most commonly used performance evaluation metrics for classifi-
cation problems. Using accuracy alone to determine the performance of a classification
model could be misleading. Calculating a confusion matrix provides a more accurate
benchmark for evaluating the performance of classification models, particularly regarding
their accuracy and suitability [54,107,204,212–214].

3. Summary and Discussion

The study of ML methodologies for measuring biomedical signals has, in recent years,
attracted increased attention. In this work, the reviewed literature provided an overview
of the application of ML approaches on PSG datasets.In this section, we summarize the
advantages, the limitations, and the current research gaps concerning these models. We
provide detailed steps for applying ML methods to multi-channel and multi-modal PSG
data. Using ML methods for feature extraction, prediction, diagnosis, and disease classifi-
cation has reduced the dependence of clinical professionals on the manual processing of
PSG datasets.
ML facilitates a more robust and deeper understanding of PSG dataset processing. The
benefits of ML for mental health diagnosis and classification have been confirmed.However,
we reviewed the literature to identify the key steps for applying ML to PSG datasets,
as well as the limitations and benefits.We discuss data-capturing, the types of datasets,
data-processing, feature extraction, model classification, and performance evaluation.

3.1. Challenges of Using ML on Multi-Channel and Multi-Modal PSG Datasets

Data-capturing: Biomedical signals are often recorded using multiple electrodes. This
leads to an increase in the dimension of recorded signals, which makes the analysis of
multi-channel PSG datasets challenging. Typically, these multi-channel EEG signals are
converted to single-channel signals for ease of analysis.
Recently, an increase in available PSG datasets have made ML research possible.
Research conducted by Guillot et al. utilized eight different datasets [185]. In most of the
articles reviewed in this work, many open-access datasets were used. The correct type of
data increased the accuracy of the results. The limitations of certain datasets concern the
issues of imbalance and noise. For satisfactory ML development, datasets must be cleaned
to remove noise, as well as prepared, before applying a classification model.
Biomedinformatics 2023, 3 208

Data-formatting: Biomedical signals are stored using different data formats, such as
the European data format (EDF), the general data format (GDF), and BrainVision (VHDR,
VMRK, EEG). It is standard practice to store PSG data in an EDF or EDF+ format, as these are
simple and flexible formats for the exchange and storage of multi-channel biological, multi-
modal, and physical signals [215,216]. However, the inconsistencies in data-formatting
create barriers for the widespread use of a dataset in the ML research community.For
example, in contrast to comma-separated-values (CSV) files that are commonly used for
data storage, EDF and EDF+ are not as accommodating as CSV files and require a special
tool to read and pre-process.
Data imbalance: PSG datasets must be processed and balanced for satisfactory clas-
sification performance. Due to the nature of these datasets, balancing all the channels in
a multi-channel dataset is always a challenge. In many cases, there is a dominant class.
We found a significant amount of literature that detailed the process of balancing a PSG
dataset. Furthermore, there are issues with multi-modal datasets that have been included
in PSG datasets, as these then have to be treated as a different modality. This also increases
the complexity of balancing a PSG dataset.
Extracting features: In this study, we showed the most commonly used methods for
feature extraction. However, different feature extraction methods can work according to
different data properties. Extensive tests are usually required. PSG datasets often contain a
significant amount of noise, making feature extraction difficult.
Classification model: Such as found in prediction models, there are training, valida-
tion, and testing datasets. Dividing PSG datasets into training, validation, and testing data
poses a challenge because of insufficient labeling, large dataset sizes, and large time stamps.
The presence of additional modalities in PSG datasets also poses a challenge for effective
data fusion.
Performance evaluation: We found in this work that the most used performance
evaluations for ML applications were sensitivity (recall), specificity, F1-score, and accuracy.
Most of the literature in this review had calculated the model accuracy as a way of deter-
mining model performance, but only calculating accuracy was not an effective evaluation
of the robustness of a model. It remained challenging to interpret information presented in
PSG datasets without the assistance of experts to determine the actual performance of a
classification model.In practice, sex and race were considered protected attributes. This
increased concerns regarding the security of subjects’ or patients’ private information since
the actual patient’s personal information was required for data-capturing. The measure of
fairness for PSG datasets has also been challenging [217]. Fairness measures used for ML
have been reported as not universally suitable [217].

3.2. Research Gaps

It was clear that ML offered significant benefits for diagnosis and classification of
mental health issues when used correctly. We showed that when implementing ML for
multi-variate multi-modal PSG datasets, a solid understanding of the steps and techniques
was required.There are many techniques and methods that are highly dependent on the
type of dataset. One major gap in the research was that the gold standard methods and
techniques have yet to be clearly defined.
Data-capturing: As compared to other data-capturing procedure for ML application,
sleep studies required an overnight period for data-capturing. Many patients reported this
as "too long". In all the articles reviewed in this work, there was little effort to reduce this
time. Furthermore, to the best of our knowledge, there has been limited research conducted
concerning the reduction in the number of electrodes or sensors used to capture these
datasets. Using a high number electrodes has been reported as uncomfortable by some
patients [218,219].
Biomedinformatics 2023, 3 209

Data-formatting: Increasing the accessibility of bio-electric datasets has been limited

by using only EDF and EDF+ file extensions.Among the 204 articles selected for this review,
none either proposed or developed a more widely accessible format.
Feature extraction: Establishing gold-standard feature-extraction techniques could
improve their scalability and ease of use.
Classification modeling: Many existing studies have been based on typical ML mod-
els. There have been some proven classification models with robust performance on PSG
datasets. SVM appeared to be a common model that many have used effectively on PSG
datasets. There were many studies that used deep-learning methods. There were a few that
used self-attention models on PSG datasets. More advanced models should be considered,
such as transfer learning, explainable machine learning, and robust learning.

4. Conclusions and Future Work

This study reviewed over 200 articles and their contributions towards the application
of ML techniques for the diagnosis of mental health issues. According to the literature
related to sleep studies and other mental health disorders, ML approaches have improved
diagnostic accuracy, as compared to traditional manual processes, and this trend is likely to
continue.More and more researchers are leveraging ML and AI tools to improve various
aspects of the healthcare environment. There is still more research needed to advance the
reliability and efficiency of these techniques. In this study, we focused on providing the
detailed steps involved when applying ML to PSG datasets.
Based on the literature reviewed in this work and on the basis of our own knowledge,
more research should be conducted regarding the application of attention-based models.
We plan on creating a benchmark for the implementation of ML techniques on PSG datasets.
Currently, there have been various techniques proposed, with most indicating robust
performance enhancements. We plan on researching methods to assist researchers in the
selection of the best classification models and feature extraction techniques.

Author Contributions: Conceptualization, J.E. and H.W.; Methodology, J.E.; Software, J.E.; Validation,
J.E.; Investigation, J.E.; Data curation, J.E.; Writing—original draft, J.E.; Writing— review & editing,
J.E. and H.W.; Visualization, J.E.; Supervision, Haifeng Wang. All authors have read and agreed to
the published version of the manuscript
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflicts of interest.

Nomenclatures
The following terms were used in this manuscript:

AASM American Academy of Sleep Medicine

ADHD Attention-Deficit Hyperactivity Disorder
AI Artificial Intelligence
ANN Artificial Neural Network
CNN Convolutional Neural Networks
ECG Electrocardiograph
EEG Electroencephalogram
ELM Extreme Learning Machine
EMG Electromyogram
Biomedinformatics 2023, 3 210

EOG Electro-oculogram
FFT Fast Fourier Transformation
HRV Heart Rate Variability
LR Logistic Regression
LSTM Long-Short Term Memory
ML Machine Learning
NREM Non-Rapid Eye Movement
REM Rapid Eye Movement
RF Random Forest
RNN Recurrent Neural Networks
RK Rechtschaffen and Kales
PCA Principal Component Analysis
PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses
PSD Power Spectral Density
SNN Spike Neural Network
PSG Polysomnography
SPO2 Saturation of Peripheral Oxygen
SVM Support Vector Machine

References
1. Goetz, C.; Bavaresco, R.; Kunst, R.; Barbosa, J. Industrial intelligence in the care of workers’ mental health: A review of status and
challenges. Int. J. Ind. Ergon. 2022, 87, 103234. [CrossRef]
2. Ramos-Lima, L.F.; Waikamp, V.; Antonelli-Salgado, T.; Passos, I.C.; Freitas, L.H.M. The use of machine learning techniques in
trauma-related disorders: A systematic review. J. Psychiatr. Res. 2020, 121, 159–172. [CrossRef] [PubMed]
3. Li, Y.; Li, N.; Zhang, L.; Liu, Y.; Zhang, T.; Li, D.; Bai, D.; Liu, X.; Li, L. Predicting PTSD symptoms in firefighters using a
fear-potentiated startle paradigm and machine learning. J. Affect. Disord. 2022, 319, 294–299. [CrossRef] [PubMed]
4. Sumathi, M.S.; Joshi, C.S.; Thomas, R.R.; Reethu, G. Analysis and Performance of Machine Learning Algorithms on Disease Diagnosis.
In Proceedings of the 2020 3rd International Conference on Energy, Power and Environment: Towards Clean Energy Technologies,
Shanghai, China, 12–13 July 2021; IEEE: Shillong, India, 2021; pp. 1–6. [CrossRef]
5. Rejaibi, E. MFCC-based Recurrent Neural Network for automatic clinical depression recognition and assessment from speech.
Biomed. Signal Process. Control. 2022, 11. [CrossRef]
6. Gore, E.; Rathi, S. Types of Data with Algorithms for Assessing Mental Health Conditions. In Proceedings of the 2019 5th
International Conference on Computing, Communication, Control and Automation (ICCUBEA), Pune, India, 19–21 September
2019; IEEE: Pune, India, 2019; pp. 1–8. [CrossRef]
7. Lu, W.; Li, J.; Li, Y.; Sun, A.; Wang, J. A CNN-LSTM-Based Model to Forecast Stock Prices. Complexity 2020, 2020, 6622927.
[CrossRef]
8. Zhang, C.; Song, D.; Chen, Y.; Feng, X.; Lumezanu, C.; Cheng, W.; Ni, J.; Zong, B.; Chen, H.; Chawla, N.V. A Deep Neural
Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. Proc. AAAI Conf. Artif. Intell.
2019, 33, 1409–1416. [CrossRef]
9. Li, J.; Izakian, H.; Pedrycz, W.; Jamal, I. Clustering-based anomaly detection in multi-variate time-series data. Appl. Soft Comput.
2021, 100, 106919. [CrossRef]
10. Manjunath, S.; Nathaniel, A.; Druce, J.; German, S. Improving the Performance of Fine-Grain Image Classifiers via Generative
Data Augmentation. arXiv 2020. [CrossRef]
11. Goldstein, M.; Uchida, S. A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data.
PLoS ONE 2016, 11, e0152173. [CrossRef]
12. Sabry, F.; Eltaras, T.; Labda, W.; Alzoubi, K.; Malluhi, Q. Machine Learning for Healthcare Wearable Devices: The Big Picture. J.
Healthc. Eng. 2022, 2022, 4653923. [CrossRef]
13. Mehdiyev, N.; Lahann, J.; Emrich, A.; Enke, D.; Fettke, P.; Loos, P. Time Series Classification using Deep Learning for Process
Planning: A Case from the Process Industry. Procedia Comput. Sci. 2017, 114, 242–249. [CrossRef]
14. Li, J. Multi-modal bio-electrical signal fusion analysis based on different acquisition devices and scene settings: Overview,
challenges, and novel orientation. Inf. Fusion 2022, 79, 229–247. [CrossRef]
15. Saeidi, M.; Karwowski, W.; Farahani, F.V.; Fiok, K.; Taiar, R.; Hancock, P.A.; Al-Juaid, A. Neural Decoding of EEG Signals with
Machine Learning: A Systematic Review. Brain Sci. 2021, 11, 1525. [CrossRef] [PubMed]
16. Heo, J.; Yoon, H.; Park, K. A Novel Wearable Forehead EOG Measurement System for Human Computer Interfaces. Sensors 2017,
17, 1485. [CrossRef] [PubMed]
Biomedinformatics 2023, 3 211

17. Hallett, M.; DelRosso, L.M.; Elble, R.; Ferri, R.; Horak, F.B.; Lehericy, S.; Mancini, M.; Matsuhashi, M.; Matsumoto, R.; Muthuraman,
M.; et al. Evaluation of movement and brain activity. Clin. Neurophysiol. 2021, 132, 2608–2638. [CrossRef] [PubMed]
18. Wang, F.; Wei, X.; Guo, J.; Zheng, Y.; Li, J.; Du, S. Research Progress of Rehabilitation Exoskeletal Robot and Evaluation
Methodologies Based on Bio-electrical Signals. In Proceedings of the 2019 IEEE 9th Annual International Conference on CYBER
Technology in Automation, Control, and Intelligent Systems (CYBER), Suzhou, China, 29 July–2 August 2019; IEEE: Suzhou,
China, 2019; pp. 826–831. [CrossRef]
19. Bleakley, L.E.; Keenan, R.J.; Graven, R.D.; Metha, J.A.; Ma, S.; Daykin, H.; Cornthwaite-Duncan, L.; Hoyer, D.; Reid, C.A.;
Jacobson, L.H. Altered EEG power spectrum, but not sleep-wake architecture, in HCN1 knockout mice. Behav. Brain Res. 2023,
437, 114105. [CrossRef] [PubMed]
20. Bi, X.; Wang, H. Early Alzheimer’s disease diagnosis based on EEG spectral images using deep learning. Neural Netw. 2019, 114,
119–135. [CrossRef]
21. Martinek, R.; Ladrova, M.; Sidikova, M.; Jaros, R.; Behbehani, K.; Kahankova, R.; Kawala-Sterniuk, A. Advanced Bio-electrical
Signal Processing Methods: Past, Present and Future Approach—Part II: Brain Signals. Sensors 2021, 21, 6343. [CrossRef]
22. Kaplan, K.A.; Hardas, P.P.; Redline, S.; Zeitzer, J.M. Correlates of sleep quality in midlife and beyond: A machine learning
analysis. Sleep Med. 2017, 34, 162–167. [CrossRef]
23. Gerla, V.; Kremen, V.; Macas, M.; Dudysova, D.; Mladek, A.; Sos, P.; Lhotska, L. Iterative expert-in-the-loop classification of sleep
PSG recordings using a hierarchical clustering. J. Neurosci. Methods 2019, 317, 61–70. [CrossRef]
24. Biswal, S.; Sun, H.; Goparaju, B.; Westover, M.B.; Sun, J.; Bianchi, M.T. Expert-level sleep scoring with deep neural networks. J.
Am. Med. Inform. Assoc. 2018, 25, 1643–1650. [CrossRef]
25. Chambon, S.; Galtier, M.N.; Arnal, P.J.; Wainrib, G.; Gramfort, A. A Deep Learning Architecture for Temporal Sleep Stage
Classification Using Multivariate and Multimodal Time Series. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 758–769. [CrossRef]
[PubMed]
26. Sharma, M.; Patel, V.; Acharya, U.R. Automated identification of insomnia using optimal bi-orthogonal wavelet transform
technique with single-channel EEG signals. Knowl.-Based Syst. 2021, 224, 107078. [CrossRef]
27. Zaffaroni, A.; Coffey, S.; Dodd, S.; Kilroy, H.; Lyon, G.; O’Rourke, D.; Lederer, K.; Fietze, I.; Penzel, T. Sleep Staging Monitoring
Based on Sonar Smartphone Technology. In Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering
in Medicine and Biology Society (EMBC), Berlin, Germany, 23–27 July 2019; IEEE: Berlin, Germany, 2019; pp. 2230–2233. [CrossRef]
28. Ravan, M. A machine learning approach using EEG signals to measure sleep quality. AIMS Electron. Electr. Eng. 2019, 3, 347–358.
[CrossRef]
29. Wang, H.; Lu, C.; Zhang, Q.; Hu, Z.; Yuan, X.; Zhang, P.; Liu, W. A novel sleep staging network based on multi-scale dual
attention. Biomed. Signal Process. Control 2022, 74, 103486. [CrossRef]
30. Ramachandran, A.; Karuppiah, A. A Survey on Recent Advances in Machine Learning Based Sleep Apnea Detection Systems.
Healthcare 2021, 9, 914. [CrossRef] [PubMed]
31. Aboalayon, K.; Faezipour, M.; Almuhammadi, W.; Moslehpour, S. Sleep Stage Classification Using EEG Signal Analysis: A
Comprehensive Survey and New Investigation. Entropy 2016, 18, 272. [CrossRef]
32. Xu, S.; Faust, O.; Seoni, S.; Chakraborty, S.; Barua, P.D.; Loh, H.W.; Elphick, H.; Molinari, F.; Acharya, U.R. A review of automated
sleep disorder detection. Comput. Biol. Med. 2022, 150, 106100. [CrossRef]
33. Acharya, U.R.; Oh, S.L.; Hagiwara, Y.; Tan, J.H.; Adeli, H.; Subha, D.P. Automated EEG-based screening of depression using deep
convolutional neural network. Comput. Methods Programs Biomed. 2018, 161, 103–113. [CrossRef]
34. Liu, S.; Shen, J.; Li, Y.; Wang, J.; Wang, J.; Xu, J.; Wang, Q.; Chen, R. EEG Power Spectral Analysis of Abnormal Cortical Activations
During REM/NREM Sleep in Obstructive Sleep Apnea. Front. Neurol. 2021, 12, 643855. [CrossRef]
35. Ay, B.; Yildirim, O.; Talo, M.; Baloglu, U.B.; Aydin, G.; Puthankattil, S.D.; Acharya, U.R. Automated Depression Detection Using Deep
Representation and Sequence Learning with EEG Signals. J. Med. Syst. 2019, 43, 205. [CrossRef]
36. Zoubek, L.; Charbonnier, S.; Lesecq, S.; Buguet, A.; Chapotot, F. Feature selection for sleep/wake stages classification using data
driven methods. Biomed. Signal Process. Control 2007, 2, 171–179. [CrossRef]
37. Rezaei, M.; Mohammadi, H.; Khazaie, H. EEG/EOG/EMG data from a cross sectional study on psychophysiological insomnia
and normal sleep subjects. Data Brief 2017, 15, 314–319. [CrossRef] [PubMed]
38. Satapathy, S.K.; Bhoi, A.K.; Loganathan, D.; Khandelwal, B.; Barsocchi, P. Machine learning with ensemble stacking model for
automated sleep staging using dual-channel EEG signal. Biomed. Signal Process. Control 2021, 69, 102898. [CrossRef]
39. Gao, W.; Xu, Y.; Li, S.; Fu, Y.; Zheng, D.; She, Y. Obstructive sleep apnea syndrome detection based on ballistocardiogram via
machine learning approach. Math. Biosci. Eng. 2019, 16, 5672–5686. [CrossRef]
40. Salama, E.S. A 3D-convolutional neural network framework with ensemble learning techniques for multi-modal emotion
recognition. Egypt. Inform. J. 2021, 22, 167–176. [CrossRef]
41. Langer, N.; Plomecka, M.B.; Tröndle, M.; Negi, A.; Popov, T.; Milham, M.; Haufe, S. A benchmark for prediction of psychiatric
multimorbidity from resting EEG data in a large pediatric sample. NeuroImage 2022, 258, 119348. [CrossRef]
Biomedinformatics 2023, 3 212

42. Uwaechia, A.N.; Ramli, D.A. A Comprehensive Survey on ECG Signals as New Biometric Modality for Human Authentication:
Recent Advances and Future Challenges. IEEE Access 2021, 9, 97760–97802. [CrossRef]
43. Rakshith, V.; Apoorv, V.; Akarsh, N.K.; Arjun, K.; Krupa, B.N.; Pratima, M.; Vedamurthachar, A. A novel approach for the
identification of chronic alcohol users from ECG signals. In Proceedings of the TENCON 201–2017 IEEE Region 10 Conference,
Penang, Malaysia, 5–8 November 2017; IEEE: Penang, Malaysia, 2017; pp. 1321–1326. [CrossRef]
44. Vuksanovic, B. Analysis of Human Electrocardiogram for Biometric Recognition Using Analytic and AR Modeling Extracted
Parameters. Int. J. Inf. Electron. Eng. 2014, 4. [CrossRef]
45. Said, S.; Nait-ali, A. Machine-Learning based Wearable Multi-Channel sEMG Biometrics Modality for User’s Identification. In
Proceedings of the 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART), Paris/Créteil,
France, 8–10 December 2021; p. 4.
46. Pigoni, A.; Delvecchio, G.; Madonna, D.; Bressi, C.; Soares, J.; Brambilla, P. Can Machine Learning help us in dealing with
treatment resistant depression? A review. J. Affect. Disord. 2019, 259, 21–26. [CrossRef]
47. Combrisson, E.; Vallat, R.; Eichenlaub, J.B.; O’Reilly, C.; Lajnef, T.; Guillot, A.; Ruby, P.M.; Jerbi, K. Sleep: An Open-Source Python
Software for Visualization, Analysis, and Staging of Sleep Data. Front. Neuroinform. 2017, 11, 60. [CrossRef] [PubMed]
48. Supratak, A.; Dong, H.; Wu, C.; Guo, Y. DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-Channel
EEG. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1998–2008. [CrossRef] [PubMed]
49. Moser, D.; Anderer, P.; Gruber, G.; Parapatics, S.; Loretz, E.; Boeck, M.; Kloesch, G.; Heller, E.; Schmidt, A.; Danker-Hopfe, H.; et al.
Sleep Classification According to AASM and Rechtschaffen & Kales: Effects on Sleep Scoring Parameters. Sleep 2009, 32, 139–149.
50. The AASM-Manual for Scoring Sleep and Associated Event. Available online: https://aasm.org/clinical-resources/scoring-
manual/ (accessed on 21 February 2023).
51. Huang, C.S.; Lin, C.L.; Ko, L.W.; Liu, S.Y.; Sua, T.P.; Lin, C.T. A hierarchical classification system for sleep stage scoring via
forehead EEG signals. In Proceedings of the 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind,
and Brain (CCMB), Singapore, 16–19 April 2013; IEEE: Singapore, 2013; pp. 1–5. [CrossRef]
52. Ghaderyan, P.; Moghaddam, F.; Khoshnoud, S.; Shamsi, M. New interdependence feature of EEG signals as a biomarker of timing
deficits evaluated in Attention-Deficit/Hyperactivity Disorder detection. Measurement 2022, 199, 111468. [CrossRef]
53. Lee, W.; Kim, G.; Yu, J.; Kim, Y. Model Interpretation Considering Both Time and Frequency Axes Given Time Series Data. Appl.
Sci. 2022, 12, 12807. [CrossRef]
54. Dev, A.; Roy, N.; Islam, M.K.; Biswas, C.; Ahmed, H.U.; Amin, M.A.; Sarker, F.; Vaidyanathan, R.; Mamun, K.A. Exploration of
EEG-Based Depression Biomarkers Identification Techniques and Their Applications: A Systematic Review. IEEE Access 2022, 10,
16756–16781. [CrossRef]
55. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.;
Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71.
[CrossRef]
56. Sharma, L.D.; Bohat, V.K.; Habib, M.; Al-Zoubi, A.M.; Faris, H.; Aljarah, I. Evolutionary inspired approach for mental stress
detection using EEG signal. Expert Syst. Appl. 2022, 197, 116634. [CrossRef]
57. Motwani, A.; Shukla, P.K.; Pawar, M. Ubiquitous and smart healthcare monitoring frameworks based on machine learning: A
comprehensive review. Artif. Intell. Med. 2022, 134, 102431. [CrossRef]
58. Barros, C.; Silva, C.A.; Pinheiro, A.P. Advanced EEG-based learning approaches to predict schizophrenia: Promises and pitfalls.
Artif. Intell. Med. 2021, 114, 102039. [CrossRef]
59. Mirchi, N.; Warsi, N.M.; Zhang, F.; Wong, S.M.; Suresh, H.; Mithani, K.; Erdman, L.; Ibrahim, G.M. Decoding Intracranial EEG
With Machine Learning: A Systematic Review. Front. Hum. Neurosci. 2022, 16, 913777. [CrossRef] [PubMed]
60. Thieme, A.; Belgrave, D.; Doherty, G. Machine Learning in Mental Health: A Systematic Review of the HCI Literature to Support
the Development of Effective and Implementable ML Systems. ACM Trans. Comput.-Hum. Interact. 2020, 27, 1–53. [CrossRef]
61. Boukobza, A.; Burgun, A.; Roudier, B.; Tsopra, R. Deep Neural Networks for Simultaneously Capturing Public Topics and
Sentiments During a Pandemic: Application on a COVID-19 Tweet Data Set. JMIR Med. Inform. 2022, 10, e34306. [CrossRef]
[PubMed]
62. Gramfort, A.; Luessi, M.; Larson, E.; Engemann, D.A.; Strohmeier, D.; Brodbeck, C.; Goj, R.; Jas, M.; Brooks, T.; Parkkonen, L.; et
al. MEG and EEG Data Analysis with MNE-Python. Front. Neurosci. 2013, 7, 1–13. [CrossRef]
63. Sekkal, R.N.; Bereksi-Reguig, F.; Ruiz-Fernandez, D.; Dib, N.; Sekkal, S. Automatic sleep-stage classification: From classical
machine learning methods to deep learning. Biomed. Signal Process. Control 2022, 77, 103751. [CrossRef]
64. Pepi, C.; Mercier, M.; Carfì Pavia, G.; de Benedictis, A.; Vigevano, F.; Rossi-Espagnet, M.C.; Falcicchio, G.; Marras, C.E.; Specchio,
N.; de Palma, L. Can Presurgical Interhemispheric EEG Connectivity Predict Outcome in Hemispheric Surgery? A Brain Machine
Learning Approach. Brain Sci. 2023, 13, 71. [CrossRef]
65. ElMoaqet, H.; Eid, M.; Ryalat, M.; Penzel, T. A Deep Transfer Learning Framework for Sleep Stage Classification with Single-
Channel EEG Signals. Sensors 2022, 22, 8826. [CrossRef]
66. Ehiabhi, J.; Wang, H. An Unsupervised Anomaly Detection Model for Multivariate Time Series Data. In Proceedings of the IISE
ANNUAL Conference, Seattle, DC, USA, 21 May 2022; p. 7.
Biomedinformatics 2023, 3 213

67. Ameera, A.; Saidatul, A.; Ibrahim, Z. Analysis of EEG Spectrum Bands Using Power Spectral Density for Pleasure and Displeasure
State. IOP Conf. Ser. Mater. Sci. Eng. 2019, 557, 012030. [CrossRef]
68. Lee, H.; Li, B.; DeForte, S.; Splaingard, M.; Huang, Y.; Chi, Y.; Linwood, S.L. A Large Collection of Real-world Pediatric Sleep
Studies. Sci. Data 2022, 9, 421. [CrossRef]
69. O’Reilly, C.; Gosselin, N.; Carrier, J.; Nielsen, T. Montreal Archive of Sleep Studies: An open-access resource for instrument
benchmarking and exploratory research. J. Sleep Res. 2014, 23, 628–635. [CrossRef]
70. Goldberger, A.L.; Amaral, L.A.N.; Glass, L.; Hausdorff, J.M.; Ivanov, P.C.; Mark, R.G.; Mietus, J.E.; Moody, G.B.; Peng, C.K.;
Stanley, H.E. PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic
Signals. Circulation 2000, 101. [CrossRef] [PubMed]
71. Memis, G.; Sert, M. Multimodal Classification of Obstructive Sleep Apnea Using Feature Level Fusion. In Proceedings of the
2017 IEEE 11th International Conference on Semantic Computing (ICSC), San Diego, CA, USA, 30 January–1 February 2017; IEEE:
San Diego, CA, USA, 2017; pp. 85–88. [CrossRef]
72. Khosla, A.; Khandnor, P.; Chand, T. Automated diagnosis of depression from EEG signals using traditional and deep-learning
approaches: A comparative analysis. Biocybern. Biomed. Eng. 2022, 42, 108–142. [CrossRef]
73. Engemann, D.A.; Mellot, A.; Höchenberger, R.; Banville, H.; Sabbagh, D.; Gemein, L.; Ball, T.; Gramfort, A. A reusable benchmark
of brain-age prediction from M/EEG resting-state signals. NeuroImage 2022, 262, 119521. [CrossRef] [PubMed]
74. Khalighi, S.; Sousa, T.; Santos, J.M.; Nunes, U. ISRUC-Sleep: A comprehensive public dataset for sleep researchers. Comput.
Methods Programs Biomed. 2016, 124, 180–192. [CrossRef] [PubMed]
75. Ranjan, R.; Sahana, B.C. Automatic Detection of Mental Health Status using Alpha Subband of EEG Data. In Proceedings of the
2022 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Messina, Italy, 2–24 June 2022; IEEE:
Messina, Italy, 2022; pp. 1–6. [CrossRef]
76. Jain, R.; Ganesan, R.A. Reliable sleep staging of unseen subjects with fusion of multiple EEG features and RUSBoost. Biomed.
Signal Process. Control 2021, 70, 103061. [CrossRef]
77. Supakar, R.; Satvaya, P.; Chakrabarti, P. A deep learning based model using RNN-LSTM for the Detection of Schizophrenia from
EEG data. Comput. Biol. Med. 2022, 151, 106225. [CrossRef] [PubMed]
78. Rasheed, K.; Qadir, J.; O’Brien, T.J.; Kuhlmann, L.; Razi, A. A Generative Model to Synthesize EEG Data for Epileptic Seizure
Prediction. IEEE Trans. Neural Syst. Rehabil. Eng. 2021, 29, 2322–2332. [CrossRef]
79. Chatterjee, R.; Maitra, T.; Hafizul Islam, S.; Hassan, M.M.; Alamri, A.; Fortino, G. A novel machine learning based feature
selection for motor imagery EEG signal classification in Internet of medical things environment. Future Gener. Comput. Syst. 2019,
98, 419–434. [CrossRef]
80. Alam, M.N.; Ibrahimy, M.I.; Motakabber, S.M.A. Feature Extraction of EEG Signal by Power Spectral Density for Motor Imagery
Based BCI. In Proceedings of the 2021 8th International Conference on Computer and Communication Engineering (ICCCE),
Kuala Lumpur, Malaysia, 9–10 April 2021; IEEE: Kuala Lumpur, Malaysia, 2021; pp. 234–237. [CrossRef]
81. Tang, M.; Zhang, Z.; He, Z.; Li, W.; Mou, X.; Du, L.; Wang, P.; Zhao, Z.; Chen, X.; Li, X.; et al. Deep adaptation network for
subject-specific sleep stage classification based on a single-lead ECG. Biomed. Signal Process. Control 2022, 75, 103548. [CrossRef]
82. Alvarez-Estevez, D.; Rijsman, R.M. Inter-database validation of a deep-learning approach for automatic sleep scoring. PLoS ONE
2021, 16, e0256111. [CrossRef]
83. Palotti, J.; Mall, R.; Aupetit, M.; Rueschman, M.; Singh, M.; Sathyanarayana, A.; Taheri, S.; Fernandez-Luque, L. Benchmark on a
large cohort for sleep-wake classification with machine learning techniques. NPJ Digit. Med. 2019, 2, 50. [CrossRef]
84. Zarei, A.; Beheshti, H.; Asl, B.M. Detection of sleep apnea using deep neural networks and single-lead ECG signals. Biomed.
Signal Process. Control 2022, 71, 103125. [CrossRef]
85. Zhao, R.; Xia, Y.; Wang, Q. Dual-modal and multi-scale deep neural networks for sleep staging using EEG and ECG signals.
Biomed. Signal Process. Control 2021, 66, 102455. [CrossRef]
86. Pisipati, M.; Nandy, A. Human Emotion Recognition using EEG Signal in Music Listening. In Proceedings of the 2021 IEEE 18th
India Council International Conference (INDICON), Guwahati, India, 19–21 December 2021; IEEE: Guwahati, India, 2021; pp.
1–6. [CrossRef]
87. Galvão, F.; Alarcão, S.M.; Fonseca, M.J. Predicting Exact Valence and Arousal Values from EEG. Sensors 2021, 21, 3414. [CrossRef]
88. Qu, W.; Kao, C.H.; Hong, H.; Chi, Z.; Grunstein, R.; Gordon, C.; Wang, Z. Single-channel EEG based insomnia detection with
domain adaptation. Comput. Biol. Med. 2021, 139, 104989. [CrossRef]
89. Abdelhameed, A.M.; Bayoumi, M. Semi-Supervised EEG Signals Classification System for Epileptic Seizure Detection. IEEE
Signal Process. Lett. 2019, 26, 1922–1926. [CrossRef]
90. Pourmohammadi, S.; Maleki, A. Stress detection using ECG and EMG signals: A comprehensive study. Comput. Methods Programs
Biomed. 2020, 193, 105482. [CrossRef] [PubMed]
91. Zhu, S.; Qi, J.; Hu, J.; Hao, S. A new approach for product evaluation based on integration of EEG and eye-tracking. Adv. Eng.
Inform. 2022, 52, 101601. [CrossRef]
92. Sridhar, N.; Shoeb, A.; Stephens, P.; Kharbouch, A.; Shimol, D.B.; Burkart, J.; Ghoreyshi, A.; Myers, L. Deep learning for automated
sleep staging using instantaneous heart rate. NPJ Digit. Med. 2020, 3, 106. [CrossRef] [PubMed]
Biomedinformatics 2023, 3 214

93. Petroff, O.A.; Spencer, D.D.; Goncharova, I.I.; Zaveri, H.P. A comparison of the power spectral density of scalp EEG and subjacent
electrocorticograms. Clin. Neurophysiol. 2016, 127, 1108–1112. [CrossRef] [PubMed]
94. Unde, S.A.; Shriram, R. Coherence Analysis of EEG Signal Using Power Spectral Density. In Proceedings of the 2014 Fourth
International Conference on Communication Systems and Network Technologies, Bhopal, India, 7–9 April 2014; IEEE: Bhopal,
India, 2014; pp. 871–874. [CrossRef]
95. Nagar, P.; Sethia, D. Brain Mapping Based Stress Identification Using Portable EEG Based Device. In Proceedings of the 2019 11th
International Conference on Communication Systems & Networks (COMSNETS), Bengaluru, India, 7–11 January 2019; IEEE:
Bengaluru, India, 2019; pp. 601–606. [CrossRef]
96. Weon, H.W.; Byun, Y.E.; Lim, H.J. Quantitative EEG (QEEG) Analysis of Emotional Interaction between Abusers and Victims in
Intimate Partner Violence: A Pilot Study. Brain Sci. 2021, 11, 570. [CrossRef] [PubMed]
97. El-Sappagh, S.; Alonso, J.M.; Islam, S.M.R.; Sultan, A.M.; Kwak, K.S. A multilayer multi-modal detection and prediction model
based on explainable artificial intelligence for Alzheimer’s disease. Sci. Rep. 2021, 11, 2660. [CrossRef] [PubMed]
98. Vidhya, R.B.; Jerritta, S. Pre-processing ECG signals for smart home material application. Mater. Today Proc. 2022, 49, 2955–2961.
[CrossRef]
99. Tian, T.; Wang, L.; Luo, M.; Sun, Y.; Liu, X. ResNet-50 based technique for EEG image characterization due to varying
environmental stimuli. Comput. Methods Programs Biomed. 2022, 225, 107092. [CrossRef] [PubMed]
100. Kora, P.; Meenakshi, K.; Swaraja, K.; Rajani, A.; Raju, M.S. EEG based interpretation of human brain activity during yoga and
meditation using machine learning: A systematic review. Complement. Ther. Clin. Pract. 2021, 43, 101329. [CrossRef] [PubMed]
101. Sharif, M.S.; Theeng Tamang, M.R.; Fu, C. Predicting the Health Impacts of Commuting Using EEG Signal Based on Intelligent
Approach. In Proceedings of the 2021 International Conference on Innovation and Intelligence for Informatics, Computing, and
Technologies (3ICT), Zallaq, Bahrain, 29–30 September 2021; IEEE: Zallaq, Bahrain, 2021; pp. 386–391. [CrossRef]
102. Uchida, S.; Maloney, T.; Feinberg, I. Sigma (12–16 Hz) and beta (20–28 Hz) EEG discriminate NREM and REM sleep. Brain Res.
1994, 659, 243–248. [CrossRef] [PubMed]
103. Quintero-Zea, A.; Lopez, J.D.; Smith, K.; Trujillo, N.; Parra, M.A.; Escudero, J. Phenotyping Ex-Combatants From EEG Scalp
Connectivity. IEEE Access 2018, 6, 55090–55098. [CrossRef]
104. Hsu, Y.L.; Yang, Y.T.; Wang, J.S.; Hsu, C.Y. Automatic sleep stage recurrent neural classifier using energy features of EEG signals.
Neurocomputing 2013, 104, 105–114. [CrossRef]
105. Patnaik, S.; Moharkar, L.; Chaudhari, A. Deep RNN learning for EEG based functional brain state inference. In Proceedings
of the 2017 International Conference on Advances in Computing, Communication and Control (ICAC3), Mumbai, India, 1–2
December 2017; IEEE: Mumbai, India, 2017; pp. 1–6. [CrossRef]
106. Kim, C.; Sun, J.; Liu, D.; Wang, Q.; Paek, S. An effective feature extraction method by power spectral density of EEG signal for
2-class motor imagery-based BCI. Med. Biol. Eng. Comput. 2018, 56, 1645–1658. [CrossRef]
107. Zhou, J.; Wang, G.; Liu, J.; Wu, D.; Xu, W.; Wang, Z.; Ye, J.; Xia, M.; Hu, Y.; Tian, Y. Automatic Sleep Stage Classification With
Single Channel EEG Signal Based on Two-Layer Stacked Ensemble Model. IEEE Access 2020, 8, 57283–57297. [CrossRef]
108. Faisal, M.A.A.; Chowdhury, M.E.; Khandakar, A.; Hossain, M.S.; Alhatou, M.; Mahmud, S.; Ara, I.; Sheikh, S.I.; Ahmed, M.U. An
investigation to study the effects of Tai Chi on human gait dynamics using classical machine learning. Comput. Biol. Med. 2022,
142, 105184. [CrossRef] [PubMed]
109. Majhi, M.K.; Pradhan, B.K.; Sarkar, P.; Sivaraman, J.; Pal, K. Can statistical and entropy-based features extracted from ECG signals
efficiently differentiate the cannabis-consuming women population from the non-consumer? Med. Hypotheses 2022, 167, 110952.
[CrossRef]
110. Rasheed, K.; Qayyum, A.; Qadir, J.; Sivathamboo, S.; Kwan, P.; Kuhlmann, L.; O’Brien, T.; Razi, A. Machine Learning for
Predicting Epileptic Seizures Using EEG Signals: A Review. IEEE Rev. Biomed. Eng. 2021, 14, 139–155. [CrossRef] [PubMed]
111. Cruz-Garza, J.G.; Darfler, M.; Rounds, J.D.; Gao, E.; Kalantari, S. EEG-based investigation of the impact of room size and window
placement on cognitive performance. J. Build. Eng. 2022, 53, 104540. [CrossRef]
112. Zhang, R.; Jia, J.; Zhang, R. EEG analysis of Parkinson’s disease using time–frequency analysis and deep learning. Biomed. Signal
Process. Control 2022, 78, 103883. [CrossRef]
113. Zhuang, Z.; Wang, F.; Yang, X.; Zhang, L.; Fu, C.H.; Xu, J.; Li, C.; Hong, H. Accurate contactless sleep apnea detection framework
with signal processing and machine learning methods. Methods 2022, 205, 167–178. [CrossRef]
114. Parhi, K.K.; Ayinala, M. Low-Complexity Welch Power Spectral Density Computation. IEEE Trans. Circuits Syst. I Regul. Pap.
2014, 61, 172–182. [CrossRef]
115. Gore, E.; Rathi, S. Surveying Machine Learning Algorithms On Eeg Signals Data For Mental Health Assessment. In Proceedings
of the 2019 IEEE Pune Section International Conference (PuneCon), Pune, India, 18–20 December 2019; IEEE: Pune, India, 2019;
pp. 1–6. [CrossRef]
116. Che Wan Fadzal, C.W.N.F.; Mansor, W.; Khuan, L.Y.; Mohamad, N.B.; Mahmoodin, Z.; Mohamad, S.; Amirin, S. Welch power
spectral density of EEG signal generated from dyslexic children. In Proceedings of the 2014 IEEE REGION 10 SYMPOSIUM,
Kuala Lumpur, Malaysia, 14–16 April 2014; IEEE: Kuala Lumpur, Malaysia, 2014; pp. 560–562. [CrossRef]
Biomedinformatics 2023, 3 215

117. Lai, D.; Heyat, M.B.B.; Khan, F.I.; Zhang, Y. Prognosis of Sleep Bruxism Using Power Spectral Density Approach Applied on EEG
Signal of Both EMG1-EMG2 and ECG1-ECG2 Channels. IEEE Access 2019, 7, 82553–82562. [CrossRef]
118. Kang, J.M.; Cho, S.E.; Moon, J.Y.; Kim, S.I.; Kim, J.W.; Kang, S.G. Difference in spectral power density of sleep electroencephalog-
raphy between individuals without insomnia and frequent hypnotic users with insomnia complaints. Sci. Rep. 2022, 12, 2117.
[CrossRef]
119. Wang, L.; Deng, X.; Lv, X.; Liu, K.; Yang, Q.; Long, C. A WeChat Mini-program System with LSTM for The Emotional EEG Signal
Recognition. In Proceedings of the 2020 2nd International Conference on Industrial Artificial Intelligence (IAI), Shenyang, China,
23–25 October 2020; IEEE: Shenyang, China, 2020; pp. 1–5. [CrossRef]
120. Vuppalapati, C.; Raghu, N.; Veluru, P.; Khursheed, S. A System To Detect Mental Stress Using Machine Learning And Mobile
Development. In Proceedings of the 2018 International Conference on Machine Learning and Cybernetics (ICMLC), Chengdu,
China, 15–18 July 2018; IEEE: Chengdu, China, 2018; pp. 161–166. [CrossRef]
121. You, Y.; Zhong, X.; Liu, G.; Yang, Z. Automatic sleep-stage classification: A light and efficient deep neural network model based
on time, frequency and fractional Fourier transform domain features. Artif. Intell. Med. 2022, 127, 102279. [CrossRef]
122. Zhang, Y.; Wang, K.; Wei, Y.; Guo, X.; Wen, J.; Luo, Y. Minimal EEG channel selection for depression detection with connectivity
features during sleep. Comput. Biol. Med. 2022, 147, 105690. [CrossRef] [PubMed]
123. Molin, N.L.; Molin, C.; Dalpatadu, R.J.; Singh, A.K. Prediction of obstructive sleep apnea using Fast Fourier Transform of
overnight breath recordings. Mach. Learn. Appl. 2021, 4, 100022. [CrossRef]
124. Chatterjee, R.; Bandyopadhyay, T.; Sanyal, D.K.; Guha, D. Dimensionality reduction of EEG signal using Fuzzy Discernibility
Matrix. In Proceedings of the 2017 10th International Conference on Human System Interactions (HSI), Ulsan, Republic of Korea,
17–19 July 2017; IEEE: Ulsan, Republic of Korea, 2017; pp. 131–136. [CrossRef]
125. Kadam, S.T.; Dhaimodker, V.M.; Patil, M.M.; Edla, D.R. EIQ: EEG based IQ test using wavelet packet transform and hierarchical
extreme learning machine. J. Neurosci. Methods 2019, 322, 71–82. [CrossRef] [PubMed]
126. Rodríguez-Bermúdez, G.; García-Laencina, P.J.; Roca-González, J.; Roca-Dorda, J. Efficient feature selection and linear discrimina-
tion of EEG signals. Neurocomputing 2013, 115, 161–165. [CrossRef]
127. Zhang, J.; Yin, Z.; Chen, P.; Nichele, S. Emotion recognition using multi-modal data and machine learning techniques: A tutorial
and review. Inf. Fusion 2020, 59, 103–126. [CrossRef]
128. Adam, A.; Shapiai, M.I.; Mohd Tumari, M.Z.; Mohamad, M.S.; Mubin, M. Feature Selection and Classifier Parameters Estimation
for EEG Signals Peak Detection Using Particle Swarm Optimization. Sci. World J. 2014, 2014, 973063. [CrossRef] [PubMed]
129. Debarnot, U.; Perrault, A.A.; Sterpenich, V.; Legendre, G.; Huber, C.; Guillot, A.; Schwartz, S. Motor imagery practice benefits
during arm immobilization. Sci. Rep. 2021, 11, 8928. [CrossRef] [PubMed]
130. Duan, L.; Ge, H.; Ma, W.; Miao, J. EEG feature selection method based on decision tree. Bio-Med. Mater. Eng. 2015, 26, S1019–S1025.
[CrossRef]
131. Duan, X.; Ying, S.; Yuan, W.; Cheng, H.; Yin, X. A Generative Adversarial Networks for Log Anomaly Detection. Comput. Syst.
Sci. Eng. 2021, 37, 135–148. [CrossRef]
132. Li, T.; Zhou, M. ECG Classification Using Wavelet Packet Entropy and Random Forests. Entropy 2016, 18, 285. [CrossRef]
133. Utomo, O.K.; Surantha, N.; Isa, S.M.; Soewito, B. Automatic Sleep Stage Classification using Weighted ELM and PSO on
Imbalanced Data from Single Lead ECG. Procedia Comput. Sci. 2019, 157, 321–328. [CrossRef]
134. Annaby, M.; Said, M.; Eldeib, A.; Rushdi, M. EEG-based motor imagery classification using digraph Fourier transforms and
extreme learning machines. Biomed. Signal Process. Control 2021, 69, 102831. [CrossRef]
135. Wang, Y.; Wang, W.; Liu, Y.; Wang, D.; Liu, B.; Shi, Y.; Gao, P. Feature Extracting of Weak Signal in Real-Time Sleeping EEG with
Approximate Entropy and Bispectrum Analysis. In Proceedings of the 2009 3rd International Conference on Bioinformatics and
Biomedical Engineering, Beijing, China, 11–16 June 2009; IEEE: Beijing, China, 2009; pp. 1–4. [CrossRef]
136. Koh, J.; Ooi, C.P.; Lim-Ashworth, N.S.; Vicnesh, J.; Tor, H.T.; Lih, O.S.; Tan, R.S.; Acharya, U.; Fung, D.S.S. Automated classification
of attention deficit hyperactivity disorder and conduct disorder using entropy features with ECG signals. Comput. Biol. Med.
2022, 140, 105120. [CrossRef] [PubMed]
137. Tautan, A.M.; Rossi, A.C.; de Francisco, R.; Ionescu, B. Automatic Sleep Stage Detection: A Study on the Influence of Various PSG
Input Signals. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology
Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; IEEE: Montreal, QC, Canada, 2020; pp. 5330–5334. [CrossRef]
138. Zhou, D.; Xu, Q.; Wang, J.; Xu, H.; Kettunen, L.; Chang, Z.; Cong, F. Alleviating Class Imbalance Problem in Automatic Sleep
Stage Classification. IEEE Trans. Instrum. Meas. 2022, 71, 1–12. [CrossRef]
139. Efe, E.; Ozsen, S. CoSleepNet: Automated sleep staging using a hybrid CNN-LSTM network on imbalanced EEG-EOG datasets.
Biomed. Signal Process. Control 2023, 80, 104299. [CrossRef]
140. Xu, Q.; Zhou, D.; Wang, J.; Shen, J.; Kettunen, L.; Cong, F. Convolutional Neural Network Based Sleep Stage Classification with
Class Imbalance. In Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 18–26 July
2022; IEEE: Padua, Italy, 2022; pp. 1–6. [CrossRef]
141. Patterson, J.; Gibson, A. Deep Learning A Practitioner’s Approach; y O’Reilly Media, Inc.: Sebastopol, CA, USA, 2017.
Biomedinformatics 2023, 3 216

142. Qiu, S.; Zhao, H.; Jiang, N.; Wang, Z.; Liu, L.; An, Y.; Zhao, H.; Miao, X.; Liu, R.; Fortino, G. Multi-sensor information fusion based
on machine learning for real applications in human activity recognition: State-of-the-art and research challenges. Inf. Fusion 2022,
80, 241–265. [CrossRef]
143. Singh, P.; Singh, N.; Singh, K.K.; Singh, A. Chapter 5—Diagnosing of disease using machine learning. In Machine Learning and the
Internet of Medical Things in Healthcare; Singh, K.K., Elhoseny, M., Singh, A., Elngar, A.A., Eds.; Academic Press: Cambridge, MA,
USA, 2021; pp. 89–111. [CrossRef]
144. Shehab, M.; Abualigah, L.; Shambour, Q.; Abu-Hashem, M.A.; Shambour, M.K.Y.; Alsalibi, A.I.; Gandomi, A.H. Machine learning
in medical applications: A review of state-of-the-art methods. Comput. Biol. Med. 2022, 145, 105458. [CrossRef]
145. Guo, K.; Mei, H.; Xie, X.; Xu, X. A Convolutional Neural Network Feature Fusion Framework with Ensemble Learning for
EEG-based Emotion Classification. In Proceedings of the 2019 IEEE MTT-S International Microwave Biomedical Conference
(IMBioC), Nanjing, China, 6–8 May 2019; IEEE: Nanjing, China, 2019; pp. 1–4. [CrossRef]
146. Shatte, A.; Hutchinson, D.; Teague, S. Machine learning in mental health: A systematic scoping review of methods and
applications. Psychol. Med. 2019, 49, 1426–1448. [CrossRef]
147. Wang, J.; Chen, Y.; Hao, S.; Peng, X.; Hu, L. Deep learning for sensor-based activity recognition: A survey. Pattern Recognit. Lett.
2019, 119, 3–11. [CrossRef]
148. Sathya, R.; Annamma, A. Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification. Int. J.
Adv. Res. Artif. Intell. 2013, 2, 1–80. [CrossRef]
149. Schmidt, F.; Suri-Payer, F.; Gulenko, A.; Wallschlager, M.; Acker, A.; Kao, O. Unsupervised Anomaly Event Detection for
Cloud Monitoring Using Online Arima. In Proceedings of the 2018 IEEE/ACM International Conference on Utility and Cloud
Computing Companion (UCC Companion), Zurich, Switzerland, 17–20 December 2018; IEEE: Zurich, Switzerland, 2018; pp.
71–76. [CrossRef]
150. Garcia-Ceja, E.; Riegler, M.; Nordgreen, T.; Jakobsen, P.; Oedegaard, K.J.; Tørresen, J. Mental health monitoring with multi-modal
sensing and machine learning: A survey. Pervasive Mob. Comput. 2018, 51, 1–26. [CrossRef]
151. Nash, C.; Nair, R.; Naqvi, S.M. Machine Learning and ADHD Mental Health Detection—A Short Survey. In Proceedings of
the 2022 25th International Conference on Information Fusion (FUSION), Linköping, Sweden, 4–7 July 2022; IEEE: Linköping,
Sweden, 2022; pp. 1–8. [CrossRef]
152. Sarkar, A.; Singh, A.; Chakraborty, R. A deep learning-based comparative study to track mental depression from EEG data.
Neurosci. Inform. 2022, 2, 100039. [CrossRef]
153. Liu, F.T.; Ting, K.M.; Zhou, Z.H. Isolation-Based Anomaly Detection. ACM Trans. Knowl. Discov. Data 2012, 6, 1–39. [CrossRef]
154. Moura, M.d.C.; Zio, E.; Lins, I.D.; Droguett, E. Failure and reliability prediction by support vector machines regression of
time-series data. Reliab. Eng. Syst. Saf. 2011, 96, 1527–1534. [CrossRef]
155. Stranges, S.; Tigbe, W.; Gómez-Olivé, F.X.; Thorogood, M.; Kandala, N.B. Sleep Problems: An Emerging Global Epidemic?
Findings From the INDEPTH WHO-SAGE Study Among More Than 40,000 Older Adults From 8 Countries Across Africa and
Asia. Sleep 2012, 35, 1173–1181. [CrossRef]
156. V, K.R.; Rajagopalan, S.S.; Bhardwaj, S.; Panda, R.; Reddam, V.R.; Ganne, C.; Kenchaiah, R.; Mundlamuri, R.C.; Kandavel, T.;
Majumdar, K.K.; et al. Machine learning detects EEG microstate alterations in patients living with temporal lobe epilepsy. Seizure
2018, 61, 8–13. [CrossRef]
157. Thamaraimanalan, T.; Mohankumar, M.; Anandakumar, H.; Deepha, M.; Priya, U.H.; Priya, G.B.; Devi, M.A. Machine Learning
based Patient Mental Health Prediction using Spectral Clustering and RBFN Algorithms. In Proceedings of the 2022 8th
International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 25–26 March
2022; IEEE: Coimbatore, India, 2022; pp. 1840–1843. [CrossRef]
158. van der Velden, B.H.; Kuijf, H.J.; Gilhuijs, K.G.; Viergever, M.A. Explainable artificial intelligence (XAI) in deep learning-based
medical image analysis. Med. Image Anal. 2022, 79, 102470. [CrossRef]
159. Ahuja, R.; Banga, A. Mental Stress Detection in University Students using Machine Learning Algorithms. Procedia Comput. Sci.
2019, 152, 349–353. [CrossRef]
160. He, Z.; Xu, X.; Deng, S. Discovering cluster-based local outliers. Pattern Recognit. Lett. 2003, 24, 1641–1650. [CrossRef]
161. Hosseini, M.P.; Hosseini, A.; Ahi, K. A Review on Machine Learning for EEG Signal Processing in Bioengineering. IEEE Rev.
Biomed. Eng. 2021, 14, 204–218. [CrossRef]
162. Suthaharan, S. Support Vector Machine. In Machine Learning Models and Algorithms for Big Data Classification. Integrated Series in
Information Systems; Springer: Boston, MA, USA, 2016; Volume 36.
163. Khatun, S.; Morshed, B.I.; Bidelman, G.M. A Single-Channel EEG-Based Approach to Detect Mild Cognitive Impairment via
Speech-Evoked Brain Responses. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 1063–1070. [CrossRef] [PubMed]
164. Raut, K.; Patil, J.; Wade, S.; Tinsu, J. Mental Health and Personality Determination using Machine Learning. In Proceedings of
the 2022 7th International Conference on Communication and Electronics Systems (ICCES), Coimbatore, India, 22–24 June 2022;
IEEE: Coimbatore, India, 2022; pp. 1231–1236. [CrossRef]
165. Subhani, A.R.; Mumtaz, W.; Saad, M.N.B.M.; Kamel, N.; Malik, A.S. Machine Learning Framework for the Detection of Mental
Stress at Multiple Levels. IEEE Access 2017, 5, 13545–13556. [CrossRef]
Biomedinformatics 2023, 3 217

166. Raj, A. The Perfect Recipe for Classification UsingLogistic Regression. Available online: https://towardsdatascience.com/the-
perfect-recipe-for-classification-using-logistic-regression-f8648e267592 (accessed on 21 February 2023).
167. Shen, C.; Lin, H.; Fan, X.; Chu, Y.; Yang, Z.; Wang, J.; Zhang, S. Biomedical event trigger detection with convolutional highway
neural network and extreme learning machine. Appl. Soft Comput. 2019, 84, 105661. [CrossRef]
168. Pawar, D.; Dhage, S. EEG-based covert speech decoding using random rotation extreme learning machine ensemble for intuitive
BCI communication. Biomed. Signal Process. Control 2023, 80, 104379. [CrossRef]
169. Cecaj, A.; Lippi, M.; Mamei, M.; Zambonelli, F. Comparing Deep Learning and Statistical Methods in Forecasting Crowd
Distribution from Aggregated Mobile Phone Data. Appl. Sci. 2020, 10, 6580. [CrossRef]
170. Guillot, A.; Sauvet, F.; During, E.H.; Thorey, V. Dreem Open Datasets: Multi-Scored Sleep Datasets to compare Human and
Automated sleep staging. IEEE Trans. Neural Syst. Rehabilit. Eng. 2020, 28, 1955–1965. [CrossRef]
171. Greff, K.; Srivastava, R.K.; Koutník, J.; Steunebrink, B.R.; Schmidhuber, J. LSTM: A Search Space Odyssey. IEEE Trans. Neural
Netw. Learn. Syst. 2017, 28, 2222–2232. [CrossRef]
172. Aydin, O.; Guldamlasioglu, S. Using LSTM networks to predict engine condition on large scale data processing framework. In
Proceedings of the 2017 4th International Conference on Electrical and Electronic Engineering (ICEEE), Ankara, Turkey, 8–10
April 2017; IEEE: Ankara, Turkey, 2017; pp. 281–285. [CrossRef]
173. Provotar, O.I.; Linder, Y.M.; Veres, M.M. Unsupervised Anomaly Detection in Time Series Using LSTM-Based Autoencoders. In
Proceedings of the 2019 IEEE International Conference on Advanced Trends in Information Theory (ATIT), Kyiv, Ukraine, 18–20
December 2019; IEEE: Kyiv, Ukraine, 2019; pp. 513–517. [CrossRef]
174. Shi, X.; Chen, Z.; Wang, H.; Yeung, D.Y.; Wong, W.k.; Woo, W.c. Convolutional LSTM Network: A Machine Learning Approach
for Precipitation Nowcasting. In Proceedings of the 28th International Conference on Neural Information Processing Systems,
Montreal, QC, Canada, 7–12 December 2015; p. 9.
175. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention Is All You Need.
arXiv 2017, arXiv:1706.03762.
176. Bhojanapalli, S.; Yun, C.; Rawat, A.S.; Reddi, S.; Kumar, S. Low-Rank Bottleneck in Multi-head Attention Models. In Proceedings
of the 37th International Conference on Machine Learning, Virtual, 13–18 July 2020; pp. 864–873.
177. Zhou, X.; Liang, W.; Wang, K.I.K.; Wang, H.; Yang, L.T.; Jin, Q. Deep-Learning-Enhanced Human Activity Recognition for Internet
of Healthcare Things. IEEE Internet Things J. 2020, 7, 6429–6438. [CrossRef]
178. Murad, A.; Pyun, J.Y. Deep Recurrent Neural Networks for Human Activity Recognition. Sensors 2017, 17, 2556. [CrossRef]
179. Inoue, M.; Inoue, S.; Nishida, T. Deep recurrent neural network for mobile human activity recognition with high throughput.
Artif. Life Robot. 2018, 23, 173–185. [CrossRef]
180. Liu, L. Objects detection toward complicated high remote basketball sports by leveraging deep CNN architecture. Future Gener.
Comput. Syst. 2021, 119, 31–36. [CrossRef]
181. Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997, 9, 1735–1780. [CrossRef] [PubMed]
182. Erdaş, Ç.B.; Güney, S. Human Activity Recognition by Using Different Deep Learning Approaches for Wearable Sensors. Neural
Process. Lett. 2021, 53, 1795–1809. [CrossRef]
183. Nath, R.K.; Thapliyal, H.; Caban-Holt, A. Machine Learning Based Stress Monitoring in Older Adults Using Wearable Sensors
and Cortisol as Stress Biomarker. J. Signal Process. Syst. 2022, 94, 513–525. [CrossRef]
184. Wu, N.; Green, B.; Ben, X.; O’Banion, S. Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case.
arXiv 2020, arXiv:2001.08317.
185. Guillot, A.; Thorey, V. RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale. IEEE Trans. Neural Syst. Rehabil.
Eng. 2021, 29, 1441–1451. [CrossRef]
186. Munir, M.; Siddiqui, S.A.; Chattha, M.A.; Dengel, A.; Ahmed, S. FuseAD: Unsupervised Anomaly Detection in Streaming Sensors
Data by Fusing Statistical and Deep Learning Models. Sensors 2019, 19, 2451. [CrossRef]
187. Thongsuwan, S.; Jaiyen, S.; Padcharoen, A.; Agarwal, P. ConvXGB: A new deep learning model for classification problems based
on CNN and XGBoost. Nucl. Eng. Technol. 2021, 53, 522–531. [CrossRef]
188. Choi, S.H.; Yoon, H.; Kim, H.S.; Kim, H.B.; Kwon, H.B.; Oh, S.M.; Lee, Y.J.; Park, K.S. Real-time apnea-hypopnea event detection during
sleep by convolutional neural networks. Comput. Biol. Med. 2018, 100, 123–131. [CrossRef]
189. Kavi, R.; Kulathumani, V.; Rohit, F.; Kecojevic, V. Multiview fusion for activity recognition using deep neural networks. J. Electron.
Imaging 2016, 25, 043010. [CrossRef]
190. Hssayeni, M.D.; Jimenez-Shahed, J.; Burack, M.A.; Ghoraani, B. Ensemble deep model for continuous estimation of Unified
Parkinson’s Disease Rating Scale III. Biomed. Eng. OnLine 2021, 20, 32. [CrossRef] [PubMed]
191. Mutegeki, R.; Han, D.S. A CNN-LSTM Approach to Human Activity Recognition. In Proceedings of the 2020 International
Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan, 19–21 February 2020; IEEE:
Fukuoka, Japan, 2020; pp. 362–366. [CrossRef]
192. Mekruksavanich, S.; Jitpattanakul, A. LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in
Smart Homes. Sensors 2021, 21, 1636. [CrossRef] [PubMed]
193. Na, Y.; Kim, D.; Kim, D.K.; Lee, J.G. Evaluation of OSA Patient Sleep Stage Classification Performance Using a Multi-Channel
PSG Dataset. In Proceedings of the 2022 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), Yeosu,
Republic of Korea, 17–20 December 2022; IEEE: Yeosu, Republic of Korea, 2022; pp. 1–4. [CrossRef]
Biomedinformatics 2023, 3 218

194. Liu, Y.; Liu, H.; Yang, B. Automatic Sleep Arousals Detection From Polysomnography Using Multi-Convolution Neural Network
and Random Forest. IEEE Access 2020, 8, 176343–176350. [CrossRef]
195. Li, F.; Yan, R.; Mahini, R.; Wei, L.; Wang, Z.; Mathiak, K.; Liu, R.; Cong, F. End-to-end sleep staging using convolutional neural
network in raw single-channel EEG. Biomed. Signal Process. Control 2021, 63, 102203. [CrossRef]
196. Howe-Patterson, M.; Pourbabaee, B.; Benard, F. Automated Detection of Sleep Arousals From Polysomnography Data Using a
Dense Convolutional Neural Network. In Proceedings of the 2018 Computing in Cardiology Conference (CinC), Maastricht, The
Netherlands, 23–26 September 2018. [CrossRef]
197. Yamazaki, K.; Vo-Ho, V.K.; Bulsara, D.; Le, N. Spiking Neural Networks and Their Applications: A Review. Brain Sci. 2022, 12,
863. [CrossRef] [PubMed]
198. Doborjeh, Z.; Doborjeh, M.; Taylor, T.; Kasabov, N.; Wang, G.Y.; Siegert, R.; Sumich, A. Spiking Neural Network Modelling Approach
Reveals How Mindfulness Training Rewires the Brain. Sci. Rep. 2019, 9, 6367. [CrossRef]
199. Pfeiffer, M.; Pfeil, T. Deep Learning With Spiking Neurons: Opportunities and Challenges. Front. Neurosci. 2018, 12, 774.
[CrossRef]
200. Saputra, N.H.; Nafi’Iyah, N. Identification of Human Stress Based on EEG Signals Using Machine Learning. In Proceedings of
the 2022 1st International Conference on Information System & Information Technology (ICISIT), Virtual, 27–28 July 2022; IEEE:
Yogyakarta, Indonesia, 2022; pp. 176–180. [CrossRef]
201. Bashar, K.; Chiaki, I.; Yoshida, H. Human identification from brain EEG signals using advanced machine learning method
EEG-based biometrics. In Proceedings of the IEEE EMBS Conference on Biomedical Engineering and Sciences, Lyon, France, 4–8
December 2016; p. 5.
202. Abdul Hamid, D.S.B.; Goyal, S.; Bedi, P. Integration of Deep Learning for Improved Diagnosis of Depression using EEG and
Facial Features. Mater. Today Proc. 2021, in press. [CrossRef]
203. Troncoso-García, A.; Martínez-Ballesteros, M.; Martínez-Álvarez, F.; Troncoso, A. Explainable machine learning for sleep apnea
prediction. Procedia Comput. Sci. 2022, 207, 2930–2939. [CrossRef]
204. Sharma, G.; Parashar, A.; Joshi, A.M. DepHNN: A novel hybrid neural network for electroencephalogram (EEG)-based screening
of depression. Biomed. Signal Process. Control 2021, 66, 102393. [CrossRef]
205. Acar, E. Unraveling Diagnostic Biomarkers of Schizophrenia Through Structure-Revealing Fusion of Multi-Modal Neuroimaging
Data. Front. Neurosci. 2019, 13, 16. [CrossRef] [PubMed]
206. Johnstone, S.J.; Parrish, L.; Jiang, H.; Zhang, D.W.; Williams, V.; Li, S. Aiding diagnosis of childhood attention-deficit/hyperactivity
disorder of the inattentive presentation: Discriminant function analysis of multi-domain measures including EEG. Biol. Psychol.
2021, 161, 108080. [CrossRef] [PubMed]
207. Bernardo, D.; Nariai, H.; Hussain, S.A.; Sankar, R.; Salamon, N.; Krueger, D.A.; Sahin, M.; Northrup, H.; Bebin, E.M.; Wu, J.Y.
Visual and semi-automatic non-invasive detection of interictal fast ripples: A potential biomarker of epilepsy in children with
tuberous sclerosis complex. Clin. Neurophysiol. 2018, 129, 1458–1466. [CrossRef] [PubMed]
208. Mencar, C.; Gallo, C.; Mantero, M.; Tarsia, P.; Carpagnano, G.E.; Foschino Barbaro, M.P.; Lacedonia, D. Application of machine
learning to predict obstructive sleep apnea syndrome severity. Health Inform. J. 2020, 26, 298–317. [CrossRef]
209. Xu, D.; Wang, Y.; Meng, Y.; Zhang, Z. An Improved Data Anomaly Detection Method Based on Isolation Forest. In Proceedings
of the 2017 10th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 9–10 December
2017; IEEE: Hangzhou, China, 2017; pp. 287–291. [CrossRef]
210. Satapathy, S.K.; Kondaveeti, H.K. Prognosis of Sleep Stage Classification Using Machine Learning Techniques Applied on
Single-channel of EEG signal of both Healthy Subjects and Mild Sleep effected Subjects. In Proceedings of the 2021 International
Conference on Artificial Intelligence and Machine Vision (AIMV), Gandhinagar, India, 28–30 June 2021; IEEE: Gandhinagar, India,
2021; pp. 1–7. [CrossRef]
211. Zhu, G.; Li, Y.; Wen, P. Analysis and Classification of Sleep Stages Based on Difference Visibility Graphs From a Single-Channel
EEG Signal. IEEE J. Biomed. Health Inform. 2014, 18, 1813–1821. [CrossRef]
212. Salman, A.A.; Kumar, D.M.S. Introducing Confusion Matrix and Accuracy in Disease Prediction on Liver Using Machine
Learning. Int. J. Comput. Sci. Trends Technol. (IJCST) 2020, 8, 5–9.
213. Rahman, A.; Chowdhury, M.E.H.; Khandakar, A.; Kiranyaz, S.; Zaman, K.S.; Reaz, M.B.I.; Islam, M.T.; Kadir, M.A. Multimodal
EEG and Keystroke Dynamics Based Biometric System Using Machine Learning Algorithms. IEEE Access 2021, 9, 19. [CrossRef]
214. Zhou, D.; Wang, J.; Hu, G.; Zhang, J.; Li, F.; Yan, R.; Kettunen, L.; Chang, Z.; Xu, Q.; Cong, F. SingleChannelNet: A model for
automatic sleep stage classification with raw single-channel EEG. Biomed. Signal Process. Control 2022, 75, 103592. [CrossRef]
215. Kemp, B.; Olivan, J. European data format ‘plus’ (EDF+), an EDF alike standard format for the exchange of physiological data.
Clin. Neurophysiol. 2003, 114, 1755–1761. [CrossRef]
216. Korompili, G.; Amfilochiou, A.; Kokkalas, L.; Mitilineos, S.A.; Tatlas, N.A.; Kouvaras, M.; Kastanakis, E.; Maniou, C.; Potirakis,
S.M. PSG-Audio, a scored polysomnography dataset with simultaneous audio recordings for sleep apnea studies. Sci. Data 2021,
8, 197. [CrossRef]
217. Le Quy, T.; Roy, A.; Iosifidis, V.; Zhang, W.; Ntoutsi, E. A survey on datasets for fairness-aware machine learning. WIREs Data
Min. Knowl. Discov. 2022, 12, e1452. [CrossRef]
Biomedinformatics 2023, 3 219

218. Ketola, E.C.; Barankovich, M.; Schuckers, S.; Ray-Dowling, A.; Hou, D.; Imtiaz, M.H. Channel Reduction for an EEG-Based
Authentication System While Performing Motor Movements. Sensors 2022, 22, 39156. [CrossRef] [PubMed]
219. Ng, C.R.; Fiedler, P.; Kuhlmann, L.; Liley, D.; Vasconcelos, B.; Fonseca, C.; Tamburro, G.; Comani, S.; Lui, T.K.Y.; Tse, C.Y.; et al.
Multi-Center Evaluation of Gel-Based and Dry Multipin EEG Caps. Sensors 2022, 22, 8079. [CrossRef] [PubMed]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.