Machine Translated by Google

Cognitive Computing (2021) 13:1–33

https://doi.org/10.1007/s12559-020-09773-x

Deep Learning in Mining Biological Data

Mufti Mahmud1,5 M. Shamim Kaiser2 T. Martin McGinnity1,3 Amir Hussain4

Received: 1 May 2020 / Accepted: 28 September 2020 / Published online: 5 January 2021
© The Author(s) 2020

Abstract
Recent technological advancements in data acquisition tools allowed life scientists to acquire multimodal data from different
biological application domains. Categorized in three broad types (ie images, signals, and sequences), these data are huge
in amount and complex in nature. Mining such an enormous amount of data for pattern recognition is a big challenge and
requires sophisticated data-intensive machine learning techniques. Artificial neural network-based learning systems are well
known for their pattern recognition capabilities, and lately their deep architectures—known as deep learning (DL)—have been
successfully applied to solve many complex pattern recognition problems. To investigate how DL—especially its different
architectures—has contributed and been utilized in the mining of biological data pertaining to those three types, a meta-
analysis has been performed and the resulting resources have been critically analysed. Focusing on the use of DL to analyze
patterns in data from diverse biological domains, this work investigates different DL architectures' applications to these data.
This is followed by an exploration of available open access data sources pertaining to the three data types along with popular
open-source DL tools applicable to these data. Also, comparative investigations of these tools from qualitative, quantitative,
and benchmarking perspectives are provided. Finally, some open research challenges in using DL to mine biological data
are outlined and a number of possible future perspectives are put forward.

Keywords Brain–Machine Interfaces Bioimaging Deep learning performance comparison Medical imaging Omics
Open access data sources Open-source tools

Introduction centuries [1]. This accelerated the development of cutting

edge tools and technologies that allow scientists to
The pursuit of understanding human behaviours, along with holistically study the biological systems as well as dig down,
the various pathologies, their early diagnosis and finding in an unprecedented manner, to the molecular details of the
cures, has driven the life sciences research in the last two living organisms [2, 3]. Increasing technological sophistication
has presented scientists with novel tools for DNA sequencing
[4], gene expression [5], bioimaging [6], neuroimaging [7],
M. Mahmud and MS Kaiser are joint first authors.
and body/brain–machine interfaces [8].
* Mufti Mahmoud These innovative approaches to study living organisms
[email protected]; [email protected] produce huge amount of data [9] and create a situation
* M. Shamim Kaiser often referred to as 'Data Deluge' [10]. Depending on the tar
[email protected] get application and experimentation, these biological big
1
data can be characterized by their inherent characteristics
Department of Computer Science, Nottingham Trent
of being hierarchical (ie data coming from different levels of
University, Clifton, NG11 8NS Nottingham, UK
two
a biological system—from molecules to cells to tissues to
Institute of Information Technology, Jahangirnagar
systems), heterogeneous (ie data acquired by different
University, Savar 1342 Dhaka, Bangladesh
3
acquisition methods—from genetics to physiology to pathol
Intelligent Systems Research Centre, Ulster University,
ogy to imaging), dynamic (ie data changes as a function of
Northern Ireland BT48 7JL Derry, UK
4
time), and complex (ie data describing nonlinear biological
School of Computing, Edinburgh, EH11 4BN Edinburgh,
UK processes) [11]. These intrinsic characteristics of biological
5
big data posed an enormous challenge to data scientists to
Medical Technology Innovation Facility, Nottingham Trent
identify patterns and analyze them to infer meaningful
University, NG11 8NS Clifton, Nottingham, UK

Vol.:(0123456789) 1 3
Machine Translated by Google

two
Cognitive Computing (2021) 13:1–33

conclusions from these data [12]. The challenges have Since the 1950s, many methods pertaining to both the
triggered the development of rational, reliable, reusable, learning paradigms (ie supervised and unsupervised) have
rigorous, and robust software tools [11] using machine been proposed. The popular methods in the supervised
learning (ML)-based methods to facilitate recognition, domain include: ANN [14] and its variants (eg
classifcation, and prediction of patterns in the biological big dataBackpropagation
[13 ]. [15], Hopfeld Networks [16], Boltzmann
Based on how a method learns from the data, the ML Machines [17], Restricted Boltzmann Machines [18],
techniques can be broadly categorized into supervised Spiking Neural Networks [19], etc.), Bayesian Statistics
and unsupervised approaches. In supervised learning, [20], Support Vector Machines [21] and other linear classifiers [22] (eg
objects in a pool are classified using a set of known Fisher's Linear Discriminant [23], Regressors [24], Naive
annotations or attributes or features, ie a supervised Bayes Classifer [25], etc.), k-Nearest Neighbors [26], Hid
algorithm learns the pattern(s) from a limited number of den Markov Model [27], and Decision Trees [28]. Popular
annotated training data and then classifies the remaining unsupervised methods include: Autoencoders [29],
testing data using the acquired knowledge. Instead, in the Expectation–Maximization [30], Information Bottleneck [31],
unsupervised learning, pattern(s) are first defined from a Self Organizing Maps [32], Association Rules [33],
subset of the unknown data and then the remaining data Hierarchical Clustering [34], k-Means [35], Fuzzy Clustering
are classified based on the defined patterns, ie an [36], and Density-based Clustering [37, 38] (eg Ordering
unsupervised algorithm first defines pattern(s) among the Points To Identify the Clustering Structure [39]). Many of
objects in a pool of data with unknown annotations or these meth ods have been successfully applied to data
attributes or features, and then uses the acquired knowledge coming from various biological sources.
edge to classify the remaining data. In addition, there is For the sake of simplicity, the vast amount of biological
another category called reinforcement learning which is out data coming from the diverse application domains have
of the scope of this work, but allows an agent to improve been categorized to a few broad data types. These data
its experience and knowledge by learning iteratively through types include Sequences (data generated by Omics
interacting with its environment. technologies, eg [gen/transcript/epigen/prote/metabol]omics [40]),

Data deep learning applications

electroencephalogram Human activity recognition

Electromyogram Behavioral monitoring
Simple to Abstract mapping/
Electrocardiogram features Classification cognitive state
Single/multiunit activity Abnormal detection
Field potentials
Image reconstruction
disease diagnosis
GIVES DBN Organ segmentation
EM Images Abnormal detection
(f/s)MRI
PET/CT scan
radiographs Cell analysis
Fundus Images Cell/tissue classification
Endoscopy Images cell count
RNN CNN disease diagnosis

Gene/DNA/RNA seq. Disease prediction

gene expression drug design
Molecular components gene
Protein structure alternative splicing
Input Output hidden Visible kernel Conv/Pool
PS interpretation

ALU control aluminum

ALU ALU

Cache

DRAM DRAM

GPU fpga
DL4J sci kit

CPUs SO C DEEPEARNING4J

Hardware Tools / Frameworks / Libraries

Fig. 1 The ecosystem of modern data analytics using advanced [Medical/ Bio]-Imaging, and signals from the [Brain/ Body]–Machine
machine learning methods with specifc focus on application of DL to Interfaces) are mined using DL with suitable architectures tailored for
biological data mining. The biological data coming from various specifc applications
sources (eg sequence data from the Omics, various images from the

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 3

Images (data generated by [bio/medical/clinical/health] the components of this ecosystem (see the various
imaging techniques containing [sub-]cellular and diagnostic components of the ecosystem in Fig. 1). The remaining
images), and Signals (electrical signals generated by the components of the ecosystem include open access data
brain and the muscles and acquired using appropriate sen sources and open source toolboxes and libraries which are
sors at the [Brain /Body]–Machine Interfaces or BMI). Each used in developing the individual methods. It is therefore of
of these data types originating at diverse biological application paramount importance to have a complete understanding of
domains have witnessed major contributions from the the availability of datasets and their characteristics, the
specified ML methods and their variants (see for Sequences capabilities and options offered by the libraries, and how they
[41], images [42–44], and signals [45–47]). compare with each other in different execution environments such as centra
In recent years, DL methods are potentially reshaping the processing unit (CPU) and graphical processing unit (GPU).
future of ML and AI [48]. It is worthy to mention here that, The current paper's novelty lies in being frst of its kind to
from a broader perspective, ML has been applied to a range comprehensively cover the complete ecosystem of modern
of tasks including anomaly detection [49, 50, 278, 283, 290], data analysis using advanced ML technique, ie, DL.
biological data mining [51, 52], detection of coro navirus [53, Therefore, with the above aim, this review provides—a
54], disease detection and patient management [55–57, 277, brief overview on DL concepts and their applications to
279–282, 284, 286, 287, 289, 291], education [58], natural various biological data types; a list of available open access
language processing [59, 285, 288], and price prediction [60]. data repositories offering data for method development; and
Despite notable popularity and applicability to diverse a list of existing open-source libraries and frameworks which
disciplines [61], there exists no comprehensive review which can be utilized to harness the power of these techniques
focuses on pattern recognition in biological data and provides along with their relative and performance comparison.
pointers to the various biological data sources and DL tools, Towards the end, some open issues are identified and some
and the performances of those tools [51]. speculative future perspectives are outlined.
The remainder of the article is organized as follows:
Also, considering the ecosystem of modern data analysis Section 2 provides the conceptual overview and introduces
systems using advanced ML techniques (such as DL), the reader to the underlying theory of DL; Section 3
describes the applications; Section 4 lists the open-source
providing information about application methods only partially covers

a b
3%1%
3.
4%
5%

5%

6%

58%
7%

8%

CNN FCN DA TRL RNN

ANN GAN DNN DBN DBM

Fig. 2 Application of different DL models to biological data. a colors of the individual feet match the colors in the wordcloud.
Wordcloud generated using author keywords extracted from research Legend—CNN: Convolutional Neural Network, FCN: Fully Connected
papers published between January 2011 and March 2020 which Network, DA[E]: Deep Autoencoder, TRL: Transfer Learning, RNN:
mentioned analysis of biological data (images, signals and sequences) Recurrent Neural Network (including Long Short-Term Memory or
using DL techniques and indexed in the Scopus database. The key LSTM), ANN: Artifcial Neural Network , GAN: Generative Adversarial
words were pruned to highlight the analysis methods. b Distribution Network, DNN: Deep Neural Network, DBN: Deep Belief Network,
of published papers mentioning the usage of top 10 techniques. the DBM: Deep Boltzmann Machine

13
Machine Translated by Google

4 Cognitive Computing (2021) 13:1–33

data repositories; Section 5 introduces the popular open- BM is that, the learning process is computationally expensive
source DL tools; and Sections 6 and 7 compare the most and quite slow. Due to this, a BM requires a long period to
popular tools from relative and performance perspectives. Sectionreach
8 equilibrium statistics [62]. However, this learning
presents the reader with some of the open issues and hints inefciency can be solved by forming a bipartite graph (ie
on the future perspectives, and finally, the article is concluded restricting to have one hidden layer and one visible layer)
in Section 9. [67]. To extend this shallow architecture to a deep one,
multiple RBMs as unitary learning elements are stacked
together and this yields the following two DL architectures.
Overview of Deep Learning
Deep Boltzmann Machine (DBM)
In DL the data representations are learned with increasing
abstraction levels, ie, at each level more abstract DBM [70] is a stack of undirected RBMs which supports a
representations are learned by defining them in terms of less feedback mechanism among the layers to facilitate inference
abstract representations at lower levels [62]. Through this from higher-level units to propagate to lower-level units. This
hierarchical learning process, a system can learn complex allows an input to be alternatively interpreted through
representations directly from the raw data [63]. concurrent competition at all levels of the model.
Though many DL architectures have been proposed in Despite this powerful inference mechanism, estimating model
the literature for various applications, there has been a parameters from data remains a challenge and can not be
consistent preference to use particular variants for biological solved using traditional gradient-based methods (eg,
data. As shown in Fig. 2, the most popular models have been persistent contrastive divergence [71]) [70]. Though this
identified as—Deep Neural Network (DNN), Deep Boltzmann learning problem is overcome by pretraining each RBM in a
Machine (DBM) and Deep Belief Network (DBN), Deep layerwise greedy fashion, with outputs of the hidden variables
Autoencoder (DA), Generative Adversarial Network (GAN) , from lower layers as input to upper layers [67], the time
Recurrent Neural Network (RNN, including LSTM), and complexity remains high and the approach may not be
Convolutional Neural Network (CNN). Each of these models' suitable for large training datasets [72].
architectures and their respective pros and cons are listed in
Table 1. The following subsections introduce each of these Deep Belief Network (DBN)
most frequently used DL architectures in mining biological
data. DBN [73], in contrast to the DBM, is formed by stacking
several RBMs together in a way that one RBM's latent layer
Deep Neural Network (DNN) is linked to the next RBM's visible layer. As the top two layers
of DBN are undirected, the connections are down ward
A DNN [64] is inspired by the brain's multilevel visual directed to its immediate lower layer [73, 74]. Thus, the DBN
processing mechanism starting with the cortical area 'V1' is a hybrid model with the first two layers as an undirected
and then to area 'V2', and so on [65]. Mimicking this, the graphical model and the rest being directed generative
traditional artificial neural network or NN is extended with model. The different layers are learned in a layerwise greedy
additional hidden layers containing nonlinear computational fashion and fne-tuned based on required output [75]; how
units in each of these hidden layers to learn a subset of the ever, the training procedure is computationally demanding.
given representations. Despite its successful usage in a
range of different applications, the main drawback has been Deep Autoencoder (DA)
the slow and cumbersome training process [66].
DA is a DL architecture [76] obtained by stacking a number
[Restricted] Boltzmann Machines ([R]BM) of data-driven Autoencoders which are unsupervised
elements. DA is also known as DAE and is designed to
[R]BM represents specifc probability distributions through an reduce data dimension by automatically projecting incom ing
undirected probabilistic generative model [67]. Considered representations to a lower-dimensional space than that of
as a nonlinear feature detector, [R]BM is trained based on the input. In an Autoencoder, equal amounts of units are
optimizing its parameters for a set of given observations to used in the input/output layers and less units in the hidden
obtain the best possible ft of the probability distribution layers. (Non)linear transformations are embodied in the
through a Markov chain Monte Carlo method known as Gibbs hidden layer units to encode the given input into smaller
sampling [68, 69 ]. With symmetrical connections among dimensions [77]. Despite the fact that it requires a pretraining
subsequent units in multiple hidden layers, BM has only one stage and suffers from a vanishing error, this architecture is
visible layer. The main drawback of the standard popular for its data compression capability

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 5

Table 1 Keypoints and applications of different deep learning architectures

Architecture pros Cons.

DNN - DNN can learn high-level feature - It requires substantial volume of training
representation and apply transfer data.
learning. - Significant computational power is
- It can be used for healthcare and required.
visual recognition. - The learning process is slow.

Input hidden Output

DBM - Graphical model, undirected links - High time complexity for interference
across a set of visible nodes and a than DBN. ÿ - Learning information does
set of hidden nodes. not reach the lower layer.
- Used mainly for dimensionality - Tends to overft.
reduction and classification.

hidden Visible

DBN - Easy to code and works sufciently - It can be trained greedily, one layer at
well for just a few layers. a time.
- High performance gain by add - Hard to deduce posterior distribution for
ing layers compared to multilayer configurations of hidden causes.
perceptron.
- Robust in classification.
hidden Visible

GIVES
- Learn data encoding, reconstruction - Requires pretraining stage due to the
and generation at the same time. chances of vanishing error.
- Training is stable without label data. -Each application requires redesigned
- Variant: sparse, denoising and with and retrained the model.
tractive DA. - The DA is sensitive to input errors.

Input hidden Output

GAN - The main benefit is data augmenta - Difficult to train as optimizing loss
tion. function is hard and requires a lot of
- GAN performs unsupervised learning. trial and error.

- GAN learns density distributions of

data.
Backfed Input hidden Input-Output

RNN - It can process inputs of any length. - Computation is slow and training can be
- RNN can use internal memory and difficult.
performs well for stream time series - Processing long sequences is difficult.
data. - Prone to problems such as exploding
and gradient vanishing.

Input hidden Output

13
Machine Translated by Google

6 Cognitive Computing (2021) 13:1–33

Table 1 (continued)
Architecture pros Cons.

CNN - CNN can capture hierarchical - Large labeled dataset is required for
information. training.
Convolution and Pooling Fully connected
- CNN can share pretrained weight - The working mechanism of CNN is
which is required for transfer not clear.
learning.
- Requires less connection compared
to DNN.

Input Output
kernel Conv/Pool hidden

Legend: DA Deep Autoencoder, DBN Deep Belief Network, RNN Recurrent Neural Network, DNN Deep Neural Network, DBM Deep Boltzmann
Machine, CNN Convolutional Neural Network.

and has many variants, eg Denoising Autoencoder [76], Sparse Convolutional Neural Network (CNN)
Autoencoder [78], Variational Autoencoder [79], and Contractive
Autoencoder [80]. CNN [90] is a multilayer NN model [91] which has gained popularity
in analyzing image-based data. Inspired by the neurobiology of
the visual cortex, the CNN consists of convolutional layer(s)
Generative Adversarial Network (GAN) containing a set of learnable flter banks and followed by fully
connected layer(s). These flter banks convolve with the input data
GAN [81] is an effective generative model. Generative models and pass the results to activation functions (eg ReLU, Sigmoid,
perform an unsupervised learning task, where they automatically and Tanh). There also exist subsampling steps in between these
discover and learn existing patterns in data and then use that layers. The CNN outperforms DNNs, which as they do not scale
knowledge to generate new examples of the learned pattern as if well with multidimensional locally correlated input data. To address
they were drawn from the original dataset. Using GAN, the problem the scaling problem of DNNs, the CNN approach has been quite
is seen as a supervised learning problem with two strands: (i) the successful in analyzing datasets with a high number of nodes and
generator, which generates new examples as trained, and (ii) the parameters (eg images). As the images are 'stationary,' convolution
discriminator, which classifes generated examples to two classes flters (CF) can easily learn data driven kernels. Applying such CF
(real or fake). These generator and discriminator models are along with a suitable pooling function reduces the features that
trained together in a zero-sum game (ie in an adver sarial fashion) are supplied to the fully connected network to classify. However,
such that the examples generated by the generator model in case of large datasets even this can be daunting and can be
maximize the loss of the discriminator model [82, 83]. solved using sparsely connected networks. Some of the popular
CNN configurations include AlexNet [92], VGGNet [93]

Recurrent Neural Network (RNN) GoogLeNet [94], etc. (see Table 2 for a complete list of CNN's
variations with relevant details).
The RNN architecture [84] is designed to detect spatio temporal
alignments in streams of data [85]. Unlike feed forward NN which
performs computations unidirectionally from input to output, an Deep Learning and Biological Data
RNN computes the current state's output depending on the outputs
of the previous states. Due to this 'memory'-like property, despite Many studies have been reported in the literature which employ
learning problems related to vanishing and exploding gradients, diverse DL architectures with related and varied parameter sets
RNN has gained popularity in many felds involving streaming data (see section 2) to analyze patterns in biological data. For most of
(eg text mining, time series, genomes, financial, etc.). In recent the DL architectures, as shown in Fig. 3, the number of publications
years, two main variants, bidirectional RNN (BRNN) [86] and Long is increasing steadily over the years. A set of randomly selected
Short-Term Memory (LSTM) [87], have also been applied [48, 88, representative studies from the large amount of reported literature
89]. are described below and summarized in Table 3. These studies
belong to

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 7

Table 2 Keypoints of different deep CNN architectures

Architecture Network Design Parameters Key points

LeNet (1998) LeNet-5 is the frst CNN architecture with 2 convolu 0.06 million - Feedforward NN.
tional and 3 fully connected layers. - Connection between layers is sparse to reduce
computational complexity. 60 million - Deeper
AlexNet (2012) AlexNet has 8 layers and consists of 5 convolutional than the LeNet and aliasing artifacts in the learned feature maps due
and 3 fully connected layers. to large flter size.
VGG-16 (2014) VGG-16 has 13 convolutional layers (and max pooling 138 million - Roughly twice deeper network can be designed
layers) and 2 fully connected layers followed by 1 compared to the AlexNet.
output layer with softmax activation. - A deeper variant of VGG-16 is VGG-19.
- Computationally expensive and cannot be used
with low resource systems.
Inception-v1 (2014) Also known as GoogleNet, it has 22 layers with 5 million - It uses sparse connections to overcome redundant
parameters (or 27 when pooling layers are information problem and omits irrelevant feature
included). Towards the end, it employs an average maps.
pooling. - High accuracy with a reduced computational cost.
- It's heterogeneous topology requires customization.
Inception-v3 (2015) Inception-v3 has 48 layers with a number of inception 23 million - It increases accuracy and reduces computational
modules (each consisting of pooling layers and complexity in comparison to Inception-v1.
convolutional flters with activation functions), - Reduces representational bottleneck.
concatenation layers and fully connected layer(s) - Replaces large size flters with smaller ones.
along with dropout and softmax. - It's architecture is complex and lacks homogeneity.
ResNet-50 (2015) ResNet-50 has 50 layers with initial convolutional 25.5 thousand - It provides an accelerated training speed.ÿ
and max-pooling layers, and final average pooling lion -Reduces the effect of Vanishing Gradient Prob
and fully connected layers. In between, there are 3, lem.
4, 6 and 3 residual blocks separated in 4 stages - Classifies images with high accuracy.
where each block contains 3 convolutional layers.
Xception (2016) The Xception architecture has 36 convolutional 22.8 million - Xception shows small gains in classifcation per
layers forming the feature extraction base of the formance on the ImageNet dataset and large gains
network. The 36 convolutional layers are structured on the JFT dataset when compared to Inception v3.
into 14 modules, all of which have linear residual
connections around them, except for the first and last
modules.
Inception-v4 (2016) Inception-v4 consists of two main sections: a feature 43 million - Deep hierarchies of features, multilevel feature
extractor and a fully connected layer. The feature representation.ÿ - Learning speed is slow.
extractor includes various convolutional blocks such
as 1 stem block, 14 inception blocks, 2 reduction
blocks and a pooling layer. The inception blocks are
divided into three categories, namely, A, B, and C
with 4, 7, and 3 blocks, respectively.
Inception-ResNet-v2 Inception-ResNet-v2 consists of 164 layers with 56 million - It improves training speed.ÿ - Deep hierarchies of features,
(2016) several convolutional blocks which include 1 stem multilevel feature representation.
block, 20 residual inception blocks, 2 reduction
blocks and a pooling layer. The residual inception
blocks are divided into three categories, namely, A,
B, and C with 5, 10, and 5 blocks, respectively.

ResNeXt-50 (2016) ResNeXt-50 has initial convolutional and max 25 million - Has homogeneous topology. ÿ - Performs grouped
pooling layers, and final average pooling and fully convolution.
connected layers. In between, there are 3, 4, 6 and
3 residual blocks separated in 4 stages where each
block contains 3 convolutional layers. In comparison
to ResNet-50, it scales up the number of parallel
towers (cardinality=32) within each residual block.

DenseNet-121 DenseNet architecture includes 4 dense blocks. Each 8 millions - Introduces depth or cross-layer dimension.ÿ -
(2016) layer in a dense block is connected to every other Ensures maximum data fow between the layers in
layer. The dense blocks, consisting of convolution, the network. ÿ - Avoids relearning of redundant
pooling, batch normalization and activation, are feature maps.
separated by transition layers.

13
Machine Translated by Google

8 Cognitive Computing (2021) 13:1–33

Images neuroendocrine carcinoma [55, 74, 105]. CNN's dual pathway

a
100 version was used by Kamnitsas et al. to segment lesions
related to tumors, traumatic injuries, and ischemic strokes
80
[109]. CNN was also used by Fritscher et al. for volume
60 segmentation [101] and by Cho et al. to fnd anatomical
40 structures (Lung nodule to classify malignancy) [106] from
Computed Tomography (CT) scans. DBN was applied on MRIs
twenty
to detect Attention Defcit Hyperactivity Disorder [96] and on
0 cardiac MRIs to segment the heart's left ventricle [107]. GANs
DNN DBM DBN DA GAN RNN CNN have gained popularity in image synthesis and data augmentation to reduce
signals GAN's application in data augmentation and image translation
b
100 has been reviewed in [143] and data augmentation in the CT
segmentation tasks was done using CycleGAN [144]. GAN
80
based framework called MedGAN was proposed for medical
60 image-to-image translation [145]. GAN was used as survival
40 prediction model for chest CT scan images of patients suffering
from idiopathic pulmonary fbrosis [146, 147]. GAN was also
twenty
used by Halicek for synthesizing hyperspectral images from
0 digitized histology of breast cancer cells [148].
DNN DBM DBN DA GAN RNN CNN

c sequences signals
100

80 A stacked DA was employed to detect emotion from

Electroencephalography (EEG) signals after extracting relevant
60
features using Principal Component Analysis (PCA) and
40 reducing non-stationary effect using covariate shift adaptation
[119]. DBN was applied to decode motor imagery through
twenty

classifying EEG signal [110]. For a similar purpose, CNN was

0 used with augmented common spatial pattern features [111].
DNN DBM DBN DA GAN RNN CNN
EEG signals were also classified using DA after features such
2015 2016 2017 2018 2019 as location, time, and frequency were extracted using CNN
[112]. Li et al. used DBN to extract low-dimensional latent
features, and select critical channels to classify affective state using EEG sig
Fig. 3 Trends in publication involving different DL architectures from
2015 to 2019 in three major types of data—images a, signals b, and Also, Jia et al. used an active learning to train DBN and
sequences c. The numbers of papers have been normalized within genera tive RBMs for the classifcation [115]. Tripathi et al.
each data type. However, it is noteworthy that the ratio of number of utilized DNN- and CNN-based model for emotion classification [116].
publications involving DL techniques applied to different data types
:
1 CNN was employed to predict seizures through synchronization
(images, signals, and sequences) are approximately—1:1 4 10
patterns classification [118]. DBN [123] and CNN [122] were
used to decode motion action from NinaPro database. the
the three data types we have considered within the context of later approach was also used on MIT-BIH, INCART, and SVDB
this paper, that is, images, signals, and sequences. repositories [122]. Moreover, the Electrocardiogram (ECG)
Arrhythmias were classifed using DBN [120, 121] from the
Images data supplied by the MIT-BIH arrhythmia database. Zhu et al.
used a GAN model with LSTM and CNN to generate ECG
CNN was used by on histology images of the breast to fnd signals with high morphological similarity [149]. Another GAN
mitosis [108, 142] and to segment neuronal structures in model, RPSeqGAN, trained with SeqGAN [150] generated
Electron Microscope Images (EMI) [103]. Havaei et al. used arrhythmic ECG data with fve periods and showed high stability
CNN to segment brain tumor from Magnetic Resonance and data quality [151]. GAN is also used by Luo and Lu for
Imaging (MRI) [100] and Hosseini et al. used it for the diagnosis EEG data augmentation [152]. You et al. [153] and Jiao et al.
of Alzheimer's disease (AD) from MRI [56, 97]. DBM [98] and [154] utilized GAN-based model for detecting seizure using
RBM [99] were used in detecting AD and mild cognitive EEG signal and Driver sleepiness using EEG and
impairment (MCI) from MRI and Positron Emission Tomography Electrooculography (EOG) signals, respectively. Singh et al.
(PET) scans. Again, CNN was used on MRI to detect proposed a new GAN framework work for ECG denoising [155].

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 9

Table 3 Deep learning applied to biological data

Type Date [base/set] DL-architecture Task

Images ABIDE DNN [95] autism disorder identification

ADHD-200 data set DBN [96] ADHD screening
ADNI dataset CNN [97], DBM [98], DBN [99] AD/MCI diagnosis
BRATS Dataset CNN [100] Brain pathology segmentation
CT data set CNN [101] Fast segmentation of 3D medical images
DRIVE, STARE datasets GAN [102] Retinal blood vessel segmentation
EM segmentation challenge dataset CNN [103] Neuronal segment membranes
LSTM [104] Biomedical volumetric image segmentation
IBSR, LPBA40, and OASIS dataset CNN [105] skull stripping
LIDC-IDRI dataset CNN [106] Lung nodule malignancy classifcation
MICCAI 2009 LV dataset DBN [107] Heart LV segmentation
MYTHS dataset CNN [108] Mitosis detection in breast cancer
PACS data set CNN [106] Medical image classification
TBI data set CNN [109] Brain lesion segmentation
signals BCI competition IV DBN [110], CNN [111–113] motion action decoding
DEAP data set DBN [114, 115] Affective state recognition
CNN [116] Emotion classification
DECAF GAN [117]

Freiburg data set CNN [118] Security prediction

MAHNOB-HCI DA [119] Emotion recognition
MIT-BIH arrhythmia database DBN [120, 121] ECG arrhythmia classification
MIT-BIH, INCART, and SVDB CNN [122] Movement decoding
NinaPro database DBN [123], CNN [122] motion action decoding
Sequences CullPDB, CB513, CASP datasets, CAMEO CNN [124] 2ps prediction
DREAM CNN [125] DNA/RNA sequence prediction
DNN [126] Predict effective drug combination
ENCODE database CNN [127, 128] Gene expression identification
ENCODE DGF dataset CNN [129] Predict noncoding variant of gene
GEO-database GAN [130] Gene expression data augmentation
GWH and UCSC datasets DBN [131] Splice junction prediction
JASPAR database and ENCODE CNN [132] Predicting DNA-binding protein
myRBoost RNN [133] micro-RNA Prediction

miRNA-mRNA pairing data repository LSTM [134] micro-RNA target prediction

Protein Data Bank (PDB) DA [135] Protein structure reconstruction

SRBCT, prostate tumor, and MLL GE sbv DBN [136] Gene/MiRNA feature selection
IMPROVER DBN [137] Human diseases and drug development
TCGA database DA [138] Cancer detection and gene identification
DBM [139]
DNN [140] Drug combination estimation
UCSC, CGHV Data, SPIDEX database CNN [141] Genetic variants identification

sequences [135]. Lee et al. applied a DBN-based unsupervised method

to perform autoprediction of splicing junction at Deoxyri
The stacked denoising DA has been used to extract bonucleic Acid (DNA) level [131]. Combining DBN with
features for cancer diagnosis and classifcation along with active learning, Ibrahim et al. devised a method to select
the identifcation of related genes from Gene Expression feature groups from genes or micro-Ribonucleic Acids (miR
(GE) data [138]. GAN was also used to identify expression NAs) based on expression profiles [136]. For translational
patterns from GE data [156]. A template-based DA learning research, bimodal DBNs were used by Chen et al. to
model was used in reconstructing the protein structures predict responses of human cells using model organisms [137]. Bread

13
Machine Translated by Google

10 Cognitive Computing (2021) 13:1–33

et al. applied a hybrid CNN-DBN model on RNAs for the information from a number of different microscopic image
prediction of RNA-binding protein (RBP) interaction sites and acquisition systems.
motifs [157], and Alipanahi et al. used CNN to predict Another image library, called the Cell Image Library (CIL)
sequence specificities of [D/R]BPs [125]. Denas and Taylor [163], presents more than 10,000 unique datasets and 20 TB
used CNN to preprocess data generated from Chromatin of images, videos, and animations data. These data belong
Immunoprecipitation followed by sequencing (ChIP-seq) and to a wide diversity of organisms, cell types, and cellular
created gene transcription factor activity profiles [127]. processes.
CNN was used by Kelley et al. to predict DNA sequence The Euro Bioimaging [164] database provides biological
accessibility [128], by Zeng et al. to predict the BPD [132], by and biomedical imaging data aiming to provide collaboration
Zhou et al. [129] and Huang et al.[141] to find non-coding among different stakeholders including scientists, industry,
gene variation, and by Wang et al. to predict secondary national and European authorities. Its mission is to give
protein structure (2ps) [124]. Park et al. used LSTM to predict access and services to state-of-the-art imaging techniques
precursor miRNA [133] and Lee et al. [134] used it to predict and bioimaging data for scientists in Europe and beyond.
miRNA precursors' targets. GAN was used by Marouf et al. Euro Bioimaging also includes image analysis tools.
for the realistic generation of single-cell RNA-seq data [130], The HAPS is a histology image database [165] contains
by Jiang et al. to predict disease gene from RNA-seq data medium-/high-resolution photograph of microscopic image of
[158], by Zhao et al. as a semi-supervised procedure for human cells and tissues which are free of any copy right.
predicting drug target binding [159], and by Wang et al. for Another image database, the Image Data Resource (IDR)
identifying expression patterns from GE data [156]. [166], contains individual datasets of cellular and tissue
images. Various categories of images include time lapse
imaging, protein localization studies, digital pathology
imaging, yeast study, human high-content screening, etc. It
Open Access Biological Data Sources
is also public API which facilitates viewing, analysis, and
sharing of multi-D image data for cell biology.
Reproducing scientific results, reported as statistically
The SICAS Medical Image Repository (SMIR) is an image
processed quantitative data or carefully selected representative
repository for medical research purpose. Two of their featured
qualitative data, has been greatly facilitated by data sharing
collections include post-mortem full-body CT [167] scan of 50
initiatives [160]. In the last few decades, many open access
anonymized subjects of different age groups and gender,
data repositories have been made available for this purpose
and CT, micro-CT, segmentation, and shape models of the
[161]. Indeed, many research funders and journals now
cochlea [183].
require data used for studies to be made openly available for
The Cancer Imaging Archive (TCIA) [168] contains CT,
verification. To facilitate method development, here we list
MRI, and nuclear medicine (eg PET) images for clinical
the leading and popular open access data repositories
diagnosis, biomarker, and cross-disciplinary investigation.
pertaining to the Sequences, Images, and Signals data which
The Stanford Tissue Microarray Database (TMA) [169] is a
are summarized in Tables 4, 5, and 6, respectively.
source for annotated microscopic tissue images and
associated expression data. The data can be used for
Images studying cell biology. The UCSB bio-segmentation bench
mark dataset [170] contains 2/3-D cellular, subcellular, and
Table 4 lists the leading open access data sources including tissue images. These datasets can be used for segmentation
databases and individual datasets that provide access to and classification tasks.
data pertaining to biological image research. For the sake of
simplicity, these sources have been grouped to four broad Disease Detection and Diagnosis
application areas—[bio/medical] image processing and
analysis, disease detection and diagnosis, neuroimage A large amount of imaging data has been acquired from
processing and analysis, and segmentation—and these are patients with neurological disorders. The Autism Brain
briefly described below. Imaging Data Exchange (ABIDE) [171] database includes
autism brain imaging datasets for studying the autism spec
Bio/Medical Image Processing and Analysis trum disorder. The other dataset pertains to the Attention
Defcit Hyperactivity Disorder (ADHD) [172] and includes 776
The Cell Centered Database (CCDB) [162] collection provides resting-state fMRI and anatomical datasets which are fused
high-resolution 3-D light and electron microscopic over the 8 independent imaging sites. The phenotypic
reconstructions of cells and subcellular structures. information includes age, sex, diagnostic status, measured
It also contains [2/3/4]-D protein distribution and structural ADHD symptom, intelligence quotient, and medication

13
Machine Translated by Google

eleven
Cognitive Computing (2021) 13:1–33

Table 4 Application-wise categorization of open access data repositories and datasets pertaining to [bio/medical/health/clinical] images

application Yam Description Ref.

Bio/medical image processing and analysis CCDB High-resolution 2/3/4-D light and electron microscope images Cell image [162]
CYL datasets and cell library app. [163]

Euro Bioimaging Biological and biomedical imaging data [164]

HAPS Microscopic image of human cells and tissues Viewing, [165]
IDR analysis, and sharing of multi-D image data Post-mortem CT [166]
SMIR scans of the whole body CT, MRI, and PET images of cancer [167]
TCIA patients Microscopic tissue images of human 2D/ 3D cellular, [168]
TMA subcellular and tissue images Autism brain imaging datasets [169]

UCSB BioSeg fMRI/anatomical datasets fused over the 8 imaging sites MCI, [170]

Disease detection and diagnosis ABIDE early AD and elderly control subjects' diagnosis data Multimodal [171]
ADHD-200 mammography and ultrasound scan data Chest X-ray scans for [172]
ADNI pneumonia Breast cancer histological images Lupus , brain, prostate [173]
BCDR MRI scans COVID-19 cases with chest X-ray/CT images fMRI datasets [174]

Kaggle CXRayP and synthesis platform Labeled chest X-ray images with diagnoses MRI [175]
MYTHS datasets and XNAT data management platform Imaging modalities and [176]
NAMIC brain diseases data CT of human temporal bones It provides [177]

nCOV-CXray neuroimaging data and toolkit software Maps of brain regions and a set [178]

Neurosynth of whole-head MRI API for collecting and sharing statistical maps of [179]
NIH brain MRI, PET, S PECT, CT, MEG/EEG and optical imaging [180]
OASIS [181]

Open NI [182]
SMIR [183]

Neuroimaging processing and analysis IXI [184]

LPBA40 [185]

NeuroVault.org [186]
NITRC [187]

OpenfMRI Multimodal MRI and EEG datasets [188]

uk dating service fMRI data set [189]

segmentation DRIVE Digital Retinal Images diabetic patient [190]

IBSR Segmentation results of MRI data [191]
STARE The dataset contains raw/labeled retinal images [192]

Legend: CXRayP Chest X-ray Pneumonia, JHDTI Johns Hopkins Diffusion Tensor Imaging

status. Imaging-based diagnostic classifcation is the main disease. On the other hand, the Open Neuroimaging (Open
aim of the ADHD 200 dataset. The ADNI (Alzheimer's NI) [182] dataset contains imaging modalities and brain
Disease Neuroimaging Initiative [173]) is a popular data diseases data which can be used to study decision support
base and contains neuroimaging datasets from neurodegen system for disease identifcation.
erative diseases, in particular, AD, MCI, early and late AD The recent novel coronavirus disease or COVID-19 pan
and elderly control subjects. The datasets offered by this demic has attracted a number of researchers to focus their
repository are mainly dedicated for development of novel attention on the detection of the novel coronavirus disease.
methods for diseases related to AD. Another dataset focus The NIH [180]
ing on AD is the Open Access Series of Imaging Studies nCOV chest X-ray database [178] contains COVID-19
(OASIS) [181] dataset. This contains MRI datasets and cases with chest X-ray/CT images. The data can be used
open-source data management platform (XNAT) to study for identifying bacterial vs viral vs COVID-19 pneumonia.
and analyze AD. Neurosynth [179] is yet another database Similar chest X-ray datasets [175] are hosted by Kag gle
which includes fMRI literature (with some datasets) and which include chest X-ray scans data for detecting traditional
synthesis platform to study brain structure, functions, and viral and bacterial pneumonia.

13
Machine Translated by Google

12 Cognitive Computing (2021) 13:1–33

Table 5 Application-wise categorization of open access data repositories and datasets pertaining to biological signals

application Yam Description Ref.

Abnormal detection SAD mc-EEG Multichannel EEG data for sustained-attention driving task [193]
TUH EEG Corpus Repository for EEG datasets, tools and documents [194]
MIT-BIH-ARH ECG database containing 48 recordings [195]
PTB D-ECG ECG database containing 549 recordings [196]
TEL ECG 250 ECG recordings with annotated QRS and artifact masks [197]
Human–Machine Interface BNCI Various BMI signal datasets [198]
EMG Data Rep Various EMG datasets [199]
sEMG facial Contains EMG data from 15 participants [200]
NinaPro database Kinematic as well as the sEMG data of 27 subjects [201]
Emotion/affective state detection DEAP Simultaneously recorded EMG/EEG data [202]
DECAF MEG, hEOG, ECG, trapezius muscle EMG, face video data [203]
imagine EEG datasets of 31 subjects while listening voice [204]
MAHNOB-HCI EMG, ECG, and respiration and skin temperature data [205]
SEED EEG dataset for emotion and vigilance [206]
Imagery classification engine EEG-BCI-MI EEG signals from 13 subjects with 60,000 MI examples [207]
EEG-MI-BCI EEG data from BCI for MI tasks [208]
EEG-MMI EEG data from PhysioNet for MI task [209]
Neurological condition evaluation V-P300 BCI 16-electrode dry EEG from 71 subjects (SP mode) [210]
32-electrode wet EEG from 50 subjects (SP mode) [211]
32-electrode wet EEG from 38 subjects (MPC mode) [212]
32-electrode wet EEG from 44 subjects (MPCC mode) [213]
Signal processing and classification BCI competition EEG, ECoG, and MEG data from a range of BCI applications [214]
BCI-NER challenge 56 channel EEG dataset decoded by a P300 speller [215]
DRYAD EEG datasets of 13 subjects recorded under various conditions [216]
PhysioNet Various EEG, ECG, EMG and sEMG datasets [217]
ICU LM Various ECG, EMG, sEMG datasets [218]

Legend: MI Motor Imagery, MMI Motor Movement/Imagery, ERP Event-Related Potentials, SADmc-EEG Sustained-Attention Driving multi channel
EEG, V-P300 Visual P300, SP Single Player, MP Multiplayer, BCI-SSVEP Steady State Visual Evoked Potentials , EMG DataRep EMG
Dataset Repository, ARH Arrhythmia, D-ECG Diagnostic ECG

Breast cancer is also another important disease which can and diffusion weighted images. These images have been
be addressed through imaging and this has attracted a number collected from three different hospitals in London, UK.
of databased hosting breast cancer images. Another database, called the Loni Probabilistic Brain Atlas
The Breast Cancer Digital Repository (BCDR) [174] database (LPBA40) [185], contains maps of brain anatomic regions of 40
contains multimodal mammography and ultra sound scan and human volunteers. Each map generates a set of whole head
patient history data collected from 1734 anonymized patients. MRI, whereas each MRI describes to identify 56 structures of
The data can be used for disease detection and diagnosis brain, most of them lies in the cortex. The study of skull-stripped
methods. Another dataset, MITOS [176], contains breast cancer MRI volumes, and classifcation of the native space MRI,
histological images (haema toxylin and eosin stained slides). probabilistic maps are key uses of LPBA40. The NeuroVault.org
The detection of mitosis and evaluation of nuclear atypia are [186] is a web-based repository (API) for collecting and sharing
key uses. statistical maps of the human brain to study human brain
regions. The Neuroimaging Informatics Tools and Resources
Neuroimaging Processing and Analysis Clearing house (NITRC) [187] provides a range of imaging data
from MRI to PET, SPECT, CT, MEG/EEG, and optical imaging
The Information eXtraction from Images (IXI) dataset [184] for analyzing functional and structural neuroimages. The Open
provides 600 MRI images from healthy subjects to study brain fMRI [188] dataset contains MRI images acquired using different
functions. These images saved in NIFTI fle format and were modalities including diffusion-weighted, T1-weighted
acquired using protocol—T1, T2, proton-density weighted magnetization prepared rapid acquisition with gradient echo
images; magnetic resonance angiography images; (MPRAGE)

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 13

Table 6 Application-wise categorization of open access data repositories and datasets pertaining to Omics data

application Yam Description Ref.

Bioassay analysis and drug design COVID-19 Gene sequence, pathway, and bioassay datasets of COVID-19 [220]
PubChem Contains compound structures, molecular datasets, and tool [221]

Genetic disorder analysis Cancer GeEx Different cancer genome datasets [222]
IGDD Mutation data on common genetic diseases [223]
TCGA Contains cancer genome data [224]
BDTNP 3D Gene expression, DNA-binding data and ChAcD [225]
Nucleic acid research ENCODE human genome dataset [226]
ESP Contains sequencing data [227]
GEO Contains high-throughput gene expression and functional genomics [228]
datasets

gnomAD Large-scale exomes and genomes sequencing data [229]

GTEx Gene expression datasets [230]
harmonize me Collection of genes and proteins datasets [231]
INSDC Contains nucleotide sequence data [232]
IGSR Genome data of various ethnicities, age and sex [233]
JASPER Transcription factor DNA-binding preferences dataset [2. 3. 4]

NIHREM Human genome datasets [235]

NSD Includes omics and health science data [236]

SysGenSim Bioinformatics tools and gene sequence dataset [237]

Protein structure analysis PDB Proteins, nucleic acids, and complex assemblies data [238]
SCOP2 Contains structural classification of proteins [239]
SCOPE [240]
MB ICU 2ps and splice–junction gene sequences [241]

Signal transduction pathway study NCI Nature Molecular interactions and reactions of cells [242]
NetPath Signal transduction pathways in humans [243]
react me Database for reactions, pathways and biological processes [244]

Single-cell omics myRBoost The genomes of eukaryotes containing at least 100 miRNAs [245]
SGD Provides biological data for budding yeast and analysis tool [246]

MRI, and multi-echo fast low-angle shot (FLASH) MRI. It Structured Analysis of the Retina (STARE) was initiated in
also contains biosignal datasets to study brain regions and 1975. The project contains datasets of 400 raw retinal
its functions. These can be used as a benchmark dataset images, 10 labeled images of artery/vein, and 80 images
in order to differentiate outcome from various neuroimaging with ground truth. Each image is annotated and features
analysis tools. The UK data service [189] contains T1/2, are shown in image by the expert. The data set can be
diffusion tensor imaging, and fMRI datasets from 22 patients used for blood vessel segmentation and optic nerve
suffering from brain tumors which can be useful for studying detection.
brain tumor surgical planning. The Internet Brain Segmentation Repository (IBSR)
gives segmentation results of MRI data. Development of
segmentation segmentation methods is the main application of this IBSR.

Segmentation is an important step in any image processing signals

pipeline. Many datasets mentioned above can be used for
segmentation purposes. Table 5 lists leading open access data repositories and
Focusing on eye diseases, the Digital Retinal Images for datasets (also referred to as data sources) pertaining to
Vessel Extraction (DRIVE) contains JPEG Com pressed biological signals. These sources are broadly mapped to
retinal images of 400 diabetic patients between 25-90 years six application areas—anomaly detection, human–machine
old. This dataset can be used to understand segmentation interfacing which includes brain–machine interfacing as
of blood vessels in retinal images and identify diabetic well as rehabilitation research, emotion/affective state
retinopathy. Another dataset called detection, motor imagery classifcation, neurological condition

13
Machine Translated by Google

14 Cognitive Computing (2021) 13:1–33

evaluation, and signal processing and classifcation—which multifinger movements datasets of 2 channels, 10 classes
are described in the following subsections. and 8 channels, 15 classes; single-/multifinger pressure on
a steering wheel; EMG controlled multifunctional upper-limb
Anomaly Detection prostheses and EMG pattern recognition datasets.
For surface EMG (sEMG), the facial sEMG dataset
Anomaly detection is one of the major application areas in contains facial sEMG signals from the muscles corrugator
which scientists have devoted much effort. In this process, a supercilii, zygomaticus major, orbicularis oris, orbicularis
number of open access data sources, largely containing oculi, and masseter. Archived data are from 15 participants
EEG and ECG data, have been frequently used. (8 females and 7 males) aged between 26 and 57 years
Starting with the EEG signals, the SAD mc-EEG [193] (mean age 40.7 ± 9.6 years). These data can be used for
dataset contains 32 channel EEG signals from 27 subjects rehabilitation research. Also, the NinaPro database includes
recorded while they were test-driving. That is, signals were kinematic as well as sEMG data of 27 subjects, while these
acquired when each subject attended two 90-minute virtual subjects were moving finger, hand, and wrist. These data
reality sessions for sustained-attention driving. can be used to study biorobotics and activity detection.
The TUH EEG corpus [194] is also an open-source
clinical EEG data repository for clinical EEG data, tool and
documentation. The major datasets include seizure detection, Emotion/Affective State Detection
abnormal EEG, EEG with artifacts (introduced by eye
movement, chewing, shivering, electrode pop, electrode Emotion and affective state detection has been a very active
static, and lead artifacts, and muscle artifacts), EEG for epi research field over the years. A combination of different
lepsy, etc. signals has been utilized in detecting emotion and affective
Regarding the ECG signals, the MIT-BIH arrhythmia [195] states, and a number of data sources providing these signals
arrhythmia database includes 2-channel ambulatory ECG are described below.
recording taken from 47 subjects for studying arrhyth mia. A Database for Emotion Analysis using Physiological
There are 48 complete ECG recordings and about 24 Signals (DEAP) provides various datasets for analyzing the
recordings are freely available. The PTB diagnostic ECG human affective states. It provides EEG and sEMG signals
database [196] comprises 549 ECG recordings taken from of 32 volunteers, while they were watching music videos to
290 subjects of age ranged from 17 to 87 years using analyze the affective states. These volunteers also rated the
conventional 12 leads and 3 Frank lead ECG recorder. Each video, and the front face was also recorded for 22 volunteers.
recording includes 15 signals coming from these leads and DECAF is a multimodal dataset for decoding user
each subject was represented in 1 to 5 records. Both the physiological responses to affective multimedia content. It
data sets can be used for anomaly detection. Another ECG contains magnetoencephalogram (MEG), horizontal
data set, the TELE-ECG dataset [197] includes 250 ECG electrooculogram (hEOG), ECG, trapezius muscle EMG,
records with annotated QRS and artifact masks. It also and near-infrared face video data to study physiological and
includes QRS and artifact detection algorithms to study QRS mental states. Another multimodal dataset is the MAH
and detect artifacts from ECG signals. NOB-HCI [205] dataset which includes ECG, respiration,
and skin temperature data in addition to 32-channel EEG
signals from 30 subjects, while they were watching movie
Human–Machine Interface clips and photos. The different sensors were synchronized
to record a synchronized multimodal dataset. The subjects
The application area of Human–Machine Interfacing focuses were asked to label their own emotion state.
on [body and brain]–machine interfacing and rehabilitation. On the other hand, the Imagined Emotion [204] dataset
This is done largely through Electromyography (EMG) and provides EEG signals recorded when subjects were listening
sometimes with EEG signals. to voice recording. The SJTU Emotion EEG Dataset [206]
The BNCI Horizon 2020 database contains more than 25 contains three individual datasets (SEED, SEED-IV and
datasets such as stimulated EEG datasets, Electrocorticog SEED-VIG) of EEG signals. In the SEED dataset EEG
raphy (ECoG)-based BCI datasets, Event Related Potential signals were recorded, while the subjects were watching
(ERP)-based BCI datasets, mental arithmetic, motor imagery movie clips and annotated their emotional state as positive,
(extracted from EEG, EOG, fNIRS, EMG) datasets, EEG/ negative and neural. In case of SEED-IV, four emotional
EOG datasets of neuroprosthetic control, speller datasets. states such as happy, sad, fear, and neutral were noted,
Modeling and designing of BMI devices are the key whereas the SEED-VIG dataset contains EEG signals related
application of this database. While the BNCI contains a to vigilance when the subjects were driving.
variety of signals, the EMG Datasets Repository [199] includes single/

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 fifteen

Imagery Classification Engine BCI-NER dataset [215]. This dataset was mainly intended for
methodological development of an error detection algorithm
Motor imagery (MI) is yet another very active area of research. suitable for the P300-based BCI systems. The BCI competition
As an outcome of a large number of community contributors, datasets include EEG datasets (eg, cortical negativity or
many datasets have been developed from which the popular positivity, feedback test trials, self-paced key typing, P300
ones are described below. speller paradigm, motor/mental imagery data, continuous EEG,
The electroencephalographic brain–computer interface EEG with eye movement), ECoG datasets (eg, fnger movement,
mental imagery (EEG-BCI-MI) [207] dataset contains 60 hours motor/mental imagery signals in the form of EEG/ECoG), and
of EEG recording from 13 subjects and 75 experiments. This MEG dataset (eg, wrist move ment). These datasets can be
contains around 60,000 mental imagery exam ples which is used for signal processing and classification methods for BMI.
approximately 4.8 hours of EEG recordings (with 4600 MI Similarly, the BCI-NER Challenge [215] dataset provides 56-
examples) per participant. The datasets can be used for the channel EEG signals from 26 subjects using a P300 speller.
rehabilitation of patients having movement ment disorders.
Another EEG dataset for MI brain–computer interface (EEG-MI- In addition to the datasets released for challenges and
BCI) [208] contains EEG signals with 3-D electrode location competitions, there are repositories which provide rich data
and EEG for non-task-related states as well. The dataset was sets for this application area. The DRYAD [216] is a versa tile
recorded from 52 partici pants which also contains [physio/ repository which has been recently unveiled. It contains a range
psycho]logical data and EMG signals in addition to the EEG. of EEG recorded datasets when 19 subjects listen to natural
The dataset can be employed to find the human factors which speech time-reversed speech, cocktail party attention, and
influence MI BCI performances. Yet another EEG signal centric noisy audiovisual speech. The PhysioNet repository [217]
data set is called EEG motor movement/imagery (EEG-MMI) contains a large number of neuroelectric and myoelectric
dataset [209] and incorporates 1500 (1–2 minutes) EEG datasets. As the name suggests, it is mainly for physiological
recordings taken from 109 volunteers. The dataset can be used data. These datasets mainly pertain to signals such as EEG,
in designing BCI systems for rehabilitation purposes. ECoG, EMG, and ECG and are acquired from many diverse
experimental settings. The UCI ML repository [218] contains a
large number of diverse datasets with direct application to
Neurological Condition Evaluation machine learning methods. Some relevant biosignal datasets
include ECG, EEG, and (s)EMG signals from diverse
A number of visual P300-based datasets are available with experimental and physiological conditions.
open-access attributes to perform a range of neurological
condition evaluation. These datasets, V-P300 BCI, are
sequences
composed of data recorded using dry or wet electrode with 16
or 32 channels while the subjects were playing the Brain
Invaders game [219]. These datasets were recorded using Table 6 lists the leading popular open access data sources
different playing modalities such as single player (16 dry pertaining to the various omics-related researches which include
electrodes [210] from 71 subjects and 32 wet electrodes [211] genomics, proteomics, and metabolomics. Grouped to six broad
from 50 subjects), multiplayer in collaborative mode (32 wet application areas, namely, bioassay analysis and drug design,
electrodes from 38 subjects [212]), and multiplayer cooperation genetic disorder analysis, nucleic acid research, protein
and competition mode (32 wet electrodes from 44 subjects structure analysis, signal transduction pathway study, and
[213]). single-cell omics, the following subsections provide brief
discussions about the leading open access omics data
sources.
Signal Processing and Classification

To solve various signal processing and classifcation problems,

Bioassay Analysis and Drug Design
a number of datasets have been made available under open-
access. Most of these problems are released to the community
Since December 2019, the world has experienced a pandemic
in the form of challenges with relevant datasets to solve them.
caused by the SARS-CoV-2 (COVID-19) virus. Triggered by
The competitions during the BCI meetings have served this
the necessity to facilitate the ongoing researches, the SARS-
purpose for several years and have released datasets (the BCI
CoV-2 [220] dataset provides gene sequence, proteins,
competition datasets [214]) which are still available with relevant
pathway, and bioassay for SARS-CoV-2 along with compounds
problem statements and sample codes for others to use. The
used in clinical trials. This dataset can be used for studying
challenge dataset provided by the IEEE Neural Engineering
biological/chemical process and drug design.
Conference (NER2015) is known as

13
Machine Translated by Google

16 Cognitive Computing (2021) 13:1–33

The PubChem database [221] contains millions of from different sequencing projects. The dataset can be used
compound structures and descriptive datasets of chemical for disease diagnosis and genetic studies. The Genotype
molecules and their activities against biological assays. Main Tissue Expression (GTEx) [230] database contains GE data
tained by the National Center for Biotechnology Information of sets of 54 healthy tissue sites collected from 1000 subjects
the United States National Institutes of Health, it can be freely and histology images. It also includes samples from GTEx
accessed through a web user interface and down loaded via biobank.
FTP. It also contains software services (such as plotting and The Harmonizome [231] database provides details about
clustering). It can be used for [gen/prote]-omics study and drug genes and proteins from 114 datasets provided by 66 online
design. resources with 71927784 associations between 295496
attributes and 56720 genes. The International Nucleotide
Sequence Database [232], popularly known as INSDC, cor
Genetic Disorder Analysis roborates biological data from three major sources: i) DNA
Databank of Japan [247], ii) European Nucleotide Archive
The cancer gene expression (GE) [222] serves as a small [248], and iii) GenBank [249]. These sources provide the
repository containing several cancer GE datasets which can spectrum of raw data reads, though alignments, and assemblies
be employed for designing tool/algorithm for cancer detection. to functional annotation, enriched with contextual information
The cancer genome atlas (TCGA) [224] repository contains relating to samples and experimental configurations. Similar
more than 2.5 petabytes of genomic, epigenomic, to this, the International Genome Sample Resource (IGSR)
transcriptomic, and proteomic data. It contains data about 33 [233] includes genome sequencing data from the 1000
different cancer types and over 20,000 samples. These data genomes project. The genome data was taken from people of
are generated by the National Cancer Institute and the National various ethnicities, age, and sex with the final dataset contains
Human Genome Research Institute. This repository is used in gene sequencing data from 2,504 individuals from 26
facilitating genomic study for improving the prevention, populations. These data can be used for disease diagnosis
diagnosis, and treatment of cancer. To analyze region-specific and genetic studies. Also, the SysGenSim [237] database
diseases, the Indian Genetic Disease Data base (IGDD) [223] includes bioinformatics tool, and Pula-Magdeburg single-gene
tracks mutations in the normal genes for genetic diseases knockout, StatSeq, and DREAM 5 benchmark datasets for
reported in India. studying Gene Sequence.
JASPAR [234] is a database for transcription factor DNA-
Nucleic Acid Research
binding profle. The data spans through six different taxonomic
groups covering Vertebrata, Nematoda, Insecta, Plantae,
The Berkeley Drosophila Transcription Network Project Fungi, and Urochordata. The database can be used for
(BDTNP) [225] database contains datasets pertaining to 3D translational genomics research.
Gene expression data, in vivo and in vitro DNA-binding data The NIH Roadmap Epigenomics Mapping repository
as well as Chromatin Accessibility data (ChAcD). Research on (NIHREM) [235] includes 2,804 datasets, ie, 1,821 his tone
GE and anomaly detection is the key application of the modification, 360 DNase, 277 DNA methylation, and 166 RNA-
datasets provided by this database. Seq datasets. The repository provides 3,174-fold 150.21 billion
The Encyclopedia of DNA Elements (ENCODE) [226] is a mapped sequencing the human and tools for analyzing these
whole-genome database curated by the ENCODE Con datasets. It can be used for stem cell mapping and selection
sortium. It contains a large number of datasets pertaining to of tissues that are responsible for human disease. Also, the
functional genomics and characterization data including meta- database known as Nature scientific data (NSD) [236] includes
data of human, worm, mouse, and fy. Another data base, datasets pertaining to omics, taxonomy and species diversity,
called the Exome Sequencing Project (ESP) [227], includes mathematical and modeling resources, cytometry, organ ism-
genome datasets which can be used to fnd lung and blood focused resources, and health science data. This can be used
disorders and their management and treatment. The Gene for studying and modeling different aspects of genomics.
Expression Omnibus (GEO) [228] is an open-access functional
genomics (microarray and sequence) data repository. This
database can be used for functional genomic and epigenomic
studies such as genome methylation, chromatin structure, and Protein StructureAnalysis
genome–protein interactions. It is supported by the National
Center for Biotechnology Information at the National Library of The Protein Data Bank (PDB) [238] contains 3D structural
Medicine of the USA [228]. The Genome Aggregation Database data proteins and nucleic acids. These data are obtained tools
(gnomAD) [229] database contains large-scale exome and such as X-ray crystallography, NMR spectroscopy, and cryo
genome sequencing data electron microscopy. It includes more than 135 thousand

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 17

data of proteins, nucleic acids, and complex assemblies. and miRNA-related pathology. Saccharomyces Genome
These can be used to understand all aspects of biomedicine Database (SGD) [246] also provides complete biological
and agriculture. information for the budding yeast Saccharomyces cerevi siae.
Structural classifcation of proteins (SCOP) is a repository They also give an open-source tool for searching and analyzing
which hosts manually classified protein structure datasets. The these data and thereby enable the discovery of functional
classification was done based on amino acid sequences and relationships between sequence and gene products in fungi and
their structural similarity. The main objective is to find the higher organisms. The study of genome expression,
evolutionary relationship between the proteins. Currently two transcriptome, and computational biology is the main function of
versions of SCOP are maintained. the SGD.
The SCOP Version 2 (SCOP2) [239] is the up-to-date SCOP
database released at the frst quarter of 2020. In contrast, the
SCOP-extended (SCOPe) [240] is an extended version of the OpenÿSource Deep Learning Tools
original SCOP maintained by UC Berkeley. SCOPe includes
many new classifed protein structures via a fusion of manual Due to emerging interest and concurrent multidisciplinary
and automation curation. efforts towards DL in recent years, several open-source libraries,
Molecular Biology Databases at the UCI (UCI MB) contain frameworks, and platforms have been made available to the
three individual databases: i) Secondary Protein Structure [241], community. However, for a new user of these tools to mine
which is a bench repository that classifies secondary structure biological data, it is not always straightforward to know their
of certain globular proteins; ii) Splice–Junction Gene Sequences characteris tics, advantages, and disadvantages. In this process,
[250], which contain primate splice–junction gene sequences one of the main hurdles for a new analyst is to select the
(DNA) with associated imperfect domain theory; and iii) Promoter appropriate DL architecture/model and relevant library providing
Gene Sequences [251], which contain E. coli promoter gene suitable implementations of the selected architecture. Towards
sequences (DNA) with partial domain theory. Objectives include introducing a beginner to the field of biological data analysis
i) sequencing and predicting the secondary structure of certain using these open-source tools, this section describes the tools
proteins; ii) studying primate splice–junction gene sequences in a tutorial style indicating their characteristics, pros, and cons.
(DNA) with associated imperfect domain theory; iii) studying E. The focus of the section has been to review and summarize the
Coli promoter gene sequences (DNA) with partial domain theory. most popular open-source tools, which aim to facilitate the
technological developments for the community. This
comprehensive collection contains tools (also developed by
Signal Transduction Pathway Study individuals) which are well maintained with a rea sonable amount
of implemented algorithms (ie, deep learning architectures). For
The NCI–Nature Pathway Interaction Database [242] hosts the sake of brevity, the individual publication references of the
cellular signaling (molecular interactions/reactions) pathways in tools are omitted and interested readers may consult them at
humans. The database can be used for cancer research. their respective websites from the provided URLs.
The database was created by the US National Cancer Institute,
NIH, with the collaboration of Nature Publishing Group and
published in the last quarter of 2006. Another database, NetPath
[243], also contains signal transduction pathways in humans. Table 7 summarizes the main features and differences of the
Created jointly by Johns Hopkins University and the Institute of various tools. To measure the impact and acceptability of a tool
Bioinformatics (IOB) in India; it includes 45 signaling path ways in the community, we provide GitHub-based measures such as
ranging from protein–protein interactions to enzyme–protein numbers of Stars, Forks, and Contributors. These numbers are
substrate reactions including 10 major pathways of the immune indicative of the popularity, maturity, and diffusion of a tool in
system and 10 pathways relevant to cancer regulation. The the community.
other one, Reactome [244], is an open access database hosting
biological pathways of metabolic processes to hormonal Coffee
signaling in humans. Created through a collaboration between
North America and Europe, it can be used for cancer research and treatment.
Cafe (http://cafe.berkeleyvision.org/) is scalable, written in C++
and provides bindings for Python as well as MATLAB.
Singleÿcell Omics Dedicated for experiment, training, and deploying general
purpose DL models, this framework allows switching between
The miRBoost dataset [245] contains the genomes of development and deployment platforms. Targeting computer
eukaryotes containing at least 100 miRNAs. This dataset is vision applications, it is considered as the fastest implementation
used for studying post-transcriptional gene regulation (PTGeR) of the CNN.

13
Machine Translated by Google

18 Cognitive Computing (2021) 13:1–33

Table 7 Summary of Open-Source Deep Learning Tools (* as of July 2020)

Tool Platform Language(s) stars* Forks* Contribute* Supported DL Architecture

Coffeeb L,M,W,A Py, C++, Ma 30100 18200 266 CNN, RNN, GAN
Chainerc L py 5300 1400 251 DA, CNN, RNN, GAN
DL4ja LMW ha 11500 4800 32 DA, CNN, RNN, RBM, LSTM, GAN
DyNeta L C++ 3000 687 117 CNN, RNN, LSTM
H2Oa LMW Ha, Py, R 4700 1700 132 CNN, RNN
Kerasc LMW py 47500 18000 816 CNN, RNN, DBN, GAN
Lasagnea L, M py 3700 980 68 CNN, RNN, LSTM, GAN
MCTcÿÿ W C++ 16720 4400 197 CNN, DBN, RNN, LSTM
MXNet L,M,W,Y,I C++ 18500 6600 780 DA, CNN, RNN, LSTM, GAN
neonaÿÿ L, M py 3800 846 78 DA, CNN, RNN, LSTM, GAN
PyTorchb L, M py 37400 9500 1345 CNN, RNN, LSTM, GAN
singha LMW Py, C++, Ja 2000 499 46 CNN, RNN, RBM, DBM
TensorFlowa LMW py, c++ 14300 80600 2450 CNN, RNN, RBM, LSTM, GAN
TF.Learnc L, M py, c++ 9400 2400 120 CNN, BRNN, RNN, LSTM, GAN
Theanob LMW py 9103 2500 332 CNN, RNN, RBM, LSTM, GAN
Torchb L,M,W,Y,I Lu, C, C++ 8495 2400 130 CNN, RNN, RBM, LSTM, GAN
Velesa L,M,W,A py 891 185 10 DA, CNN, RNN, LSTM, RBM

L Linux/Unix, M MacOSX, W Windows, A Android, I iOS, CP Cross-platform, Py Python, Ja Java, Lu Lua, Ma Matlab
*GitHub parameters (as of April 1, 2020)
a
Apache2 License
b
BSD License
c
MIT License

pros Cons.

– Easy to deploy; – Open Computing Language framework/Open Multi

– Pretrained models are available; Processing API is not supported.
– Faster training speed;
– Used for feedforward networks. DeepLearning4j

Cons. DeepLearning4j (DL4J, https://deeplearning4j.org/), written

in Java with core libraries in C/C++, is a distributed
– Requires writing code for generating new layers; framework for quick prototyping that targets mainly non-
– Less support for recurring networks; researchers. Compatible with JVM supported languages (eg, Scala/
– No support for distributed training. Clojure), it works on distributed processing frameworks (eg,
Hadoop and Spark). Through Keras (see section 5.6) as a
Chainers Python API, it allows importing existing DL models from
other frameworks. It allows creation of NN architectures by
Chainer (http://chainer.org/) is a DL framework provided as combining available shallow NN architectures.
Python library. Besides the availability of popular optimization pros
techniques and NN related computations (eg, convolution,
loss, and activation functions), dynamic creation of graphs – Supports integration with Big Data frameworks Apache
makes Chainer powerful. It supports a wide range of DL Spark and Hadoop;
architectures including CNN, GAN, RNN, and DA. – Supports distributed GPU and CPU platforms and capa
pros ble to work with tensor.

– One of the tools for leading dynamic computation graphs/ Cons.

networks;
– Notably faster than other Python-oriented frameworks. – Open Computing Language framework is not supported;

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 19

– GUI is supported for workflow and visualization. pros

DyNet – Rich documentation;

– A high-level API for neural networks;
The DyNet library (https://dynet.readthedocs.io/), write ten in C+ – Ability to run on top of state-of-the-art deep learning libraries/
+ with Python bindings, is the successor of the 'C++ neural network frameworks such as TensorFlow, CNTK, or Theano.
library'. In DyNet, computational graphs are dynamically created
for each training example; thus, it is computationally efficient and
flexible. Targeting NLP applications, its specialty is in CNN, RNN, Cons.
and LSTM.
– Cannot use multi-GPU directly;
pros – Requires Theano as backend for OpenMP support and Theano/
TensorFlow/PlaidML as backend for OpenCL.
– Designed to be efficient for running on CPU or GPU.
– Dynamic computation graph like PyTorch and Chainer. lasagne

Cons. Lasagne (http://lasagne.readthedocs.io) DL library is built on top

of Theano. It allows multiple input, output, and auxiliary classifiers.
– In terms of TensorFlow, limited functions are available. It supports user-defined cost functions and provides many
optimization functions. Lasagne supports CNN, GAN, RNN, and
H2O LSTM.
pros
H2O (http://www.h2o.ai) is an ML software that includes DL and
data analysis. It provides a unified interface to other DL frameworks – Lasagne is a lightweight library to build and train DL algorithms
like TensorFlow, MXNet, and Caffe. It also supports training of DL in Theano;
models (CNN and RNN) designed in R, Python, Java, and Scala. – Layers, regularizers, and optimizers can be used independently
dently;
pros – Clear documentation is available;
– Supports training the network on a GPU.
– Due to its in-memory distributed parallel processing capacities,
it can be used for real-time data; Cons.
– GUI is supported (called Flow) for workfow and visu
alization; – Small community than TensorFlow.
– GPU support for Deep Water and NVIDIA;
– Fast training, memory-efficient DataFrame manipulation; Microsoft Cognitive Toolkit
– Easy-to-use algorithms and well documented;
Replacing CNTK, the Microsoft Cognitive Toolkit (MCT, https
Cons. ://cntk.ai/) is mainly coded in C++. It provides implementations of
various learning rules and supports different DL architectures
– Lacks the data manipulation capabilities of R and Pandas including DNN, CNN, RNN, and LSTM.
DataFrames; pros
– Slow in learning and supports limited model running at
on time – It is a framework for feedforward DNNs, CNN and RNN;
– Can train production systems very fast;
Keras – Can achieve state-of-the-art performance on benchmark
tasks tasks;

The Python-based Keras (https://keras.io/) library is used on top – Allow directed graph visualization.
of Theano or TensorFlow. Its models can be imported to DL4J
(see section 5.3). It was developed as a user friendly tool enabling Cons.
fast experimentation, and easy and fast prototyping. Keras
supports CNN, GAN, RNN, and DBN [252]. – Less community support;
– Difficult to install;

13
Machine Translated by Google

twenty
Cognitive Computing (2021) 13:1–33

– Draw lass interest among the research community. – Easy to create a layer and run on GPU.

MX Net Cons.

MXNet (https://mxnet.io/) framework allows defining, training, – Requires writing training code;
and deploying deep NN (DA, CNN, GAN, RNN and LSTM) on – Limited documentation.
a wide range of devices—from cloud infrastructure to mobile or
even embedded devices (eg Raspberry Pi). Written in C++, it sing
is memory efficient and supports Go, JavaScript, Julia,
MATLAB, Perl, Python, R, and Scala. Singa (https://singa.incubator.apache.org/), it is a distributed
pros DL platform written in C++, Java, and Python.
Its flexible architecture allows synchronous, asynchronous,
– A DL framework which has a high-performance rule and hybrid training frameworks to run. It supports a wide range
have API; of DL architectures including CNN, RNN, RBM, and DBM.
– Rich Language support;
– MXNet features advanced GPU support; pros
– Highly scalable.
– Pretrained models are available;
Cons. – Supports model/data or hybrid partitioning, and synchronous/
asynchronous/hybrid training;
– Small community than TensorFlow; – Distributed deep learning system and handle Big data.
– Poor API documentation; – Widely used for healthcare data analytics.
– Less popular with the research community.
Cons.
Neon
– No Open Multi-Processing support.
Neon (www.nervanasys.com/technology/neon/) is a DL
framework written in Python. It provides implementations of TensorFlow
various learning rules, along with functions for optimization and
activation. Its support for DL architecture includes CNN, GAN, TensorFlow (www.tensorflow.org), written in C++ and Python,
RNN, LSTM, and DA. was developed by Google and supports very large scale deep
pros NN. Amended recently as 'TensorFlow Fold', its capability to
dynamically create graphs made the architecture flexible,
– Better visualization properties than other frameworks; allowing deployment to a wide range of devices (eg, multi-CPU/
– Apply optimization at data loading level, GPU desktop, server, mobile devices, etc.) without code
rewriting [ 253, 254]. Also it contains a data visualization tool
Cons. named TensorBoard and supports many DL architectures
including CNN, GAN, RNN, LSTM, and RBMs [255].
– Small community than TensorFlow;
– Less popular with the research community. pros

PyTorch – Handles large-scale data and operate in heterogeneous

environments;
PyTorch (http://pytorch.org/) provides Torch modules in Python. – Faster compile time than Theano;
More than a wrapper, its deep integration allows exploiting the – Computational graph abstraction;
powerful features of Python. Inspired by Chainer, it allows – Supports parallelism.
dynamic network creation for variable workload and supports – TensorBoard is used for workflow and visualization.
CNN, GAN, RNN and LSTM.
pros Cons.

– Pretrained models are available; – Large memory footprint;

– OpenCL support via separately maintained package. – Less number of pretrained models are available;
– Easily combine modular pieces; – Computational graph can be slow;

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 twenty-one

– No support for matrix operations; Recently, Facebook's DL modules (fbcunn) focusing on

– Difficulties in debugging. CNN have been open-sourced as a plug-in to Torch.
pros
TF.Learn
– User-friendly;
TF.Learn (www.tflearn.org) is a TensorFlow (see section – Convenient to employ with GPUs;
5.13)-based high-level Python API. It supports fast prototyping – Pretrained models are available;
with modular NN layers and multiple optimizers, inputs, and – Highly modular;
outputs. Supported DL architectures include CNN, GAN, – Easy to create a layer and run on GPU.
BRNN, and LSTM.
pros Cons.

– Modular and transparent DL library built on the top of – Special data format and requires conversion;
TensorFlow; – Require to write training code;
– Provides a higher-level API to TensorFlow. – Less documentation available.

Cons. candles

– Slower compared to its competitors. Veles (https://github.com/Samsung/veles) is a Python-based

distributed platform for rapid DL application development. It
Theano provides machine learning and data processing services
and supports IPython notebooks. Developed by Samsung,
Theano (www.deeplearning.net/software/theano/) is a one of its advantages is that it supports OpenCL for cross-
Python library that builds on core packages like NumPy and platform parallel programming, and allows execution across
SymPy. It defines, optimizes, and evaluates mathematical hetero ogenous platforms (eg servers, PC, mobile, and
expressions with tensors and served as foundation for many embedded devices). The supported DL architectures include
DL libraries. DA, CNN, RNN, LSTM, and RBM.
pros pros

– High flexibility; – Distributed platform support;

– High computational stability; – Supports Jupyter Notebook;
– Well suited for tensor-based mathematical expressions; – Supports OpenCL for cross-platform parallel program
– Open-source libraries such as Keras, Lasagne and Blocks ming.
built on the top of Theano;
– Able to visualize convolutional filters, images, and Cons.
graphs;
– High-level wrappers like Keras and Lasagne increases – Less community support;
usability. – Draws lass interest from the research community.

Cons.
Relative Comparison of DL Tools
– Difficult to learn;
– Difficult to deploy; To perform relative comparison among the available open
– Deployed on single GPU; source DL tools, we selected four metrics which are detailed
– Slower compilation time than TensorFlow. below: trend in their usage, community participation in their
development, interoperability among themselves, and their
torch scalability (Fig. 4).

Started in 2000, Torch (http://torch.ch/), a ML library and trendy

scientific computing framework, has evolved as a powerful
DL library. Core functions are implemented in C and the rest To assess the popularity and trend of the various DL tools
via LuaJIT scripting language made Torch superfast. among the DL consumers, we looked into two different
Software giants like Facebook and Google use Torch extensively.sources to assess the utilization of the tools.

13
Machine Translated by Google

22 Cognitive Computing (2021) 13:1–33

a b MXNet Theano Others d

Caffe Keras Torch Theano Tensorflow Coffee
100%
80%
mention
(%)

60%
Keras
40% DL4J
DEEPEARNING4J
python
twenty%

0% 2015-01 2015-04 2015-07 2015-10 2016-01 2016-04 2016-07 2016-10 2017-01 2017-04 2017-07 2017-10 2018-01 2018-04 2018-07 2018-10 2019-01 2019-04 2019-07 2019-10

TensorFlow

Time (year-month)

c and

Memory app.
160 logic I/O

Settings

140 tensor flow SO C

DLL PLL I/O
Keras
Memory Macro
40
thousands)
Github
stars
(in

Coffee fpga
pyTorch

computing
Increased
capability

twenty MX Net MCTs ALU control aluminum

DRAM
torch Theano TF.Learn ALU ALU
DL4j H2O _
SingaChainer Cache GPU
0 candles Neon
lasagne DyNet DRAM

CPUs
2008 2014 2015 2016
Year of release Increased energy efficiency

Fig. 4 Relative comparison of DL tools. a Popularity trend of individual contributors. d As for the interoperability among the DL tools, Keras
DL tools as per mention in google search generated globally (data allows model importing from Cafe, MCT (CNTK), Theano, and Ten
courtesy: Google Trend). b Mention in articles submitted to arXiv sorFlow and lets DL4j to import. e Regarding hardware-based scal
preprint server during the frst quarter of 2020. c The effect of ability of the DL tools, most of the tools provide CPU and GPU sup
community's participation on individual tools is shown by the bub ble port, whereas FPGA and ASIC can mainly execute pretrained models
size, which is product of normalized number of GitHub forks and

Firstly, we extracted globally generated search data from Community

Google Trends1 for five years (January 2015 to December
2019) related to search terms consisting of ÿ[toolname] + The community-based development score for each tool
DeepLearningÿ. The data showed a progressive increase discussed in Section 5 was calculated from repository
of search about TensorFlow since its release followed by population parameters of GitHub (https://github.com/) (ie,
Keras (Fig. 4a). Secondly, mining the content of around star, fork, and contributors). The bubble plot shown in Fig.
2,000 papers submitted to arXiv's cs.[CV | CL | LG | AI | 4c depicts community involvement in the development of
NE], and stat.ML categories, during the first quarter of 2020 the tools indicating the year of initial stable release. Each
(ie January to March), for the presence of the tool names bubble size in the fgure, pertaining to a tool, represents the
[256]. As seen in Fig. 4b which shows the percentage of normal ized combined effect of fork and contributors of that
each individual tool's mention in the papers, the top six tool. It is clearly seen that a very large part of the community
tools were identified as: PyTorch, TensorFlow, Keras, Cafe, comfort is concentrated on TensorFlow, followed by Keras and Cafe.
MXNet, and Theano.
Interoperability

In today's cross-platform development environments, an

important measure to judge a tool's fexibility is its
interoperability with other tools. In this respect, Keras is the
most flexible one whose high-level neural networks are
1
https://trends.google.com capable of running on top of either Tensor or Theano. Alternatively,

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 23

DL4j model imports neural network models originally con Stacked autoencoders or DA were benchmarked using
fgured and trained using Keras that provides abstraction the experimental setup number 1 in Table 8. To estimate
layers on top of TensorFlow, Theano, Cafe, and CNTK the performance of the various tools on implementing DA,
backends (Fig. 4d). three autoencoders (number of hidden layers: 400, 200,
and 100, respectively) were stacked with tied weights and
Scalability sigmoid activation functions. A two-step network training
was per formed on the MNIST dataset [261]. As reported in Fig. 5
Hardware-based scalability is an important feature of the (a, b), the performances of various DL tools are evaluated
individual tools (Fig. 4e). Today's hardware for computing using forward runtime and training time. The forward
devices are dominated by graphics processing units (GPUs) runtime refers to the required time for evaluating the
and central processing units (CPUs). But considering information fow through the full network to produce the
increased computing capacity and energy efficiency, the intended out put for an input batch, dataset, and network.
coming years are expected to witness expanded role for In contrast, the gradient computation time measures the
other chipset types including application-specific integrated time that is required to train DL tools. The results suggest
circuits (ASICs) and feld-programmable gate arrays that, regardless of the number of CPU threads used or
(FPGAs). So far DL has been predominantly used through GPU, Theano and Torch outperform TensorFlow in both
software. The requirement for hardware acceleration, gradient and forward times (Fig. 5 a, b).
energy efficiency, and higher performance has driven the Experimental setup number 2 (Table 8) was used in
development of chipset-based DL systems. benchmarking RNN. The adapted LSTM network [262] was
designed with 10000 input and output units with two layers
and ÿ13 million parameters. As the performance of RNN
Performance of Tools and Benchmark depends on the input length, an input length of 32 was used
for the experiment. As the results indicate (Fig. 5 cf), MCT
The power of DL methods lies in their capability to outperforms other tools on both CPU and all three GPU
recognize patterns for which they are trained. Despite the platforms. On CPUs, TensorFlow performs little better than
availability of several accelerating hardware (eg, multi core Torch (Fig. 5c). On GPUs, Torch is the slowest with
[C/G]PUs/FPGAs), this training phase is very time TensorFlow and MXNet performing similarly (Fig. 5df ).
consuming, cumbersome, and computationally challenging.
Moreover, as each tool provides implementations of several
DL architectures and often emphasizing separate
Table 8 Hardware configuration of the evaluating setup
components of them on different hardware platforms,
selecting an appropriate tool suitable for an application is ESN Processor Memory
getting increasingly difficult. Besides, different DL tools
1 CPU: E5-1650a @ 3.50GHz 32GB
have different targets, eg, Caffe targets applications,
GPU: Nvidia GeForce GTX Titan Xbÿÿ
whereas Torch and Theano are more for DL research.
two CPU: E5-2630c @ 2.20GHz 128GB
To facilitate scientists in picking the right tool for their
GPU: Nvidia GeForce GTX 980dÿÿ
application, scientists benchmarked the performances of
GPU: Nvidia GeForce GTX 1080eÿÿ
the popular tools concerning their training times [257, 258].
GPU: Tesla K80 accelerator with GK210 GPUsf
Moreover, to the best of our knowledge, there exist two
3 CPU: E5-2690c @ 2.60GHz 256GB
main efforts that provide the benchmarking details of the
GPU: Tesla P100 acceleratorgÿÿ
various DL tools and frameworks publicly [259, 260].
GPU: Tesla M40 acceleratorhÿÿ
Summarizing those seminal works, below we provide the
GPU: Tesla K80 accelerator with GK210 GPUsfÿÿ
time required to complete the training process as a
performance measure of four different DL architectures (eg, ESN Experimental Setup Numbers
FCN, CNN, RNN, and DA) among the popular tools (eg, a
Intel Xeon CPU v2
Caffe, CNTK, MXNET , Theano, TensorFlow, and Torch) b
3072 cores, 1000 MHz base clock, 12 GB memory
on multicore [C/G]PU platforms. c
Intel Xeon CPU v4
Table 8 lists the experimental setups used in bench d
2048 cores, 1126 MHz base clock, 4 GB memory
marking the specified tools. Mainly three different set ups, and

2560 cores, 1607 MHz base clock, 8 GB memory

each with Intel Xeon E5 CPU, were utilized during the F
Tesla K80 accelerator has two Tesla GK210 GPUs with 2496 cores,
process. Though the CPU was similar, the GPU hardware 560 MHz base clock, 12 GB memory
was different: GeForce GTX Titan X, GTX 980, GTX 1080, g 3584 cores, 1189 MHz base clock, 16 GB memory
Tesla K80, M40, and P100. h
3072 cores, 948 MHz base clock, 12 GB memory

13
Machine Translated by Google

24 Cognitive Computing (2021) 13:1–33

a Stacked Autoencoder (CPU) b Stacked Autoencoder (GPU-GTX Titan X)

256 Gradient forward 64 256 64
Gradient forward
128 1 6 12 1 6 12 32
128 32

64 64 16
16 8
32 32
4
16 8 16
two

8 8
4 1
4 4 0.5
two
two two
0.25
1 1 1 0.125

c d
LSTM (CPU) LSTM (GPU-GTX980)
16384 1024 tensor flow
1248 16 32 64 128 256 512 1024
4096
256
1024 Theano

256 64
torch
64 16
16
MCTs
4
4

1 1 MX Net
F
and
LSTM (GPU-GTX1080) LSTM (GPU-Tesla K80)
4096 16384
64 128 256 512 1024 64 128 256 512 1024
1024 4096

1024
256
256
64
64
16
16
4 4

1 1

Fig. 5 Benchmarking stacked autoencoder or DA (a, b) and LSTM (cf) in CPU and GPU platforms. The numbers in (a, c) denote the number of
CPU threads employed in the benchmarking process, and in (df) denote the batch size. In case of DA the batch size was 64

Still a large portion of the pattern analysis is done using Theano, and Torch) [264] clearly suggest that network
CNN; therefore, we further focused on CNN and investigated training process is much accelerated in GPUs in comparison
how the leading tools performed and scaled in training to CPUs. Moreover, another important message is that, all
different CNN networks in different GPU platforms. Time GPUs are not the same and all tools don't scale up at the
speedup of GPU over CPU is considered as a metric for same rate. The time required to train a neural network
this purpose. The individual values are calculated using the strongly depends on which DL frame work is being used.
benchmark scripts of DeepMark [259] on experimental As for the hardware platform, the Tesla P100 accelerator
setup number 3 (Table 8) for one training iteration per provides the best speedup with Tesla M40 being the
batch. The time needed to execute a training iteration per second and Tesla K80 being the last among the three. In
batch equals the time taken to complete a forward CPUs, TensorFlow achieves the least training time
propagation operation followed by a backpropagation indicating a quicker training of the network.
operation. Figure 6 summa rizes the training time per In GPUs, Caffe usually provides the best speedup over
iteration per batch for both CPU and GPUs (left y-axis) and CPU but TensorFlow and Torch perform faster training
the corresponding GPU speedup over CPU (right y-axis). than Caffe. Though TensorFlow and Torch have similar
These findings for four different CNN network models performances (indicated by the height of the lines), Torch
(ie Alexnet [92], GoogLeNet [94], Overfeat [263], and VGG slightly outperforming TensorFlow in most of the networks.
[93]) available in four tools (ie Caffe, TensorFlow, Finally, most of the tools outperform Theano.

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 25

Open Issues and Future Perspectives identify below shortcomings and bottlenecks of the popular
methods, open research questions, and challenges and
The brain has the capability to recognize and understand outline possible directions which require attention in the
patterns almost instantaneously. Over several decades, near future.
scientists have been trying to decode the biological First of all, DL methods usually require large datasets.
mechanism of natural pattern recognition that takes place Though the computing cost is declining with increasing
in the brain and translate those principles into AI systems. computational power and speed, it is not worthwhile to
The increasing knowledge about the brain's information apply DL methods in cases of small to moderate sized
processing policies enabled this analogy to be adopted and datasets. This is particularly so as considering that many of
implemented in computing systems. Recent technological the DL methods perform continuous geometric
breakthroughs, seamless integration of diverse techniques, transformations of one data manifold to another with an
better understanding of the learning systems, decline of assumption that there exist learnable transfer functions
computing costs, and expansion of computational power which can per form the mapping [266]. However, in cases
empowered computing systems to reach human-level when the relationships among the data are causal or very
computation in certain scenarios [265]. Nonetheless, many complex to be learned by the geometric transformations,
of these methods require improvements. Though admittedly, the DL methods fail regardless of the size of the dataset
there are distinctions on how a DL-based method can be [267]. Also, interpreting high-level outcomes of DL methods
used and applied on biological data, however, the common is difficult due to inadequate in-depth understanding of the
open issues and challenges are equally applicable and importantDL
fortheories which
biological causes
data. We many of such models to be considered as 'Bla

a alexnet b GoogLeNet
CPUs GPU speed-up CPUs GPU speed-up
8192 100 32768 100

4096 16384

2048 75 8192 75

1024 4096
512 fifty 2048 fifty

256 1024
Tesla P100 Tesla P100
128 Tesla K80 25 Tesla K80 25
512
Tesla M40 Tesla M40
64 256
32 0 128 0

Torch Torch Torch Torch Torch Torch

Theean or Theean or Theean or Theean or Theean or Theean or
Caffe Ten so rflo w Caffe Ten so rflo w Caffe Ten so rflo w Caffe Ten so rflo w Caffe Ten so rflo w Caffe Ten so rflo w

c Overfeat d VGG
CPUs GPU speed-up CPUs GPU speed-up
16384 100 65536 100

8192 32768

4096 75 16384 75
8192
2048
4096
1024 fifty fifty
2048
512
Tesla P100 1024 Tesla P100
256 Tesla K80 25 512 Tesla K80 25
Tesla M40 Tesla M40
128 256
64 0 128 0

Torch Torch Torch Torch Torch Torch

Theean or Theean or Theean or Theean or Theean or Theean or
Caffe Ten so rflo w Caffe Ten so rflo w Caffe Ten so rflo w Caffe Ten sor flo w Caffe Ten so rflo w Caffe Ten so rflo w

Fig. 6 The speedup of CNN training in different DL tools across various GPUs in comparison to CPU. The reported values were calculated for a
batch size of 128, except for VGG for which the batch size was 64

13
Machine Translated by Google

26 Cognitive Computing (2021) 13:1–33

[268]. Moreover, like many other ML techniques, DL is also DL tools (eg, Keras, Chainer, Lasagne) and architectures
susceptible to misclassifcation [269] and overclassifcation (eg, DBN) which need to be benchmarked providing the
[270]. users with a more comprehensive list to choose. Also, the
Additionally, the ability to exploit the full benefits offered currently available benchmarks are mostly performed on
by open access data repositories, in terms of data sharing non-biological data, and their scalability to biological data is
and reuse, is often hampered by the lack of unified reporting poor; thus, specialized benchmarking on biological data are
data standards and non-uniformity of reported information needed.
[271]. Data provenance, curation, and annotation of these In order to derive insights from an image, a sequence or
biological big data are huge challenges too [272]. a signal analysis problem, a selected DL algorithm using a
Furthermore, except for very few large enterprises, the library or a tool (eg, TensorFlow, Keras, PyTorch, etc.) may
power of distributed and parallel computation through cloud need to integrate with a big data framework (eg , Hadoop,
computing remains largely unexplored for the DL techniques. Spark, etc). In such cases, troubleshooting in the model and
Due to the fact that the DL techniques require retraining for debugging the code may be very challenging for the system
different datasets, repeated training becomes a bottleneck designer due to the parallel execution of multiple threads
for cloud computing environments. Also, in such distributed which may not always execute in an orderly fashion. The
environments, data privacy and security concerns are still lack of documentation and model transparency of these
prevailing [273], and real-time process ing capability of libraries may make it impossible for the project manager to
experimental data is underdeveloped [274]. estimate eforts required in successful completion of a project.

To mitigate the shortcomings and address the open

issues, the existing theoretical foundations of the DL
methods need to be improved. The DL models are required Conclusion
not only to be able to describe specifc data but also
generalize them on the basis of experimental data which is The diverse biological data coming from different application
crucial to quantify the performances of individual NN models domains are multimodal, multidimensional, and complex in
[275]. These improvements should take place in several nature. At present, a huge amount of such data is publicly
directions and address issues like quantitative assessment available. The affordable access to these data came with a
of individual model's learning efciency and associated huge challenge to analyze and recognize patterns in them
computational complexity in relation to well-defined which require sophisticated ML tools to do the job. As a
parameter tuning strategies, the ability to generalize and result, many ML-based analytical tools have been developed
topologically self organize based on data-driven properties. and reported over the last decades and this process has
Also, to facilitate intuitive and less cumbersome interpretation been greatly facilitated by the decrease of computational
of the analysis results, novel tools for data visualization costs, increase of computing power, and availability of cheap
should be incorporated in the DL frameworks. storage. With the help of these learning techniques,
Recent developments in combined methods pertaining machines have been trained to understand and decipher
to deep reinforcement learning (deep RL) have been complex patterns and interactions of variables in biological
popularly applied to many application domains (for a review data. To facilitate a wider dissemination of DL techniques
on deep RL, see [276]). However, deep RL methods have applied to biological data and serve as a reference point,
not yet been applied to biological pattern recognition problems. this article provides a comprehensive survey of the literature
For example, analyzing and aggregating dynamically on those techniques' application on biological data and the
changing patterns in biological data coming from multiple relevant open-access data repositories. It also lists existing
levels could help to remove data redundancy and discover open-source tools and frameworks implementing various DL
novel biomarkers for disease detection and prevention. methods and compares these tools for their popularity and
Also, novel deep RL methods are needed to reduce the performance. Finally, it concludes by pointing out some
currently required large set of labeled training data. open issues and proposing some future perspectives.
Renewing eforts are required for standardization,
annotation, curation, and provenance of data and their Acknowledgments The authors would like to thank the members of
the acslab (http://www.acslab.info/) for valuable discussions.
sources along with ensuring uniformity of information among
the different repositories. Additionally, to keep up with the Author Contributions This work was carried out in close collaboration
rapidly growing big data, powerful and secure computational among all authors. MM and MSK conceived the idea, developed the
infrastructures in terms of distributed, cloud, and parallel method and experiments, analyzed the obtained data, and wrote the
computing tailored to such well-understood learning manuscript. TMM and AH edited the manuscript. All authors have
contributed to, seen, and approved the paper.
mechanisms are badly needed. Lastly, there are many other popular

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 27

Compliance with Ethical Standards 16. Hopfeld JJ. Neurons with graded response have collective
computational properties like those of two-state neurons.
Conflict of Interest The authors declare that the research was PNAS. 1984;81(10):3088–92.
17. Ackley DH, Hinton GE, Sejnowski TJ. A learning algorithm for
conducted in the absence of any commercial or financial relationships
Boltzmann machines. Cogn Sci. 1985;9(1):147–69.
that could be constructed as a potential conflict of interest.
18. Salakhutdinov R, Mnih A, Hinton G. Restricted Boltzmann
machines for collaborative fltering. In: Proc. ICML; 2007. p. 791–
Ethical Approval This article does not contain any studies with human
798.
participants or animals.
19. Maass W. Networks of spiking neurons: The third generation of
neural network models. Neural Netw. 1997;10(9):1659–71.
Informed Consent As this article does not contain any studies with
20. Heckerman D. A Tutorial on learning with Bayesian net works.
human participants or animals, the informed consent is not applicable
In: Jordan MI, editor. Learning in Graphical Models.
89. SpringerNetherlands; 1998. p. 301–354.
Open Access This article is licensed under a Creative Commons 21. Cortes C, Vapnik V. Support-vector networks. MachLearn.
Attribution 4.0 International License, which permits use, sharing, 1995;20(3):273–97.
adaptation, distribution and reproduction in any medium or format, as 22. Yuan GX, Ho CH, Lin CJ. Recent advances of large-scale lin ear
long as you give appropriate credit to the original author(s) and the classification. IEEE Proc. 2012;100(9):2584–603.
source, provide a link to the Creative Commons license, and indicate 23.Fisher RA. The use of multiple measurements in taxonomic
if changes were made. The images or other third party material in this problems. Ann Eugenics. 1936;7(2):179–88.
article are included in the article's Creative Commons license, unless 24. Uysal I, Güvenir HA. An overview of regression techniques for
otherwise indicated in a credit line to the material. If material is not knowledge discovery. Knowl Eng Rev. 1999;14(4):319–40.
included in the article's Creative Commons license and your intended 25. Rish I. An empirical study of the naive Bayes classifer. In: Proc.
use is not permitted by statutory regulation or exceeds the permitted 2001 IJCAI. vol. 3; 2001. p. 41–46.
use, you will need to obtain permission directly from the copyright 26. Cover T, Hart P. Nearest neighbor pattern classification. IEEE
TransInf Theory. 1967;13(1):21–7.
holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
27. Rabiner L, Juang B. An introduction to hidden Markov models.
IEEE ASSP Mag. 1986;3(1):4–16.
28. Kohavi R, Quinlan JR. Data mining tasks and methods:
classification: decision-tree discovery. In: Klösgen W, Zytkow
references JM, editors. Handbook of data mining and knowledge discovery.
New York, NY, USA: Oxford University Press, Inc.; 2002. p. 267–
1. Coleman W. Biology in the nineteenth century: problems of form, 276.
function, and transformation. cambridge; New York: Cambridge 29. Hinton GE. Connectionist learning procedures. Artif Intell.
University Press; 1977. 1989;40(1–3):185–234.
2. Magner LN. A history of the life sciences. 3rd ed. New York: M. 30. Dempster AP, Laird NM, Rubin DB. Maximum likelihood of
Decker; 2002. incomplete data via the EM algorithm. JR Stat Soc Series B
3. Brenner S. History of science. The revolution in the life sciences. Methodol. 1977;39(1):1–38.
Science. 2012;338(6113):1427–8. 31. Tishby N, Pereira FC, Bialek W. The information bottleneck
4. Shendure J, Ji H. Next-generation DNA sequencing. Nat Biotech method. In: Proc. 37th ACCCC. 1999. p. 368-377.
nol. 2008;26(10):1135–45. 32. Kohonen T. Self-organized formation of topologically correct
5. Metzker ML. Sequencing technologies the next generation. Nat feature maps. Biol Cybernet. 1982;43(1):59–69.
Rev Genet. 2010;11(1):31–46. 33. Agrawal R, Imieliÿski T, Swami A. Mining association rules
6. Vadivambal R, Jayas DS. Bio-imaging: principles, techniques, between sets of items in large databases. In: Proc. ACM SIG
and applications. Boca Raton, FL: CRC Press, Taylor & Francis MOD '93. 1993. p. 207–216.
Group; 2016. 34. Gordon AD. A review of hierarchical classification. JR Stat Soc
7. Poldrack RA, Farah MJ. Progress and challenges in testing the Series A General. 1987;150(2):119–37.
human brain. Nature. 2015;526(7573):371–9. 35. Ball G, Hall D. ISODATA, a novel method of data analysis and
8. Lebedev MA, Nicolelis MAL. Brain-machine interfaces: from basic pattern classifcation. Stanford, CA: Stanford Research Institute;
science to neuroprostheses and neurorehabilitation. Phys Rev. 1965.
2017;97(2):767–837. 36. Dunn JC. A fuzzy relative of the ISODATA process and its use in
9. Quackenbush J. Extracting biology from high-dimensional detecting compact well-separated clusters. J Cybernet.
biological data. J Exp Biol. 2007;210:1507–17. 1973;3(3):32–57.
10. Mattmann CA. Computing: A vision for data science. Nature. 37. John AH. Clustering algorithms. New York, NY, USA: John Wiley
2013;493(7433):473–5. & Sons, Inc.; 1975.
11. Li Y, Chen L. Big biological data: challenges and opportunities. 38. Kriegel HP, Kroger P, Sander J, Zimek A. Density-based cluster
Genomics Proteomics Bioinformatics. 2014;12(5):187–9. eng. WIRES Data Min Knowl. 2011;1(3):231–40.
12. Marx V. Biology: The big challenges of big data. Nature. 39. Ankerst M, Breunig MM, Kriegel HP, Sander J. OPTICS: ordering
2013;498(7453):255–60. points to identify the clustering structure. In: Proc. ACM
13. Tarca AL, Carey VJ, Chen Xw, Romero R, Draghici S. Machine SIGMOD'99. 1999. p. 49–60.
learning and its applications to biology. PLoS Comput Biol. 40. Horgan RP, Kenny LC. Omic technologies: genomics,
2007;3(6):e116. transcriptomics, proteomics and metabolomics. Obstet Gynecol.
14. Hopfeld JJ. Artificial neural networks. IEEE Circuits Devices Mag. 2011;13(3):189–95.
1988;4(5):3–10. 41. Libbrecht MW, Noble WS. Machine learning applications in
15. Hecht-Nielsen R. Theory of the backpropagation neural net genetics and genomics. Nat Rev Genet. 2015;16(6):321–32.
work. In: Proc. IJCNN 1989; 1989. p. 593–605.

13
Machine Translated by Google

28 Cognitive Computing (2021) 13:1–33

42. Lemm S, Blankertz B, Dickhaus T, Müller KR. Introduction to 63. Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge, USA:
machine learning for brain imaging. NeuroImage. 2011;56(2):387– MIT Press; 2016.
99. 64. Saxe AM, McClelland JL, Ganguli S. Exact solutions to the nonlinear
43. Erickson BJ, Korfatis P, Akkus Z, Kline TL. Machine learning for dynamics of learning in deep linear neural nets. In: Proc.
medical imaging. RadioGraphics. 2017;37(2):505–15. ICLR; 2014. p. 1–22.
44. Kan A. Machine learning applications in cell image analysis. 65. Schmidhuber J. Deep learning in neural networks: an overview.
Immunol Cell Biol. 2017;95(6):525–30. Neural Netw. 2015;61:85–117.
45. Vidaurre C, Sannelli C, Müller KR, Blankertz B. Machine-learning- 66. Zeng D, Zhao F, Shen W, Ge S. Compressing and accelerating
based coadaptive calibration for brain-computer interfaces. neural network for facial point localization. Cogn Comput.
Neural Computation. 2010;23(3):791–816. 2018;10(2):359–67.
46. Mala S, Latha K. Feature selection in classification of eye movements 67. Salakhutdinov R, Hinton GE. Deep Boltzmann machines. In: Proc.
using electrooculography for activity recognition. Com Math Met AISTATS2009; 2009. p. 448–455.
Med. 2014 Dec;2014. 68. Geman S, Geman D. Stochastic relaxation, Gibbs distributions, and
47. Mahmud M. Vassanelli S. Processing and analysis of multichannel the Bayesian restoration of images. IEEE Trans Pattern Anal Mach
extracellular neuronal signals: state-of-the-art and challenges. Intell. 1984;6(6):721–41.
Front Neurosci. 2016. p. 10. 69. Fischer A, Igel C. An introduction to restricted Boltzmann machines.
48. Lecun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015 In: Proc. CIARP 2012; 2012. p. 14–36.
5;521(7553):436–444. 70. Desjardins G, Courville AC, Bengio Y. On training deep Boltz mann
49. Yahaya SW, Lotf A, Mahmud M. A consensus novelty detection machines. CoRR. 2012; abs/1203.4416.
ensemble approach for anomaly detection in activities of daily living. 71. Tieleman T. Training restricted Boltzmann machines using
Applied Soft Computing. 2019;83:105613. approximations to the likelihood gradient. In: Proc. ICML; 2008. p.
50. Fabietti M, Mahmud M, Lotf A, Averna A, Guggenmo D, Nudo R, et 1064–1071.
al. Neural network-based artifact detection in local feld potentials 72. Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS. Deep learning
recorded from chronically implanted neural probes. for visual understanding: A review. Neurocomputing. 2016;187:27–
In: Proc. IJCNN; 2020. p. 1–8. 48.
51. Mahmud M, Kaiser MS, Hussain A, Vassanelli S. Applications of 73. Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep
deep learning and reinforcement learning to biological data. belief nets. Neural Comput. 2006;18(7):1527–54.
IEEE Trans Neural Netw Learn Syst. 2018;29(6):2063–79. 74. Bi X, Zhao X, Huang H, Chen D, Ma Y. Functional brain net work
52. Mahmud M, Kaiser MS, Hussain A. Deep learning in mining biological classification for Alzheimer's disease detection with deep features
data. arXiv:200300108 [cs, q-bio, stat]. 2020 Feb;p. and extreme learning machine. Cogn Comput. 2020;12:513–527.
1–36. ArXiv: 2003.00108. Available from: http://arxiv.org/
abs/2003.00108. 75. Ravi D, Wong C, Deligianni F, Berthelot M, et al. Deep learning for
53. Dey N, Rajinikanth V, Fong SJ, Kaiser MS, Mahmud M. health informatics. IEEE J Biomed Health Inform. 2017;21(1):4–21.
Social-group-optimization assisted Kapur's entropy and mor
phological segmentation for automated detection of COVID-19 76. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA.
infection from computed tomography images. Cogn Comput. Stacked denoising autoencoders: learning useful representations in
2020;12(5):1011–1023. a deep network with a local denoising criterion. J Mach Learn Res.
54. Aradhya MVN, Mahmud M, Guru D, S Agrawal B, Kaiser MS. 2010;11:3371–408.
One shot cluster based approach for the detection of COVID-19 77. Baldi P. Autoencoders, unsupervised learning and deep architecture
from chest X-ray images. Cogn Comput. 2020;p. 1–8. tours. In: Proc. ICUTLW; 2012. p. 37–50.
55. Noor MBT, Zenia NZ, Kaiser MS, Mahmud M, Al Mamun S. 78. Ranzato M, Poultney C, Chopra S, Cun YL. Efficient learning of
Detecting neurodegenerative disease from MRI: a brief review on a sparse representations with an energy-based model. In: Proc.
deep learning perspective. In: Liang P, Goel V, Shan C, eds. NIPS; 2006. p. 1137–1144.
Brain Inform. Cham: Springer International Publishing; 2019. p. 115– 79. Kingma DP, Welling M. Auto-encoding variational bayes.
125. CoRR. 2014; abs/1312.6114.
56. Ali HM, Kaiser MS, Mahmud M. Application of convolutional neural 80. Rifai S, Vincent P, Muller X, Glorot X, Bengio Y. Contractive auto-
network in segmenting brain regions from MRI data. In: Liang P, encoders: explicit invariance during feature extraction.
Goel V, Shan C, eds. Brain Inform. Cham: Springer International In: Proc. ICML; 2011. p. 833–840.
Publishing; 2019. p. 136–146. 81. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D,
57. Miah Y, Prima CNE, Seema SJ, Mahmud M, Kaiser MS. Perform Ozair S, et al. Generative adversarial nets. In: Advances in neural
comparison of machine learning techniques in identifying dementia information processing systems; 2014. p. 2672–2680.
from open access clinical datasets. In: Proc. ICATION. 82. Isola P, Zhu JY, Zhou T, Efros AA. Image-to-image translation with
Springer, Singapore; 2020. p. 79–89. conditional adversarial networks. In: Proc.IEEE CVPR; 2017. p.
58. Watkins J, Fabietti M, Mahmud M. SENSE: a student performance 1125–1134.
quantifer using sentiment analysis. In: Proc. IJCNN; 2020. p. 1–6. 83. Wang Z, Healy G, Smeaton AF, Ward TE. Use of neural signals to
evaluate the quality of generative adversarial net work performance
59. Rabby G, Azad S, Mahmud M, Zamli KZ, Rahman MM. TeKET: a in facial image generation. Cogn Comput. 2020;12(1):13–24.
tree-based unsupervised keyphrase extraction technique. Cogn
Comput. 2020;12(5):811–33. 84. Pascanu R, Gulcehre C, Cho K, Bengio Y. How to construct deep
60. Orojo O, Tepper J, McGinnity TM, Mahmud M. A multi-recur rent recurrent neural networks. In: Proc. ICLR; 2014. p. 1–13.
network for crude oil price prediction. In: Proc. SSCI; 2019. p. 2940– 85. Elman JL. Finding structure in time. Cognitive Sci. 1990;14(2):179–
2945. 21111.
61. Ching T, et al. Opportunities and obstacles for deep learning in 86. Schuster M, Paliwal KK. Bidirectional recurrent neural net works.
biology and medicine. bioRxiv. 2017;p. 142760. IEEE Tran Signal Processes. 1997;45(11):2673–81.
62. Bengio Y. Learning deep architectures for AI. Found Trends Mach 87. Hochreiter S, Schmidhuber J. Long short-term memory. Neural
Learn. 2009;2(1):1–127. Comput. 1997;9(8):1735–80.

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 29

88. Lipton ZC, Berkowitz J, Elkan C. A critical review of recurring neural 109. Kamnitsas K, Ledig C, Newcombe VFJ, Simpson J, et al. Efficient multi-
networks for sequence learning. CoRR. 2015 May; CoRR: 1506.00019. scale 3D CNN with fully connected CRF for accurate brain lesion
segmentation. Med Image Anal. 2017;36:61–78.
89. Ma Y, Peng H, Khan T, Cambria E, Hussain A. Sentic LSTM: a hybrid 110. Lu N, Li T, Ren X, Miao H. A deep learning scheme for motor imagery
network for targeted aspect-based sentiment analysis. classifcation based on restricted Boltzmann machines.
Cogn Comput. 2018;10(4):639–50. IEEE Trans Neural Syst Rehabilitation Eng. 2016; 99
90. Wiatowski T, Bölcskei H. A mathematical theory of deep con volutional 111. Yang H, Sakhavi S, Ang KK, Guan C. On the use of convolutional neural
neural networks for feature extraction. IEEE TransInf Theory. networks and augmented CSP features for multi class motor imagery
2017;64(3):1845–66. of EEG signals classification. In: Proc. 37th IEEE EMBC; 2015. p. 2620–
91. LeCun Y, Bengio Y. Convolutional networks for images, speech, and 2623.
time series. In: Arbib MA, editor. The handbook of brain theory and 112. Tabar YR, Halici U. A novel deep learning approach for classifcation of
neural networks. Cambridge, MA, USA: MIT Press; 1998. p. 255–258. EEG motor imagery signals. J Neural Eng. 2017;14(1):016003.

92. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep 113. Sakhavi S, Guan C, Yan S. Parallel convolutional-linear neural network
convolutional neural networks. In: Proc. NIPS; 2012. p. 1097–1105. for motor imagery classifcation. In: Proc. EUSIPCO; 2015. p. 2786–
2790.
93. Simonyan K, Zisserman A. Very deep convolutional networks for large- 114. Li K, Li X, Zhang Y, Zhang A. Affective state recognition from EEG with
scale image recognition. CoRR. 2014; abs/1409.1556. deep belief networks. In: Proc. BIBM; 2013. p. 305–310.
94. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going
deeper with convolutions. In: Proc. CVPR2015; 2015. p. 1–9. 115. Jia X, Li K, Li X, Zhang A. A novel semi-supervised deep learning
framework for affective state recognition on EEG signals. In: Proc.
95. Heinsfeld AS, Franco AR, Craddock RC, Buchweitz A, Meneguzzi F. IEEE BIBE; 2014. p. 30–37.
Identifcation of autism spectrum disorder using deep learning and the 116. Tripathi S, Acharya S, Sharma R, Mittal S, et al. Using deep and
ABIDE dataset. NeuroImage: Clin. 2018;17:16 – 23. convolutional neural networks for accurate emotion classification on
DEAP dataset. In: Proc. 29th IAAI; 2017. p. 4746–4752.
96. Kuang D, He L. Classification on ADHD with deep learning. 117. Chen G, Zhu Y, Hong Z, Yang Z. EmotionalGAN: generating ECG to
In: Proc. CCBD; 2014. p. 27–32. enhance emotion state classification. In: Proc. AICS 2019. 2019. p.
97. Hosseini Asl E, Gimelfarb GL, El-Baz A. Alzheimer's disease diagnostics 309-313.
by a deeply supervised adaptive 3D convolutional network. CoRR. 118. Mirowski P, Madhavan D, LeCun Y, Kuzniecky R. Classifcation of
2016; abs/1607.00556. patterns of EEG synchronization for seizure prediction. Clin
98. Suk HI, Lee SW, Shen D. Hierarchical feature representation and Neurophysiol. 2009;120(11):1927–40.
multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage. 119. Jirayucharoensak S, Pan-Ngum S, Israsena P. EEG-based emotion
2014;101:569–82. recognition using deep learning network with principal component
99. Li F, Tran L, Thung KH, Ji S, Shen D, Li J. A robust deep model for based covariate shift adaptation. Scientific World J. 2014;p. 1–10.
improved classifcation of AD/MCI patients. IEEE J Biomed Health
Inform. 2015;19(5):1610–6. 120. Wu Z, Ding X, Zhang G. A novel method for classifcation of ECG
100. Havaei M, Guizard N, Larochelle H, Jodoin PM. Deep learning trends arrhythmias using deep belief networks [journal article]. J Comp Intel
for focal brain pathology segmentation in MRI. In: Holzinger A, editor. Appl. 2016;15:1650021.
Machine learning for health informatics: state-of-the-art and future 121. Yan Y, Qin X, Wu Y, Zhang N, Fan J, et al. A restricted Boltz mann
challenges. Cham: Springer; 2016. p. 125–148. machine based two-lead electrocardiography classification.
In: Proc. BSN; 2015. p. 1–9.
101. Fritscher K, Raudaschl P, Zafno P, Spadea MF, Sharp GC, et al. Deep 122. Atzori M, Cognolato M, Müller H. Deep learning with convolutional
neural networks for fast segmentation of 3D medical images. In: Proc. neural networks applied to electromyography data: a resource for the
MICCAI; 2016. p. 158–165. classifcation of movements for prosthetic hands.
102. Iqbal T, Ali H. Generative adversarial network for medical images (MI- Front Neurorobot. 2016;10:9.
GAN). J Med Syst. 2018;42(11):231. 123. Huanhuan M, Yue Z. Classification of electrocardiogram signals with
103. Ciresan D, Giusti A, Gambardella L, Schmidhuber J. Deep neural nets DBN. In: Proc. IEEE CSE; 2014. p. 7–12.
segment neuronal membrane in electron microscopy images. 124. Wang S, Peng J, Ma J, Xu J. Protein secondary structure prediction
In: Proc. NIPS; 2012. p. 2843–2851. using deep convolutional neural felds. Scientific Reports.
104. Stollenga MF, Byeon W, Liwicki M, Schmidhuber J. Parallel multi- 2016 Nov;6(1).
dimensional LSTM, with application to fast biomedical volumetric image 125. Alipanahi B, Delong A, Weirauch MT, Frey BJ. Predicting the sequence
segmentation. In: Proc. NIPS; 2015. p. 2980–88. specifcities of DNA- and RNA-binding proteins by deep learning. Nature
Biotechnol. 2015;33(8):831–8.
105. Kleesiek J, Urban G, Hubert A, Schwarz D, Maier-Hein K, Bendszus M, 126. Chen G, Tsoi A, Xu H, Zheng WJ. Predict effective drug com bination
et al. Deep MRI brain extraction: A 3D convolutional neural network for by deep belief network and ontology fngerprints. J Biomed Inform.
skull stripping. NeuroImage. 2016;129:460–9. 2018;85:149–54.
106. Cho J, Lee K, Shin E, Choy G, Do S. Medical image deep learning with 127. Denas O, Taylor J. Deep modeling of gene expression regulation in an
hospital PACS dataset. CoRR. 2015; abs/1511.06348. Erythropoiesis model. In: Proc. ICMLRL; 2013. p. 1–5.
107. Ngo T, et al. Combining deep learning and level set for the auto mated 128. Kelley DR, Snoek J, Rinn JL. Basset: learning the regulatory code of the
segmentation of the left ventricle of the heart from cardiac cine mr. Med accessible genome with deep convolutional neural networks. Genome
Image Anal. 2017;35:159–71. Res. 2016;26(7):990–9.
108. Ciresan D, Giusti A, Gambardella L, Schmidhuber J. Mitosis detection in 129. Zhou J, Troyanskaya OG. Predicting effects of noncoding variables with
breast cancer histology images with deep neural networks. In: Proc. deep learning-based sequence model. NatureMethods. 2015;12(10):931–
MICCAI; 2013. p. 411–4188. 4.

13
Machine Translated by Google

30 Cognitive Computing (2021) 13:1–33

130. Marouf M, et al. Realistic in silico generation and augmentation of 151. Ye F, Zhu F, Fu Y, Shen B. ECG generation with sequence generative
single-cell RNA-seq data using generative adversarial networks. Nat adversarial nets optimized by policy gradient. IEEE Access.
Commun. 2020;11:166. 2019;7:159369–159378.
131. Lee T, Yoon S. Boosted categorical restricted boltzmann machine for 152. Luo Y, Lu BL. EEG data augmentation for emotion recognition using a
computational prediction of splice junctions. In: Proc. ICML; 2015. p. conditional Wasserstein GAN. In: 2018 IEEE EMBC; 2018. p. 2535–
2483–2492. 2538. ISSN: 1558-4615.
132. Zeng H, Edwards MD, Liu G, Giford DK. Convolutional neural network 153. YouS, et al. Unsupervised automatic seizure detection for focal onset
architectures for predicting DNA-protein binding. Bioinformatics. seizures recorded with behind-the-ear EEG using an abnormally-
2016;32(12):121–7. detecting generative adversarial network. Comput Methods Programs
133. Park S, Min S, Choi H, Yoon S. deepMiRGene: Deep neural network Biomed. 2020;p. 105472.
based precursor microRNA prediction. CoRR. 2016; abs/1605.00017. 154. Jiao Y, Deng Y, Luo Y, Lu BL. Driver sleepiness detection from EEG
and EOG signals using GAN and LSTM networks. neuro computation.
134. Lee B, Baek J, Park S, Yoon S. deepTarget: end-to-end learning 2020;408:100-111.
framework for miRNA target prediction using deep recurrent neural 155. Singh P, Pradhan G. A new ECG denoising framework using generative
networks. CoRR. 2016; abs/1603.09123. adversarial network. IEEE/ACM Trans Comput Biol Bioinform. 2020;p.
135. Li H. A template-based protein structure reconstruction method using 3114–3128.
DA learning. J Proteomics Bioinform. 2016;9(12). 156. Wang X, Ghasedi Dizaji K, Huang H. Conditional generative adversarial
136. Ibrahim R, Yousri NA, Ismail MA, El-Makky NM. Multi-level gene/ network for gene expression inference. Bioinformatics. 2018
MiRNA feature selection using deep belief nets and active learning. 09;34(17):i603–i611.
In: Proc. IEEE EMBC; 2014. p. 3957–3960. 157. Pan X, Shen HB. RNA-protein binding motifs mining with a new hybrid
137. Chen L, Cai C, Chen V, Lu X. Trans-species learning of cellular deep learning based cross-domain knowledge integration approach.
signaling systems with bimodal deep belief networks. Bioinformatics. BMC Bioinform. 2017;18(1).
2015 Sep;31(18):3008–3015. 158. Jiang X, Zhao J, Qian W, Song W, Lin GN. A generative advisory
138. Danaee P, Ghaeini R, Hendrix DA. A deep learning approach for network model for disease gene prediction with RNA-seq data. IEEE
cancer detection and relevant gene identification. In: Proc. Pac. Access. 2020;8:37352–60.
Symp. Biocomput.. vol. 22; 2016. p. 219–229. 159. Zhao L, Wang J, Pang L, Liu Y, Zhang J. GANsDTA: predicting drug-
139. Li Y, Fauteux F, Zou J, Nantel A, Pan Y. Personalized prediction of target binding afnity using GANs. Front Genet. 2020;10:1243.
genes with tumor-causing somatic mutations based on multi-modal
deep Boltzmann machine. Neurocomputing. 2019;324:51–62. 160. Publisher. Sharing data. Nat Cell Biol. 2009 11;11(11):1273.
161. Lord PW, et al. Large-scale data sharing in the life sciences: data
140. Zhang T, Zhang L, Payne PRO, Li F. Synergistic drug combination standards, incentives, barriers and funding models (the 'Joint Data
prediction by integrating multi-omics data in deep learning models. Standards Study'). 2005. Available from: http://www.nesc.
arXiv:181107054 [cs, q-bio, stat]. 2018 Nov;ArXiv: 1811.07054. ac.uk/technical_papers/UKeS-2006-02.pdf.
Available from: http://arxiv.org/abs/1811.07054. 162. Martone ME, Ellisman MH, Sosinsky GE, Gupta A, Tran J, Wong W,
141. Huang Y, Gulko B, Siepel A. Fast, scalable prediction of deleterious et al. Cell Centered Database. UC San Diego Library Digital
noncoding variants from functional and population genomic data. Collections. 2017. https://doi.org/10.6075/J0S180PX.
Nature Genet. 2017;49:618–24. 163. Ellisman M, et al. Cell Image Library. 2016. Available from: http://
142. Le EPV, Wang Y, Huang Y, Hickman S, Gilbert FJ. Artificial intelligence www.cellimagelibrary.org/. Accessed on 4 Jan 2020.
in breast imaging. Clin Radiol. 2019;74(5):357–66. 164. ERIC. EuroBioimaging. 2016. Available from: http://www.
143. Yi X, Walia E, Babyn P. Generative adversarial network in medical eurobioimaging.eu/. Accessed on 4 Jan 2020.
imaging: A review. Med Image Anal. 2019;58:101552. 165. Karkow W. HAPS Histology Image Database. 2008. Available from:
144. Sandfort V, Yan K, Pickhardt PJ, Summers RM. Data augmentation http://hapshistology.wikifoundry.com/. Accessed on 23 Jan 2017.
using generative adversarial networks (CycleGAN) to improve
generalizability in CT segmentation tasks. Sci Rep. 2019;9(1):1–9. 166. Dundee U. IDR: Image Data Resource. 2016. Available from:
https://idr.openmicroscopy.org/.
145. Armanious K, et al. MedGAN: Medical image translation using GANs. 167. Kistler M. SMIR Full Body CT. SMIR. 2017. Available from: https://
Computed Med Imaging Graph. 2020;79:101684. doi.org/10.1007/BF00337288.
146. Uemura T, et al. GAN-based survival prediction model from CT images 168. Arkansas for Medical Sciences U. The Cancer Imaging Archive.
of patients with idiopathic pulmonary fbrosis. In: Chen PH, Deserno 2015. Available from: https://www.cancerimagingarchive.
TM, editors. Medical Imaging 2020: Imaging Information for net/. Accessed on 4 Jan 2020.
Healthcare, Research, and Applications. vol. 11318. 169. Marinelli RJ, et al. The Stanford Tissue Microarray Database.
SPIE; 2020. p. 354–359. 2007. Available from: https://tma.stanford.edu. Accessed on 23 Jan
147. Thambawita V, Hammer HL, Riegler M, Halvorsen P. GANEx: A 2017.
complete pipeline of training, inference and benchmarking GAN 170. University of California SB. UCSB Bio-Segmentation Bench mark
experiments. In: Proc. 2019 CBMI. 2019. p. 1–4. dataset. 2008. Available from: https://bioimage.ucsb.edu/
148. Halicek M, et al. Conditional generative adversarial network for research/bio-segmentation. Accessed on 23 Jan 2017.
synthesizing hyperspectral images of breast cancer cells from 171. ABIDE. Autism Brain Imaging Data Exchange. 2012. Available from:
digitized histology. In: Tomaszewski JE, Ward AD, editors. Medical https://goo.gl/n694sN. Accessed on 4 Jan 2020.
Imaging 2020: Digital Pathology, Vol. 11320. SPIE; 2020. p. 198–205. 172. Milham MP. ADHD200. 2011. Available from: https://doi.
org/10.1007/BF00337288. Accessed on 4 Jan 2020.
149. Zhu F, Ye F, Fu Y, Liu Q, Shen B. Electrocardiogram generation with 173. ANDI. Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets.
a bidirectional LSTM-CNN generative adversarial network. 2009. Available from: https://adni.loni.usc.edu/.
Sci Rep. 2019;9(1):1–11. Accessed on 4 Jan 2020.
150. Yu L, Zhang W, Wang J, Yu Y. Seqgan: Sequence generative 174. Lopez M. Breast Cancer Digital Repository. 2008. Available from:
adversarial nets with policy gradient. In: Proc. 31st AAAI Conf. https://bcdr.eu/. Accessed on 8 April 2020.
AI; 2017. p. 2852–2858.

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 31

175. Mooney P. Chest X-Ray Images (Pneumonia) | Kaggle. UNSW QRS detection algorithms. 2016. Available from: http://
2018. Available from: https://doi.org/10.1007/BF00337288. arxiv.org/abs/1811.07054.
Accessed on 4 Jan 2020. 198. 2020 BH. BNCI Horizon 2020. 2015. Available from: http://arxiv.
176. MYTHS-ATYPIA. MYTHS-ATYPIA-14 - Dataset. 2012. Avail org/abs/1811.07054. Accessed on 4 Jan 2020.
Available from: https://doi.org/10.1007/BF00337288. Accessed on 199. Khushaba RM. Electromyogram (EMG) Repository. 2012. Available
4 Jan 2020. from: http://arxiv.org/abs/1811.07054. Accessed on 6 April 2020.
177. NAMIC. MIDAS - Community National Alliance for Medical Image
Computing (NAMIC). 2010. Available from: http://arxiv. 200. Rantanen V, et al. Mimetic Interfaces: Facial Surface EMG Data set
org/abs/2003.00108. 2015. 2015. Available from: http://arxiv.org/abs/1811.07054.
178. Cohen JP, Morrison P, Dao L. COVID-19 image data collection. Accessed on 4 Jan 2020.
2020. Available from: http://arxiv.org/abs/2003.00108. 201. Atzori M. NinaPro database non-invasive adaptive hand prosthetics.
Accessed on 4 Jan 2020. 2012. Available from: http://arxiv.org/abs/1811.07054.
179. Yarkoni T. Neurosynth. 2012. Available from: http://arxiv.org/ Accessed on 4 Jan 2020.
abs/2003.00108. Accessed on 4 Jan 2020. 202. Koelstra S, et al. Database for emotion analysis using physiological
180. NIH. NIH chest x-ray datasets. 2017. Available from: http:// signals. 2011. Available from: http://arxiv.org/
arxiv.org/abs/2003.00108. Accessed on 4 Jan 2020. abs/1811.07054. Accessed on 4 Jan 2020.
181. LaMontagne PJ, et al. Open Access Series of Imaging Studies 203. Abadi MK, et al. MEG-based multimodal database for decoding
(OASIS). 2019. Available from: http://arxiv.org/abs/2003.00108 affective physiological responses. 2007. Available from: http://
182. Muschelli J. Open Neuroimaging Datasets. 2015. Available from: arxiv.org/abs/1811.07054. Accessed on 4 Jan 2020.
http://arxiv.org/abs/2003.00108. Accessed on 4 Jan 2020. 204. Head IT of University of California SD. Imagined emotion. 2009.
183. Reyes M. The HEAR-EU multiscale imaging and modeling data set Available from: http://arxiv.org/abs/1811.07054. Accessed on 4 Jan
of the human inner ear. SMIR. 2017. Available from: http:// 2020.
arxiv.org/abs/2003.001086. 205. Soleymani M, Lichtenauer J, Pun T, M P. HCI tagging database.
184. Dataset I. Brain development datasets. 2014. Available from: http:// 2012. Available from: http://arxiv.org/abs/1811.07054. Accessed on
arxiv.org/abs/2003.00108. Accessed on 4 Jan 2020. 4 Jan 2020.
185. Shattuck DW, Mirza M, Adisetiyo V, Hojatkashani C, Salamon G, et 206. Lu PBL. SEED Datasets. 2013. Available from: http://arxiv.org/
al. Construction of a 3D probabilistic atlas of human cortical abs/1811.07054. Accessed on 4 Jan 2020.
structures. NeuroImage. 2008 Feb;39(3):1064–1080. 207. Kaya M, Binli MK, Ozbay E, Yanar H, Mishchenko Y. A large
186. Gorgolewski KJ, et al. NeuroVault. 2015. Available from: http:// electroencephalographic motor imagery dataset for
arxiv.org/abs/2003.00108. Accessed on 4 Jan 2020. electroencephalographic brain computer interfaces. fgshare.
187. Boekel W. Neuroimaging informatics tools and resources 2018;Collection. Available from: http://www.nesc.ac.uk/technical_papers/
clearinghouse dataset. 2015. Available from: https://doi.org/10.1007/ UKeS-2006-02.pdf.
s12559-020-09751-3. Accessed on 4 Jan 2020. 208. Cho MH amd Ahn, Ahn S, Kwon M, C JS. Supporting data for EEG
188. Poldrack, et al. OPEN fMRI: A multi-subject, multi-modal human datasets for motor imagery brain computer interface.
neuroimaging dataset. 2015. Available from: https://doi. Giga Science Database. 2017. Available from: http://www.nesc.
org/10.1007/s12559-020-09751-3. Accessed on 4 Jan 2020. ac.uk/technical_papers/UKeS-2006-02.pdf.
189. Pernet C, Gorgolewski K, Ian W. Neuroimaging dataset of brain 209. Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR.
tumor patients. 2016. Available from: https://doi.org/10.1007/ EEG Motor Movement/Imagery Dataset. 2009. Available from: http://
s12559-020-09751-3. Accessed on 4 Jan 2020. www.nesc.ac.uk/technical_papers/UKeS-2006-02.
190. van Ginneken B, Kerkstra S, Meakin J. DRIVE - Grand Chal lenge. pdf. Accessed on 6 April 2020.
2004. Available from: https://doi.org/10.1007/s12559- 210. Korczowski L, Ostaschenko E, Andreev A, Cattan G, Rodrigues PC,
020-09751-3. Gautheret V, et al. Brain Invaders calibration-less P300-
191. IBS Repository. NITRC: IBSR: Tool/Resource Info. 2007. based BCI using dry EEG electrodes Dataset (bi2014a). 2019.
Available from: https://doi.org/10.1007/s12559-020-09751-3. Available from: http://www.nesc.ac.uk/technical_papers/UKeS
Accessed on 4 Jan 2020. 2006-02.pdf. Accessed on 4 Jan 2020.
192. Goldbaum M. The STARE Project. 1975. Available from: https:// 211. Korczowski L, Cederhout M, Andreev A, Cattan G, Rodrigues PL,
doi.org/10.1007/s12559-020-09751-3. Gautheret V, et al. Brain Invaders calibration-less P300-
193. Cao Z, Chuang M, King JT, Lin CT. Multi-channel EEG recordings based BCI with modulation of fash duration Dataset (bi2015a).
during a sustained-attention driving task. figshare. 2019. Available from: http://www.nesc.ac.uk/technical_papers/
2019;Collection. Available from: https://doi.org/10.1007/s12559- UKeS-2006-02.pdf. Accessed on 4 Jan 2020.
020-09751-3. 212. Korczowski L, Ostaschenko E, Andreev A, Cattan G, Rodrigues PC,
194. Picone J. Temple University EEG Corpus. 2011. Available from: Gautheret V, et al. Brain Invaders Solo versus Collaboration: Multi-
https://doi.org/10.1007/s12559-020-09751-3. Accessed on 7 April User P300-based Brain-Computer Interface Dataset (bi2014b).
2020. 2019. Available from: http://www.nesc.ac.uk/technical_
195. GB M, RG M. MIT-BIH Arrhythmia Database. 1999. Available from: papers/UKeS-2006-02.pdf. Accessed on 4 Jan 2020.
https://doi.org/10.1007/s12559-020-09751-3. Accessed on 4 Jan 213. Korczowski L, Cederhout M, Andreev A, Cattan G, Rodrigues PL,
2020. Gautheret V, et al. Brain Invaders Cooperative versus Competitive:
196. Goldberger A, et al. PhysioBank, PhysioToolkit, and PhysioNet: Multi-User P300-based Brain-Computer Interface Dataset (bi2015b).
components of a new research resource for complex physiologic 2019. Available from: http://www.nesc.ac.uk/technical_
signals. 2003. Circulation. 101(23):e215-e220. Available from: papers/UKeS-2006-02.pdf. Accessed on 4 Jan 2020.
https://doi.org/10.1007/s12559-020-09751-3. 214. BCI Competitions. BCI Competition datasets. 2008. Available from:
197. Khamis H, Weiss R, Xie Y, Chang CW, Lovell NH, Redmond SJ. http://www.nesc.ac.uk/technical_papers/UKeS-2006-02.
TELE ECG Database: 250 telehealth ECG records (collected using pdf. Accessed on 4 Jan 2020.
dry metal electrodes) with annotated QRS and artifact masks, and 215. BCI Challenge NER2015. BCI Challenge @ NER 2015. 2015.
MATLAB code for the UNSW artifact detection and Available from: http://www.nesc.ac.uk/technical_papers/UKeS
2006-02.pdf. Accessed on 4 Jan 2020.

13
Machine Translated by Google

32 Cognitive Computing (2021) 13:1–33

216. Broderick MPea. Electrophysiological correlates of semantic dis 242. Schaefer CFAKKSBJDMHTBKH. NCI-Nature pathway inter action
similarity reflect the comprehension of natural, narrative speech, v3, database. 2009. Available from: http://hapshistology.
Dryad; 2020. Available from: http://www.nesc.ac.uk/technical_ wikifoundry.com/. Accessed on 10 April 2020.
papers/UKeS-2006-02.pdf. Accessed on 4 Jan 2020. 243. Kandasamy K, et al. NetPath. 2010. Available from: http://
217. for Complex Physiologic Signals RR. Physionet. 1999. Available from: hapshistology.wikifoundry.com/. Accessed on 10 April 2020.
http://www.cellimagelibrary.org/. Accessed on 4 Jan 2020. 244. Stein L, D'Eustachio P, Hermjakob H, Wu G. Reactome. 2010.
218. Aha D. UCI ML repository. 1987. Available from: http://www. Available from: http://hapshistology.wikifoundry.com/. Accessed on
cellimagelibrary.org/. Accessed on 4 Jan 2020. 10 April 2020.
219. Congedo M, et al. ”Brain Invaders”: a prototype of an open source 245. Tran RV. myRBost. 2015. Available from: http://hapshistology.
P300-based video game working with the OpenViBE platform. In: wikifoundry.com/. Accessed on 4 Jan 2020.
Proc. BCI 2011; 2011. p. 280–283. 246. SGD. Saccharomyces Genome Database. 2012. Available from: http://
220. PubChem. PubChem Data Sources. 2020. Available from: http:// hapshistology.wikifoundry.com/. Accessed on 4 Jan 2020.
www.cellimagelibrary.org/. Accessed on 4 Jan 2020. 247. DNAD-J. DNA Databank of Japan. 1980. Available from: http://
221. PubChem. PubChem Data Sources. 2005. Available from: http:// hapshistology.wikifoundry.com/. Accessed on 4 Jan 2020
www.cellimagelibrary.org/. Accessed on 4 Jan 2020. 248. ENA. European Nucleotide Archive. 1990. Available from: http://
222. Biolab. Bioinformatics Laboratory. 1999. Available from: http:// hapshistology.wikifoundry.com/. Accessed on 4 Jan 2020.
www.cellimagelibrary.org/. Accessed on 4 Jan 2020. 249. GenBank. GenBank; 2013. Available from: https://idr.
223. Pradhan S, et al. Indian Genetic Disease Database. 2011. Available openmicroscopy.org/. Accessed on 4 Jan 2020.
from: http://www.cellimagelibrary.org/. Accessed on 4 Jan 2020. 250. Noordewier MO, Towell GG, Shavlik JW. UCI Molecular Biol ogy (UCI
MB) splice-junction gene sequences data set. 1981.
224.Atlas TCG. The Cancer Genome Atlas Home Page [nciHome]. Available from: https://idr.openmicroscopy.org/. Accessed on 4 Jan
2005. Available from: http://www.cellimagelibrary.org/. 2020.
Accessed on 4 Jan 2020. 251. ICU-MB. UCI Molecular Biology (UCI MB) promoter gene sequences
225. Network BDT. Berkeley Drosophila Transcription Network Project. data set. 1985. Available from: https://idr.openmicroscopy.
2001. Available from: http://www.cellimagelibrary.org/. org/. Accessed on 4 Jan 2020.
226. ENCODE. Encyclopedia of DNA Elements. 2003. Available from: http:// 252. Manaswi NK. Understanding and working with Keras. In: Deep learning
www.cellimagelibrary.org/. Accessed on 4 Jan 2020. with applications using Python. Springer; 2018. p. 31–43.
227. NHLBI GO ESP. Exome Variant Server. 2011. Available from: http://
www.cellimagelibrary.org/. Accessed on 6 April 2020. 253. Kunkel R, et al. TensorSCONE: a secure TensorFlow framework using
228. GEO. Gene Expression Omnibus. 2000. Available from: http:// Intel SGX. CoRR. 2019. p. 1–12.
www.eurobioimaging.eu/. Accessed on 4 April 2020. 254. Sun X, Peng X, Ding S. Emotional human-machine conversation
229. Abreu M, et al. gnomAD. 2016. Available from: http://www. generation based on long short-term memory. Cogn Comput.
eurobioimaging.eu/. Accessed on 6 April 2020. 2018;10(3):389–97.
230. of MIT TBI, Harvar. GTEx Portal. 2012. Available from: http:// 255. Hao L, Liang S, Ye J, Xu Z. TensorD: A tensor decomposition library
www.eurobioimaging.eu/. Accessed on 4 Jan 2020. in TensorFlow. Neurocomputing. 2018;318:196–200.
231. Rouillard AD, Gundersen GW, Fernandez NF, Wang Z, Monteiro CD, 256. Karpathy A. A peek at trends in machine learning. 2017. Available
McDermott MG, et al. The harmonizome: a collection of processed from: https://idr.openmicroscopy.org/.
datasets gathered to serve and mine knowledge about genes and 257. Bahrampour S, Ramakrishnan N, Schott L, Shah M. Comparative
proteins. Database 2016. 2016;07:baw100. study of deep learning software frameworks. CoRR. 2016; abs/
232. INSDC. The International Nucleotide Sequence Database 1511.06435. ArXiv: 1511.06435.
Collaboration. 2016. Available from: http://www.eurobioimaging. 258. Shi S, et al. Benchmarking state-of-the-art deep learning software
eu/. Accessed on 4 Jan 2020. tools. CoRR. 2016; abs/1608.07249.
233. Resource TIGS. 1000 Genome Project. 2008. Available from: http:// 259.Deepmark. The deep learning benchmarks. 2017. Available from:
www.eurobioimaging.eu/. Accessed on 6 April 2020. https://idr.openmicroscopy.org/. Accessed on 17 Dec 2017.
234. JASPAR. JASPAR 2018: An open-access database of transcription 260. Narang S. The source code and experimental data of benchmarking
factor binding professionals. 2008. Available from: http://www. state-of-the-art deep learning software tools. 2017. Available from:
eurobioimaging.eu/. Accessed on 4 Jan 2020. https://idr.openmicroscopy.org/. Accessed on 17 Dec 2017.
235. NREM Consortium. Roadmap Epigenomics Project - Data. 2007. 261. LeCun Y, Cortes C, Burges CJC. The MNIST database of hand
Available from: http://www.eurobioimaging.eu/. Accessed on 6 April written digits. 1998. Available from: https://idr.openmicroscopy.
2020. org/. Accessed on 4 Jan 2020.
236. NSD. Nature Scientific data. 2014. Available from: http://www. 262. Zaremba W, Sutskever I, Vinyals O. Recurrent neural network
eurobioimaging.eu/. Accessed on 4 Jan 2020. regularization. CoRR. 2014; abs/1409.2329.
237. SysGENSIM. SysGenSIM - Benchmark datasets. 2013. Available 263. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y.
from: http://www.eurobioimaging.eu/. Accessed on 4 Jan 2020. OverFeat: integrated recognition, localization and detection using
238. in Research BMSEB, Education. RCSB Protein Data Bank - RCSB convolutional networks. CoRR. 2013; abs/1312.6229.
PDB. 2015. Available from: http://www.eurobioimaging. 264. Murphy J. Deep learning benchmarks of NVIDIA Tesla P100 PCIe,
eu/. Accessed on 4 Jan 2020. Tesla K80, and Tesla M40 GPUs; 201. Available from: https://
239. Murzin AG, Brenner SE, Hubbard TJP, Chothia C. Structural idr.openmicroscopy.org/. Accessed on 4 Jan 2020.
classifcation of proteins database 2. 2020. Available from: http:// 265. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, et al.
hapshistology.wikifoundry.com/. Accessed on 10 April 2020. Human-level control through deep reinforcement learning.
240. Fox NK, Brenner SE, Chandonia JM. Structural classifcation of pro Nature. 2015;518(7540):529–33.
teins database - extended. 2018. Available from: http://hapshistology. 266. Chollet F. The limitations of deep learning. 2017. Available from: https://
wikifoundry.com/. Accessed on 10 April 2020. idr.openmicroscopy.org/. Accessed on 12 Dec 2017.
241. Qian N, Sejnowski TJ. UCI Molecular Biology (UCI MB) protein 267. Zenil H, et al . An algorithmic information calculus for causal
secondary structure data set. 1988. Available from: http://hapshistology. discovery and reprogramming systems. bioRxiv. 2017;p. 185637.
wikifoundry.com/. Accessed on 4 Jan 2020.

13
Machine Translated by Google

Cognitive Computing (2021) 13:1–33 33

268. Shwartz-Ziv R, Tishby N. Opening the black box of deep neural 282. Nahiduzzaman Md, Tasnim M, Newaz NT, Kaiser MS, Mah mud
networks via information. CoRR. 2017 Mar;abs/1703.00810. M. Machine learning based early fall detection for elderly people
269. Nguyen AM, Yosinski J, Clune J. Deep neural networks are easily with neurological disorder using multimodal data fusion.
fooled: High confidence predictions for unrecognizable images. In: Mahmud M, Vassanelli S, Kaiser MS, Zhong N, editors.
In: Proc. VCRP; 2015. p. 427–436. Brain Inform. Cham: Springer International Publishing; 2020. p.
270. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, 204–14.
Goodfellow IJ, et al. Intriguing properties of neural networks. In: 283. Farah L, Hussain A, Kerrouche A, Ieracitano C, Ahmad J, Mah
CoRR. vol. abs/1312.6199; 2013. p. 1–10. mud M. A highly-efficient fuzzy-based controller with high
271. Baker NA, Klemm JD, Harper SL, Gaheen S, Heiskanen M, reduction inputs and membership functions for a grid-connected
Rocca-Serra P, et al. Standardizing data. Nat Nanotechnol. photovoltaic system. IEEE Access. 2020;8:163225–37.
2013;8(2):73. 284. Fabietti M, Mahmud M, Lotf A. Machine learning in analyzing
272. Wittig U, Rey M, Weidemann A, Muller W. Data management and invasively recorded neuronal signals: available open access data
data enrichment for systems biology projects. J Biotechnol. sources. In: Mahmud M, Vassanelli S, Kaiser MS, Zhong N,
2017;261:229–237. editors. Brain Inform. Cham: Springer International Publishing;
273. Mahmud M, Rahman MM, Travalin D, Raif P, Hussain A. 2020. p. 151–62.
Service oriented architecture based web application model for 285. Chen L, Yan J, Chen J, Sheng Y, Xu Z, Mahmud M. An event
collaborative biomedical signal analysis. Biomed Tech (Berl). based topic learning pipeline for neuroimaging literature mining.
2012;57:780–3. 2020 Oct 23 [cited 2020 Nov 22]; Available from: https://www.
274. Mahmud M, Pulizzi R, Vasilaki E, Giugliano M. A web-based researchsquare.com/article/rs-95392/v1
framework for semi-online parallel processing of extracellular 286. Nahian MJA, Ghosh T, Uddin MN, Islam MdM, Mahmud M, Kaiser
neuronal signals recorded by microelectrode arrays. In: Proc. MS. Towards artificial intelligence driven emotion aware fall
MEAMEETING; 2014. p. 202–203. monitoring framework suitable for elderly people with neurological
275. Angelov P, Sperduti A. Challenges in deep learning. In: Proc. disorder. In: Mahmud M, Vassanelli S, Kaiser MS, Zhong N,
ESANN; 2016. p. 489–495. editors. Brain Inform. Cham: Springer International Publishing;
276. Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA. Deep 2020. p. 275–86.
reinforcement learning: a brief survey. IEEE Signal Process Mag. 287. Al Banna MdH, Ghosh T, Taher KA, Kaiser MS, Mahmud M. A
2017;34(6):26–38. monitoring system for patients of autism spectrum disorder using
277. Noor MB, Zenia NZ, Kaiser MS, Al Mamun S, Mahmud M. artificial intelligence. In: Mahmud M, Vassanelli S, Kaiser MS,
Application of deep learning in detecting neurological disorders Zhong N, editors. Brain Inform. Cham: Springer International
from magnetic resonance images: a survey on the detection of Publishing; 2020. p. 251–62.
Alzheimer's disease, Parkinson's disease and schizophrenia. 288. Adiba FI, Islam T, Kaiser MS, Mahmud M, Rahman MA. Effect of
Brain Inform. 2020;7(1):1-21. corpora on classifcation of fake news using naive Bayes clas
278. Al Banna MH, Taher KA, Kaiser MS, Mahmud M, Rahman MS, sifer. International Journal of Automation, Artificial Intelligence
Hosen AS, Cho GH. Application of artificial intelligence in and Machine Learning. 2020;1(1):80–92.
predicting earthquakes: state-of-the-art and future challenges. 289. Sumi AI, Zohora MostF, Mahjabeen M, Faria TJ, Mahmud M,
IEEE Access. 2020;8:192880–192923. Kaiser MS. fASSERT: A fuzzy assistive system for children with
279. Jesmin S, Kaiser MS, Mahmud M. Artifcial and internet of autism using internet of things. In: Wang S, Yamamoto V, Su J,
healthcare things based Alzheimer care during COVID 19. In: Yang Y, Jones E, Iasemidis L, et al., eds. Brain Inform. Cham:
Mahmud M, Vassanelli S, Kaiser MS, Zhong N, editors. Brain Springer International Publishing; 2018. p. 403–12.
Inform. Cham: Springer International Publishing; 2020. p. 263– 290. Mahmud M, Kaiser MS, Rahman MM, Rahman MA, Shabut A, Al-
274. Mamun S, et al. A brain-inspired trust management model to
280. Ruiz J, Mahmud M, Modasshir M, Shamim Kaiser M, Alzheimer's assure security in a cloud based IoT framework for neuroscience
disease neuroimaging initiative for the 3D DenseNet ensemble applications. Cogn Comput. 2018 Oct 1;10(5):864–73.
in 4-way classifcation of Alzheimer's disease. In: Mahmud M, 291. Kaiser MS, Chowdhury ZI, Mamun SA, Hussain A, Mahmud M. A
Vassanelli S, Kaiser MS, Zhong N, editors. Brain Inform. Cham: neuro-fuzzy control system based on feature extraction of surface
Springer International Publishing; 2020. p. 85–96. electromyogram signal for solar-powered wheelchair.
Cogn Comput. 2016 Oct 1;8(5):946–54.
281. Rahman S, Sharma T, Mahmud M. Improving alcoholism
diagnosis: comparing instance-based classifiers against neural
net works for classifying EEG signal. In: Mahmud M, Vassanelli
S, Kaiser MS, Zhong N, editors. Brain Inform. Cham: Springer
International Publishing; 2020. p. 239–50.

13