Hang Wan, Rui Xu, Meng Zhang, Yanpeng Cai, Jian Li, Xia Shen,

A novel model for water quality prediction caused by non-point sources pollution
based on deep learning and feature extraction methods,
Journal of Hydrology,
Volume 612, Part A,
2022,
128081,
ISSN 0022-1694,
https://doi.org/10.1016/j.jhydrol.2022.128081.
(https://www.sciencedirect.com/science/article/pii/S0022169422006564)
Abstract: Non-point source (NPS) pollution is an important factor affecting the
quality of water environment. In recent years, a large number of online water
quality monitoring stations have been used to obtain continuous time series water
quality monitoring data. These data provide the necessary basis for the application
of deep learning methods in water quality prediction. However, the prediction
accuracy of traditional deep learning methods is low, especially in predicting the
water quality with NPS pollution. Aiming to address this limitation, a novel deep
learning model named SOD-VGG-LSTM with the simulation-observation difference (SOD)
modular based on physical process, the visual geometry (VGG) modular reflecting
spatial characteristics, and the long short-term memory (LSTM) modular based on
deep learning method was developed to improve the accuracy of the water quality
prediction with NPS pollution. The established model can overcome the problem that
mechanism models can not predict the changes of water quality on the hourly or
minute time scale. The model was applied in Lijiang River watershed. Experimental
results indicated that the proposed model had the highest accuracy in the extreme
value prediction compared with the mechanism model and LSTM model. The maximum
relative errors between the predicted and observed results for DO, CODMn, NH3-N,
and TP were 8.47%, 19.76%, 24.1%, and 35.4%, respectively. The model evaluation
demonstrated thatthe established SOD-VGG-LSTM model achieved superior computational
performance compared to Auto Regression Integreate Moving Average model (ARIMA),
Support Vector Regression model (SVR), and Recurrent Neural Network model (RNN).
The evaluation results showed that SOD-VGG-LSTM achieved 3.2–39.3% higher R2 than
ARIMA, SVR and RNN. The proposed model can provide a new method for water quality
prediction with NPS pollution.
Keywords: Non-point source pollution; Spatial characteristics; LSTM; VGG

Xinming Ren, Huaxi Gu, Wenting Wei,

Tree-RNN: Tree structural recurrent neural network for network traffic
classification,
Expert Systems with Applications,
Volume 167,
2021,
114363,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2020.114363.
(https://www.sciencedirect.com/science/article/pii/S0957417420310435)
Abstract: Network traffic classification plays an important role in network
monitoring and network management. With the continuous development of network
technology, traditional methods of traffic classification have more limitations in
accuracy to deal with encrypted traffic. Fortunately, deep neural network (DNN) is
an effective method for handling traffic classification due to its ability to learn
inherent data features. However, this method generally classifies network traffic
with only the single classifier, which makes it relatively less effective in some
classes for the problem of large classification. In this paper, we propose a tree
structural recurrent neural network (Tree-RNN), which divides a large
classification into small classifications by using the tree structure. A specific
classifier is set for each small classification after division. With multiple
classifiers employed, Tree-RNN can complement each other in classification
performance, and the problem of the single classifier is solved. Since multiple
classifiers are all end-to-end frameworks, Tree-RNN can automatically learn the
nonlinear relationship between input data and output data without feature
extraction. To verify the validity of our model, we compare Tree-RNN with state-of-
the-art methods using the ISCX public traffic dataset. Experimental results show
that Tree-RNN can achieve higher performance in less training time. The average
accuracy of Tree-RNN is 4.88% higher than other state-of-the-art methods, and it
has higher average precision and average recall.
Keywords: Network traffic classification; Deep learning; Recurrent neural network;
Tree structure; End-to-end

Min Zhao, Weizheng Yan, Na Luo, Dongmei Zhi, Zening Fu, Yuhui Du, Shan Yu, Tianzi
Jiang, Vince D. Calhoun, Jing Sui,
An attention-based hybrid deep learning framework integrating brain connectivity
and activity of resting-state functional MRI data,
Medical Image Analysis,
Volume 78,
2022,
102413,
ISSN 1361-8415,
https://doi.org/10.1016/j.media.2022.102413.
(https://www.sciencedirect.com/science/article/pii/S1361841522000652)
Abstract: Functional magnetic resonance imaging (fMRI) as a promising tool to
investigate psychotic disorders can be decomposed into useful imaging features such
as time courses (TCs) of independent components (ICs) and functional network
connectivity (FNC) calculated by TC cross-correlation. TCs reflect the temporal
dynamics of brain activity and the FNC characterizes temporal coherence across
intrinsic brain networks. Both features have been used as input to deep learning
approaches with decent results. However, few studies have tried to leverage their
complementary information to learn optimal representations at multiple facets.
Motivated by this, we proposed a Hybrid Deep Learning Framework integrating brain
Connectivity and Activity (HDLFCA) together by combining convolutional recurrent
neural network (C-RNN) and deep neural network (DNN), aiming to improve
classification accuracy and interpretability simultaneously. Specifically, C-RNNAM
was proposed to extract temporal dynamic dependencies with an attention module (AM)
to automatically learn discriminative knowledge from TC nodes, while DNN was
applied to identify the most group-discriminative FNC patterns with layer-wise
relevance propagation (LRP). Then, both prediction outputs were concatenated to
build a new feature matrix, generating the final decision by logistic regression.
The effectiveness of HDLFCA was validated on both multi-site schizophrenia (SZ, n ∼
1100) and public autism datasets (ABIDE, n ∼ 1522) by outperforming 12 alternative
models at 2.8-8.9% accuracy, including 8 models using either static FNC or TCs and
4 models using dynamic FNC. Appreciable classification accuracy was achieved for HC
vs. SZ (85.3%) and HC vs. Autism (72.4%) respectively. More importantly, the most
group-discriminative brain regions can be easily attributed and visualized,
providing meaningful biological interpretability and highlighting the great
potential of the proposed HDLFCA model in the identification of valid neuroimaging
biomarkers.
Keywords: Attention mechanism; Deep learning; Brain connectivity and activity; fMRI

Qingtai Wang, Hongquan Huang, Xingke Ma, Zhiwen Shen, Chenglin Zhong, Weicheng
Ding, Wei Zhou, Jianbin Zhou,
Trapezoidal pile-up nuclear pulse parameter identification method based on deep
learning transformer model,
Applied Radiation and Isotopes,
Volume 190,
2022,
110515,
ISSN 0969-8043,
https://doi.org/10.1016/j.apradiso.2022.110515.
(https://www.sciencedirect.com/science/article/pii/S0969804322004006)
Abstract: Pile-up between adjacent nuclear pulses is unavoidable in the actual
detection process. Some scholars have tried to apply deep learning techniques to
identify pile-up nuclear pulse parameters. However, traditional deep learning
recurrent neural networks (RNNs) suffer from inefficient pulse recognition and poor
recognition of pile-up nuclear pulses with short intervals between adjacent pulses.
In this paper, a Transformer model with an attention mechanism as the core to
recognize pile-up nuclear pulses is innovatively applied, aiming to provide a more
accurate and efficient method for pile-up nuclear pulse recognition. Thus, it gives
a better help for the spectrum correction with a high count rate.
Keywords: Nuclear pulse; Attention mechanism; Transformer

Masayuki Nigo, Hong Thoai Nga Tran, Ziqian Xie, Han Feng, Bingyu Mao, Laila Rasmy,
Hongyu Miao, Degui Zhi,
PK-RNN-V E: A deep learning model approach to vancomycin therapeutic drug
monitoring using electronic health record data,
Journal of Biomedical Informatics,
Volume 133,
2022,
104166,
ISSN 1532-0464,
https://doi.org/10.1016/j.jbi.2022.104166.
(https://www.sciencedirect.com/science/article/pii/S1532046422001782)
Abstract: Vancomycin is a commonly used antimicrobial in hospitals, and therapeutic
drug monitoring (TDM) is required to optimize its efficacy and avoid toxicities.
Bayesian models are currently recommended to predict the antibiotic levels. These
models, however, although using carefully designed lab observations, were often
developed in limited patient populations. The increasing availability of electronic
health record (EHR) data offers an opportunity to develop TDM models for real-world
patient populations. Here, we present a deep learning-based pharmacokinetic
prediction model for vancomycin (PK-RNN-V E) using a large EHR dataset of 5,483
patients with 55,336 vancomycin administrations. PK-RNN-V E takes the patient’s
real-time sparse and irregular observations and offers dynamic predictions. Our
results show that RNN-PK-V E offers a root mean squared error (RMSE) of 5.39 and
outperforms the traditional Bayesian model (VTDM model) with an RMSE of 6.29. We
believe that PK-RNN-V E can provide a pharmacokinetic model for vancomycin and
other antimicrobials that require TDM.
Keywords: Vancomycin; Recurrent neural network; Pharmacokinetics; Deep learning;
Bayesian model

Mohit Dua, Rohit Yadav, Divya Mamgai, Sonali Brodiya,

An Improved RNN-LSTM based Novel Approach for Sheet Music Generation,
Procedia Computer Science,
Volume 171,
2020,
Pages 465-474,
ISSN 1877-0509,
https://doi.org/10.1016/j.procs.2020.04.049.
(https://www.sciencedirect.com/science/article/pii/S1877050920310152)
Abstract: It is very well known that Sheet Music is one of the most effective
medium for musicians, professional artists and amateurs to access the chords to any
songs. Many systems have already been developed which generate sheet music by
taking song as input. This paper presents one such system that aims to improve the
accuracy of sheet music generated by previous works. Improvement is achieved by
working on source separation and chord estimation modules of the previous system.
The proposed work utilizes Deep Learning techniques such as Recurrent Neural
Network (RNN) with Gated Recurrent Units (GRU) and Long Short Term Memory (LSTM).
In source separation module, multi-layered GRU cells for implementing RNN and in
chord estimation module, LSTM cells for implementing RNN are used for the
implementation. In source separation module, the number of sources that it can
separate are also increased to improve the accuracy of chord estimation module.
Keywords: Deep Learning; RNN; LSTM; Sheet music; Source Separation; Chord
Estimation

Harmandeep Singh Gill, Baljit Singh Khehra,

An integrated approach using CNN-RNN-LSTM for classification of fruit images,
Materials Today: Proceedings,
Volume 51, Part 1,
2022,
Pages 591-595,
ISSN 2214-7853,
https://doi.org/10.1016/j.matpr.2021.06.016.
(https://www.sciencedirect.com/science/article/pii/S2214785321043509)
Abstract: With the advancement in technology, Computer and machine vision system is
getting involved in the agriculture sector for the last few years. Deep Learning is
a recent advancement in the Artificial Intelligence field. In the present era, many
researchers have used deep learning applications for the classification of images,
and is found to be one of the emerging areas in computer vision. In the
classification of fruit images, the main goal is to improve the accuracy of the
classification system. The accuracy of the classifier depends on various factors
like the nature of acquired images, the number of features, types of features,
selection of optimal features from extracted features, and type of classifiers
used. In the proposed article, integration of CNN, RNN, and LSTM for the
classification of fruit images are defined. In this approach, CNN and RNN are
employed for the development of discriminative characteristics and sequential-
labels respectively. LSTM presents an explanation by integrating a memory cell to
encode learning at each interval of classification. Key parameters: accuracy, F-
measure, sensitivity, and specificity are applied to assess the achievement of the
proposed scheme. From empirical results, it has been declared that the offered
classification method provides efficient results.
Keywords: CNN; RNN; LSTM; Integrated Approach; Fruit classification

Na Zhao, Zhen Long, Jian Wang, Zhi-Dan Zhao,

AGRE: A knowledge graph recommendation algorithm based on multiple paths embeddings
RNN encoder,
Knowledge-Based Systems,
2022,
110078,
ISSN 0950-7051,
https://doi.org/10.1016/j.knosys.2022.110078.
(https://www.sciencedirect.com/science/article/pii/S0950705122011741)
Abstract: More and more researches have focused on the use of knowledge graphs (KG)
to solve the sparsity problem of traditional collaborative filtering recommendation
systems. However, most KG based recommendation algorithms focus on independent
paths connecting users and items, or iteratively propagate user preferences in KG.
Therefore, in this study, we propose a knowledge graph recommendation system
algorithm for the multiple paths RNN encoder (AGRE), which fully considers the
association between paths. Specifically, the paths between the user and the item
are coded by a specified RNN (MRNN) to accurately learn the user’s preferences.
Traditional RNNs can encode multiple paths without considering the association
between paths, but our RNN can encode multiple paths with considering the
association between paths. Our RNNs are encoded with full consideration of the
association between paths. We have compared AGRE with other state-of-the-art
algorithms on three real-world datasets, and achieved good results in terms of AUC
and Precision@K. This indicates that AGRE could solve the problem of sparse
interaction between users and items, and could make full use of the knowledge graph
for recommendation.
Keywords: Knowledge graph; Recommendation algorithm; Multiple paths; MRNN
Anupong Wongchai, Durga rao Jenjeti, A. Indira Priyadarsini, Nabamita Deb, Arpit
Bhardwaj, Pradeep Tomar,
Farm monitoring and disease prediction by classification based on deep learning
architectures in sustainable agriculture,
Ecological Modelling,
Volume 474,
2022,
110167,
ISSN 0304-3800,
https://doi.org/10.1016/j.ecolmodel.2022.110167.
(https://www.sciencedirect.com/science/article/pii/S030438002200268X)
Abstract: Agriculture is necessary for all human activities to survive.
Overpopulation and resource competitiveness are major challenges that threaten the
planet's food security. Smart farming as well as precision agriculture advancements
provide critical tools for addressing agricultural sustainability concerns and
addressing the ever-increasing complexity of difficulties in agricultural
production systems. This research proposed novel technique in agricultural farm
monitoring and crop disease prediction using deep learning architectures. Here the
monitored data has been collected based on IoT module along with the historical
data of cultivation farm image data. This data has been processed for removal of
noise removal and image resizing. The features of processed data has been extracted
using deep attention layer based convolutional learning (DAL_CL) in which the
features of data has been extracted. This extracted data has been classified using
recursive architecture based on neural networks (RNN). The suggested system may use
data categorization and deep learning to exploit obtained data and anticipate when
a plant will (or will not) get a disease with a high degree of precision, with
ultimate goal of making agriculture more sustainable.Experimental results shows the
accuracy of 96%, precision of 89%, specificity of 89%, F-1 score of 75% and AUC of
66%.
Keywords: Agriculture; Farm monitoring; Crop disease prediction; Deep learning;
Features; Classification

Zhiguo Zhu, Mengru Yan, Xiaoyi Deng, Ming Gao,

Rating prediction of recommended item based on review deep learning and rating
probability matrix factorization,
Electronic Commerce Research and Applications,
Volume 54,
2022,
101160,
ISSN 1567-4223,
https://doi.org/10.1016/j.elerap.2022.101160.
(https://www.sciencedirect.com/science/article/pii/S1567422322000448)
Abstract: With a sharp improvement in E-commerce and data, the precise rating
prediction of recommended items under user preferences has been a hot research
topic in the EC intelligence domain. The rating data Matrix Factorization based
methods have been widely used in item rating predictions in e-commerce
recommendation systems. However, “cold start” and “data sparsity” have seriously
restricted the accuracy of such methods. In addition to the rating data, as side
information, the massive reviews posted by users rich in semantic and emotional
information express user preferences and item characteristics, and will certainly
improve the accuracy of the rating prediction. Accordingly, this paper combines the
deep learning for the review text and the matrix factorization method for rating
data to predict the rating of the recommended items accurately. Firstly, based on
the Deep Learning methods, self-attention mechanism and bi-directional RNN
(Recurrent Neural Network) with the core of GRU (Gated Recurrent Unit), the deep
nonlinear features of users and items are learned from review texts. Then, these
features are introduced as a prior mean into the classical rating-based probability
matrix factorization model to obtain the latent factor vectors of users and items
with the rating of the recommended item accurately predicted. Finally, adopting MSE
and MAE as the indicators, the extensive experiments conducted on four real
datasets verify that the proposed model TFRMF (Topical Features Regularized Matrix
Factorization) performs better than other classical counterparts. The achievements
of this work will provide powerful methods and decision supports for accurate and
personalized e-commerce recommendation practices.
Keywords: Precise recommendation; Ratings and reviews; Deep learning; Bi-
directional GRU network; Self-attention mechanism; Probability matrix factorization

S. Ali Naqvi, Meagan T.M. King, Trevor J. DeVries, Herman W. Barkema, Rob Deardon,
Data considerations for developing deep learning models for dairy applications: A
simulation study on mastitis detection,
Computers and Electronics in Agriculture,
Volume 196,
2022,
106895,
ISSN 0168-1699,
https://doi.org/10.1016/j.compag.2022.106895.
(https://www.sciencedirect.com/science/article/pii/S0168169922002125)
Abstract: With growing adoption of precision dairy technologies, the use of big
data is becoming increasingly common in the dairy industry. The speed at which data
are generated has led to increased interest in developing detection and predictive
models for animal health and disease events using real time records. When combining
data from multiple sources, statistical methods exist to account for the underlying
heterogeneity in data collected from commercial farms, although its impact on
predictive models is not known. We investigated how 4 different issues commonly
seen in these large datasets impact the performance of deep recurrent neural
networks (RNNs) trained to detect the onset of clinical mastitis (CM) in dairy
cows. Data were simulated by first sampling from real-world data and adding noise,
then defining the association between predictor variables and CM while
incorporating parameters to reflect underlying heterogeneity: 1) random effects to
reflect unmeasured variability at the farm level (3 levels – none, moderate, high);
2) random effects to reflect unmeasured variability at the cow level (3 levels –
none, moderate, high); 3) missed recording of CM cases (3 false-negative rates –
0.10, 0.25, 0.50); and 4) incomplete observations due to certain farms not having a
somatic cell count sensor (SCC data missing vs SCC data included). At baseline
(moderate farm and cow random effects; moderate misclassification; 42% herds with
SCC sensor) the model achieved a sensitivity and specificity of 86% and 90%
respectively. Higher levels of unmeasured variability at the farm and cow levels
resulted in reduced model performance (sensitivity and specificity of 76% and 85%
at the highest levels), indicating that data collection and feature selection
should be informed by previous knowledge of the associations between the outcome
and predictors when possible, and that model performance may be limited when
predictors are selected only from routinely collected data. However, even when 50%
of CM cases were incorrectly recorded as CM-negative, model performance did not
decrease, demonstrating that deep RNNs are robust to the level of misclassification
that would be typically encountered in dairy datasets. RNNs were also able to
accurately detect CM onset even when a highly predictive variable, somatic cell
count, was excluded from training and test data, but the models took longer to
train. The effect of unmeasured variability on model performance demonstrates how
predictors should be selected for RNNs, whereas RNNs appear to be very robust to
misclassification in training data as well as missing variables. Researchers
developing studies using deep learning should therefore focus their attention more
on predictor selection than on reducing or standardizing outcome recording, since
RNNs appear to be robust to the latter, while being more strongly impacted by the
former.
Keywords: Mastitis; Detection; Robotic milking systems; Simulation; Dairy

G. Sivapriya, V. Praveen, P. Gowri, S. Saranya, S. Sweetha, Kukunoor Shekar,

Segmentation of Hard exudates for the detection of Diabetic Retinopathy with RNN
based sematic features using fundus images,
Materials Today: Proceedings,
Volume 64, Part 1,
2022,
Pages 693-701,
ISSN 2214-7853,
https://doi.org/10.1016/j.matpr.2022.05.189.
(https://www.sciencedirect.com/science/article/pii/S2214785322034691)
Abstract: One of the diabetes complications that affects the eyes is Diabetic
retinopathy and it is caused by blood vessels damage in the retina. The DR can be
detected by finding the Hard Exudate present in it. The deep networks are becoming
more deeper and more complex. So that adding a greater number of layers to a neural
network can make it stronger for image related tasks. But the main drawback in
adding more layers is that, it may greatly reduce the accuracy of the image and
also the data models are complex. In order to overcome this drawback, Recurrent
Neural Network can be introduced. The main aim for applying the recurrent neural
network is that it can model a collection of records in such a way that each
pattern is assumed to be dependent on the previous one. It can process inputs of
any length. Even if the input size is large, the model size will not change. It
makes the training process faster and attain more accuracy while compared to other
neural networks. It greatly reduces the loss of accuracy because each lower knows
the information of the top layers while updating the weights. This Recurrent has a
greater number of parameters, so it is obvious that it can produce better result as
compared to other net with the accuracy of 97.28%.
Keywords: Diabetic retinopathy; RNN classifier; GLCM; Hard exudates

Mohammad-Reza Mohammadi, Fahimeh Hadavimoghaddam, Saeid Atashrouz, Ali Abedi,

Abdolhossein Hemmati-Sarapardeh, Ahmad Mohaddespour,
Modeling of nitrogen solubility in unsaturated, cyclic, and aromatic hydrocarbons:
Deep learning methods and SAFT equation of state,
Journal of the Taiwan Institute of Chemical Engineers,
Volume 131,
2022,
104124,
ISSN 1876-1070,
https://doi.org/10.1016/j.jtice.2021.10.024.
(https://www.sciencedirect.com/science/article/pii/S1876107021005861)
Abstract: Background
An accurate understanding of the nitrogen solubility in hydrocarbons is crucial for
developing enhanced oil recovery (EOR) by nitrogen injection and the design of
thermal separation and chemical conversion processes in chemical industries and oil
refineries. Unsaturated, cyclic, and aromatic hydrocarbons as important compounds
of crude oil are common solvents in the chemical industry that have received less
attention compared to normal alkanes.
Methods
In this paper, four deep learning models, including recurrent neural network (RNN),
long short-term memory (LSTM), deep belief network (DBN), and convolutional neural
network (CNN) were developed for estimating the nitrogen solubility in unsaturated,
cyclic, and aromatic hydrocarbons. To this end, 673 experimental nitrogen
solubility data for 23 various hydrocarbons were collected from the literature in a
wide ranges of operating pressure (0.03–100.1 MPa) and temperature (78–662.8 K).
The input parameters to the models were considered critical temperature, critical
pressure, and molecular weight of hydrocarbon solvents accompanying operating
conditions of temperature and pressure. Also, the performance of deep learning
models was compared with Soave-Redlich-Kwong (SRK), Peng-Robinson (PR), and
statistical associating fluid theory (SAFT) equations of state (EOSs).
Significant findings
The CNN model is able to estimate the experimental values of nitrogen solubility
with a root mean square error (RMSE) of 0.0211. Also, SAFT EOS outperformed the two
cubic EOSs. Based on sensitivity analysis, pressure has the greatest impact on
nitrogen solubility in unsaturated, cyclic, and aromatic hydrocarbons, followed by
temperature and the critical temperature of hydrocarbons. Nitrogen solubility in
unsaturated, cyclic, and aromatic hydrocarbons increases with the increase in
pressure and temperature, and it lowers by the increase in the critical pressure,
critical temperature, and molecular weight of hydrocarbon solvents. Finally, high
reliability of the experimental data and statistically high validity of the CNN
model were proved by the Leverage approach. The findings of this study can have
implications in increasing the efficiency of gas processing units, thermal
separation, and chemical conversion processes.
Keywords: Nitrogen solubility; Hydrocarbon; Deep learning; Convolutional neural
network; Recurrent neural network; Statistical associating fluid theory

Run-Ze Xu, Jia-Shun Cao, Jing-Yang Luo, Qian Feng, Bing-Jie Ni, Fang Fang,
Integrating mechanistic and deep learning models for accurately predicting the
enrichment of polyhydroxyalkanoates accumulating bacteria in mixed microbial
cultures,
Bioresource Technology,
Volume 344, Part B,
2022,
126276,
ISSN 0960-8524,
https://doi.org/10.1016/j.biortech.2021.126276.
(https://www.sciencedirect.com/science/article/pii/S0960852421016187)
Abstract: The enrichment of polyhydroxyalkanoates (PHA) accumulating bacteria (PAB)
in mixed microbial cultures (MMC) is extremely difficult to be predicted and
optimized. Here we demonstrate that mechanistic and deep learning models can be
integrated innovatively to accurately predict the dynamic enrichment of PAB. Well-
calibrated activated sludge models (ASM) of the PAB enrichment process provide
time-dependent data under different operating conditions. Recurrent neural network
(RNN) models are trained and tested based on the time-dependent dataset generated
by ASM. The accurate prediction performance is achieved (R2 > 0.991) for three
different PAB enrichment datasets by the optimized RNN model. The optimized RNN
model can also predict the equilibrium concentration of PAB (R2 = 0.944) and
corresponding time, which represents the end of the PAB enrichment process. This
study demonstrates the strength of integrating mechanistic and deep learning models
to predict long-term variations of specific microbes, helping to optimize their
selection process for PHA production.
Keywords: Recurrent neural networks (RNN); Long short-term memory (LSTM); Activated
sludge model (ASM); Polyhydroxyalkanoates (PHA); Microbial enrichment

Bihter Das, Ömer Osman Dursun, Suat Toraman,

Prediction of air pollutants for air quality using deep learning methods in a
metropolitan city,
Urban Climate,
Volume 46,
2022,
101291,
ISSN 2212-0955,
https://doi.org/10.1016/j.uclim.2022.101291.
(https://www.sciencedirect.com/science/article/pii/S2212095522002097)
Abstract: Air quality forecasting is very difficult in metropolitan areas due to
emissions, high population density, and uncertainty in defining meteorological
areas. The use of incomplete information during the training phase and the poor
model selection to be used restrict the air quality estimation. In this study,
predictions of PM10 and SO2 air pollutants in 2022 were made by using Long Short-
Term Memory Networks (LSTM), Recurrent Neural Network (RNN), and Multilayer
Perceptron (MLP) by revising the error term of traditional methods and completing
the missing data. The data of Basaksehir district of Istanbul province, where
industrialization and population are very concentrated, were obtained from the
national air quality monitoring network. When PM10 and SO2 estimation results
obtained are compared with the real values, 15.15 real data belonging to PM10, is
estimated as 15.11 in LSTM. Likewise, 4.65 real data belonging to SO2, is estimated
as 5.18 in LSTM. As a result of the application, LSTM predicts PM10 and SO2 better
than the MLP and RNN models. The results were compared with other studies in the
literature, and the proposed LSTM deep learning architecture performed well
compared to studies using data sets and location information under similar
conditions.
Keywords: Air pollutants; Long short-term memory; Air quality; Forecasting; Machine
learning; Deep learning

Jian Huang, Bin Liu, Jianhua Tao,

Learning long-term temporal contexts using skip RNN for continuous emotion
recognition,
Virtual Reality & Intelligent Hardware,
Volume 3, Issue 1,
2021,
Pages 55-64,
ISSN 2096-5796,
https://doi.org/10.1016/j.vrih.2020.11.005.
(https://www.sciencedirect.com/science/article/pii/S2096579620301157)
Abstract: Background
One of the most critical issues in human-computer interaction applications is
recognizing human emotions based on speech. In recent years, the challenging
problem of cross-corpus speech emotion recognition (SER) has generated extensive
research. Nevertheless, the domain discrepancy between training data and testing
data remains a major challenge to achieving improved system performance.
Methods
This paper introduces a novel multi-scale discrepancy adversarial (MSDA) network
for conducting multiple timescales domain adaptation for cross-corpus SER, i. e.,
integrating domain discriminators of hierarchical levels into the emotion
recognition framework to mitigate the gap between the source and target domains.
Specifically, we extract two kinds of speech features, i.e., handcraft features and
deep features, from three timescales of global, local, and hybrid levels. In each
timescale, the domain discriminator and the emotion classifier compete against each
other to learn features that minimize the discrepancy between the two domains by
fooling the discriminator.
Results
Extensive experiments on cross-corpus and cross-language SER were conducted on a
combination dataset that combines one Chinese dataset and two English datasets
commonly used in SER. The MSDA is affected by the strong discriminate power
provided by the adversarial process, where three discriminators are working in
tandem with an emotion classifier. Accordingly, the MSDA achieves the best
performance over all other baseline methods.
Conclusions
The proposed architecture was tested on a combination of one Chinese and two
English datasets. The experimental results demonstrate the superiority of our
powerful discriminative model for solving cross-corpus SER.
Keywords: Continuous emotion recognition; Skip RNN; Temporal contexts; Redundancy

Madini O. Alassafi, Mutasem Jarrah, Reem Alotaibi,

Time series predicting of COVID-19 based on deep learning,
Neurocomputing,
Volume 468,
2022,
Pages 335-344,
ISSN 0925-2312,
https://doi.org/10.1016/j.neucom.2021.10.035.
(https://www.sciencedirect.com/science/article/pii/S0925231221015150)
Abstract: COVID-19 was declared a global pandemic by the World Health Organisation
(WHO) on 11th March 2020. Many researchers have, in the past, attempted to predict
a COVID outbreak and its effect. Some have regarded time-series variables as
primary factors which can affect the onset of infectious diseases like influenza
and severe acute respiratory syndrome (SARS). In this study, we have used public
datasets provided by the European Centre for Disease Prevention and Control for
developing a prediction model for the spread of the COVID-19 outbreak to and
throughout Malaysia, Morocco and Saudi Arabia. We have made use of certain
effective deep learning (DL) models for this purpose. We assessed some specific
major features for predicting the trend of the existing COVID-19 outbreak in these
three countries. In this study, we also proposed a DL approach that includes
recurrent neural network (RNN) and long short-term memory (LSTM) networks for
predicting the probable numbers of COVID-19 cases. The LSTM models showed a 98.58%
precision accuracy while the RNN models showed a 93.45% precision accuracy. Also,
this study compared the number of coronavirus cases and the number of resulting
deaths in Malaysia, Morocco and Saudi Arabia. Thereafter, we predicted the number
of confirmed COVID-19 cases and deaths for a subsequent seven days. In this study,
we presented their predictions using the data that was available up to December
3rd, 2020.
Keywords: Prediction; RNN; LSTM; COVID-19; Time series

Ebubekir Buber, Banu Diri,

Web Page Classification Using RNN,
Procedia Computer Science,
Volume 154,
2019,
Pages 62-72,
ISSN 1877-0509,
https://doi.org/10.1016/j.procs.2019.06.011.
(https://www.sciencedirect.com/science/article/pii/S187705091930780X)
Abstract: Web page classification is an information retrieval application that
provides useful information that can be a basis for many different application
domains. In this study, a deep learning-based system has been developed for the
classification of web pages. The meta tag information contained in the web page is
used to classify a web page. The meta tags used are title, description and
keywords. RNN based deep learning architecture was used during the tests. Transfer
learning is the name given to the approach to building a machine learning model
with the use of pre-trained parameters to solve a problem. The effect of using
transfer learning on the system has also been examined. According to the results
obtained, success rate of web page classification system is approximately 85%. It
is not observed that transfer learning has significant contribution to the success
rates. However, the use of transfer learning has reduced the consumed system
resources.
Keywords: web page classification; classification; categorization; deep learning;
RNN; transfer learning

Xinxing Zhao, Joel Weijia Lai, Andrew Fu Wah Ho, Nan Liu, Marcus Eng Hock Ong, Kang
Hao Cheong,
Predicting hospital emergency department visits with deep learning approaches,
Biocybernetics and Biomedical Engineering,
Volume 42, Issue 3,
2022,
Pages 1051-1065,
ISSN 0208-5216,
https://doi.org/10.1016/j.bbe.2022.07.008.
(https://www.sciencedirect.com/science/article/pii/S0208521622000729)
Abstract: Overcrowding in emergency department (ED) causes lengthy waiting times,
reduces adequate emergency care and increases rate of mortality. Accurate
prediction of daily ED visits and allocating resources in advance is one of the
solutions to ED overcrowding problem. In this paper, a deep stacked architecture is
being proposed and applied to the daily ED visits prediction problem with deep
components such as Long Short Term Memory (LSTM), Gated Recurrent Units (GRU) and
simple Recurrent Neural Network (RNN). The proposed architecture achieves very high
mean accuracy level (94.28–94.59%) in daily ED visits predictions. We have also
compared the performance of this architecture with non-stacked deep models and
traditional prediction models. The results indicate that deep stacked models
outperform (4–7%) the traditional prediction models and other non-stacked deep
learning models (1–2%) in our prediction tasks. The application of deep neural
network in ED visits prediction is novel as this is one of the first studies to
apply a deep stacked architecture in this field. Importantly, our models have
achieved better prediction accuracy (in one case comparable) than the state-of-the-
art in the literature.
Keywords: Deep learning; Deep stacked architecture; ED Prediction; LSTM; RNN; GRU

Junhua Zheng, Chao Wu, Qingqiang Sun, Zhihuan Song, Le Zhou,

Deep learning of complex process data for fault classification based on sparse
probabilistic dynamic network,
Journal of the Taiwan Institute of Chemical Engineers,
Volume 138,
2022,
104498,
ISSN 1876-1070,
https://doi.org/10.1016/j.jtice.2022.104498.
(https://www.sciencedirect.com/science/article/pii/S1876107022002978)
Abstract: Background
The dynamic and nonlinear characteristics of process data have become the major
issue in data-driven process monitoring. Traditional data-driven methods are often
only able to extract a single feature in process data. Therefore, how to
effectively extract multi-dimensional features has become the focus of current
research.
Methods
Sparse probabilistic dynamic network (SPDN) is a deep learning model proposed in
this paper for the purpose of fault classification. The method mainly takes the
advantages of the sparse Gaussian-Bernoulli Restricted Boltzmann Machine (GRBM) and
the recurrent neural network (RNN) with long-short term memory (LSTM) units. First,
the sparse GRBM is used for nonlinear feature extraction in an unsupervised way.
Then, LSTM is introduced to realize the modeling of sequence data which can
effectively handle the dynamic feature of the data.
Findings
In the Tennessee-Eastman benchmark process, the classification accuracies of the
proposed method are proved to be far superior to MLP, RNN and PDN. Meanwhile, in
order to prove the influence of the data dynamics and the internal parameters of
the structure on the fault classification results, two additional experiments were
carried out.
Keywords: Process monitoring; Fault classification; Gaussian-Bernoulli restricted
Boltzmann machine; Recurrent neural network; Dynamic modeling; Nonlinear feature
extraction

Phuong Thao Thi Ngo, Mahdi Panahi, Khabat Khosravi, Omid Ghorbanzadeh, Narges
Kariminejad, Artemi Cerda, Saro Lee,
Evaluation of deep learning algorithms for national scale landslide susceptibility
mapping of Iran,
Geoscience Frontiers,
Volume 12, Issue 2,
2021,
Pages 505-519,
ISSN 1674-9871,
https://doi.org/10.1016/j.gsf.2020.06.013.
(https://www.sciencedirect.com/science/article/pii/S1674987120301687)
Abstract: The identification of landslide-prone areas is an essential step in
landslide hazard assessment and mitigation of landslide-related losses. In this
study, we applied two novel deep learning algorithms, the recurrent neural network
(RNN) and convolutional neural network (CNN), for national-scale landslide
susceptibility mapping of Iran. We prepared a dataset comprising 4069 historical
landslide locations and 11 conditioning factors (altitude, slope degree, profile
curvature, distance to river, aspect, plan curvature, distance to road, distance to
fault, rainfall, geology and land-sue) to construct a geospatial database and
divided the data into the training and the testing dataset. We then developed RNN
and CNN algorithms to generate landslide susceptibility maps of Iran using the
training dataset. We calculated the receiver operating characteristic (ROC) curve
and used the area under the curve (AUC) for the quantitative evaluation of the
landslide susceptibility maps using the testing dataset. Better performance in both
the training and testing phases was provided by the RNN algorithm (AUC = 0.88) than
by the CNN algorithm (AUC = 0.85). Finally, we calculated areas of susceptibility
for each province and found that 6% and 14% of the land area of Iran is very highly
and highly susceptible to future landslide events, respectively, with the highest
susceptibility in Chaharmahal and Bakhtiari Province (33.8%). About 31% of cities
of Iran are located in areas with high and very high landslide susceptibility. The
results of the present study will be useful for the development of landslide hazard
mitigation strategies.
Keywords: CNN; RNN; Deep learning; Landslide; Iran

Muralitharan Krishnan, Yongdo Lim, Seethalakshmi Perumal, Gayathri Palanisamy,

Detection and defending the XSS attack using novel hybrid stacking ensemble
learning-based DNN approach,
Digital Communications and Networks,
2022,
,
ISSN 2352-8648,
https://doi.org/10.1016/j.dcan.2022.09.024.
(https://www.sciencedirect.com/science/article/pii/S2352864822001997)
Abstract: Existing web-based security applications have failed in many situations
due to the great intelligence of attackers. Among web applications, Cross-Site
Scripting (XSS) is one of the dangerous assaults experienced while modifying an
organization's or user's information. To avoid these security challenges, this
article proposes a novel, all-encompassing combination of machine learning (NB,
SVM, k-NN) and deep learning (RNN, CNN, LSTM) frameworks for detecting and
defending against XSS attacks with high accuracy and efficiency. Based on the
representation, a novel idea for merging stacking ensemble with web applications,
termed “hybrid stacking”, is proposed. In order to implement the aforementioned
methods, four distinct datasets, each of which contains both safe and unsafe
content, are considered. The hybrid detection method can adaptively identify the
attacks from the URL, and the defense mechanism inherits the advantages of URL
encoding with dictionary-based mapping to improve prediction accuracy, accelerate
the training process, and effectively remove the unsafe JScript/JavaScript keywords
from the URL. The simulation results show that the proposed hybrid model is more
efficient than the existing detection methods. It produces more than 99.5% accurate
XSS attack classification results (accuracy, precision, recall, f1_score, and ROC)
and is highly resistant to XSS attacks. In order to ensure the security of the
server's information, the proposed hybrid approach is demonstrated in a real-time
environment.
Keywords: Machine learning; Deep neural networks; Classification; Stacking
ensemble; XSS attack; URL encoding; JScript/JavaScript; Web security

Lulu Qi, Jialuo Du, Yue Sun, Yongzhao Xiong, Xinyao Zhao, Daodong Pan, Yueru Zhi,
Yali Dang, Xinchang Gao,
Umami-MRNN: Deep learning-based prediction of umami peptide using RNN and MLP,
Food Chemistry,
Volume 405, Part A,
2023,
134935,
ISSN 0308-8146,
https://doi.org/10.1016/j.foodchem.2022.134935.
(https://www.sciencedirect.com/science/article/pii/S0308814622028977)
Abstract: Umami components are an important part of food condiments, and the use of
umami peptides in the condiment industry has received great attention. However,
traditional methods for umami peptide identification are time-consuming, labor-
intensive, and difficult to achieve high throughput. Therefore, it is essential to
develop an effective algorithm to identify potential umami peptides. In this study,
we proposed a prediction method for umami peptides called Umami-MRNN. We
constructed a merged model for the Multi-layer Perceptron and Recurrent Neural
Network. We then developed predictors with six feature vectors as the input. We
trained the neural networks using the training dataset and selected hyperparameters
of machine learning models via a 10-fold cross-validation. The independent tests
showed that Umami-MRNN achieved an accuracy of 90.5% and a Matthews correlation
coefficient value of 0.811. To assist the scientific community, we also developed a
publicly accessible web server at https://umami-mrnn.herokuapp.com/.
Keywords: Umami peptides; Deep learning; Multi-layer perceptron; Recurrent neural
network; Predictor

Xuan Li, Wei Zhang,

Physics-informed deep learning model in wind turbine response prediction,
Renewable Energy,
Volume 185,
2022,
Pages 932-944,
ISSN 0960-1481,
https://doi.org/10.1016/j.renene.2021.12.058.
(https://www.sciencedirect.com/science/article/pii/S0960148121017791)
Abstract: Subjected to strong cyclic wind and wave loads, wind turbines could
experience severe fatigue damages and possibly fail to function normally due to
accumulated damages at certain critical locations. Therefore, fatigue damage
evaluation and prediction are essential and important to be conducted, which could
involve massive numerical simulations and computational costs due to dynamic
analyses of the wind turbines under various environmental conditions. To reduce the
calculation cost related to the time-consuming dynamic analysis, sequence models
such as the recurrent neural network (RNN) and the long-short term memory model
(LSTM) originated from the deep learning topic are good and promising candidates to
predict structural dynamic responses at multiple critical locations under different
environmental scenarios. However, the training cost and prediction accuracy of
these deep learning models might not be satisfiable since these models are purely
data-driven and require significant amount of training data and a large number of
training parameters. To reduce the computational cost and improve the prediction
accuracy, a hybrid method that integrates the physical information of the
underlying wind turbine system into the data-driven model is implemented in the
present study as a computationally efficient simulation model. Structural
properties and linearized representations of the wind turbine system are served as
the physical constraints and applied in a recently proposed deep residual recurrent
neural network (DR-RNN) to form as a physics-informed deep learning model. This
physics-informed model is first applied to a frame structure with four degrees of
freedom as a benchmark study to show the accuracy and efficiency of this model. The
applicability of this physics-informed model to a complex wind turbine system is
then investigated, and the performance of the developed physics-informed model on
the structural response prediction is also compared with a regular data-driven
model.
Keywords: Physics-informed deep learning model; Wind turbine; Recurrent neural
network (RNN); Long-short term memory (LSTM); Structure linearization; Response
prediction

Rohaifa Khaldi, Abdellatif El Afia, Raddouane Chiheb, Siham Tabik,

What is the best RNN-cell structure to forecast each time series behavior?,
Expert Systems with Applications,
2022,
119140,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.119140.
(https://www.sciencedirect.com/science/article/pii/S0957417422021583)
Abstract: It is unquestionable that time series forecasting is of paramount
importance in many fields. The most used machine learning models to address time
series forecasting tasks are Recurrent Neural Networks (RNNs). Typically, those
models are built using one of the three most popular cells, ELMAN, Long-Short Term
Memory (LSTM), or Gated Recurrent Unit (GRU) cells, each cell has a different
structure and implies a different computational cost. However, it is not clear why
and when to use each RNN-cell structure. Actually, there is no comprehensive
characterization of all the possible time series behaviors and no guidance on what
RNN cell structure is the most suitable for each behavior. The objective of this
study is two-fold: it presents a comprehensive taxonomy of all-time series
behaviors (deterministic, random-walk, nonlinear, long-memory, and chaotic), and
provides insights into the best RNN cell structure for each time series behavior.
We conducted two experiments: (1) The first experiment evaluates and analyzes the
role of each component in the LSTM-Vanilla cell by creating 11 variants based on
one alteration in its basic architecture (removing, adding, or substituting one
cell component). (2) The second experiment evaluates and analyzes the performance
of 20 possible RNN-cell structures. To evaluate, compare, and select the best
model, different statistical metrics were used: error-based metrics, information
criterion-based metrics, naïve-based metric, and direction change-based metric. To
further improve our confidence in the models’ interpretation and selection,
Friedman Wilcoxon-Holm signed-rank test was used. Our results showed that the MUT2,
SCRN, and ELMAN cells are the most recommended to forecast time series data with
deterministic, random-walk, and nonlinear behaviors, respectively. Whereas, the
MGU-SLIM2 and the LSTM-SLIM3 are the most suitable for the long-memory and chaotic
behaviors, respectively.
Keywords: Forecasting; Time series; Times series behavior; RNN models; LSTM cells;
Performance evaluation metrics

Jingyang Wang, Xiaolei Li, Jiazheng Li, Qiuhong Sun, Haiyao Wang,
NGCU: A New RNN Model for Time-Series Data Prediction,
Big Data Research,
Volume 27,
2022,
100296,
ISSN 2214-5796,
https://doi.org/10.1016/j.bdr.2021.100296.
(https://www.sciencedirect.com/science/article/pii/S2214579621001131)
Abstract: With the rapid development of machine learning, a possibility is provided
for high-precision prediction of time-series. This paper proposes a new unit which
is called New Gate Control Unit (NGCU) based on Recurrent Neural Networks (RNN).
The proposal of NGCU is mainly used for prediction of time-series data. NGCU
alleviates the problems of gradient disappearance and explosion of traditional RNN.
Compared with Long Short-Term Memory (LSTM) and Gate Recurrent Unit (GRU), NGCU
improves not only the computational complexity of gating unit but also the
sensitivity of model learning. To verify the accuracy, efficiency and feasibility
of NGCU, this paper uses RNN, LSTM and GRU to conduct comparative experiments, and
uses three different data of air quality, Hang Seng Index, and gold future price to
prove the generalization of NGCU. Mean Absolute Error (MAE), Mean Squared Error
(MSE), Explained Variance Score (EVS), R2 and training time are used to evaluate
experimental results. Among the three different data prediction results, the R2 of
NGCU is 0.9736, 0.9872, and 0.9231, respectively. And NGCU's MAE, MSE, EVS are also
the best. Compared with LSTM and GRU, NGCU has the least training time, which is
323.5261s, 53.3257s, and 43.4814s respectively.
Keywords: NGCU; RNN; Time-series; Prediction

Mohammad Eslami, Julia A. Kim, Miao Zhang, Michael V. Boland, Mengyu Wang, Dolly S.
Chang, Tobias Elze,
Visual Field Prediction: Evaluating the Clinical Relevance of Deep Learning Models,
Ophthalmology Science,
Volume 3, Issue 1,
2023,
100222,
ISSN 2666-9145,
https://doi.org/10.1016/j.xops.2022.100222.
(https://www.sciencedirect.com/science/article/pii/S2666914522001117)
Abstract: Purpose
Two novel deep learning methods using a convolutional neural network (CNN) and a
recurrent neural network (RNN) have recently been developed to forecast future
visual fields (VFs). Although the original evaluations of these models focused on
overall accuracy, it was not assessed whether they can accurately identify patients
with progressive glaucomatous vision loss to aid clinicians in preventing further
decline. We evaluated these 2 prediction models for potential biases in
overestimating or underestimating VF changes over time.
Design
Retrospective observational cohort study.
Participants
All available and reliable Swedish Interactive Thresholding Algorithm Standard 24-2
VFs from Massachusetts Eye and Ear Glaucoma Service collected between 1999 and 2020
were extracted. Because of the methods’ respective needs, the CNN data set included
54 373 samples from 7472 patients, and the RNN data set included 24 430 samples from
1809 patients.
Methods
The CNN and RNN methods were reimplemented. A fivefold cross-validation procedure
was performed on each model, and pointwise mean absolute error (PMAE) was used to
measure prediction accuracy. Test data were stratified into categories based on the
severity of VF progression to investigate the models’ performances on predicting
worsening cases. The models were additionally compared with a no-change model that
uses the baseline VF (for the CNN) and the last-observed VF (for the RNN) for its
prediction.
Main Outcome Measures
PMAE in predictions.
Results
The overall PMAE 95% confidence intervals were 2.21 to 2.24 decibels (dB) for the
CNN and 2.56 to 2.61 dB for the RNN, which were close to the original studies’
reported values. However, both models exhibited large errors in identifying
patients with worsening VFs and often failed to outperform the no-change model.
Pointwise mean absolute error values were higher in patients with greater changes
in mean sensitivity (for the CNN) and mean total deviation (for the RNN) between
baseline and follow-up VFs.
Conclusions
Although our evaluation confirms the low overall PMAEs reported in the original
studies, our findings also reveal that both models severely underpredict worsening
of VF loss. Because the accurate detection and projection of glaucomatous VF
decline is crucial in ophthalmic clinical practice, we recommend that this
consideration is explicitly taken into account when developing and evaluating
future deep learning models.
Keywords: Deep learning; Artificial intelligence; Glaucoma; Visual fields;
Prediction

Mumtaz Ali, Ramendra Prasad, Yong Xiang, Adarsh Sankaran, Ravinesh C. Deo, Fuyuan
Xiao, Shuyu Zhu,
Advanced extreme learning machines vs. deep learning models for peak wave energy
period forecasting: A case study in Queensland, Australia,
Renewable Energy,
Volume 177,
2021,
Pages 1031-1044,
ISSN 0960-1481,
https://doi.org/10.1016/j.renene.2021.06.052.
(https://www.sciencedirect.com/science/article/pii/S0960148121009186)
Abstract: The peak period of an energy-generating wave is one of the most important
parameters that describe the spectral shape of the oceanic wave, as this indicates
the duration for which the waves prevail with respect to their maximum extractable
energy. In this paper, a half-hourly peak wave energy period (TP) forecast model is
constructed using a suite of statistically significant lagged inputs based on the
partial auto-correlation function with an extreme learning machine model developed
and its predictive utility is benchmarked against deep learning models, i.e.,
convolutional neural network (CNN/CovNet) and recurrent neural network (RNN) models
and other traditional M5tree, Conditional Maximization based Multiple Linear
Regression (MLR-ECM) and MLR models. The objective model (ELM) vs. the comparison
models (CNN, RNN, M5tree, MLR-ECM, and MLR) were trained and validated
independently on the test dataset obtained from coastal zones of eastern Australia
that have a high potential for implementation of wave energy generation systems.
The outcomes ascertain that the ELM model can generate significantly accurate
predictions of the half-hourly peak wave energy period, providing a good level of
accuracy relative to deep learning models in selected coastal study zones. The
study establishes the practical usefulness of the ELM model as being a noteworthy
methodology for the applications in renewable and sustainable energy resource
management systems.
Keywords: Deep learning; RNN; CNN; ELM; Peak wave energy period; Coastal waves

Raghavendra Chalapathy, Nguyen Lu Dang Khoa, Subbu Sethuvenkatraman,

Comparing multi-step ahead building cooling load prediction using shallow machine
learning and deep learning models,
Sustainable Energy, Grids and Networks,
Volume 28,
2021,
100543,
ISSN 2352-4677,
https://doi.org/10.1016/j.segan.2021.100543.
(https://www.sciencedirect.com/science/article/pii/S2352467721001144)
Abstract: Accurate building cooling load prediction is beneficial in managing
optimal operation to conserve energy user and operational cost. Several physics-
based and data-driven models proposed to forecast building cooling load focus on
one-step-ahead prediction. Deep learning-based Long-term memory (LSTM) models are
shown to perform well for one-step short-term (1Hour Ahead) building cooling load
prediction. However, no prior studies have examined the prediction performance of
shallow machine learning methods over deep learning models to forecast building
cooling demand over multi-steps across diverse real-world datasets. A multi-step
model learns a single parametric function from input time series and forecasts an
array of building cooling load values (multi-step) simultaneously. A comprehensive
study has been carried out to evaluate the performance of six data-driven models (2
shallow learning, 3 deep sequential learning, and 1 heuristic method) to predict
multi-step long-term (1Day Ahead) building cooling load. Our results demonstrate
variant of the LSTM model, the Recurrent Neural Network Multi-Input Multi-Output
(RNN-MIMO) network architecture, performs consistently well compared to its deep
learning counterparts and shallow machine learning techniques, both tree boosting
and support vector regression. Notable conclusions from results obtained are
twofold: Firstly, Long short-term memory (LSTM) based RNN-MIMO architecture
performs well in both short-term (1Hour Ahead) and long-term (1Day Ahead) multi-
step forecast horizon. RNN-MIMO is up to 33% more accurate, on average, in terms of
mean absolute error over existing, state-of-the-art shallow machine learning models
both Support Vector Regression (SVR) and tree boosting techniques (XGBoost). Our
findings have significant implications for practice. Notably, machine learning
models trained on one-step-ahead predictions cannot be deployed readily to predict
multiple time steps into the future since longer prediction horizons impose
additional training and fine-tuning efforts over each of the multiple steps. RNN-
MIMO model’s ability to predict multiple time steps simultaneously eliminates the
need for manual fine-tuning of individual models for each required forecast
horizon.
Keywords: Deep recurrent; Shopping centre, office building, hospital; HVAC; Cooling
load prediction; Machine-learning; Multi-step

Rinku Supakar, Parthasarathi Satvaya, Prasun Chakrabarti,

A deep learning based model using RNN-LSTM for the Detection of Schizophrenia from
EEG data,
Computers in Biology and Medicine,
Volume 151, Part A,
2022,
106225,
ISSN 0010-4825,
https://doi.org/10.1016/j.compbiomed.2022.106225.
(https://www.sciencedirect.com/science/article/pii/S0010482522009337)
Abstract: Normal life can be ensured for schizophrenic patients if diagnosed early.
Electroencephalogram (EEG) carries information about the brain network connectivity
which can be used to detect brain anomalies that are indicative of schizophrenia.
Since deep learning is capable of automatically extracting the significant features
and make classifications, the authors proposed a deep learning based model using
RNN-LSTM to analyze the EEG signal data to diagnose schizophrenia. The proposed
model used three dense layers on top of a 100 dimensional LSTM. EEG signal data of
45 schizophrenic patients and 39 healthy subjects were used in the study.
Dimensionality reduction algorithm was used to obtain an optimal feature set and
the classifier was run with both sets of data. An accuracy of 98% and 93.67% were
obtained with the complete feature set and the reduced feature set respectively.
The robustness of the model was evaluated using model performance measure and
combined performance measure. Outcomes were compared with the outcome obtained with
traditional machine learning classifiers such as Random Forest, SVM, FURIA, and
AdaBoost, and the proposed model was found to perform better with the complete
dataset. When compared with the result of the researchers who worked with the same
set of data using either CNN or RNN, the proposed model's accuracy was either
better or comparable to theirs.
Keywords: Schizophrenia; Electroencephalogram; Deep learning; RNN-LSTM; CNN

Ali Caglayan, Nevrez Imamoglu, Ahmet Burak Can, Ryosuke Nakamura,

When CNNs meet random RNNs: Towards multi-level analysis for RGB-D object and scene
recognition,
Computer Vision and Image Understanding,
Volume 217,
2022,
103373,
ISSN 1077-3142,
https://doi.org/10.1016/j.cviu.2022.103373.
(https://www.sciencedirect.com/science/article/pii/S1077314222000133)
Abstract: Recognizing objects and scenes are two challenging but essential tasks in
image understanding. In particular, the use of RGB-D sensors in handling these
tasks has emerged as an important area of focus for better visual understanding.
Meanwhile, deep neural networks, specifically convolutional neural networks (CNNs),
have become widespread and have been applied to many visual tasks by replacing
hand-crafted features with effective deep features. However, it is an open problem
how to exploit deep features from a multi-layer CNN model effectively. In this
paper, we propose a novel two-stage framework that extracts discriminative feature
representations from multi-modal RGB-D images for object and scene recognition
tasks. In the first stage, a pretrained CNN model has been employed as a backbone
to extract visual features at multiple levels. The second stage maps these features
into high level representations with a fully randomized structure of recursive
neural networks (RNNs) efficiently. To cope with the high dimensionality of CNN
activations, a random weighted pooling scheme has been proposed by extending the
idea of randomness in RNNs. Multi-modal fusion has been performed through a soft
voting approach by computing weights based on individual recognition confidences
(i.e. SVM scores) of RGB and depth streams separately. This produces consistent
class label estimation in final RGB-D classification performance. Extensive
experiments verify that fully randomized structure in RNN stage encodes CNN
activations to discriminative solid features successfully. Comparative experimental
results on the popular Washington RGB-D Object and SUN RGB-D Scene datasets show
that the proposed approach achieves superior or on-par performance compared to
state-of-the-art methods both in object and scene recognition tasks. Code is
available at https://github.com/acaglayan/CNN_randRNN.
Keywords: Convolutional Neural Networks; Randomized neural networks; Transfer
learning; RGB-D object recognition; RGB-D scene recognition

Colin Bonatti, Bekim Berisha, Dirk Mohr,

From CP-FFT to CP-RNN: Recurrent neural network surrogate model of crystal
plasticity,
International Journal of Plasticity,
Volume 158,
2022,
103430,
ISSN 0749-6419,
https://doi.org/10.1016/j.ijplas.2022.103430.
(https://www.sciencedirect.com/science/article/pii/S074964192200208X)
Abstract: Recurrent Neural Network (RNN) based surrogate models constitute an
emerging class of reduced order models of history-dependent material behavior.
Recently, the authors have proposed an alternative RNN formulation that provides
stress-responses independent of the time-discretization of the input-path, making
it appropriate for integration into explicit finite element (FE) frameworks.
Herein, we apply the same methodology to 2D and 3D datasets corresponding to the
effective mechanical behavior of an aluminum alloy as obtained through Crystal
Plasticity simulations. In both cases, we obtain reasonable approximations of the
behavior using RNN models of size ranging from 5'000 to 100’000 parameters. We also
develop a methodology to reduce observed numerical instabilities of the finite
element implementations.
Keywords: Crystal plasticity; Plasticity; Homogenization; Recurrent neural networks

Kotha Manohar, E. Logashanmugam,

Hybrid deep learning with optimal feature selection for speech emotion recognition
using improved meta-heuristic algorithm,
Knowledge-Based Systems,
Volume 246,
2022,
108659,
ISSN 0950-7051,
https://doi.org/10.1016/j.knosys.2022.108659.
(https://www.sciencedirect.com/science/article/pii/S0950705122003021)
Abstract: Speech emotion recognition is the crucial stream in emotional computing
and also create few issues owing to its complication in processing. The efficiency
of the acoustic methods and their speech features are improved using various
existing methods. Yet, the conventional acoustic methods are not effective in
handling speech emotion recognition because of their drawbacks. The main intend of
this research is to implement a new speech emotion recognition using the hybrid
deep learning model. Initially, few speech emotion recognition dataset is gathered
from the public sources and is put forwarded for pre-processing using artifacts
removal and filtering techniques. Then, the feature extraction of the speech
signals is performed by the Mel-Frequency Cepstral Coefficients (MFCC), mel-scale
spectrogram, tonal power, and spectral flux. In the aim of decreasing the feature
size for boosting up the learning performance, for selecting the optimal feature is
adopted by the Deer Hunting with Adaptive Search (DH-AS) algorithm. These optimal
features are used for the emotion classification by the Hybrid Deep Learning (HDL)
with “Deep Neural Network (DNN) and Recurrent Neural Network (RNN)”. These two
networks are enhanced by the developed DH-AS, thus could reach high classification
accuracy while classifying the emotions like “happy, sad, anger, fear, calm etc”.
The performance of the suggested DH-AS-HDL correspondingly improves 3.15%, 5.37%,
4.25% and 4.81% better accuracy than the PSO-HDL, GWO-HDL, WOA-HDL and DHOA-HDL,
when the learning rate as 85. The achieved results prove that the developed model
obtains superior performance by evaluating its performance through various
performance metrics.
Keywords: Speech emotion recognition; Deer hunting with adaptive search; Optimal
feature selection; Recurrent Neural Network; Deep Neural Network; Hybrid Deep
Learning

Mohammed Farsi,
Application of ensemble RNN deep neural network to the fall detection through IoT
environment,
Alexandria Engineering Journal,
Volume 60, Issue 1,
2021,
Pages 199-211,
ISSN 1110-0168,
https://doi.org/10.1016/j.aej.2020.06.056.
(https://www.sciencedirect.com/science/article/pii/S1110016820303239)
Abstract: The emerging of new models in machine learning enhances the performance
of algorithms proposed to address several challenging tasks such as object
recognition, classification and identification purpose. Nowadays. the deep learning
algorithms are playing a massive role in accurately addressing complex problems due
to their capability of learning various complex features from data. One limitation
of deep learning is a lack of sufficient data for training. In this study, we
proposed variants of Long Short Term Memory (LSTM) model and ensemble learning
methods such as XGBoost, AdaBoost, Bagging, Stacking and Random forest. The
experimentation is carried out on Time series data generated from the Internet of
Things (IoT) devices. To validate the proposed method, we have used a freely
available dataset on the web namely Smart-Fall datasets. To measure the performance
of the proposed method, we have used standard performance measures namely,
accuracy, precision, recall, f-score, specificity, geometric mean and confusion
matrix. A set of experimental details are carried out on the SmartFall dataset and
the experimental results exhibit that the Random forest algorithm performs better
when compared with a single deep LSTM model and different ensemble techniques.
Keywords: Ensemble methods; Deep learning; Recurrent neural network; Fall
detection; Time series; IoT

Abhijit Singh, Ananya Nath, Shibendu Shekhar Roy, Aditya Kumar Lohar,
Modeling of laser aided direct metal deposition of stainless steel using supervised
deep learning algorithms,
Materials Today: Proceedings,
Volume 62, Part 1,
2022,
Pages 360-366,
ISSN 2214-7853,
https://doi.org/10.1016/j.matpr.2022.03.468.
(https://www.sciencedirect.com/science/article/pii/S2214785322017291)
Abstract: Laser Direct Metal Deposition (LDMD) is a very promising additive
manufacturing methodology which provides metal cladding layer over the substrate
with good corrosion resistance, desired surface finish and very accurate geometry.
In this work, an attempt has been made to develop a forward and reverse model of
laser aided direct metal deposition utilizing deep learning algorithms. Forward
model aims at predicting the deposited bead height and width from the known set of
LDMD process parameters such as laser power, scanning velocity, and powder flow
rate. Reverse modelling shows the effectiveness of each input parameter over the
chosen response factors. Therefore, in the reverse modelling, the optimized input
parameters values have been tried to predict to get the desired deposition
characteristics during experimentation. Three different neural networks based deep
learning architectures, namely, Multi-Layer Feed Forward Neural Network (MLFFNN),
Recurrent Neural Networks (RNN) and Radial Basis Function Neural Network (RBFNN),
have been put forward to develop the said predictive models. The supervised
learning algorithms are used to train the neural networks. The performances of
MLFFNN, RNN and RBFNN models are compared among themselves. The results show that
all the models are capable of making better predictions and the models can be
effectively used in shop floor in optimal selection of most influential parameters
for the desired outputs.
Keywords: Direct metal deposition; Forward modeling; Reverse modeling; Additive
Manufacturing; Supervised learning; Deep learning algorithms

Jihong Deng, Xujie Zhang, Meichen Li, Hui Jiang, Quansheng Chen,
Feasibility study on Raman spectra-based deep learning models for monitoring the
contamination degree and level of aflatoxin B1 in edible oil,
Microchemical Journal,
Volume 180,
2022,
107613,
ISSN 0026-265X,
https://doi.org/10.1016/j.microc.2022.107613.
(https://www.sciencedirect.com/science/article/pii/S0026265X22004416)
Abstract: Edible oil is one of the three major nutritional sources of human body,
and its quality is directly related to human health. This study proposes two novel
deep learning architectures to achieve qualitative and quantitative monitoring of
the degree and level of aflatoxin B1 (AFB1) contamination synthetically of edible
oil in laboratory level. Based on Raman spectra acquired, the structures of a
convolutional neural network (CNN) and a recurrent neural network (RNN) were
designed, respectively; and deep learning models were established to achieve
qualitative identification of the AFB1 contamination degree and quantitative
detection of the AFB1 contamination level of edible oil samples. The results showed
that both the CNN model and the RNN model showed perfect recognition performance
when identifying the contamination degree of edible oil samples by the AFB1, and
their recognition accuracy reached 100% when predicting the independent samples in
the prediction set. The RNN model has better detection performance than the CNN
model when detecting the AFB1 contamination level of edible oil samples. In the
prediction set, the coefficient of determination (RP2) and ration of prediction to
deviation (RPD) of the RNN model were 0.95 and 4.86, respectively. The feasibility
study results demonstrate that deep learning combined with Raman spectroscopy can
achieve high-precision monitoring of edible oil mycotoxins. In addition, deep
learning has a good promising tool in the field of spectral chemometrics analysis.
Keywords: Edible oil; Aflatoxin B1; Raman spectroscopy; Convolutional neural
network; Recurrent neural network

K.E. ArunKumar, Dinesh V. Kalaga, Ch. Mohan Sai Kumar, Masahiro Kawaji, Timothy M
Brenza,
Forecasting of COVID-19 using deep layer Recurrent Neural Networks (RNNs) with
Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTM) cells,
Chaos, Solitons & Fractals,
Volume 146,
2021,
110861,
ISSN 0960-0779,
https://doi.org/10.1016/j.chaos.2021.110861.
(https://www.sciencedirect.com/science/article/pii/S0960077921002149)
Abstract: In December 2019, first case of the COVID-19 was reported in Wuhan, Hubei
province in China. Soon world health organization has declared contagious
coronavirus disease (a.k.a. COVID-19) as a global pandemic in the month of March
2020. Over the span of eleven months, it has rapidly spread out all over the world
with total confirmed cases of ~ 41.39 M and causing a total fatality of ~1.13 M. At
present, the entire mankind is facing serious threat and it is believed that COVID-
19 may have been around for quite some time. Therefore, it has become imperative to
forecast the global impact of COVID-19 in the near future. The present work
proposes state-of-art deep learning Recurrent Neural Networks (RNN) models to
predict the country-wise cumulative confirmed cases, cumulative recovered cases and
the cumulative fatalities. The Gated Recurrent Units (GRUs) and Long Short-Term
Memory (LSTM) cells along with Recurrent Neural Networks (RNN) were developed to
predict the future trends of the COVID-19. We have used publicly available data
from John Hopkins University's COVID-19 database. In this work, we emphasize the
importance of various factors such as age, preventive measures, and healthcare
facilities, population density, etc. that play vital role in rapid spread of COVID-
19 pandemic. Therefore, our forecasted results are very helpful for countries to
better prepare themselves to control the pandemic.
Keywords: Forecasting COVID-19 pandemic; Time series analysis; Gated Recurrent
Units (GRUs); Long Short-Term Memory (LSTM); Recurrent Neural Networks (RNNs)

Siva Sankari Subbiah, Jayakumar Chinnappan,

Deep learning based short term load forecasting with hybrid feature selection,
Electric Power Systems Research,
Volume 210,
2022,
108065,
ISSN 0378-7796,
https://doi.org/10.1016/j.epsr.2022.108065.
(https://www.sciencedirect.com/science/article/pii/S0378779622002875)
Abstract: The reliable and an economic operation of the power system rely on an
accurate prediction of short term load. In this paper, a deep learning based Long
Short Term Memory (LSTM) with hybrid feature selection namely RMR-HFS-LSTM, is
proposed. The objective of this study is to reduce the curse of dimensionality,
reduce the overfitting and improve the accuracy of short term load forecasting. The
RMR-HFS is a combination of filter and wrapper feature selection introduced for
identifying optimal subset of features. The instance based RReliefF and information
theoretic based mutual information filter feature selection are utilized to reduce
curse of dimensionality by finding and eliminating irrelevant features. The
selected features of filter feature selection is tuned by using Recursive Feature
Elimination (RFE) wrapper feature selection to reduce overfitting. The deep
learning based LSTM improves the accuracy by handling uncertainty issues. The
experiment was conducted on European weather and electricity load data using python
on Tensorflow environment. The performance of the proposed RMR-HFS-LSTM model is
compared against Multilayer Perceptron (MLP) and Recurrent Neural Network (RNN) in
terms of Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE).
The result shows that the proposed RMR-HFS-LSTM model outperforms other models.
Keywords: Deep learning; Hybrid feature selection; Long short term memory; Power
system; Recurrent neural network

Jinyong Wang, Ce Zhang,

Software reliability prediction using a deep learning model based on the RNN
encoder–decoder,
Reliability Engineering & System Safety,
Volume 170,
2018,
Pages 73-82,
ISSN 0951-8320,
https://doi.org/10.1016/j.ress.2017.10.019.
(https://www.sciencedirect.com/science/article/pii/S0951832017303538)
Abstract: Different software reliability models, such as parameter and non-
parameter models, have been developed in the past four decades to assess software
reliability in the software testing process. Although these models can effectively
assess software reliability in certain testing scenarios, no single model can
accurately predict the fault number in software in all testing conditions. In
particular, modern software is developed with more sizes and functions, and
assessing software reliability is a remarkably difficult task. The recently
developed deep learning model, called deep neural network (NN) model, has suitable
prediction performance. This deep learning model not only deepens the layer levels
but can also adapt to capture the training characteristics. A comprehensive, in-
depth study and feature excavation ultimately shows the model can have suitable
prediction performance. This study utilizes a deep learning model based on the
recurrent NN (RNN) encoder–decoder to predict the number of faults in software and
assess software reliability. Experimental results show that the proposed model has
better prediction performance compared with other parameter and NN models.
Keywords: Deep learning model based on RNN encoder–decoder; Model comparison;
Neural network models; Parameter models; Software reliability

Jingqiu Guo, Yangzexi Liu, Qingyan (Ken) Yang, Yibing Wang, Shouen Fang,
GPS-based citywide traffic congestion forecasting using CNN-RNN and C3D hybrid
model,
Transportmetrica A Transport Science,
Volume 17, Issue 2,
2021,
Pages 190-211,
ISSN 2324-9935,
https://doi.org/10.1080/23249935.2020.1745927.
(https://www.sciencedirect.com/science/article/pii/S2324993522001531)
Abstract: ABSTRACT
Traffic congestion prediction in citywide road networks is a challenging research
field in metropolitan transportation operation and management. Recent advances in
GPS technology offer great opportunities to improve upon the limitations on the
availability and quality of traffic data. Motivated by the success of deep neural
networks and considering the spatial dependencies and temporal evolutions of
network traffic, we propose an innovative deep learning-based mapping to cube
architecture for network-wide urban traffic forecasting. Experiments using real
Taxi GPS vehicle trajectory data confirm the accuracy and effectiveness of the
proposed approach combining 3-Dimensional Convolutional Networks (C3D) with
Convolutional Neuron Networks (CNNs) and Recurrent Neuron Networks (RNNs), called
CRC3D as a hybrid method integrating CNN-RNNs and C3Ds. We also compared a variety
of recurrent neural network architectures. Results show that CRC3D succeeds in
inheriting the advantages of C3D and CNN-RNN, and show its consistent and
satisfactory results in urban complex system.
Keywords: Traffic congestion; deep learning; C3D
Mahdi Panahi, Abolfazl Jaafari, Ataollah Shirzadi, Himan Shahabi, Omid Rahmati,
Ebrahim Omidvar, Saro Lee, Dieu Tien Bui,
Deep learning neural networks for spatially explicit prediction of flash flood
probability,
Geoscience Frontiers,
Volume 12, Issue 3,
2021,
101076,
ISSN 1674-9871,
https://doi.org/10.1016/j.gsf.2020.09.007.
(https://www.sciencedirect.com/science/article/pii/S1674987120302085)
Abstract: Flood probability maps are essential for a range of applications,
including land use planning and developing mitigation strategies and early warning
systems. This study describes the potential application of two architectures of
deep learning neural networks, namely convolutional neural networks (CNN) and
recurrent neural networks (RNN), for spatially explicit prediction and mapping of
flash flood probability. To develop and validate the predictive models, a
geospatial database that contained records for the historical flood events and geo-
environmental characteristics of the Golestan Province in northern Iran was
constructed. The step-wise weight assessment ratio analysis (SWARA) was employed to
investigate the spatial interplay between floods and different influencing factors.
The CNN and RNN models were trained using the SWARA weights and validated using the
receiver operating characteristics technique. The results showed that the CNN model
(AUC = 0.832, RMSE = 0.144) performed slightly better than the RNN model
(AUC = 0.814, RMSE = 0.181) in predicting future floods. Further, these models
demonstrated an improved prediction of floods compared to previous studies that
used different models in the same study area. This study showed that the spatially
explicit deep learning neural network models are successful in capturing the
heterogeneity of spatial patterns of flood probability in the Golestan Province,
and the resulting probability maps can be used for the development of mitigation
plans in response to the future floods. The general policy implication of our study
suggests that design, implementation, and verification of flood early warning
systems should be directed to approximately 40% of the land area characterized by
high and very susceptibility to flooding.
Keywords: Spatial modeling; Machine learning; Convolutional neural networks;
Recurrent neural networks; GIS; Iran

Zhenchao Wei, Xu Ji, Li Zhou, Yagu Dang, Yiyang Dai,

A novel deep learning model based on target transformer for fault diagnosis of
chemical process,
Process Safety and Environmental Protection,
Volume 167,
2022,
Pages 480-492,
ISSN 0957-5820,
https://doi.org/10.1016/j.psep.2022.09.039.
(https://www.sciencedirect.com/science/article/pii/S0957582022008138)
Abstract: Deep learning is a powerful tool for feature representation, and many
methods based on convolutional neural networks (CNNs) and recurrent neural networks
(RNNs) have been applied on fault diagnoses for chemical processes. However, unlike
attention mechanisms, these networks are inefficient when extracting features of
long-term dependencies. The transformer method employs a self-attention mechanism
and sequence-to-sequence model originally designed for natural language processing
(NLP). This approach has attracted significant attention in recent years due to its
great success in NLP fields. The fault diagnosis of a chemical process is a task
based on multi-variable time series, which are similar to text sequences with a
greater focus on long-term dependencies. This paper proposes a modified transformer
model called Target Transformer, which includes not only a self-attention
mechanism, but also a target-attention mechanism for chemical process fault
diagnoses. The Tennessee Eastman (TE) process was used to evaluate our method’s
performance.
Keywords: Fault diagnosis; Deep learning; Attention mechanism; Transformer

Mahmoud M. Bassiouni, Ripon K. Chakrabortty, Omar K. Hussain, Humyun Fuad Rahman,

Advanced deep learning approaches to predict supply chain risks under COVID-19
restrictions,
Expert Systems with Applications,
Volume 211,
2023,
118604,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.118604.
(https://www.sciencedirect.com/science/article/pii/S0957417422016566)
Abstract: The ongoing COVID-19 pandemic has created an unprecedented predicament
for global supply chains (SCs). Shipments of essential and life-saving products,
ranging from pharmaceuticals, agriculture, and healthcare, to manufacturing, have
been significantly impacted or delayed, making the global SCs vulnerable. A better
understanding of the shipment risks can substantially reduce that nervousness.
Thenceforth, this paper proposes a few Deep Learning (DL) approaches to mitigate
shipment risks by predicting ”if a shipment can be exported from one source to
another”, despite the restrictions imposed by the COVID-19 pandemic. The proposed
DL methodologies have four main stages: data capturing, de-noising or pre-
processing, feature extraction, and classification. The feature extraction stage
depends on two main variants of DL models. The first variant involves three
recurrent neural networks (RNN) structures (i.e., long short-term memory (LSTM),
Bidirectional long short-term memory (BiLSTM), and gated recurrent unit (GRU)), and
the second variant is the temporal convolutional network (TCN). In terms of the
classification stage, six different classifiers are applied to test the entire
methodology. These classifiers are SoftMax, random trees (RT), random forest (RF),
k-nearest neighbor (KNN), artificial neural network (ANN), and support vector
machine (SVM). The performance of the proposed DL models is evaluated based on an
online dataset (taken as a case study). The numerical results show that one of the
proposed models (i.e., TCN) is about 100% accurate in predicting the risk of
shipment to a particular destination under COVID-19 restrictions. Unarguably, the
aftermath of this work will help the decision-makers to predict supply chain risks
proactively to increase the resiliency of the SCs.
Keywords: Supply chain risk; COVID-19; Deep learning; Convolutional network;
Temporal convolutional network; Classifiers

Mohammad Masum, M.A. Masud, Muhaiminul Islam Adnan, Hossain Shahriar, Sangil Kim,
Comparative study of a mathematical epidemic model, statistical modeling, and deep
learning for COVID-19 forecasting and management,
Socio-Economic Planning Sciences,
Volume 80,
2022,
101249,
ISSN 0038-0121,
https://doi.org/10.1016/j.seps.2022.101249.
(https://www.sciencedirect.com/science/article/pii/S0038012122000271)
Abstract: The COVID-19 pandemic has caused a global crisis with 47,209,305
confirmed cases and 1,209,505 confirmed deaths worldwide as of November 2, 2020.
Forecasting confirmed cases and understanding the virus dynamics is necessary to
provide valuable insights into the growth of the outbreak and facilitate policy-
making regarding virus containment and utilization of medical resources. In this
study, we applied a mathematical epidemic model (MEM), statistical model, and
recurrent neural network (RNN) variants to forecast the cumulative confirmed cases.
We proposed a reproducible framework for RNN variants that addressed the stochastic
nature of RNN variants leveraging z-score outlier detection. We incorporated
heterogeneity in susceptibility into the MEM considering lockdowns and the dynamic
dependency of the transmission and identification rates which were estimated using
Poisson likelihood fitting. While the experimental results demonstrated the
superiority of RNN variants in forecasting accuracy, the MEM presented
comprehensive insights into the virus spread and potential control strategies.
Keywords: COVID-19 forecasting; Management; Mathematical epidemic model;
Statistical modeling; Deep learning

Xishuang Dong, Lijun Qian,

Semi-supervised bidirectional RNN for misinformation detection,
Machine Learning with Applications,
Volume 10,
2022,
100428,
ISSN 2666-8270,
https://doi.org/10.1016/j.mlwa.2022.100428.
(https://www.sciencedirect.com/science/article/pii/S2666827022001037)
Abstract: Misinformation refers to inaccurate information created to misguide the
readers. It spreads on social platforms like Twitter with various presentations
such as fake news and rumors that usually contain numbers, categorical information,
texts, images, etc., which has become a global issue of cybersecurity. We propose a
semi-supervised deep model based on bidirectional recurrent neural networks (Bi-
RNN) to detect misinformation with limited labeled data and large unlabeled data.
The proposed model consists of three components, namely, shared Bi-RNN, supervised
Bi-RNN, and unsupervised Bi-RNN. Specifically, the shared Bi-RNN provides common
features that input to the supervised Bi-RNN and unsupervised Bi-RNN, and they
jointly optimize two losses, namely, cross-entropy loss and mean-square-error loss,
using both labeled data and a large amount of unlabeled data. We validate our
proposed model by testing on two benchmark datasets of misinformation: LIAR and
PHEME. It is observed that the proposed model is able to achieve promising
performance even with very limited labeled data for training when compared to
baselines with supervised deep learning.
Keywords: Misinformation detection; Semi-supervised learning; Joint optimization;
Social media

Muhammad Naveed Akhter, Saad Mekhilef, Hazlie Mokhlis, Raza Ali, Muhammad Usama,
Munir Azam Muhammad, Anis Salwa Mohd Khairuddin,
A hybrid deep learning method for an hour ahead power output forecasting of three
different photovoltaic systems,
Applied Energy,
Volume 307,
2022,
118185,
ISSN 0306-2619,
https://doi.org/10.1016/j.apenergy.2021.118185.
(https://www.sciencedirect.com/science/article/pii/S0306261921014562)
Abstract: The integration of photovoltaic energy into a grid demands accurate power
output forecasting. In this research, an hour ahead prediction of power output is
performed on an annual basis over real data period (2016–2019) for three different
PV systems based on polycrystalline, monocrystalline, and thin-film technologies.
The solar radiation, ambient temperature, module temperature and wind speed are the
considered input parameters, while the power output of each PV system is the output
parameter. A hybrid deep learning (DL) method (SSA-RNN-LSTM) is proposed for an
hour ahead prediction of output power for each PV system. The proposed technique is
compared with GA-RNN-LSTM, PSO-RNN-LSTM and RNN-LSTM. The considered forecasting
accuracy measurement parameters are RMSE, MSE, MAE and coefficient of determination
(R2). The findings elaborate that SSA-RNN-LSTM has shown better forecasting
accuracy with the lowest (RMSE and MSE), highest (R2) and highest convergence speed
compared to other methods. The proposed model has shown testing (RMSE and MAE) of
(19.14% and 21.57%), (15.4% and 10.81%) and (22.9% and 25.2%) lower than RNN-LSTM
for polycrystalline, monocrystalline and thin-film PV systems respectively.
Furthermore, the proposed model is found more robust in predicting the power output
for three different PV systems over four years data period.
Keywords: An hour ahead power output forecasting; Hybrid deep learning; PV systems;
SSA-RNN-LSTM; PSO-RNN-LSTM; GA-RNN-LSTM

Yue Ma, Yingjie Hu, Glenn R. Moncrieff, Jasper A. Slingsby, Adam M. Wilson, Brian
Maitner, Ryan Zhenqi Zhou,
Forecasting vegetation dynamics in an open ecosystem by integrating deep learning
and environmental variables,
International Journal of Applied Earth Observation and Geoinformation,
Volume 114,
2022,
103060,
ISSN 1569-8432,
https://doi.org/10.1016/j.jag.2022.103060.
(https://www.sciencedirect.com/science/article/pii/S1569843222002485)
Abstract: Open (i.e., non-forest) ecosystems, such as savannas, shrublands, and
grasslands, contain over 40 % of the global total ecosystem organic carbon and
harbor a substantial portion of the world’s biodiversity. Accurately forecasting
vegetation dynamics is critical for managing biodiversity, fire, water, and carbon
in these open ecosystems. Unlike forests or other relatively stable ecosystems,
open ecosystems can have dramatically changing vegetation states since they are
prone to natural disturbances, long-term trends, and short-term events.
Consequently, it is challenging to accurately predict vegetation state in this type
of ecosystems. This paper investigates the use of deep learning based approaches
for forecasting vegetation dynamics in an open ecosystem, the fynbos shrublands of
the Cape Floristic Region of South Africa, a global biodiversity hotspot. We
experiment with different deep learning models and examine the ability of thirteen
environmental variables, such as precipitation, fire history, and temperature, to
enhance the forecasting. We find that the ConvLSTM model can forecast vegetation
state more accurately than four other compared baseline approaches. The
environmental variable mean precipitation in July (winter) provides the most
prominent enhancement for forecasting among the tested variables. Finally, we
discuss the pros and cons of using a deep learning based approach for vegetation
forecasting in open ecosystems from a conservation management perspective.
Keywords: Open ecosystems; NDVI; RNN; ConvLSTM; Cape Floristic Region; GeoAI

Simranjit Singh, Singara Singh Kasana,

Quantitative estimation of soil properties using hybrid features and RNN variants,
Chemosphere,
Volume 287, Part 1,
2022,
131889,
ISSN 0045-6535,
https://doi.org/10.1016/j.chemosphere.2021.131889.
(https://www.sciencedirect.com/science/article/pii/S0045653521023614)
Abstract: Estimating soil properties is important for maximizing the production of
crops in sustainable agriculture. The hyperspectral data next input depends upon
the previous one, and the current techniques do not take advantage of this
sequential nature of hyperspectral signatures. The variants of RNN can learn the
short-term and long-term dependencies between data. This paper proposes a deep
learning hybrid framework for quantifying the soil minerals like Clay, CEC, pH of
H2O, Nitrogen, Organic Carbon, Sand of European Union from the LUCAS library. The
hyperspectral signatures contain the data in the range of 400–2500 nm captured from
the FOSS spectroscope in the laboratory. As hyperspectral data is high dimensional,
Principal Component Analysis and Locality Preserving Projections are utilized to
form the hybrid features, which have low dimensions containing the local and global
information of the original dataset. These hybrid features are passed on to Long
Short Term Memory Networks, a deep learning framework for building an effective
prediction model. The effectiveness of the prepared models is demonstrated by
comparing it to existing state-of-the-art techniques.
Keywords: Deep learning; Hyperspectral data; Inceptisols; Quantification; Entisols;
LSTMs; Hybrid features

Amer M. Johri, Krishna V. Singh, Laura E. Mantella, Luca Saba, Aditya Sharma, John
R. Laird, Kumar Utkarsh, Inder M. Singh, Suneet Gupta, Manudeep S. Kalra, Jasjit S.
Suri,
Deep learning artificial intelligence framework for multiclass coronary artery
disease prediction using combination of conventional risk factors, carotid
ultrasound, and intraplaque neovascularization,
Computers in Biology and Medicine,
Volume 150,
2022,
106018,
ISSN 0010-4825,
https://doi.org/10.1016/j.compbiomed.2022.106018.
(https://www.sciencedirect.com/science/article/pii/S0010482522007405)
Abstract: Objective
Cardiovascular disease (CVD) is a major healthcare challenge and therefore early
risk assessment is vital. Previous assessment techniques use either “conventional
CVD risk calculators (CCVRC)” or machine learning (ML) paradigms. These techniques
are ad-hoc, unreliable, not fully automated, and have variabilities. We, therefore,
introduce AtheroEdge-MCDLAI (AE3.0DL) windows-based platform using multiclass Deep
Learning (DL) system.
Methods
Data was collected on 500 patients having both carotid ultrasound and corresponding
coronary angiography scores (CAS), measured as stenosis in coronary arteries and
considered as the gold standard. A total of 39 covariates were used, clubbed into
three clusters, namely (i) Office-based: age, gender, body mass index, smoker,
hypertension, systolic blood pressure, and diastolic blood pressure; (ii)
Laboratory-based: Hyperlipidemia, hemoglobin A1c, and estimated glomerular
filtration rate; and (iii) Carotid ultrasound image phenotypes: maximum plaque
height, total plaque area, and intra-plaque neovascularization. Baseline
characteristics for four classes (target labels) having significant (p < 0.0001)
values were calculated using Chi-square and ANOVA. For handling the cohort's
imbalance in the risk classes, AE3.0DL used the synthetic minority over-sampling
technique (SMOTE). AE3.0DL used Recurrent Neural Network (RNN) and Long Short-Term
Memory (LSTM) DL models and the performance (accuracy and area-under-the-curve) was
computed using 10-fold cross-validation (90% training, 10% testing) frameworks.
AE3.0DL was validated and benchmarked.
Results
The AE3.0DL using RNN and LSTM showed an accuracy and AUC (p < 0.0001) pairs as
(95.00% and 0.98), and (95.34% and 0.99), respectively, and showed an improvement
of 32.93% and 9.94% against CCVRC and ML, respectively. AE3.0DL runs in <1 s.
Conclusion
DL algorithms are a powerful paradigm for coronary artery disease (CAD) risk
prediction and CVD risk stratification.
Keywords: Coronary artery disease prediction; Carotid ultrasound; Artificial
intelligence; Machine learning; Deep learning; Performance evaluation

Al-Ani Mustafa Majid, Ahmed Jamal Alshaibi, Evgeny Kostyuchenko, Alexander

Shelupanov,
A review of artificial intelligence based malware detection using deep learning,
Materials Today: Proceedings,
2021,
,
ISSN 2214-7853,
https://doi.org/10.1016/j.matpr.2021.07.012.
(https://www.sciencedirect.com/science/article/pii/S2214785321048586)
Abstract: Malware propagation by adversaries has witnessed many issues across the
globe. Often it is found that malware is released in different countries for
monetary gains. With the proliferation of malware spreading activities, it is made
possible that now we have malware patterns that are used for training machine
learning models. Thus machine learning became indispensable for malware detection.
The traditional machine learning models have limitations in performance as the
training depth is limited. The emergence of deep learning models paved way for more
training possibilities and improvement in detection accuracy with least false
positives. This paper reviews literature on deep learning techniques that are used
for malware detection. The deep learning methods used for malware detection include
CNN, RNN, LSTM and auto encoders. LSTM is found to have memory in the cell to have
better possibilities. Auto encoders are found to have better unsupervised approach
with encoding and decoding to arrive at abnormalities (malware) detection. There
are many contributions found using machine learning and deep learning towards
Android malware detection. This paper provides knowledge that leads to further
research in deep learning which is essential to improve the state of the art.
Keywords: Malware detection; Machine learning; Deep learning; Artificial
intelligence; Data science

Lihua Lin, Min Li, Li Ma, Aliasghar Baziar, Ziad M. Ali,

Hybrid RNN-LSTM deep learning model applied to a fuzzy based wind turbine data
uncertainty quantization method,
Ad Hoc Networks,
Volume 123,
2021,
102658,
ISSN 1570-8705,
https://doi.org/10.1016/j.adhoc.2021.102658.
(https://www.sciencedirect.com/science/article/pii/S1570870521001736)
Abstract: In addition to the plenty of advantages that the penetration of wind
turbine (WT) brings to the smart networks, uncertainty problems can be considered
as an unavoidable phenomenon that requires to be addressed. The results of high
uncertainty are able to lead to the instability of management plans and schedules
that is able to lead to the serious issues for operators and users. The following
case study tries to propose a novel reinforcement learning based hybrid-based
quantification technique to capture the prediction fault into the output power of
WT. The offered approach has been applied the hybrid recurrent neural network (RNN)
and long-short term memory (LSTM) layout with the aim of learning the utmost
efficient Spatio-temporal properties of WT's output power. Because of the wide
complication of the information, a novel optimization approach according to the
modified sine cos algorithm has been suggested to aid in further steady layout
training. The possibility and efficacy of the model are evaluated via the test
analysis of two datasets in wind lands of Australian.
Keywords: Optimization; Reinforcement Deep Learning; Hybrid LSTM and RNN;
Uncertainty; Wind Turbine

Mohammad Ehsan Basiri, Shahla Nemati, Moloud Abdar, Erik Cambria, U. Rajendra
Acharya,
ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis,
Future Generation Computer Systems,
Volume 115,
2021,
Pages 279-294,
ISSN 0167-739X,
https://doi.org/10.1016/j.future.2020.08.005.
(https://www.sciencedirect.com/science/article/pii/S0167739X20309195)
Abstract: Sentiment analysis has been a hot research topic in natural language
processing and data mining fields in the last decade. Recently, deep neural network
(DNN) models are being applied to sentiment analysis tasks to obtain promising
results. Among various neural architectures applied for sentiment analysis, long
short-term memory (LSTM) models and its variants such as gated recurrent unit (GRU)
have attracted increasing attention. Although these models are capable of
processing sequences of arbitrary length, using them in the feature extraction
layer of a DNN makes the feature space high dimensional. Another drawback of such
models is that they consider different features equally important. To address these
problems, we propose an Attention-based Bidirectional CNN-RNN Deep Model (ABCDM).
By utilizing two independent bidirectional LSTM and GRU layers, ABCDM will extract
both past and future contexts by considering temporal information flow in both
directions. Also, the attention mechanism is applied on the outputs of
bidirectional layers of ABCDM to put more or less emphasis on different words. To
reduce the dimensionality of features and extract position-invariant local
features, ABCDM utilizes convolution and pooling mechanisms. The effectiveness of
ABCDM is evaluated on sentiment polarity detection which is the most common and
essential task of sentiment analysis. Experiments were conducted on five review and
three Twitter datasets. The results of comparing ABCDM with six recently proposed
DNNs for sentiment analysis show that ABCDM achieves state-of-the-art results on
both long review and short tweet polarity classification.
Keywords: Sentiment analysis; Deep learning; Convolutional neural network; Long
short-term memory; Attention mechanism

Dingming Wu, Xiaolong Wang, Shaocong Wu,

Jointly modeling transfer learning of industrial chain information and deep
learning for stock prediction,
Expert Systems with Applications,
Volume 191,
2022,
116257,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2021.116257.
(https://www.sciencedirect.com/science/article/pii/S0957417421015669)
Abstract: The prediction of stock price has always been a main challenge. The time
series of stock price tends to exhibit very strong nonlinear characteristics. In
recent years, with the rapid development of deep learning, the ability to
automatically extract nonlinear features has significantly attracted scholars’
attention. However, the majority of the relevant studies have concentrated on
prediction of the changes of stock market based on the data of the specific stock
(e.g., transaction data, financial data, etc.), while those studies ignored the
interaction between stocks of different industries, especially the interaction
between the stocks of upstream enterprises and downstream enterprises in the
industrial chain. This paper aims to propose a combination of transfer learning of
industrial chain information and deep learning models, including multilayer
perceptron (MLP), recurrent neural network (RNN), long short-term memory (LSTM),
and gated recurrent unit (GRU), for stock market prediction. These models are used
to predict the trend of the 379 stock market indices by industry in China, and the
DM test was employed for validation of the prediction results. It can be concluded
that RNNs are not necessarily such an optimal choice for the prediction when
dealing with specific time series data, and it could be justified by using the
local interpretable model-agnostic explanations (LIME) algorithm. Hence, the MLP
was selected to effectively improve the accuracy of the prediction of the stock
market indices based on the transfer learning of industrial chain information. The
investment strategy is constructed according to the prediction results, and the
yield of maturity exceeds that of the buy-and-hold strategy.
Keywords: Stock prediction; Deep learning; Industrial chain; Transfer learning

Xijun Wu, Yudong Niu, Shibo Gao, Zhilei Zhao, Baoran Xu, Renqi Ma, Hailong Liu,
Yungang Zhang,
Identification of antioxidants in edible oil by two-dimensional correlation
spectroscopy combined with deep learning,
LWT,
Volume 162,
2022,
113436,
ISSN 0023-6438,
https://doi.org/10.1016/j.lwt.2022.113436.
(https://www.sciencedirect.com/science/article/pii/S0023643822003711)
Abstract: The quality of edible oil is an essential part of food safety which is
highly concerned by people. In this study, perturbation Raman spectroscopy combined
with deep learning was used to identify antioxidants in edible oils. Convolutional
neural network (CNN) and recurrent neural network (RNN) are two classical network
structures in deep learning. First of all, we explored the identification effect of
antioxidants in edible oils using one-dimensional Raman data combined with one-
dimensional CNN and RNN. At the same time, we also compared the identification
effect of the data set under a single heating time disturbance. Then two-
dimensional correlation spectroscopy combined with a two-dimensional CNN model was
used to identify the types of antioxidants. It was found that the final
classification accuracy reached 97%, which was nearly 10% higher than the one-
dimensional CNN model. This showed that the two-dimensional correlation spectral
analysis based on external disturbance can “amplify” the subtle differences in the
spectral data. In addition, the traditional chemometric method, partial least
squares discriminant analysis (PLS-DA), was used as a control experiment. According
to this study, it can be seen that the perturbation spectrum combined with deep
learning was feasible in the detection of trace substances in edible oils.
Keywords: Edible oil; Antioxidant; Perturbation spectrum; Two-dimensional
correlation spectra; Deep learning

Rawan AlSaad, Qutaibah Malluhi, Ibrahim Janahi, Sabri Boughorbel,

Predicting emergency department utilization among children with asthma using deep
learning models,
Healthcare Analytics,
Volume 2,
2022,
100050,
ISSN 2772-4425,
https://doi.org/10.1016/j.health.2022.100050.
(https://www.sciencedirect.com/science/article/pii/S2772442522000181)
Abstract: Pediatric asthma is a leading cause of emergency department (ED)
utilization, which is expensive and often preventable. Therefore, development of ED
utilization predictive models that can accurately predict patients at high-risk of
frequent ED use and subsequently steering their treatment pathway towards more
personalized interventions, has high clinical utility. In this paper, we
investigate the extent to which deep learning models, specifically recurrent neural
networks (RNNs), coupled with routinely collected electronic health record (EHR)
clinical data can predict the frequency of emergency department utilization among
children with asthma. We use retrospective longitudinal EHR data of 87,413 children
with asthma aged 0–18 years, who were attributed to one or more healthcare facility
for at least 2 consecutive years between 2000–2013. The models were trained for the
task of predicting the frequency of emergency department visits in the next 12
months. We compared prediction results of three recurrent neural network (RNN)
models: bidirectional long short-term memory (BiLSTM), bidirectional gated
recurrent unit (BiGRU), and reverse time attention model (RETAIN), to a baseline
multinomial logistic regression model. We assessed the predictive accuracy of the
models using receiver operating characteristic curve (AUC–ROC), precision–recall
curve (AUC-PR), and F1-score. The results indicated that all RNN models have
similar performances reaching AUC–ROC: 0.85, AUC-PR: 0.74, and F1-score: 0.61,
compared to AUC–ROC: 0.81, AUC-PR: 0.69, and F1-score: 0.56 for a baseline
multinomial logistic regression. Predictive models created from large routinely
available EHR data using RNN models can accurately identify children with asthma at
high-risk of repeated ED visits, without interacting with the patient or collecting
information beyond the patient’s EHR.
Keywords: Deep learning; Predictive models; Electronic health record; Emergency
medicine; Asthma

A. Ajitha, Maitri Goel, Mohit Assudani, Sudha Radhika, Sanket Goel,

Design and development of Residential Sector Load Prediction model during COVID-19
Pandemic using LSTM based RNN,
Electric Power Systems Research,
Volume 212,
2022,
108635,
ISSN 0378-7796,
https://doi.org/10.1016/j.epsr.2022.108635.
(https://www.sciencedirect.com/science/article/pii/S0378779622007088)
Abstract: Covid-19 pandemic and resulting lockdown has created a wide impact on
social life, including sudden rise in residential load demand. Utilities, for
better load scheduling and economic operations, rely on different prediction models
among which neural networks proved to be more appropriate. For such unforeseen
situations, the non-availability of prior predictions elevated the utility
challenges. Moreover, the stringency of lockdowns caused due to mutated COVID-19
virus, necessitates accurate lockdown load predictions. This paper proposes a
Recurrent Neural Network based Long Short-Term Memory (RNN-LSTM) model, trained to
produce such predictions for two areas of residential sector. The model uses real-
time residential load data from the year 2020, with and without weather parameters.
The correlation factor (R) of proposed method 0.9683 outperformed the ARIMA's value
0.703. The model is evaluated with correlation factors of 0.9683 and 0.9235 without
temp; 0.90361 and 0.913662 with temperature for Apurupa and Jyothi colonies
respectively located in Hyderabad, India. In addition, the error metrics namely,
Mean absolute percentage error (MAPE) and Mean absolute error (MAE) are 2.0464 and
138.576 for Apurupa colony; 0.015 and 201.648 for Jyothi colony respectively.
However, the prediction error metrics increased slightly with temperature data. The
proposed framework will assist utilities for effective load predictions during
situations such as pandemic lockdown.
Keywords: Load forecasting; Lockdown period; Artificial neural networks; Recurrent
neural network- long-short term memory (RNN-LSTM); Energy forecast; Back
propagation; Time series model; COVID-19 pandemic

Huai Su, Enrico Zio, Jinjun Zhang, Mingjing Xu, Xueyi Li, Zongjie Zhang,
A hybrid hourly natural gas demand forecasting method based on the integration of
wavelet transform and enhanced Deep-RNN model,
Energy,
Volume 178,
2019,
Pages 585-597,
ISSN 0360-5442,
https://doi.org/10.1016/j.energy.2019.04.167.
(https://www.sciencedirect.com/science/article/pii/S036054421930800X)
Abstract: The rapid development of big data and smart technology in the natural gas
industry requires timely and accurate forecasting of natural gas consumption on
different time horizons. In this work, we propose a robust hybrid hours-ahead gas
consumption method by integrating Wavelet Transform, RNN-structured deep learning
and Genetic Algorithm. The Wavelet Transform is used to reduce the complexity of
the forecasting tasks by decomposing the original series of gas loads into several
sub-components. The RNN-structured deep learning method is built up via combining a
multi-layer Bi-LSTM model and a LSTM model. The multi-layer Bi-LSTM model can
comprehensively capture the features in the sub-components and the LSTM model is
used to forecast the future gas consumption based on these abstracted features. To
enhance the performance of the RNN-structured deep learning model, Genetic
Algorithm is employed to optimize the structure of each layer in the model.
Besides, the dropout technology is applied in this work to overcome the potential
problem of overfitting. In this case study, the effectiveness of the developed
method is verified from multiple perspective, including graphical examination,
mathematical errors analysis and model comparison, on different data sets.
Keywords: Natural gas demand forecasting; Deep learning; Recurrent neural network;
Genetic algorithm; Long short time memory model

J.I. Christy Eunaicy, S. Suguna,

Web attack detection using deep learning models,
Materials Today: Proceedings,
Volume 62, Part 7,
2022,
Pages 4806-4813,
ISSN 2214-7853,
https://doi.org/10.1016/j.matpr.2022.03.348.
(https://www.sciencedirect.com/science/article/pii/S2214785322016911)
Abstract: Due to the network access and security vulnerabilities of web
applications, web applications are often targets of cyber-attacks. Attacks against
web applications can be extremely dangerous. A lot of damage has been done because
of the vulnerability of the application, which lets them access the Web Application
database. Monitoring web attacks and generating alarms when a challenge to an
attack is detected. This work uses deep learning models (ANN, CNN & RNN) to detect
web attacks automatically. To identify the time when the attack on the payload
occurred, the work first analyses the web log information provided by the user. To
make an attack prediction, the log information is pre-processed. Web-log
information is pre-processed to remove duplicate values and missing values and to
get the payload information. To encode the fields and normalize (Min-Max) that
converts into unique format while predicting and the encoding value also applied.
To construct the prediction model for the detection of web attacks, the pre-
processed dataset is incorporated into the deep learning classifiers. In the
performance evaluation, RNN provided 94% accuracy and 6% error rate, higher than
other method.
Keywords: Web attack detection; Machine learning; Web applications; ANN; CNN; RNN

Shahid Farah, Wood David A, Nisar Humaira, Zameer Aneela, Eger Steffen,
Short-term multi-hour ahead country-wide wind power prediction for Germany using
gated recurrent unit deep learning,
Renewable and Sustainable Energy Reviews,
Volume 167,
2022,
112700,
ISSN 1364-0321,
https://doi.org/10.1016/j.rser.2022.112700.
(https://www.sciencedirect.com/science/article/pii/S1364032122005895)
Abstract: In recent years, wind power has emerged as an important source of
renewable energy. When onshore and offshore wind farm regions are connected to the
grid for power generation, consistent multi-location short-term wind power
predictions are extremely valuable in terms of assuring the power system's safety,
sustainability, and economic operation. An abrupt variation in wind power
generation influences the efficiency of the regional power grid. This makes
accurate short-term forecasting essential for high-level planning and scheduling of
power grids. To address the issue, this paper presents two variants of recurrent
neural networks (RNN): gated recurrent unit (GRU) and long short-term memory (LSTM)
models considering substantially better prediction accuracy to forecast a country-
wide (Germany) wind power data for daily (t + 1), and multi-step (t + 3, t + 5, and
t + 12) hours ahead. In addition, wind velocities [m/s] measured at heights of 2,
10, and 50-m (above ground level) are exploited as an essential characteristic
among the available input variables and evaluated each feature subset based on four
training divisions (80-20%, 70-30%, 60-40%, and 50-50%) and compared the results
with ARIMA and SVR approaches in the literature. The findings reveal that the RNN-
GRU model not only can achieve higher predicting accuracy but also has a faster
learning speed over long sequences.
Keywords: Wind power prediction; Feature selection; Gated recurrent unit; Long
short term memory; Lagged time-series analysis

Xiaohan Zhang, Lu Liu, Guodong Long, Jing Jiang, Shenquan Liu,

Episodic memory governs choices: An RNN-based reinforcement learning model for
decision-making task,
Neural Networks,
Volume 134,
2021,
Pages 1-10,
ISSN 0893-6080,
https://doi.org/10.1016/j.neunet.2020.11.003.
(https://www.sciencedirect.com/science/article/pii/S0893608020303889)
Abstract: Typical methods to study cognitive function are to record the electrical
activities of animal neurons during the training of animals performing behavioral
tasks. A key problem is that they fail to record all the relevant neurons in the
animal brain. To alleviate this problem, we develop an RNN-based Actor–Critic
framework, which is trained through reinforcement learning (RL) to solve two tasks
analogous to the monkeys’ decision-making tasks. The trained model is capable of
reproducing some features of neural activities recorded from animal brain, or some
behavior properties exhibited in animal experiments, suggesting that it can serve
as a computational platform to explore other cognitive functions. Furthermore, we
conduct behavioral experiments on our framework, trying to explore an open question
in neuroscience: which episodic memory in the hippocampus should be selected to
ultimately govern future decisions. We find that the retrieval of salient events
sampled from episodic memories can effectively shorten deliberation time than
common events in the decision-making process. The results indicate that salient
events stored in the hippocampus could be prioritized to propagate reward
information, and thus allow decision-makers to learn a strategy faster.
Keywords: Actor–Critic; Prefrontal cortex-basal ganglia circuit; Episodic memory;
Reinforcement learning

Jian Duan, Xi Zhang, Tielin Shi,

A Hybrid Attention-Based Paralleled Deep Learning model for tool wear prediction,
Expert Systems with Applications,
Volume 211,
2023,
118548,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.118548.
(https://www.sciencedirect.com/science/article/pii/S0957417422016177)
Abstract: In modern manufacturing process, tool condition significantly affects
work efficiency, machinery downtime and operating profit. Convolutional neural
network (CNN), recurrent neural network (RNN) or other deep learning models are
widely adopted to learn sensitive features individually or sequentially from
enormous samples for tool status monitoring. However, these models only learn
partial features due to their inherent structures. And features extraction
performance of the model with simple sequential combination is also restricted by
inner mutual block interference. In this paper, a novel deep learning network named
Hybrid Attention-Based Parallel Deep Learning (HABPDL) model is proposed to address
these problems. Specifically, ResNet and BiLSTM blocks individually learn features.
Their corresponding attention layers, namely convolutional block attention module
(CBAM) and general attention unit in BiLSTM, are stacked in sequence to highlight
extracted features. And global average pooling (GAP) is applied to reduce
superfluous spatial features and increase model interpretability after CBAM layer.
Finally, these features maps from CNN and RNN parts are concatenated to predict
tool wear value more accurately. Life cycle milling experiments are conducted, and
vibration signals are acquired for model training and validation. After model
hyperparameters optimization, comparison experiment results validate that the
proposed model can learn more complete features without any inner interference, and
own brilliant prediction performance due to well-designed parallel structure and
block-attention units. Proposed HABPDL model achieves the best prediction results,
and MAPE, MAE, RMSE and R2 reach 10.8%, 6.072, 7.955, and 0.933, respectively. The
model also outperforms other models even under noisy environment.
Keywords: Attention mechanism; Convolution neural network; Deep learning; Recurrent
neural network; Tool condition monitoring

Akhilesh Kumar Mishra, Puneet Mishra, H.D. Mathur,

A deep learning assisted adaptive nonlinear deloading strategy for wind turbine
generator integrated with an interconnected power system for enhanced load
frequency control,
Electric Power Systems Research,
Volume 214, Part B,
2023,
108960,
ISSN 0378-7796,
https://doi.org/10.1016/j.epsr.2022.108960.
(https://www.sciencedirect.com/science/article/pii/S0378779622010094)
Abstract: The existing linear and quadratic deloading strategies with constant
deloading factor, fail to effectively handle the nonlinear characteristics of WTGs.
This work proposes a novel deep learning assisted adaptive nonlinear deloading (DL-
AND) methodology based on a Newtonian interpolated polynomial for WTG integrated
with an interconnected power system to provide effective load frequency control
(LFC). The key feature of the proposed technique is its ability to adapt the
deloading factor in accordance with wind speed to optimize the reserve power margin
of the WTG. In this work, a deep learning-based recurrent neural network (RNN) with
long short-term memory (LSTM) technique has been proposed for wind speed
forecasting, as using a wind speed measurement device is expected to incorporate
measurement lag, leading to the deterioration of the deloading operation. The
proposed novel DL-AND technique for WTGs is used along with a fractional-order
fuzzy-based PID (FFOPID) control structure as a supplementary controller for
handling uncertainties in order to provide effective LFC. Further, Exhaustive
simulation studies have been carried out to investigate the proposed technique and
results show the effectiveness of proposed novel DL-AND strategy with FFOPID in
terms WTG reserve power margin, frequency support and performance index for all the
considered case studies.
Keywords: Doubly fed induction generator (DFIG); Load frequency control (LFC);
Recurrent neural network (RNN); Long short-term memory (LSTM); Fractional-order
fuzzy controller

Arjun Singh Saud, Subarna Shakya,

Analysis of look back period for stock price prediction with RNN variants: A case
study on banking sector of NEPSE,
Procedia Computer Science,
Volume 167,
2020,
Pages 788-798,
ISSN 1877-0509,
https://doi.org/10.1016/j.procs.2020.03.419.
(https://www.sciencedirect.com/science/article/pii/S1877050920308851)
Abstract: Stock market prediction is an attempt of determining the future value of
a stock traded on a stock exchange. Stock market investors try to predict the
stock’s future price to make trading decisions such that optimum profit can be
earned. Deep learning models are found most successful in predicting stock prices.
This paper has performed a novel analysis of the parameter look-back period used
with recurrent neural networks and also compared stock price prediction performance
of three deep learning models: Vanilla RNN, LSTM, and GRU for predicting stock
prices of the two most popular and strongest commercial banks listed on Nepal Stock
Exchange (NEPSE). From the experiments performed, it is found that GRU is most
successful in stock price prediction. In addition, the research work has suggested
suitable values of the look-back period that could be used with LSTM and GRU for
better stock price prediction performance.
Keywords: Stock Price Prediction; Deep Learning; LSTM; GRU; NEPSE

Ravi Kiran Inapakurthi, Srinivas Soumitri Miriyala, Kishalay Mitra,

Deep learning based dynamic behavior modelling and prediction of particulate matter
in air,
Chemical Engineering Journal,
Volume 426,
2021,
131221,
ISSN 1385-8947,
https://doi.org/10.1016/j.cej.2021.131221.
(https://www.sciencedirect.com/science/article/pii/S1385894721028023)
Abstract: Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM)
networks are utilized to capture the dynamic trends of 15 environmental parameters
including particulate matter and pollutants in the atmosphere that cause long-term
health hazards. Despite having the capability for capturing the long-term
dependencies and nonlinearities in dynamic data, these deep learning based models
suffer from overfitting if hyper-parameters are not determined optimally. For this
purpose, a novel evolutionary algorithm for neural architecture search balancing
the accuracy-complexity trade-off through a multi-objective optimization is
proposed. This algorithm not only designs optimal deep-RNNs, but also ensures
simultaneous determination of activation function and truncated backpropagation
length. Analysis of many-to-one and many-to-many styled RNNs concluded that latter
style is more effective. Subsequently it is compared with that of LSTMs to achieve
an overall accuracy between 85.612% and 99.56%. To further minimize this error,
multi-variate modelling is proposed. However, since it is important to identify the
most significant features, which can be considered as inputs to multi-variate deep
RNNs, Monte Carlo based Global Sensitivity Analysis is performed. It proved the
hypothesis with sufficient statistical evidence that pH of rain (whose univariate
modelling accuracy was least among all) is affected by methane, carbon monoxide,
non-methane hydrocarbons and total hydrocarbons, thus improving the modelling
accuracy to 98.97%. These models not only can help policymakers make informed
decisions and mitigate climate change, but also the approach can be extended for
other time-series modelling related applications due to its generic nature.
Keywords: Particulate matter; Machine learning; Intelligent systems; Multi-
objective optimization; Air quality monitoring; Neural architectural search

Zengwei Yao, Zihao Wang, Weihuang Liu, Yaqian Liu, Jiahui Pan,
Speech emotion recognition using fusion of three multi-task learning-based
classifiers: HSF-DNN, MS-CNN and LLD-RNN,
Speech Communication,
Volume 120,
2020,
Pages 11-19,
ISSN 0167-6393,
https://doi.org/10.1016/j.specom.2020.03.005.
(https://www.sciencedirect.com/science/article/pii/S0167639319300329)
Abstract: Speech emotion recognition plays an increasingly important role in
emotional computing and is still a challenging task due to its complexity. In this
study, we developed a framework integrating three distinctive classifiers: a deep
neural network (DNN), a convolution neural network (CNN), and a recurrent neural
network (RNN). The framework was used for categorical recognition of four discrete
emotions (i.e., angry, happy, neutral and sad). Frame-level low-level descriptors
(LLDs), segment-level mel-spectrograms (MS), and utterance-level outputs of high-
level statistical functions (HSFs) on LLDs were passed to RNN, CNN, and DNN,
separately. Three individual models of LLD-RNN, MS-CNN, and HSF-DNN were obtained.
In the models of MS-CNN and LLD-RNN, the attention mechanism based weighted-pooling
method was utilized to aggregate the CNN and RNN outputs. To effectively utilize
the interdependencies between the two approaches of emotion description (discrete
emotion categories and continuous emotion attributes), a multi-task learning
strategy was implemented in these three models to acquire generalized features by
simultaneously operating classification of discrete categories and regression of
continuous attributes. Finally, a confidence-based fusion strategy was developed to
integrate the power of different classifiers in recognizing different emotional
states. Three experiments on emotion recognition based on the IEMOCAP corpus were
conducted. Our experimental results show that the weighted pooling method based on
attention mechanism endowed the neural networks with the capability to focus on
emotionally salient parts. The generalized features learned in the multi-task
learning helped the neural networks to achieve higher accuracies in the tasks of
emotion classification. Furthermore, our proposed fusion system achieved weighted
accuracy of 57.1% and unweighted accuracy of 58.3%, which were significantly higher
than those of each individual classifier. The effectiveness of the proposed
approach based on classifier fusion was thus validated.
Keywords: Speech emotion recognition; Attention mechanism; Multi-task learning;
Classifier fusion

Abtin Shahkarami, Mansoor Yousefi, Yves Jaouën,

Complexity reduction over Bi-RNN-based nonlinearity mitigation in dual-pol fiber-
optic communications via a CRNN-based approach,
Optical Fiber Technology,
Volume 74,
2022,
103072,
ISSN 1068-5200,
https://doi.org/10.1016/j.yofte.2022.103072.
(https://www.sciencedirect.com/science/article/pii/S1068520022002553)
Abstract: Bidirectional recurrent neural networks (bi-RNNs), in particular
bidirectional long short term memory (bi-LSTM), bidirectional gated recurrent unit,
and convolutional bi-LSTM models, have recently attracted attention for
nonlinearity mitigation in fiber-optic communication. The recently adopted
approaches based on these models, however, incur a high computational complexity
which may impede their real-time functioning. In this paper, by addressing the
sources of complexity in these methods, we propose a more efficient network
architecture, where a convolutional neural network encoder and a unidirectional
many-to-one vanilla RNN operate in tandem, each best capturing one set of channel
impairments while compensating for the shortcomings of the other. We deploy this
model in two different receiver configurations. In one, the neural network is
placed after a linear equalization chain and is merely responsible for nonlinearity
mitigation; in the other, the neural network is directly placed after the chromatic
dispersion compensation and is responsible for joint nonlinearity and polarization
mode dispersion compensation. For a 16-QAM 64 GBd dual-polarization optical
transmission over 14×80km standard single-mode fiber, we demonstrate that the
proposed hybrid model achieves the bit error probability of the state-of-the-art
bi-RNN-based methods with greater than 50% lower complexity, in both receiver
configurations.
Keywords: Optical fiber communications; Kerr nonlinearity; Equalization;
Convolutional recurrent neural networks; Complexity reduction
Zineb Cheker, Saad Chakkor, Ahmed EL Oualkadi, Mostafa Baghouri, Rachid Belfkih,
Jalil Abdelkader El Hangouche, Jawhar Laameche,
Performance analysis of VEP signal discrimination using CNN and RNN algorithms,
Neuroscience Informatics,
Volume 2, Issue 3,
2022,
100087,
ISSN 2772-5286,
https://doi.org/10.1016/j.neuri.2022.100087.
(https://www.sciencedirect.com/science/article/pii/S2772528622000498)
Abstract: The visual evoked potential as an electrophysiological signal is mainly
used in the neurophysiological exploration of the optic nerves. Traditionally,
medical doctors base their diagnosis of specific pathologies related to the time
delay of the nerve flow on the time scale. In this context, the VEP latency P100
that reflects a temporal notion is considered the main characteristic on which
human interpretation is based. However, its value is influenced by different
factors and remains a limited method. This insufficiency triggers our interest
instead in deep learning architectures, taking into consideration and adapting to
the specificity of each particularity related to the laboratory of the
neurophysiological exploration unit in the hospital. The comparison between the
results obtained from Matlab by the application of the CNN as well as the RNN,
based on the evaluation parameters calculated after k-fold cross-validation,
confirms that the CNN-1D architecture can be considered powerful in terms of
reliability of classification between signals that are related to pathological
subjects and normal ones, which privileges the use of this architecture compared
with recurrent neural networks that are less reliable and require more time for
execution, subsequently the use of the CNN will allow us to avoid even the
extraction of attributes for the discrimination between the two classes object of
classification, with the possibility to progressively improve the performance of
the solution over time based on the new signals acquired in the VEP analysis
laboratory.
Keywords: Visual evoked potential; Latency P100; Deep learning; Recurrent neural
network; Convolutional neural network

Noopur Singh, Ravindra Nath, Dev Bukhsh Singh,

Splice-site identification for exon prediction using bidirectional LSTM-RNN
approach,
Biochemistry and Biophysics Reports,
Volume 30,
2022,
101285,
ISSN 2405-5808,
https://doi.org/10.1016/j.bbrep.2022.101285.
(https://www.sciencedirect.com/science/article/pii/S2405580822000851)
Abstract: Machine learning methods played a major role in improving the accuracy of
predictions and classification of DNA (Deoxyribonucleic Acid) and protein
sequences. In eukaryotes, Splice-site identification and prediction is though not a
straightforward job because of numerous false positives. To solve this problem,
here, in this paper, we represent a bidirectional Long Short Term Memory (LSTM)
Recurrent Neural Network (RNN) based deep learning model that has been developed to
identify and predict the splice-sites for the prediction of exons from eukaryotic
DNA sequences. During the splicing mechanism of the primary mRNA transcript, the
introns, the non-coding region of the gene are spliced out and the exons, the
coding region of the gene are joined. This bidirectional LSTM-RNN model uses the
intron features that start with splice site donor (GT) and end with splice site
acceptor (AG) in order of its length constraints. The model has been improved by
increasing the number of epochs while training. This designed model achieved a
maximum accuracy of 95.5%. This model is compatible with huge sequential data such
as the complete genome.
Keywords: Splice-site; Intron; Exon; Machine learning; Deep learning; Bidirectional
LSTM-RNN

Mengbo Liu, Shaoming Liao, Yifeng Yang, Yanqing Men, Junzuo He, Yongliang Huang,
Tunnel boring machine vibration-based deep learning for the ground identification
of working faces,
Journal of Rock Mechanics and Geotechnical Engineering,
Volume 13, Issue 6,
2021,
Pages 1340-1357,
ISSN 1674-7755,
https://doi.org/10.1016/j.jrmge.2021.09.004.
(https://www.sciencedirect.com/science/article/pii/S1674775521001323)
Abstract: Tunnel boring machine (TBM) vibration induced by cutting complex ground
contains essential information that can help engineers evaluate the interaction
between a cutterhead and the ground itself. In this study, deep recurrent neural
networks (RNNs) and convolutional neural networks (CNNs) were used for vibration-
based working face ground identification. First, field monitoring was conducted to
obtain the TBM vibration data when tunneling in changing geological conditions,
including mixed-face, homogeneous, and transmission ground. Next, RNNs and CNNs
were utilized to develop vibration-based prediction models, which were then
validated using the testing dataset. The accuracy of the long short-term memory
(LSTM) and bidirectional LSTM (Bi-LSTM) models was approximately 70% with raw data;
however, with instantaneous frequency transmission, the accuracy increased to
approximately 80%. Two types of deep CNNs, GoogLeNet and ResNet, were trained and
tested with time-frequency scalar diagrams from continuous wavelet transformation.
The CNN models, with an accuracy greater than 96%, performed significantly better
than the RNN models. The ResNet-18, with an accuracy of 98.28%, performed the best.
When the sample length was set as the cutterhead rotation period, the deep CNN and
RNN models achieved the highest accuracy while the proposed deep CNN model
simultaneously achieved high prediction accuracy and feedback efficiency. The
proposed model could promptly identify the ground conditions at the working face
without stopping the normal tunneling process, and the TBM working parameters could
be adjusted and optimized in a timely manner based on the predicted results.
Keywords: Deep learning; Transfer learning; Convolutional neural network (CNN);
Recurrent neural network (RNN); Ground detection; Tunnel boring machine (TBM)
vibration; Mixed-face ground

M. Nandhini, K.U. Kala, M. Thangadarshini, S. Madhusudhana Verma,

Deep Learning model of sequential image classifier for crop disease detection in
plantain tree cultivation,
Computers and Electronics in Agriculture,
Volume 197,
2022,
106915,
ISSN 0168-1699,
https://doi.org/10.1016/j.compag.2022.106915.
(https://www.sciencedirect.com/science/article/pii/S0168169922002320)
Abstract: Plantain tree is the most popular crop grown all over the world and
banana (Musa spp.) is the most marketable fruit. It is the leading food in many
countries, especially in developing countries. Plant diseases are significant
aspects that result in a serious reduction in the quantity and quality of fruit
crops. Plantain tree cultivation is affected by various diseases such as Black
Sigatoka/Yellow sigatoka, Panama, Bunchy top, Moko, chlorosis, etc. Rapid and novel
approaches for the apt discovery of diseases help farmers in developing better
decisions and efficient control measures. Convolutional Neural Networks (CNN) and
Recurrent Neural Network (RNN) have been proved their efficiency in several fields
and it has recently moved in the field of crop disease classification and
detection. The objective of this research work is to create a Deep Learning Model
for the disease classification and its early prediction to support farmers in
plantain tree cultivation. A new sequential image classification model is proposed
to detect the diseases by combining RNN and CNN, which is named as Gated-Recurrent
Convolutional Neural Network (G-RecConNN). The input to the proposed model is the
sequences of plant images. The experiments are carried out in real-time datasets
collected from the state named Tamil Nadu situated in the Southern part of India.
This method aims at numerous advantages such as reduced pre-processing of the data,
easy online performance evaluation and advancements with less real data, etc. The
experimental results inspired the utilization of the G-RecConNN model with farmer
support systems that will process continuous banana tree images as part or whole
for the early detection of banana tree diseases.
Keywords: Banana; Disease Classification; Deep Learning; Plantain Tree Cultivation;
Recurrent Convolutional Neural Network

K. Ananthajothi, K. Karthikayani, R. Prabha,

Explicit and implicit oriented Aspect-Based Sentiment Analysis with optimal feature
selection and deep learning for demonetization in India,
Data & Knowledge Engineering,
Volume 142,
2022,
102092,
ISSN 0169-023X,
https://doi.org/10.1016/j.datak.2022.102092.
(https://www.sciencedirect.com/science/article/pii/S0169023X22000830)
Abstract: Aspect-Based Sentiment Analysis (ABSA) is a popular scheme that looks for
the prediction of the sentiment of positive characteristics in text. The sentiment
of text sequences is analyzed by deep neural networks and attained noteworthy
results. Conversely, these models also have some problems with the limitation of
past-training word embeddings and lack of communication between the context and the
particular characteristic of the attention scheme. The main part of this task is to
develop the novel ABSA concerning both explicit and implicit aspects using
demonetization dataset reviews from India. Initially, the pre-processing of online
tweets is performed by stop word removal, tokenization, lower case conversion, and
stemming. Further, the explicit aspects are extracted, as it is simple to extract
from the sentence and the polarity score is computed. A machine learning algorithm
termed as Neural Network (NN) is utilized that helps for training the data
regarding the implicit aspects, and further, helps to differentiate properly for
the testing data with exact polarity score. Optimal feature selection is performed
using the Self Adaptive Beetle Swarm Optimization (SA-BSO). These optimal features
are given to a deep structured architecture called Recurrent Neural Network (RNN)
with hidden neuron optimization by SA-BSO, which categorizes the demonetization
reviews into positive, negative, or neutral. While taking the findings, the
accuracy of the offered SA-BSO-RNN is secured at 4.67%, 6.56%, 3.54%, and 7.12%
progressed than PSO-RNN, FF-RNN, CSA-RNN, and BSO-RNN, at 3-fold analysis for
dataset 1. Results show that the designed ABSA concerning both explicit and
implicit aspects using the demonetization method that provides enriched performance
with diverse performance metrics.
Keywords: Demonetization; Aspect-Based Sentiment Analysis; Polarity; Holoentropy;
Neural Network; Self Adaptive Beetle Swarm Optimization; Recurrent Neural Network

Yiqi Jiang, Chaolin Li, Lu Sun, Dong Guo, Yituo Zhang, Wenhui Wang,
A deep learning algorithm for multi-source data fusion to predict water quality of
urban sewer networks,
Journal of Cleaner Production,
Volume 318,
2021,
128533,
ISSN 0959-6526,
https://doi.org/10.1016/j.jclepro.2021.128533.
(https://www.sciencedirect.com/science/article/pii/S0959652621027426)
Abstract: Point source pollution in urban drainage networks, which is difficult to
monitor and control, has been regarded as an intractable problem. To solve the
problem, key water quality indicators must be tracked in the evaluation and
prediction of sewer water quality. However, some of these important chemical
indicators (e.g. biological oxygen demand (BOD5), chemical oxygen demand (COD),
ammonia nitrogen (NH4+-N), total nitrogen (TN), and total phosphorus (TP)) require
a great deal of time and effort to measure, which will adversely affect the
prediction in a sewage network. Existing statistical methods and machine learning
algorithms cannot effectively solve the detection time problem or provide limited
accuracy. Moreover, the lack of various factors taken into account in these methods
results in unsatisfactory predictive performance. Few studies consider the impact
of urban multi-source data on water quality prediction of sewer networks while
developing statistical methods or machine learning algorithms. To address this
problem, we propose a deep learning approach based on multi-source data fusion.
This approach takes into account the following indicators to comprehensively
analyze and predict drainage water quality: environmental indicators (such as area
and diameter); social indicators (such as population); water quantity indicators
(such as drinking water supply, sewage flow, water velocity, and liquid level); and
easily monitored water quality criteria indicators (such as pH, temperature, and
conductivity). To test the effectiveness of this method, we conducted a case study
in a city in southern China. By comparing this new method with the linear method
(multiple linear regression, MLR) and traditional learning algorithm (multilayer
perception, MLP), it is found that the deep learning algorithm—which includes
recurrent neural network (RNN), long-short term memory (LSTM), and gated recurrent
unit (GRU)—has good predictive performance, in which GRU shows superior ability in
predicting the chemical index of water quality and the learning curve is faster.
The results showed that the GRU achieved 0.82%–5.07% higher R2 than RNN and LSTM,
9.13%–15.03% higher R2 than traditional machine learning algorithms, and 37.26%–
43.38% higher R2 than linear methods.
Keywords: Urban sewer networks; Water quality prediction; Deep learning; Multi-
source data fusion

I. Kadek Sastrawan, I.P.A. Bayupati, Dewa Made Sri Arsa,

Detection of fake news using deep learning CNN–RNN based methods,
ICT Express,
Volume 8, Issue 3,
2022,
Pages 396-408,
ISSN 2405-9595,
https://doi.org/10.1016/j.icte.2021.10.003.
(https://www.sciencedirect.com/science/article/pii/S2405959521001375)
Abstract: Fake news is inaccurate information that is intentionally disseminated
for a specific purpose. If allowed to spread, fake news can harm the political and
social spheres, so several studies are conducted to detect fake news. This study
uses a deep learning method with several architectures such as CNN, Bidirectional
LSTM, and ResNet, combined with pre-trained word embedding, trained using four
different datasets. Each data goes through a data augmentation process using the
back-translation method to reduce data imbalances between classes. The results
showed that the Bidirectional LSTM architecture outperformed CNN and ResNet on all
tested datasets.
Keywords: Fake news detection; Deep learning; CNN; Bidirectional LSTM; ResNet

Liwen Xing, Ningbo Cui, Chunwei Liu, Lu Zhao, Li Guo, Taisheng Du, Cun Zhan,
Zongjun Wu, Shenglin Wen, Shouzheng Jiang,
Estimation of daily apple tree transpiration in the Loess Plateau region of China
using deep learning models,
Agricultural Water Management,
Volume 273,
2022,
107889,
ISSN 0378-3774,
https://doi.org/10.1016/j.agwat.2022.107889.
(https://www.sciencedirect.com/science/article/pii/S037837742200436X)
Abstract: Quantitatively characterizing and accurately predicting plant
transpiration are of great significance, but directly measuring transpiration is
impractical, time-consuming, and labor-intensive. This study compared the
transpiration estimation performance of multiple linear regression (MLR), modified
Jarvis–Stewart (MJS), and Shuttleworth–Wallace (S-W) with deep belief network
(DBN), long short-term memory recurrent neural network (LSTM-RNN), and LSTM-RNN
improved with multiple restricted Boltzmann machines (R-L-RNN) using 31 input
combinations comprising complete subsets of Vapor pressure deficit (VPD), Net solar
radiation (Rn), Average air temperature (Ta), Soil water content (SWC), and Leaf
area index (LAI) observations collected at Wuwei, Changwu and Taigu stations on the
Loess Plateau in China. The results showed that R-L-RNN obtained the most accurate
estimations in the partial canopy stage, dense canopy stage, and whole growth
stage, compared to MLR, MJS, S-W, DBN, and LSTM-RNN. The accuracy of the deep
learning models (DNN) increased exponentially as the number of input variables
increased, and the importance of the input variables followed the orders of: LAI
> VPD > Rn > Ta > SWC in the partial and whole canopy stage, and VPD > Rn > Ta
> LAI > SWC in the dense canopy stage. The apple tree transpiration models were
more accurate in the partial and dense canopy stages than the whole growth stage.
The coefficient of determination and Nash-Sutcliffe efficiency coefficient for the
R-L-RNN model increased by 8.1–13.1% and 11.2–25.4% in the partial canopy stage,
respectively, and by 2.6–6.9% and 14.7–20.1% in the dense canopy stage, whereas the
relative root mean square error decreased by 8.7–28.6% and 17.3–38.2%. Overall, R-
L-RNN is the most recommended model for estimating the apple tree transpiration,
because it is such a simple method that agricultural water managers can easily
determine the water consumption of apple trees using limited accessible
observational data.
Keywords: Hybrid deep learning model; Empirical model; Path analysis; Growth stage;
Input variable

Yasin Kutuk, Lina Barokas,

Multivariate CDS risk premium prediction with SOTA RNNs on MI[N]T countries,
Finance Research Letters,
Volume 45,
2022,
102198,
ISSN 1544-6123,
https://doi.org/10.1016/j.frl.2021.102198.
(https://www.sciencedirect.com/science/article/pii/S154461232100266X)
Abstract: In this study, CDS risk premiums of Mexico, Indonesia and Turkey were
predicted by applying state-of-the-art forecasters in deep learning recurrent
neural networks architectures which are the most recent ground-breaking predictors
in the time series setting. The predictive power of each sota forecaster is
compared, and the results are differentiated by country and type of sota
predictors. While the long short-term memory model is better to predict Mexico’s
CDS risk premiums, the nonlinear autoregressive network with exogenous inputs model
is found to be more suitable for Indonesia and Turkey. The results of Turkey model
reached the highest forecast accuracy.
Keywords: Credit default swap; Forecasting; Time series; Recurrent neural networks;
Deep learning

Kan Ngamakeur, Sira Yongchareon, Jian Yu, Saiful Islam,

Passive infrared sensor dataset and deep learning models for device-free indoor
localization and tracking,
Pervasive and Mobile Computing,
2022,
101721,
ISSN 1574-1192,
https://doi.org/10.1016/j.pmcj.2022.101721.
(https://www.sciencedirect.com/science/article/pii/S1574119222001341)
Abstract: Location estimation or localization is one of the key components in IoT
applications such as remote health monitoring and smart homes. Amongst device-free
localization technologies, passive infrared (PIR) sensors are one of the promising
options due to their low cost, low energy consumption, and good accuracy. However,
most of the existing systems are complexly designed and difficult to deploy in real
life, in addition, there is no public dataset available for researchers to
benchmark their proposed localization and tracking methods. In this paper, we
propose a system and a dataset collected from our PIR system consisting of
commercial-of-the-shelf (COTS) sensors without any modification. Our dataset
includes profile data of 36 classes that have over 1,000 samples of different
walking directions and test data consisting of multiple scenarios with a sequence
length of over 2,000 timesteps. To evaluate our system and dataset, we implement
various deep learning methods such as CNN, RNN, and CNN-RNN. Our results prove the
applicability and feasibility of our system and illustrate the viability of deep
learning methods for PIR-based localization and tracking. We also show that our
dataset can be converted for coordinate estimation so that deep learning methods
and particle filter approaches can be applied to estimate coordinates. As a result,
the best performer achieves a distance error of 0.25 m.
Keywords: PIR; Location estimation; Indoor localization; Device-free localization;
PIR dataset

Bhargava K Reddy, Dursun Delen,

Predicting hospital readmission for lupus patients: An RNN-LSTM-based deep-learning
methodology,
Computers in Biology and Medicine,
Volume 101,
2018,
Pages 199-209,
ISSN 0010-4825,
https://doi.org/10.1016/j.compbiomed.2018.08.029.
(https://www.sciencedirect.com/science/article/pii/S0010482518302567)
Abstract: Hospital readmission is one of the critical metrics used for measuring
the performance of hospitals. The HITECH Act imposes penalties when patients are
readmitted to hospitals if they are diagnosed with one of the six conditions
mentioned in the Act. However, patients diagnosed with lupus are the sixth highest
in terms of rehospitalization. The heterogeneity in the disease and patient
characteristics makes it very hard to predict rehospitalization. This research
utilizes deep learning methods to predict rehospitalization within 30 days by
extracting the temporal relationships in the longitudinal EHR clinical data.
Prediction results from deep learning methods such as LSTM are evaluated and
compared with traditional classification methods such as penalized logistic
regression and artificial neural networks. The simple recurrent neural network
method and its variant, gated recurrent unit network, are also developed and
validated to compare their performance against the proposed LSTM model. The results
indicated that the deep learning method RNN-LSTM has a significantly better
performance (with an AUC of .70) compared to traditional classification methods
such as ANN (with an AUC of 0.66) and penalized logistic regression (with an AUC of
0.63). The rationale for the better performance of the deep learning method may be
due to its ability to leverage the temporal relationships of the disease state in
patients over time and to capture the progression of the disease—relevant clinical
information from patients' prior visits is carried forward in the memory, which may
have enabled the higher predictability for the deep learning methods.
Keywords: Readmission; Lupus; Machine learning; Predictive analytics; Deep
learning; LSTM

A. Ramana Kumari, Suryakari Nagaraja Rao, P. Ramana Reddy,

Design of hybrid dental caries segmentation and caries detection with meta-
heuristic-based ResneXt-RNN,
Biomedical Signal Processing and Control,
Volume 78,
2022,
103961,
ISSN 1746-8094,
https://doi.org/10.1016/j.bspc.2022.103961.
(https://www.sciencedirect.com/science/article/pii/S1746809422004608)
Abstract: The early identification of dental caries is necessary for corresponding
treatments and the above reason, the bitewing radiography is utilized to offer
initial caries detection. In clinical imaging, the usage of deep structured
architectures with renowned neural network schemes helps process the vast amount of
images, which has been researched actively in recent years and provided competitive
performance. Therefore, deep learning approaches have attained remarkable diagnosis
efficiency in the domain of radiology. Owing to this emerging intelligence, this
paper aims to use the deep learning method for dental caries segmentation in an
effective way. At first, contrast enhancement via Contrast Limited Adaptive
Histogram Equalization (CLAHE), and noise filtering via bilateral filtering are
performed under the pre-processing phase. Further, the segmentation of the caries
is performed by the Fused Optimal Centroid K-means with K-Mediods Clustering (FOC-
KKC), which will be enhanced by the Hybrid Sea Lion-Squirrel Search Optimization
(HSLnSSO), inducing the best parameter optimization. Once the caries are segmented,
the post-pre-processing of images is done by morphological operations. Finally, the
detection of caries from the segmented image is employed by the meta-heuristic-
based ResneXt with Recurrent Neural Network (RNN) (M−ResneXt−RNN), where the
architecture modification is performed by the HSLnSSO algorithm. The new
segmentation model and well-trained M−ResneXt−RNN for caries detection have
exhibited superior performance when compared to the conventional techniques.
Keywords: Dental Caries Segmentation and Detection; Fused Optimal Centroid K-means
with K-Mediods Clustering; Meta-heuristic-based ResneXt-RNN; Hybrid Sea Lion-
Squirrel Search Optimization

Lei Sun, Tianyuan Liu, Yonghui Xie, Di Zhang, Xinlei Xia,

Real-time power prediction approach for turbine using deep learning techniques,
Energy,
Volume 233,
2021,
121130,
ISSN 0360-5442,
https://doi.org/10.1016/j.energy.2021.121130.
(https://www.sciencedirect.com/science/article/pii/S0360544221013785)
Abstract: Accurate power forecasting is of great importance to the turbine control
and predictive maintenance. However, traditional physics models and statistical
models can no longer meet the needs of precision and flexibility when thermal power
plants frequently undertake more and more peak and frequency modulation tasks. In
this study, the recurrent neural network (RNN) and convolutional neural network
(CNN) for power prediction are proposed, and are applied to predict real-time power
of turbine based on DCS data (recorded for 719 days) from a power plant. In
addition, the performances of two deep learning models and five typical machine
learning models are compared, including prediction deviation, variance and time
cost. It is found that deep learning models outperform other shallow models and RNN
model performs best in balancing the accuracy-efficient trade-off for power
prediction (the relative prediction error of 99.76% samples is less than 1% in all
load range for test 216 days). Moreover, the influence of training size and input
time-steps on the performance of RNN model is also explored. The model can achieve
remarkable performance by learning only 30% samples (about 216 days) with 3 input
time-steps (about 60 s). Those results of the proposed models based on deep-
learning methods indicated that deep learning is of great help to improve the
accuracy of turbine power prediction. It is therefore convinced that those models
have a high potential for turbine control and predictable maintenance in actual
industrial scenarios.
Keywords: Power prediction; Deep learning; Machine learning; Recurrent neural
network; Convolutional neural network; Power plant

Jiahuan Lu, Rui Xiong, Jinpeng Tian, Chenxu Wang, Chia-Wei Hsu, Nien-Ti Tsou,
Fengchun Sun, Ju Li,
Battery degradation prediction against uncertain future conditions with recurrent
neural network enabled deep learning,
Energy Storage Materials,
Volume 50,
2022,
Pages 139-151,
ISSN 2405-8297,
https://doi.org/10.1016/j.ensm.2022.05.007.
(https://www.sciencedirect.com/science/article/pii/S2405829722002446)
Abstract: Accurate degradation trajectory and future life are the key information
of a new generation of intelligent battery and electrochemical energy storage
systems. It is very challenging to obtain accurate predictions against uncertain
application conditions by using only a few known historical data. In this article,
we extend the widely studied remaining useful life (RUL) prediction to the
prediction of charge and discharge capacity trajectories under both fixed and
random future operating conditions. This is achieved by developing a general deep
learning framework cored by recurrent neural network (RNN) which integrates future
current plan and few early capacity-voltage data as inputs. As a case study, we
have experimented with 77 commercial batteries cycled under fixed and random
operating conditions. We demonstrate that the median root mean square error (RMSE)
of prediction can be within 2.4% for NMC/graphite batteries and 2.3% for
LFP/graphite batteries by using 3.8% of the whole life data only. Compared with the
existing methods, the proposed framework predicts more accurately and has a very
balanced performance for both fixed and random future conditions. This work
highlights the promise of actively forecasting the future of batteries based on
RNN.

Yang Wang, Teng Li, Weizhi Lu, Qihang Cao,

Attention-inspired RNN Encoder-Decoder for Sensory Time Series Forecasting,
Procedia Computer Science,
Volume 209,
2022,
Pages 103-111,
ISSN 1877-0509,
https://doi.org/10.1016/j.procs.2022.10.104.
(https://www.sciencedirect.com/science/article/pii/S1877050922015538)
Abstract: As an emerging application of biomimetic intelligence, attention
mechanism in deep learning has been actively studied for intelligent monitoring.
The present paper proposes an attention-based recurrent neural network that can
make multi-step prediction of the target parameters using historical multivariate
sensory time series. Sequence-to-sequence learning is achieved through a developed
RNN encoding-decoding architecture, which embeds a multi-layer attention mechanism.
The attention layers incorporate inter-site correlations over spatially distributed
observation sites, parameter-wise dependencies among heterogeneous parameters, and
temporal correlations over time variation. The analysis of the developed
methodology is demonstrated using real-world data collected by an air quality
monitoring network. The experimental results show that the proposed deep neural
network model can provide superior prediction performance compared to the state-of-
the-art baseline models.
Keywords: Multivariate time series; Multi-step-ahead forecasting; Attention
mechanism; RNN encoder-decoder; Environmental monitoring

Xiangyu Fu, Jiafei Fang, Shilin Xiao, Xiaoyu Liu, Lizhuo Zheng, Weisheng Hu,
Dual-RNN based polar decoder with tanh-modified LLR over FSO turbulence channel,
Optics Communications,
Volume 527,
2023,
128965,
ISSN 0030-4018,
https://doi.org/10.1016/j.optcom.2022.128965.
(https://www.sciencedirect.com/science/article/pii/S0030401822006125)
Abstract: In this paper, the deep learning-based Belief Propagation (BP) decoding
is first investigated in a polar-coded free space optical (FSO) communication
system. On the basis of neural Cyclic redundancy check-Polar Belief Propagation
(NCPBP) decoding scheme, we propose a dual-recurrent neural network (RNN) based
decoder for polar codes, which adopts a new training loss function, RNN structure
and tanh-modified input. The presented decoder exhibits better performance to the
NCPBP decoder under different intensities of turbulence. Furthermore, the decoder
trained under fixed turbulence intensity shows higher turbulence adaptability and
its advantage increases with the intensity of turbulence, reaching a decoding gain
of 1 dB under strong turbulence when Rytov variance is 3.5.
Keywords: Polar codes; Free-space optical communication; Deep learning; Belief
propagation; Recurrent neural network

Jamal Abdul Nasir, Osama Subhani Khan, Iraklis Varlamis,

Fake news detection: A hybrid CNN-RNN based deep learning approach,
International Journal of Information Management Data Insights,
Volume 1, Issue 1,
2021,
100007,
ISSN 2667-0968,
https://doi.org/10.1016/j.jjimei.2020.100007.
(https://www.sciencedirect.com/science/article/pii/S2667096820300070)
Abstract: The explosion of social media allowed individuals to spread information
without cost, with little investigation and fewer filters than before. This
amplified the old problem of fake news, which became a major concern nowadays due
to the negative impact it brings to the communities. In order to tackle the rise
and spreading of fake news, automatic detection techniques have been researched
building on artificial intelligence and machine learning. The recent achievements
of deep learning techniques in complex natural language processing tasks, make them
a promising solution for fake news detection too. This work proposes a novel hybrid
deep learning model that combines convolutional and recurrent neural networks for
fake news classification. The model was successfully validated on two fake news
datasets (ISO and FA-KES), achieving detection results that are significantly
better than other non-hybrid baseline methods. Further experiments on the
generalization of the proposed model across different datasets, had promising
results.
Keywords: Deep learning; Fake news detection; Misinformation; Disinformation;
Rumours; CNN-RNN

C. Anuradha, M. Ponnavaikko,
A RNN based offloading scheme to reduce latency and preserve energy using RNNBOS,
Measurement: Sensors,
Volume 24,
2022,
100429,
ISSN 2665-9174,
https://doi.org/10.1016/j.measen.2022.100429.
(https://www.sciencedirect.com/science/article/pii/S2665917422000630)
Abstract: Mobile cloud computing is currently evolving quickly in today's trend and
it provides infinite number of applications to the people those who are using
regularly.MCC means the mobile gadgets are strongly tied up with cloud technology
to execute various application for attaining many tasks. Mobile devices contain
different application according to its own capacity to hold each application. In
which many applications are in need of connecting with cloud storage. A new
proposed technique named RNNBOS (Recurrent Neural Network Based Offloading scheme)
is used to compute calculations in terms of energy source of mobile device along
with active conditions of network, Load computations, delay possibility of request
from device and quantitative amount of data being transferred for this purpose. We
have simulated the above technique using python tool and observed RNN based
offloading scheme is good in execution of application using MCC.
Keywords: Computational offloading; Mobile edge computing; Deep neural network;
Energy consumption and mobile cloud computing

Rajib Ghosh,
A Recurrent Neural Network based deep learning model for offline signature
verification and recognition system,
Expert Systems with Applications,
Volume 168,
2021,
114249,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2020.114249.
(https://www.sciencedirect.com/science/article/pii/S0957417420309659)
Abstract: With the recent advancement in information technology field, the demand
to develop a person authentication system through verifying their offline
signatures is gradually increasing. This type of system may be used to verify
various official documents through verifying the signatures of the concerned
persons present in the documents. This article proposes a Recurrent Neural Network
(RNN), a deep learning network, based method to verify and recognize offline
signatures of different persons. Various structural and directional features have
been extracted locally from each signature sample and the generated feature vectors
have been studied using two different models of RNN—long-short term memory (LSTM)
and bidirectional long–short term memory (BLSTM). The performance of the proposed
system has been tested on six widely used public signature databases—GPDS
synthetic, GPDS-300, MCYT-75, CEDAR, BHSig260 Hindi, and BHSig260 Bengali.
Experiment has also been performed using Convolutional Neural Network (CNN) to have
a comparison with RNN based results. Experimental results demonstrate that the
proposed RNN based signature verification and recognition system is superior over
CNN and also outperforms the existing state-of-the-art results in this regard.
Keywords: Offline signature; Verification and recognition; Deep learning; RNN; LSTM
and BLSTM

Haowei Jiang, Feiwei Qin, Jin Cao, Yong Peng, Yanli Shao,
Recurrent neural network from adder’s perspective: Carry-lookahead RNN,
Neural Networks,
Volume 144,
2021,
Pages 297-306,
ISSN 0893-6080,
https://doi.org/10.1016/j.neunet.2021.08.032.
(https://www.sciencedirect.com/science/article/pii/S0893608021003452)
Abstract: The recurrent network architecture is a widely used model in sequence
modeling, but its serial dependency hinders the computation parallelization, which
makes the operation inefficient. The same problem was encountered in serial adder
at the early stage of digital electronics. In this paper, we discuss the
similarities between recurrent neural network (RNN) and serial adder. Inspired by
carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it
possible for RNN to run in parallel. Then, we design the method of parallel RNN
computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes
advantages in parallelism and flexible receptive field. Through a comprehensive set
of tests, we verify that CL-RNN can perform better than existing typical RNNs in
sequence modeling tasks which are specially designed for RNNs. Code and models are
available at: https://github.com/WinnieJiangHW/Carry-lookahead_RNN.
Keywords: Deep learning; Carry-lookahead; Parallel computation; Sequence modeling

Zakaria Chekakta, Abdelhafid Zenati, Nabil Aouf, Olivier Dubois-Matra,

Robust deep learning LiDAR-based pose estimation for autonomous space landers,
Acta Astronautica,
Volume 201,
2022,
Pages 59-74,
ISSN 0094-5765,
https://doi.org/10.1016/j.actaastro.2022.08.049.
(https://www.sciencedirect.com/science/article/pii/S0094576522004635)
Abstract: Accurate relative pose estimation of a spacecraft during space landing
operation is critical to ensure a safe and successful landing. This paper presents
a 3D Light Detection and Ranging (LiDAR) based AI relative navigation architecture
solution for autonomous space landing. The proposed architecture is based on a
hybrid Deep Recurrent Convolutional Neural Network (DRCNN) combining a
Convolutional Neural Network (CNN) with an Recurrent Neural Network (RNN) based on
a Long–Short Term Memory (LSTM) network. The acquired 3D LiDAR data is converted
into a multi-projected images and feed the DRCNN with depth and other multi-
projected imagery. The CNN module of the architecture allows an efficient
representation of features, and the RNN module, as an LSTM, provides robust
navigation motion estimates. A variety of landing scenarios are considered,
simulated and experimented to evaluate the efficiency of the proposed architecture.
A LiDAR based imagery data (Range, Slope, and Elevation) is initially created using
PANGU (Planet and Asteroid Natural Scene Generation Utility) software and an
evaluation of the proposed solution using this data is conducted. Tests using an
instrumented Aerial Robot in Gazebo software to simulate landing scenarios on a
synthetic but representative lunar terrain (3D digital elevation model) is
proposed. Finally, real experiments using a real flying drone equipped with a
Velodyne VLP16 3D LiDAR sensor to generate real 3D scene point clouds while landing
on a designed down scaled lunar moon landing surface are conducted. All the test
results achieved show that the suggested architecture is capable of delivering good
6 Degree of Freedom (DoF) pose precision with a good and reasonable computation.
Keywords: Space landing operations; Robotics; Deep Neural Network; Relative pose
estimation; LiDAR navigation

Mohsen Saffari, Mahdi Khodayar, Mohammad E. Khodayar,

Deep recurrent extreme learning machine for behind-the-meter photovoltaic
disaggregation,
The Electricity Journal,
Volume 35, Issue 5,
2022,
107137,
ISSN 1040-6190,
https://doi.org/10.1016/j.tej.2022.107137.
(https://www.sciencedirect.com/science/article/pii/S104061902200063X)
Abstract: In recent years, sustainable sources of energies attract significant
interest due to the serious environmental issues of fossil fuels. Rooftop
photovoltaic (PV) panels are among the important power generation technologies
exploited in many modern countries. Most PV panels are installed behind-the-meter
(BTM), resulting in a lack of observability of actual load and PV power generation
in a power distribution system. This paper proposes a novel supervised
spatiotemporal approach to accurately disaggregate the net-load data of a set of
neighboring residential units. To this end, spatiotemporal correlations of a group
of neighboring residential units are modeled using a weighted undirected graph
where the nodes store the temporal features. The edges reflect the spatial
correlation between neighboring residential units and are determined by the
information analysis technique. Afterward, a generative graph attention recurrent
neural network (RNN) is devised for capturing highly nonlinear patterns of input
graphs using an RNN encoder. In addition, to boost the generalization capacity and
robustness of the proposed model, we reconstruct the input graphs using sparse
contractive decoders. Finally, an extreme learning machine (ELM) neural network is
employed to disaggregate the input net-load time-series of a set of residential
units using the extracted complex spatiotemporal patterns. Experimental results on
the real-world Pecan Street dataset demonstrate the superiority of the proposed
RNN-ELM method over the recent BTM disaggregation techniques.

Asif Iqbal Middya, Sarbani Roy,

Spatio-temporal variation of Covid-19 health outcomes in India using deep learning
based models,
Technological Forecasting and Social Change,
Volume 183,
2022,
121911,
ISSN 0040-1625,
https://doi.org/10.1016/j.techfore.2022.121911.
(https://www.sciencedirect.com/science/article/pii/S0040162522004334)
Abstract: Deep learning methods have become the state of the art for spatio-
temporal predictive analysis in a wide range of fields, including environmental
management, public health, urban planning, pollution monitoring, and so on. Despite
the fact that a variety of powerful deep learning-based models can address various
problem-specific issues in different research domain, it has been found that no
single optimal model can outperform everywhere. Now, in the last two years, various
deep learning-based studies have provided a variety of best-performing techniques
for predicting COVID-19 health outcomes. In this context, this study attempts to
perform a case study that investigates the spatio-temporal variation in the
performance of deep-learning-based methods for predicting COVID-19 health outcomes
in India. Various widely applied deep learning models namely CNN (convolutional
neural network), RNN (recurrent neural network), Vanilla LSTM (long short-term
memory), LSTM Autoencoder, and Bidirectional LSTM are considered to investigate
their spatio-temporal performance variation. The effectiveness of the models is
assessed using various metrics based on COVID-19 mortality time-series from 36
states and union territories of India.
Keywords: Covid-19; Deep learning; Spatio-temporal variation

Bo Wang, Jihong Deng, Hui Jiang, Quansheng Chen,

Electronic nose signals-based deep learning models to realize high-precision
monitoring of simultaneous saccharification and fermentation of cassava,
Microchemical Journal,
Volume 182,
2022,
107929,
ISSN 0026-265X,
https://doi.org/10.1016/j.microc.2022.107929.
(https://www.sciencedirect.com/science/article/pii/S0026265X22007573)
Abstract: This study innovatively proposes a high-precision monitoring method for
key parameters in the process of ethanol production from simultaneous
saccharification and fermentation (SSF) by electronic nose technology combined with
recurrent neural network (RNN). A PEN3 electronic nose system was employed to
acquire the odor information of the fermented samples, and four deep learning
algorithms based on the RNN architecture were employed to design reasonable network
structures to realize the deep learning of the electronic nose signal features and
model calibration. The results obtained showed that each deep learning model based
on the RNN architecture has good generalization performance for the determination
of cassava SSF process parameters. Among them, the bidirectional long short-term
memory network (BiLSTM) model has the best monitoring effect on ethanol content,
with root mean square error of prediction (RMSEP) of 3.7 mg·mL−1 and coefficient of
predictive determination (RP2) of 0.98 and the relative percent deviation (RPD) of
8.1. The bidirectional gated recurrent unit (BiGRU) model had the best monitoring
effect on glucose content, and its RMSEP, RP2 and RPD were 2.9 mg·mL−1, 0.99 and
9.1, respectively. The overall results reveal that deep learning algorithms have
promising application prospects in the feature mining and model calibration of
electronic nose signals, which provides an effective analysis tool for in-situ
monitoring of electronic nose technology in modern industrial fermentation
processes.
Keywords: Cassava; Simultaneous saccharification and fermentation; Electronic nose;
Recurrent neural network; Monitoring

Jyothis Joseph, S. Vineetha, N.V. Sobhana,

A survey on deep learning based sentiment analysis,
Materials Today: Proceedings,
Volume 58, Part 1,
2022,
Pages 456-460,
ISSN 2214-7853,
https://doi.org/10.1016/j.matpr.2022.02.483.
(https://www.sciencedirect.com/science/article/pii/S2214785322011439)
Abstract: This survey focus on sentiment analysis using various Deep learning
methodologies namely Convolutional neural network, Recurrent neural network, Long
Short Term Memory, Gated Recurrent Unit and its variants. Sentiment analysis is
used to analyse opinions or sentiments of people about entities such as products,
services, individuals. Currently it has become a very active research area since a
vast amount of data is generated daily in various forms such as text, audios and
videos in the social media on the world wide web. Sentiment analysis categorizes
opinions into positive, negative, or neutral. Deep learning network perform better
than SVMs and conventional neural networks for sentiment analysis since it can
handle huge amount of data. Out of various deep learning models Recurrent neural
networks perform better than Convolutional Neural networks for sentiment analysis.
LSTM and GRU both are better than Simple RNN because they can catch Long-Term
Dependencies.
Keywords: Sentiment analysis; Deep learning; Natural language processing;
Convolutional neural network; Recurrent neural network; Long Short Term Memory

Maxwell Fordjour Antwi-Afari, Yazan Qarout, Randa Herzallah, Shahnawaz Anwer,

Waleed Umer, Yongcheng Zhang, Patrick Manu,
Deep learning-based networks for automated recognition and classification of
awkward working postures in construction using wearable insole sensor data,
Automation in Construction,
Volume 136,
2022,
104181,
ISSN 0926-5805,
https://doi.org/10.1016/j.autcon.2022.104181.
(https://www.sciencedirect.com/science/article/pii/S0926580522000541)
Abstract: Among the numerous work-related risk factors, construction workers are
often exposed to awkward working postures that may lead them to develop work-
related musculoskeletal disorders (WMSDs). To mitigate WMSDs among construction
workers, awkward working posture recognition is the first step in proactive WMSD
prevention. Several researchers have proposed wearable sensor-based systems and
machine learning classifiers for awkward posture recognition. However, these
wearable sensor-based systems (e.g., surface electromyography) are either intrusive
or require attaching multiple sensors on workers' bodies, which may lead to
workers' discomfort and systemic instability, thus, limiting their application on
construction sites. In addition, machine learning classifiers are limited to human-
specific shallow features which influence model performance. To address these
limitations, this study proposes a novel approach by using wearable insole pressure
system and recurrent neural network (RNN) models, which automate feature extraction
and are widely used for sequential data classification. Therefore, the research
objective is to automatically recognize and classify different types of awkward
working postures in construction by using deep learning-based networks and wearable
insole sensor data. The classification performance of three RNN-based deep learning
models, namely: (1) long-short term memory (LSTM), (2) bidirectional LSTM (Bi-
LSTM), and (3) gated recurrent units (GRU), was evaluated using plantar pressure
data captured by a wearable insole system from workers on construction sites. The
experimental results show that GRU model outperforms the other RNN-based deep
learning models with a high accuracy of 99.01% and F1-score between 93.19% and
99.39%. These results demonstrate that GRU models can be employed to learn
sequential plantar pressure patterns captured by a wearable insole system to
recognize and classify different types of awkward working postures. The findings of
this study contribute to wearable sensor-based posture-related recognition and
classification, thus, enhancing construction workers' health and safety.
Keywords: Awkward working postures; Deep learning networks; Wearable insole
pressure system; Work-related musculoskeletal disorders; Work-related risk
recognition

Manyu Dhyani, Rajiv Kumar,

An intelligent Chatbot using deep learning with Bidirectional RNN and attention
model,
Materials Today: Proceedings,
Volume 34, Part 3,
2021,
Pages 817-824,
ISSN 2214-7853,
https://doi.org/10.1016/j.matpr.2020.05.450.
(https://www.sciencedirect.com/science/article/pii/S221478532034030X)
Abstract: This paper shows the modeling and performance in deep learning
computation for an Assistant Conversational Agent (Chatbot). The utilization of
Tensorflow software library, particularly Neural Machine Translation (NMT) model.
Acquiring knowledge for modeling is one of the most important task and quite
difficult to preprocess it. The Bidirectional Recurrent Neural Networks (BRNN)
containing attention layers is used, so that input sentence with large number of
tokens (or sentences with more than 20–40 words) can be replied with more
appropriate conversation. The dataset used in the paper for training of model is
used from Reddit. The model is developed to perform English to English translation.
The main purpose of this work is to increase the perplexity and learning rate of
the model and find Bleu Score for translation in same language. The experiments are
conducted using Tensorflow using python 3.6. The perplexity, leaning rate, Bleu
score and Average time per 1000 steps are 56.10, 0.0001, 30.16 and 4.5
respectively. One epoch is completed at 23,000 steps. The paper also study MacBook
Air as a system for neural network and deep learning.
Keywords: Deep learning; Chatbot; Bidirectional RNN and Attention model;
Tensorflow; Neural Machine Translation

Wen Tang, Emilie Chouzenoux, Jean-Christophe Pesquet, Hamid Krim,

Deep transform and metric learning network: Wedding deep dictionary learning and
neural network,
Neurocomputing,
Volume 509,
2022,
Pages 244-256,
ISSN 0925-2312,
https://doi.org/10.1016/j.neucom.2022.08.069.
(https://www.sciencedirect.com/science/article/pii/S0925231222010608)
Abstract: On account of its many successes in inference tasks and imaging
applications, Dictionary Learning (DL) and its related sparse optimization problems
have garnered a lot of research interest. In DL area, most solutions are focused on
single-layer dictionaries, whose reliance on handcrafted features achieves a
somewhat limited performance. With the rapid development of deep learning, improved
DL methods called Deep DL (DDL), have been recently proposed an end-to-end flexible
inference solution with a much higher performance. The proposed DDL techniques
have, however, also fallen short on a number of issues, namely, computational cost
and the difficulties in gradient updating and initialization. While a few
differential programming solutions have been proposed to speed-up the single-layer
DL, none of them could ensure an efficient, scalable, and robust solution for DDL
methods. To that end, we propose herein, a novel differentiable programming
approach, which yields an efficient, competitive and reliable DDL solution. The
novel DDL method jointly learns deep transforms and deep metrics, where each DL
layer is theoretically reformulated as a combination of one linear layer and a
Recurrent Neural Network (RNN). The RNN is also shown to flexibly account for the
layer-associated approximation together with a learnable metric. Additionally, our
proposed work unveils new insights into Neural Network (NN) and DDL, bridging the
combinations of linear and RNN layers with DDL methods. Extensive experiments on
image classification problems are carried out to demonstrate that the proposed
method can not only outperform existing DDL several counts including, efficiency,
scaling and discrimination, but also achieve better accuracy and increased
robustness against adversarial perturbations than CNNs.
Keywords: Deep dictionary learning; Deep neural network; Metric learning; Transform
learning; Proximal operator; Differentiable programming

Aratrik Chattopadhyay, Angshuman Paul, Dipti Prasad Mukherjee,

Detail preserving conditional random field as 2-D RNN for gland segmentation in
histology images,
Pattern Recognition Letters,
Volume 159,
2022,
Pages 38-45,
ISSN 0167-8655,
https://doi.org/10.1016/j.patrec.2022.05.001.
(https://www.sciencedirect.com/science/article/pii/S0167865522001532)
Abstract: Grading of cancer offers crucial insights for treatment planning.
Morphology of glands in histology images is of prime importance for grading several
types of cancers. Therefore, accurate segmentation of glands plays a pivotal role
in planning the treatment in case of such cancers. We introduce a first-of-its-kind
detail preserving conditional random field for gland segmentation from histology
images. Our design involves a novel formulation of Gibbs energy that captures the
spatial interaction between neighboring pixels through the hidden state of a 2-D
recurrent neural network (2-D RNN). We show that the iterative training of the 2-D
RNN results in the minimization of the Gibbs energy leading to accurate gland
segmentation. Experiments on publicly available histology image datasets show the
efficacy of the proposed method in accurate gland segmentation. Our model achieves
at least 7% improvement in terms of Hausdorff distance for gland segmentation
compared to a number of state-of-the-art techniques.
Keywords: 2-D RNN; Conditional random field; Detail preservation; Gland
segmentation; Histology

Mou Wang, Sylwan Rahardja, Pasi Fränti, Susanto Rahardja,

Single-lead ECG recordings modeling for end-to-end recognition of atrial
fibrillation with dual-path RNN,
Biomedical Signal Processing and Control,
Volume 79, Part 1,
2023,
104067,
ISSN 1746-8094,
https://doi.org/10.1016/j.bspc.2022.104067.
(https://www.sciencedirect.com/science/article/pii/S1746809422005353)
Abstract: Atrial fibrillation (AF) is the most common type of sustained cardiac
arrhythmia, and is associated with stroke, coronary artery disease and mortality.
Thus, early detection is crucial to avoid serious complications. Existing methods
require specialized equipment and technical expertise, and accurate machine
learning diagnosis of AF remains a dream. In this paper, we propose an end-to-end
AF recognition method with dual-path recurrent neural network (DPRNN) from single-
lead ECG. The model takes the whole ECG as input, and DPRNN splits the ECG into
shorter segments and models the sequence between intra- and inter-segment
iteratively. A mix-up operation is used for data augmentation, which overcomes the
issue of limited data. We evaluated our method on the dataset from PhysioNet
Challenge 2017. Experimental results shows that the proposed method can both
effectively recognize AF with ECG signal without any human expertise, and
outperforms state-of-the-art baseline methods. This demonstrates that dual-path
model is effective for ECG analysis. We postulate that this framework can be
generalized for other medical sequence signal.
Keywords: Atrial fibrillation; ECG; Deep learning; Recognition

Jun Xiang, Ziyuan Huang, Xiaoping Jiang, Jianhua Hou,

Similarity learning with deep CRF for person re-identification,
Pattern Recognition,
Volume 135,
2023,
109151,
ISSN 0031-3203,
https://doi.org/10.1016/j.patcog.2022.109151.
(https://www.sciencedirect.com/science/article/pii/S0031320322006306)
Abstract: The core of person re-identification (Re-ID) lies in robustly estimating
similarities for each probe-gallery image pair. A common practice in existing works
is to calculate the similarity of each image pair independently, ignoring relations
between different probe-gallery pairs. In this paper, we present a deep learning
conditional random field (Deep-CRF) graph to model group-wise similarities within a
batch of images, and regard the Re-ID task as a CRF node labeling problem. Unlike
the existing deep CRF based approach where the CRF inference is only involved in
the training stage, our method intends to fully exploit the potential of CRF model,
exhibiting inference consistency in both training and testing. Specifically, we
design unary potentials for computing each probe-gallery similarity separately. To
efficiently encode relationships between different probe-gallery pairs, pairwise
potentials are built on an arbitrary node pair whose learning is achieved by a
joint matching strategy using bidirectional LSTM. We pose the CRF inference as a
RNN learning process, where unary and pairwise potentials are jointly optimized in
an end-to-end manner. Extensive experiments on three large-scale person Re-ID
datasets demonstrate the effectiveness of the proposed method. Our Deep-CRF
achieves the best results compared with the previous graph-based deep learning
approaches and substantially exceeds the existing deep CRF framework by 8% in Rank1
accuracy on CUHK03 dataset. It also behaves competitive among the current state-of-
the-art methods.
Keywords: Person re-identification; Deep learning; Conditional random field (CRF);
Group-wise similarities

Wendong Ge, Jin-Won Huh, Yu Rang Park, Jae-Ho Lee, Young-Hak Kim, Guohai Zhou,
Alexander Turchin,
Using deep learning with attention mechanism for identification of novel temporal
data patterns for prediction of ICU mortality,
Informatics in Medicine Unlocked,
Volume 29,
2022,
100875,
ISSN 2352-9148,
https://doi.org/10.1016/j.imu.2022.100875.
(https://www.sciencedirect.com/science/article/pii/S2352914822000296)
Abstract: Background and objectives
Changes in a patient's condition over time are a backbone of clinical decision
making. However, most currently used methods for identification of patients in
intensive care units (ICUs) at high risk for death do not make effective use of the
temporal dimension of available data. We therefore conducted a study to determine
whether longitudinal data analysis using recurrent neural networks (RNN) with
attention mechanism can identify novel temporal data patterns predictive of adverse
outcomes.
Methods
We analyzed data on patients admitted to the Medical Intensive Care Unit (MICU) of
Asan Medical Center between 2010 and 2017. Static (demographics, diagnoses,
procedures, medications) and longitudinal (vitals, laboratory tests, Glasgow Coma
Scale) variables were included in the analysis. We used an RNN model with long
short-term memory (RNN-LSTM) with attention mechanism to identify and test novel
data patterns predictive of ICU death. We also compared accuracy of prediction of
ICU mortality between a logistic regression and RNN-LSTM models with and without
attention.
Results
Among 4896 patients admitted to the MICU, 548 (11.19%) died. RNN-LSTM model with
attention identified several high-risk longitudinal variable patterns that were
predictive of ICU mortality in a confirmatory analysis, including sustained low
blood oxygen content (OR 2.33; 95% CI 1.16 to 4.70) and high frequency of serum
sodium measurements (OR 1.27; 95% CI 1.04 to 1.56). RNN-LSTM models with and
without attention achieved numerically, but not statistically significantly higher
c-statistics for prediction of ICU mortality compared to logistic regression.
Conclusions
RNN-LSTM model with attention identified novel temporal data patterns predictive of
ICU mortality. These predictors were both statistically significant and clinically
plausible, likely representing progressive respiratory failure (sustained low
oxygen saturation) and close monitoring of a clinically deteriorating patient
(frequent sodium measurements).
Keywords: Predictive modeling; Temporal data; Deep learning; Recurrent neural
networks; Attention

Mouaad Boulakhbar, Markos Farag, Kawtar Benabdelaziz, Tarik Kousksou, Malika Zazi,
A deep learning approach for prediction of electrical vehicle charging stations
power demand in regulated electricity markets: The case of Morocco,
Cleaner Energy Systems,
Volume 3,
2022,
100039,
ISSN 2772-7831,
https://doi.org/10.1016/j.cles.2022.100039.
(https://www.sciencedirect.com/science/article/pii/S2772783122000371)
Abstract: The transport sector is a prominent source of increasing fuel consumption
and greenhouse gas (GHG) emissions. Electric vehicle (EV) is deemed an appealing
solution for those problems. However, due to the variation in charging demands, the
high penetration of electric vehicles may cause different problems to the
distribution network and its reliability. Therefore, several approaches are
employed to predict the EVs charging demand and avoid the corresponding challenges.
This paper compares the performance of four well-known deep learning models, namely
artificial neural networks (ANN), recurrent neural networks (RNNs), long short-term
memory (LSTM), and gated recurrent units (GRUs), in predicting charging demand for
EV users after a charging session begins. We use a dataset consisting of 2000
observations of charging events collected from two public charging stations in
Morocco. According to numerical data results, the first layer of the GRU regression
approach marginally beats the other three methods in estimating power charging
needs. Specifically, the GRU regression model has an RMSE and MAPE of 1.39% and
0.50% in the training stage and 2.90% and 0.76% in the testing stage, respectively.
These findings can assist the National Office of Electricity and Water in Morocco
in ensuring the reliability of grid utility in the short run and guiding them to
construct additional charging stations in the long run.
Keywords: Load forecasting; Deep learning; Electric vehicles; Regulated electricity
market; Morocco

Adeel Shiraz Hashmi, Tanvir Ahmad,

GP-ELM-RNN: Garson-pruned extreme learning machine based replicator neural network
for anomaly detection,
Journal of King Saud University - Computer and Information Sciences,
Volume 34, Issue 5,
2022,
Pages 1768-1774,
ISSN 1319-1578,
https://doi.org/10.1016/j.jksuci.2019.09.007.
(https://www.sciencedirect.com/science/article/pii/S1319157819306809)
Abstract: Replicator Neural Network (RNN) is a popular algorithm for anomaly
detection, but finding optimal number of hidden layers and then finding optimal
number of neurons in each hidden layer is quite a challenging and time-consuming
task. Extreme Learning Machines (ELM) are neural networks with single-hidden layer
but the learning algorithm is different and faster than back-propagation. ELM-based
RNNs solve our problem of determining the number of hidden layers and the learning
algorithm is also faster than gradient-descent based RNN. The problem of
identifying the optimal number of neurons in the hidden layer can be solved by
Garson algorithm. In this work, the author propose an optimal Replicator Neural
Network which is optimized using ELM learning and Garson algorithm for anomaly
detection. The experimental results show that the proposed method is fast as well
as highly accurate.
Keywords: Anomaly detection; Big data; Garson algorithm; Replicator neural network;
Extreme learning machine