Expert Systems With Applications 173 (2021) 114598

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Predictive maintenance system for production lines in manufacturing: A

machine learning approach using IoT data in real-time
Serkan Ayvaz a, *, Koray Alpay b
a
Department of Artificial Intelligence Engineering, Bahcesehir University, Besiktas, Istanbul, Turkey
b
Department of Gebze Plant Digital Team, Gebze Development Center, Procter & Gamble, Kocaeli, Turkey

A R T I C L E I N F O A B S T R A C T

Keywords: In this study, a data driven predictive maintenance system was developed for production lines in manufacturing.
Predictive maintenance By utilizing the data generated from IoT sensors in real-time, the system aims to detect signals for potential
Internet of things failures before they occur by using machine learning methods. Consequently, it helps address the issues by
Manufacturing systems
notifying operators early such that preventive actions can be taken prior to a production stop. In current study,
Artificial intelligence
Machine learning
the effectiveness of the system was also assessed using real-world manufacturing system IoT data. The evaluation
Big data results indicated that the predictive maintenance system was successful in identifying the indicators of potential
failures and it can help prevent some production stops from happening. The findings of comparative evaluations
of machine learning algorithms indicated that models of Random Forest, a bagging ensemble algorithm, and
XGBoost, a boosting method, appeared to outperform the individual algorithms in the assessment. The best
performing machine learning models in this study have been integrated into the production system in the factory.

1. Introduction 1.1. Data collection and connectivity through IoT

With the digital transformation experienced today, applications of Technological advancements have transformed how individuals ac­
Big Data and Artificial Intelligence (AI) shape our daily lives and in­ cess to information in recent decades. Now, data are being collected
crease the efficiency of working processes in all areas. Unprecedented from variety of resources. Mobile phones, wearable technologies, and
technological and scientific developments are particularly taking place other smart devices have become essential part of daily life. While data
in the fields of AI and Internet of Things (IoT), which are at the heart of is being collected more rapidly, it has a more complex structure now.
the fourth industrial revolution (Lee, Davari, Singh, & Pandhare, 2018). The key challenge is analyzing such a big data collected from various
In this era, AI and big data analytics play a vital role in innovation environments and resources in a timely manner, and utilizing it in
independently of the industry by providing solutions to complex prob­ making better decisions in important issues such as management of real-
lems. Due to global challenges and intense competitiveness in the mar­ time operations, processing and evaluation of data produced by devices,
kets, the firms constantly face a pressing need for increasing efficiency, and instant detection of risks and faults (Wang, Wan, Zhang, Li, &
reducing costs and delivering better solutions nowadays (Furman & Zhang, 2016).
Seamans, 2019). Institutions and countries that successfully carry out Thanks to rapid developments in data protocols, smart devices and
the processes of understanding, analyzing big data, producing fast re­ technologies, the IoT has been transforming the manufacturing pro­
sults, developing AI driven new and useful tools, and adapting data cesses by integrating systems and connecting devices, systems and
centered decision-making mechanisms are one step ahead in the people (Ansari, Erol, & Sihn, 2018). At the center of industry 4.0 revo­
competitive world (Bolton, Machová, Kovacova, & Valaskova, 2018; lution, the IoT facilitates greater productivity in manufacturing by
Wamba-Taguimdje, Wamba, Kamdjoug, & Wanko, 2020). enabling connectivity and data exchange between production systems
(Lu, 2017). Modern manufacturing processes are typically composed of
very complex autonomous robotic machinery, which are embedded with
sophisticated software systems and surrounded by the IoT devices. With

* Corresponding author at: Yildiz mh. Ciragan cd. No: 4-6, Besiktas, 34349 Istanbul, Turkey.
E-mail addresses: [email protected] (S. Ayvaz), [email protected] (K. Alpay).

https://doi.org/10.1016/j.eswa.2021.114598
Received 10 September 2020; Received in revised form 14 November 2020; Accepted 8 January 2021
Available online 16 January 2021
0957-4174/© 2021 Elsevier Ltd. All rights reserved.
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

integration of robotic and autonomous equipment, the efficiency of 2019; Yu et al., 2019; Erfani et al., 2016; Fernandes, Canito, Corchado,
manufacturing systems has increased tremendously around the world in & Marreiros, 2019). Some studies have treated PdM from diverse per­
recent years (Stock & Seliger, 2016). spectives such as supervised classification (Traini et al., 2019; Edward,
Dariusz, Żabiński, Prucnal, & Se, 2020, anomaly detection Rabatel,
1.2. Production stops and preventive maintenance Bringay, & Poncelet, 2011; Yu et al., 2019; Rivera, Scholz, Bühl, Krauss,
& Schilling, 2019), regression in high dimensional data (Zhang, Wang,
The production environment rely on equipment to work properly. A Yan, & Gao, 2018; Chen et al., 2020; Zhang et al., 2018), reinforcement
fault in a component or sub-system may cause a stop in entire production learning for system modeling (Huang, Chang, & Arinez, 2020) and un­
line. The production stops are associated with huge costs in many folds supervised learning problem (Erfani et al., 2016; Zhao et al., 2017).
including but not limited to the loss of production and time in downtime,
losses of efforts in identification of the cause of the failure and repair, 2.1. Statistical approaches to predictive maintenance
wastes of products produced right after bringing back the system until
normal operations due to low quality, costs of repairs and deterioration Based on current developments in production and technology, esti­
of equipment (Froger, Gendreau, Mendoza, Pinson, & Rousseau, 2016). mating remaining useful life of machinery has begun to play an
The stops may occur as a consequence of various types of issues such important role for ensuring machine condition monitoring, productivity,
as malfunctions of equipment, operator errors or environmental factors. reliability and safety. Different theoretical and practical applications
Although it may not be possible to eliminate all system failures, pre­ have been proposed. Frequently reported are combinations of different
ventable faults make of a large portion of the production stops (He, Gu, deep learning models that combine multiple aspects in one application.
Chen, & Han, 2017). If predicted early, the equipment or sub-system In addition, the need for real-time processing of complex data and data
leading to potential faults can be addressed without a production stop, streams has been demonstrated in some application scenarios. Similarly,
which would result in significant saving in associated costs (Yu, Dillon, Autoregressive Integrated Moving Average (ARIMA) models have also
Mostafa, Rahayu, & Liu, 2019). been used for error detection and predictive maintenance forecasts in
Predictive maintenance (PdM) has become common objective in the time series (Ho, Xie, & Goh, 2002; Ramos, Oliveira, & Silva, 2014;
industry to reduce maintenance costs and ensure sustainable operational Kanawaday & Sane, 2017; Fernandes et al., 2019; Francis & Mohan,
management (Stock & Seliger, 2016). The essence of the PdM is to 2019). However, when the accuracy of prediction performance is
predict the next error in a manner that preventive maintenance can be compared, it is known that ARIMA methods are not as successful and
performed before the failure takes place. PdM also has the potential to flexible as the machine learning counterparts. Hence, machine learning
promote sustainable practices in production by maximizing the useful methods were considered in our approach only.
lives of production (Lee, Kao, & Yang, 2014).
For PdM, data flexibility is seen as a critical issue that can compro­ 2.2. Data-driven AI applications to failure predictions
mise algorithm performance for modeling based on data (Wang, Zhang,
Duan, & Gao, 2017). Processing raw data from sensors presents another Zhang et al. investigated developing models using temporally
challenge since sensor data are not labeled. In addition, data sparseness dependent sensor data for Nasa’s aircraft engine performance decline
may affect the algorithm performance in existing approaches when monitoring and engine useful life expectancy (Zhang et al., 2018). Due
addressing the problem of data tagging. Moreover, it is often very to their wide use in most aircraft engines, 6 built-in sensors were
difficult to clearly identify the anomalies and patterns in high dimen­ selected and focused on a Long Short Term Memory (LSTM) based model
sional IoT data (Erfani, Rajasegarar, Karunasekera, & Leckie, 2016). to characterize system failure behavior and estimate the useful life
When considering big data environments, it is even more challenging remaining. In addition, the performance of LSTM’s ability to capture
due to noise in device generated data (Yu et al., 2019). temporal dependence was compared with various machine learning
techniques. Because the recurrent neural network (RNN) algorithms do
1.3. AI applications in manufacturing not store long-term memory, the LSTM cell was chosen for this problem.
In the experiments, 200 synthetic cycles were created to test LSTM
Nevertheless, data driven AI applications utilizing data collected performance. It has been shown that the trained model can effectively
from IoT devices can help in predictive maintenance (Kanawaday & predict the actual behavior of the test set and the accuracy of the pre­
Sane, 2017; Yu et al., 2019; Wang et al., 2017). The aim of this study is to diction increases as the length of the prediction decreases. Compared to
develop a predictive maintenance system that produces realistic pre­ various machine learning prediction models, LSTM performed better in
dictions of potential failures for production lines in manufacturing useful life estimation in both datasets than alternative machine learning
before occurring using machine learning methods. In order to obtain the methods including Support Vector Regression (SVR), Multilayer Sensor
most suitable model to address this problem, multiple algorithms were (MLP), and Deep Evolution Neural Network (DCNN) (Zhang et al.,
explored in details and compared using a real-world dataset. 2018).
The paper is organized as follows: Section 2 reviews existing solu­ In a similar study, Lee et al. worked on data-driven artificial intel­
tions and the related work in the area of predictive maintenance. In ligence based modeling approach to predict spindle motor, cutting
Section 3, the methods and materials used in the study are explained in machine wear and malfunctions (Lee et al., 2019). In their experiments,
detail. Section 4 presents a real-world application of the proposed sys­ SVR and artificial neural network models have yielded successful results
tem in a manufacturing plant as a use case. Section 5 illustrates the re­ in predicting the condition and remaining life of the systems. In another
sults of the evaluations assessing the efficiency of the proposed system. related work, Traini and others proposed a modeling framework to es­
Section 6 provides an interpretation of the results obtained and discusses timate maintenance on grinder machines (Traini et al., 2019). They
the significance of the study. Finally, it is followed by conclusion. evaluated several algorithms for regression and classification purposes.
Rivera et al. examined the errors in production systems from the
2. Related work point of view of anomaly detection (Rivera et al., 2019). Accordingly,
they provided a statistical method based on kernel density estimation
With continuing development in big data, AI and IoT technologies and the evolution of distribution over time, which could lead to the
that are driving forces of industry 4.0, the field of predictive mainte­ possibility of predicting maintenance. However, when analyzed from an
nance has attracted many researchers from research community recently expert point of view, it was noticed that there were not many
(Rieger, Regier, Stengel, & Clarke, 2019; Li, Wang, & Wang, 2017; outstanding findings in the data. On the other hand, the problem of
Zhang, Wang, Yan, & Gao, 2018; Traini, Bruno, D’Antonio, & Lombardi, reinterpretability of the results has emerged. Because it was not clear

2
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

why the algorithm marked the particular cycle as abnormal, they had algorithms are successful for time series data and LSTM algorithm,
difficulty deciding whether it was correct to throw the cycle as an which is a version of RNN for data with long-term dependencies, pro­
anomaly for further analysis. vides good performance (Mohammadi, Al-Fuqaha, Sorour, & Guizani,
In Rivera et al. (2019), the authors pointed out the difficulties of 2018). Different from these aforementioned RNN based approaches, we
using raw sensor data and stated that without using expert knowledge, deal with a problem that the exact time to a failure is not known at the
the reference data may deviate too much and may cause the proposed time of prediction in production setting. Thus, the RNN algorithms are
system to make false predictions. The most important finding that they not applicable to our case because these approaches necessitate avail­
obtained was the importance of data quality and the use of expert ability of actual data at previous time steps due to sequence-to-sequence
knowledge. analysis of data.
Li et al. also proposed a data mining framework for maintenance and While many applications were developed for real-time processing of
error prediction in production machines (Li et al., 2017). However, they deep learning based predictive maintenance models, there are also
stated that there were still many challenges in the application of the critical voices that criticize the absolute focus on predictive success and
framework that they proposed for fault diagnosis in machine centers. recommend more focus on practical use and robust applications (Li
The main benefits of the Artificial Neural Networks (ANN) algorithm et al., 2017).
that they used in the application were fault tolerance, generalization and Numerous deep learning approaches described in the literature are
adaptability, but besides, the limitation was that it is not possible to not suitable for practical use. This is due to long processing times,
determine the effect of various factors and the disadvantages of not computational complexity and excessive power consumption. Canziani
having an explanation function. and others propose to pay more attention to performance problems in
their articles because performance problems are key in practical deep
2.3. Deep learning applications in predictive maintenance learning practices (Canziani, Paszke, & Culurciello, 2016). Due to per­
formance implications of deep learning methods in real-time practical
From deep learning perspective, Song et al. and Xie et al. have also applications, we have decided to use data-driven machine learning al­
confirmed that RNN and LSTM algorithms perform well for sequential gorithms in our predictive maintenance system.
data, time series with long-term dependencies, and IoT flow data (Song,
Kanasugi, & Shibasaki, 2016; Xie, Wu, Liu, & Li, 2017). The authors in
Xie et al. (2017), suggested that a combination of LSTM and Naïve Bayes 2.4. Predictive maintenance in sustainability of cyber-manufacturing
model may capture trends and make forecasts successfully. Naïve Bayes
served as an anomaly detector in the output of the LSTM model. Predictive maintenance has also attracted attention from the
Contrarily, Mocanu and others stated that the Factored Conditional perspective of sustainability in Cyber-Manufacturing Systems (Jasiule­
Restricted Boltzmann Machine (FCRBM) method performs slightly bet­ wicz-Kaczmarek, Legutko, & Kluk, 2020; Song & Moon, 2017; Jena,
ter than other algorithms in comparing deep learning methods in their Mishra, & Moharana, 2020; Machado, Winroth, & Ribeiro da Silva,
applications involving the prediction of building energy consumption in 2020). Recent technological advancements and digitalization have
smart cities (Mocanu, Nguyen, Gibescu, & Kling, 2016). In addition, begun to transform manufacturing towards Industry 4.0 (Jeschke,
when Gensler and others examined deep learning methods in an IoT Brecher, Meisen, Özdemir, & Eschert, 2017) by making systems and
application that includes solar panels prediction, in addition to LSTM, processes intelligent connected through IoT to such an extent that
Deep Belief Network (DBN) method also showed good results in time physical manufacturing systems can be represented by a digital twin (Lu
series data (Gensler, Henze, Sick, & Raabe, 2016). et al., 2020) in cloud, which enables monitoring and intelligent decision
Rieger et al. reviewed and categorized the deep learning methods making in real-time (He & Bai, 2020).
used in predictive maintenance in IoT devices (Rieger et al., 2019). Applying data-driven predictive maintenance as a function of smart
Temporal delay in failure prediction in industrial production processes manufacturing offers sustainability benefits (Song & Moon, 2017) in
is a problem. Therefore, it is treated as a real-time process. In such es­ many ways, including increasing reliability and lifetime of equipment,
timates, RNNs have been proposed due to sequential data development reducing downtime and energy consumption due to stopping, and
in spatial–temporal data types, because of their success in these types. lowering cost of materials and inventory due to production wastes
However, for these data types with long-term dependencies, RNNs were (Jasiulewicz-Kaczmarek et al., 2020). Consequently, overall productiv­
not considered a good choice as they did not learn the previous situation ity and quality of products can be increased (Song & Moon, 2018).
and results. For such problems, the combination of RNN with LSTM is
preferred because LSTM adds labeling and prediction functionality. 3. Proposed predictive maintenance system
In a research study, a new approach called Cox proportional hazard
deep learning (CoxPHDL) was proposed to address data flexibility and Making machines, devices and systems connected is the most
data censorship, which are commonly mentioned in the analysis of fundamental step for an IoT application implementation. An efficient
operational maintenance data (Chen et al., 2020). The main idea was to and scalable data processing infrastructure is required to use the data
offer an integrated solution using deep learning and reliability analysis. streaming continuously and instantly from the sensors placed in the
To begin with, an automatic encoder was adopted to convert nominal factory in decision making processes. The connectivity of devices in
data into a solid representation. Secondly, a Cox proportional hazard existing production environments can be established by adding sensors
model (Cox PHM) was investigated to estimate Time-Between-Failure to the machines, controllers and installing gateway devices for
from censored data. A long-term memory (LSTM) network had been communication.
created to train the Time-to-Failure prediction model based on pre- The IoT system must have a framework that can be integrated with
processed maintenance data. The experimental studies using a large external systems. With an integrated data streaming prediction model, a
real-world maintenance dataset demonstrated the advantages of their PdM system may be able to detect the failures, create alarms by applying
proposed approach, in which algorithm performance was based on the certain rules, execute commands on the production systems, and send
LSTM network (Chen et al., 2020). warning messages to the authorized officials in real-time.
Mohammadi et al. also compiled a large number of deep learning In this study, the infrastructure was first improved by making ma­
algorithms used in the processing of flowing IoT data in his review chines connected through sensors and gateways, establishing the
article. It is mentioned that Restricted Boltzmann Machine (RBM) Internet of Things platform, making the data streaming from the pro­
method is useful in feature extraction and dimensional reduction in case duction lines constantly accessible and integrating system components
of multidimensionality in IoT data. It has been stated that RNN prior to developing the predictive maintenance system.

3
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

3.1. Overview of proposed system architecture Application interfaces were developed to dynamically present the
relevant content and outputs of the prediction model to the end users as
The system comprises an IoT platform to enable communication warnings, visual notifications depending on the user roles and authori­
between machines, collect data from devices, monitor live data, and zations. The end-user interfaces for monitoring and error reporting op­
manage historical data. The IoT platform monitors the status and sensor erations are web applications running on web browsers. The web
data on the machines and periodically record them in the historical interfaces were customarily built and integrated into the IoT platform.
database and create the basis data for the failure prediction model. In our system, all the computational tasks are executed on-premise
Since data are provided from a large number of IoT devices at the servers due to data security and privacy concerns. We also followed
factory, no data loss must occur, the data collected from all sensors must the best practices and tuning in order to make efficient use of the
be processed instantly and evaluated in decision processes. The system architectural setup.
contains a database for connected systems, which allows real-time
querying of streaming data that is used for machine learning (ML) 4. Implementing proposed system in a consumer goods
models. After collecting the data from the sensors and making the manufacturing plant
necessary data conversions automatically during the data preprocessing
process, a fault detection analysis is performed by ML algorithms. To assess the effectiveness of the proposed system, we have imple­
Fig. 1 shows an architectural overview of proposed predictive mented the system in a real-world manufacturing plant in Turkey as a
maintenance system components. The data from the sensors are use case in an additive layered manufacturing process. The plant pro­
collected on the private cloud system in a database that allows recording duces personal care goods such as baby care, feminine hygiene and
data in distributed systems by gathering them through the Internet of home care products for a world leader consumer goods production
Things (IoT) and converting them into a single data type using the company. The following subsections present the application of the
Message Queuing Telemetry Transport (MQTT) protocol, which is a proposed system and the underlying machine learning modeling for
standard messaging protocol designed by OASIS for IoT. MQTT is chosen describing our general approach in a concrete example.
for connecting the IoT sensors in our case because it is a lightweight,
easy to implement and open messaging protocol (Stanford-Clark & 4.1. Data preprocessing
Hunkeler, 1999; Locke, 2010).
Using the data collected in real-time from sensors, the system aims to 4.1.1. Data collection
automatically detect the signals for potential errors using machine The dataset was obtained from assembly lines producing baby di­
learning algorithms independently from the human interventions and to apers in a real-world production plant. The variables reflect device
guide operational preventive actions. The machine learning models in generated data values that change over time. The dataset consisted data
the system calculate the optimum values according to the changes in the readings from various types of built-in IoT sensors that are primarily
input parameters resulting from instantaneous measurements automat­ detecting properties such as motion, speed, weight, temperature, elec­
ically. Algorithms were trained to optimize the weights of the input trical current, vacuum and air pressure on different equipment in the
parameters, and capture the data patterns in on the stream data suc­ manufacturing lines. The data points were collected from IoT sensors
cessfully. The best performing ML models then were integrated into the monitoring the production system in every 3 to 6 s periods. In total, the
production system in the factory. dataset contained 101 features and 8,389,515 rows.

Fig. 1. Architectural overview of the predictive maintenance system.

4
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

Most of the columns in the dataset were filled. There were only a few magnitude less than of the normal (No-failure) operation cases. More­
columns with a small ratio of missing values, which consisted approxi­ over, the distribution of classes may be imbalanced among the failure
mately one percent of data. The missing values might have occurred due types as it was the case in our situation. Some of the failure types were
to errors in the process of logging the sensor reading in database. Since very rare with a few examples only while other failure types occur more
the proportion of the missing values was very low, those were auto­ commonly. To build a meaningful predictive model, we omitted rare
matically filled with median values of the respective columns. All col­ failure types in the dataset and focused on two of failure types only,
umns except timestamps were numerical. This provided great namely, Weight low 1 and Weight low 2 errors that originate from
convenience in the analysis methods. adjacent zones in the assembly line and make up the majority of the
However, the ranges of the sensor readings were significantly cases. These errors are called “Weight low” due to the fact that failures
different, in some cases an order of magnitude difference. Normalizing originally arise from insufficient raw material feeding into injection
features that are not in the same scale plays an important role in elim­ system.
inating potential optimization problems during data modeling. The In our case, we have applied bagging and boosting methods as
normalization process makes modeling less sensitive to the scale of balancing strategies.
features and improves overall quality of the analysis. Thus, the data
standardization was applied to the features during pre-processing phase 4.1.3. Correlation analysis
of the study. One of the most important outputs of the study is data discovery. The
descriptive correlation analysis demonstrates inherent associations
4.1.2. Handling imbalanced data among different features. In this research, feature discovery and corre­
In general, machine learning algorithms can capture patterns in data lation analyzes have been explored. We examined the relationships be­
better from a balanced data distribution. When class distribution is tween the data points that were collected from the production lines and
imbalanced, the ML classifiers are prone to have bias towards large class. the remaining useful lifetime before the two Weight low failure types.
For handling imbalanced datasets, some data sampling strategies have The features that were descriptively important in preventive mainte­
been developed such as applying random under-sampling and over- nance have been investigated for decision making processes and taken
sampling, syntatic over-sampling (Chawla, Bowyer, Hall, & Kegel­ into account in modeling task.
meyer, 2002), bagging and boosting. Fig. 2 illustrates the results of correlation levels between all features
Due to the nature of data distribution in predictive maintenance, the in the analysis. It is clear from the figure that some features have high
data distribution is almost always going to be highly imbalanced. degree of correlation, e.g., X5 through X12 and X13 through X17, and
Because the frequency of the failure is often expected to be an order of might lead to Multicollinearity. However, this is not surprising because

Fig. 2. Correlation of variables.

5
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

the highly correlated variables were derived from different sensors on principal components 1 and 2.
the same components and represented the corresponding subsystem of Moreover, the variable importancies were calculated based on the
production unit. Since very highly correlated features could be repre­ predictive power of features in Random Forest algorithm (Svetnik et al.,
sented by a smaller form, the correlation results were considered for 2003) using Gini impurity measure in the study. Gini-based importance
dimensionality reduction in the feature selection phase. of a variable is computed based on the sum of squared errors (SSE)
difference if the variable is excluded (Strobl, Boulesteix, Kneib, Augus­
4.1.4. Feature selection tin, & Zeileis, 2008; Menze et al., 2009). Among all variables, top 10
In machine learning, feature selection is an essential task for important features were identified as shown in Fig. 4.
modeling high dimensional big data successfully. High dimensionality The findings of PCA variable contributions were mostly in line with
problem, also known as curse of dimensionality (Bellman, 1966; Powell, the variable importancies. We have also considered the expert knowl­
2007), can muddle the ML models since it often causes numerous edge in identifying relevant sensor variables mostly associated with
challenging issues such as overfitting, multicollinearity, exponential common error types. After taking into account the factor analysis and
computational complexity, data sparseness, and reduced interpretability domain expertise, the modeling dataset was finally reduced to 19 fea­
(Altman & Krzywinski, 2018). By carefully applying dimensionality tures; 18 independent variables and the dependent variable, which is the
reduction strategies, feature selection process can help decrease poten­ remaining useful lifetime before the next stop associated with the two
tial ML model prediction errors in high dimensional data. error types.
In the study, the dataset contained 94 sensor generated variables Before proceeding to the modeling phase, the process of splitting the
about the production line and 7 data points associated with the failure dataset into training and test set has been applied with appropriately. It
type, a total of 101 features. The variables related to the two failure is the key process for identifying potential modeling issues such as
types included data points such as timestamp, error type, uptime, overfitting and underfitting. After the split, 80% of the dataset was
downtime, team, work shift at the time of stops. Since each sensor allocated to training dataset and the rest held out to the test set. More­
reading was accompanied with a timestamp variable showing the last over, 5-fold cross validation technique has been applied in training data
time the sensor recorded a value, the dataset contained many duplicated to avoid the overfitting due to sampling issues. To prevent the risk of
timestamp values. To eliminate the duplicate features, we mapped 47 involuntarily leaking information from the test set to the training and
timestamps to a single unique timestamp. After the mapping and indirectly provoking overfitting, the test set was hidden from hyper­
removal of redundant timestamps, the dataset was reduced to 58 parameter tuning process. For performing hyperparameter tuning, we
features. used random search with 5-fold cross validation technique, in which 1-
Principal component analysis (PCA) is a dimension reduction tech­ fold of training data was held out as a validation set and the remaining
nique that projects a set of variables into a smaller set of linearly folds were used for model training.
orthogonal new artificial dimensions, called principle components that
capture the largest variances among original observations (Wall, 4.2. Selecting and training ML models
Rechtsteiner, & Rocha, 2003). The original variables can be constructed
by using the weight vectors of the PCs. PCA is especially useful when In predictive maintenance, predicting the class of an instance as a
dealing with high dimensional space because the dataset can be repre­ failure or normal operation only is not sufficient. It is more important to
sented by a much smaller subspace without having to loose a lot of correctly estimate the remaining time well before the potential failure to
variation in original data samples (Shaw, 2009; Al-Kandari & Jolliffe, a certain extent that the maintenance or other preventive actions can be
2005). taken ahead of time. Hence, the problem can be considered as a
The dataset in our case was high dimensional and required applying regression task rather than a classification problem. In other words, the
feature selection. As a dimension reduction technique, PCA was also objective of the model was to predict remaining useful time before the
applied during feature selection phase. The results of PCA indicated that failure in a data-driven approach.
more than 95% of the variance in the dataset can be explained by 17 Although the prediction problem in this case may appear like a time
principle components only. Fig. 3 illustrates PCA variance contributions series problem, it is not a suitable approach to consider it as one because
of the features. It demonstrates how each variable contributes to the model can’t verify actual time to failure values at the run time until
the next failure happen. Since it is not possible to calculate the predic­
tion errors from input time steps for propagation through time, the
predictions will be based on the predictions of previous input time-steps.
To fully understand a particular predictive modeling problem,
various configurations, which are also called hyperparameters in

Fig. 3. PCA variable variance contributions. Fig. 4. Top 10 important variables.

6
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

machine learning, must be explored and compared against a benchmark. 5. Results

In the analysis, the selection of the algorithms and corresponding
hyperparameters determines the success of the model. In this study, In the evaluations, the prediction performances of the algorithms
multiple ML algorithms and hyperparameters were examined for the were also assessed using R2 (R-squared), MAE (Mean Absolute Error),
purpose of finding the most suitable prediction models. As some MAPE (Mean Absolute Percentage Error) and RMSE (Root Mean Squared
equipment and data points varied in production lines, different ML Error) metrics as illustrated in Table 1.
models were developed for each production line. For hyperparameter R2 is the proportion of the total variability in the dependent variable
tuning, random search with 5-fold cross validation technique was used. that the model was able to reduce. In other words, it measures the model
The random search was chosen over grid search method due to extreme quality by the fraction of overall uncertainty that the model has helped
computation cost and inefficiencies of the latter. explain. A higher R2 value indicates a better fit for the model in general
For the evaluations, six different algorithms were used: four (Draper & Smith, 1998). As shown in Eq. 1, R2 is defined as:
ensemble and two constituent machine learning algorithms. The
∑
n
ensemble methods, namely were Random Forest, which is a bagging (yi − ŷi )2
ensemble method, and XGBoost, Gradient Boosting and AdaBoost, R = 1 − i=1
2
(1)
∑n
which are boosting ensemble algorithms. Moreover, Multilayer Per­ (yi − y)2
ceptron (MLP) Regressor, a Neural Network model and Support Vector i=1

Regression (SVR) algorithm were assessed.

MAE measures the average difference between the model predictions
Furthermore, we have explored the three other widely known linear
from the actual failures the test set in terms of absolute values (Willmott
regression algorithms for modeling purposes: Multiple Regression, Lasso
& Matsuura, 2005). MAE is calculated as shown in Eq. 2:
Regression and Ridge Regression. However, these algorithms performed
poorly due to non-linear nature of our problem. These algorithms were ∑
n
|yi − ŷi |
unable to capture any variance in the data. In other words, the models
MAE = i=1 (2)
were not able to perform any better than predicting the mean RUL of the n
sample dataset. Therefore, we omitted the linear models from the
evaluations in the study. In the formula, ŷi refers to the predicted remaining useful time value by
Ultimately, the algorithms used in the study were selected primarily the algorithms and yi denotes the actual value in the test dataset. If the
for the following reasons: (1) since prediction of RUL is treated as MAE value is smaller, it indicates that the algorithm provides better
regression problem, the chosen ML algorithms must be applicable to predictions for potential failures. MAPE is very similar to MAE. The main
regression; (2) they can be applied to non-linear data as it is in our case; difference is that MAPE measures the variation in percentage as opposed
(3) the algorithms need to have high prediction power according to to the absolute values (Chai & Draxler, 2014).
standard performance metrics in literature; (4) the models need to be √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
√∑
√n
robust and scalable in high-dimensional data. √ (yi − ŷi )2
√
After evaluating several different options within the hyperparameter RMSE = i=1 (3)
space for the models, the following parameters were found optimal n
under the specified conditions. For the XGBoost Regressor model, we
RMSE is another commonly used metric that measures the standard
noticed that the hyperparemeters of maximum depth of 5, the learning
deviation of the prediction errors. Eq. 3 demonstrates the formula used
rate of 0.3 and the number of estimators of 100 were the best options.
for calculating the RMSE measure (Hyndman & Koehler, 2006; Chai &
Likewise, for training the Random Forest Regressor model, various al­
Draxler, 2014). Similarly, a smaller RMSE value is preferred as it in­
ternatives were tested to optimize the number of tree estimators. The
dicates that the model prediction is more accurate.
model parameter with 51 tree estimators was found to be performing
As a result of evaluations and comparisons, the best results were
well.
obtained using Random Forest with a 0.982 R2 value in the testing
Similarly, based on the hyperparameter optimizations in the MLP
dataset. Although the XGBoost method had a very close R2 value with
Regressor model training, it was observed that the MLP Regressor model
0.979, it lagged slightly behind Random Forest. On the other hand,
with relu activation function, 50 hidden layer nodes, epsilon of 1e-08
Neural Network model MLP Regressor with had performed worse when
and adam optimizer appeared to fit well to the dataset.
compared to the other three ensemble learning methods and shown to be
When fitting the Support Vector Regression (SVR) model to the
insufficient for this prediction process. The worst performance was
dataset, the most important SVR parameter is kernel type. Since the
observed when using Adaboost and SVR algorithms with 0.338 and
dataset was not linearly separable, the linear SVR performed poorly.
Upon explorations, we observed that SVR with Radial Basis Function 0.347 R2 values, respectively.
(RBF) kernel appeared to fit relatively well in our case. Fig. 5 illustrates a comparison of algorithm results in terms of pre­
For Gradient Boosting model, the hyperparemeters of maximum diction performance based on the test dataset. An excerpt of records
depth of 2, the learning rate of 0.05, and the number of estimators of from the evaluations were demonstrated in the figure only for visuali­
1000 performed the best. On the other hand, Adaboost model was un­ zation purposes. As shown in the figure, Random Forest and XGBoost
able to fit the data adequately. Among various tuning options, the algorithms clearly achieved better prediction results than the Gradient
hyperparemeters of maximum depth of 2, the learning rate of 0.05, and
the number of estimators of 1000 resulted in its best performance.
In the study, Python programming language, Keras and Sci-kit learn Table 1
Evaluation results for accuracy of ML algorithms on test dataset.
machine learning libraries were used for the model development and
evaluations. During optimization of methods, learning speed and mo­ Algorithm R2 MAE MAPE RSME
mentum values were kept as default values. The machine learning
Random forest 0.982 51.97 3.27 147.19
models were deployed into production using Flask web-development
XGBoost 0.979 82.09 5.16 157.28
framework. Flask API exposes the deployed ML models over REST web
Gradient boosting 0.776 394.52 24.79 523.91
services interface with the intention that the models can be accessed
MLP regressor 0.675 466.32 29.30 632.69
from the IoT platform securely via HTTP protocol in real-time.
SVR 0.347 682.43 42.88 896.07
AdaBoost 0.338 752.95 47.32 902.85

7
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

Fig. 5. Comparison of algorithms results: actual vs predicted.

Boosting, AdaBoost, MLP and SVR models. high-dimensional fast data in real-time. This is important as many ap­
proaches in the literature are developed based on syntactic data and are
6. Discussion not suitable for practical use in factory settings. Many approaches fail
due to unforeseen challenges associated with the manufacturing
In manufacturing, traditional maintenance strategies of production environments.
lines typically involve replacement of equipment occurring after a fail­ Additionally, the proposed predictive maintenance system has
ure or at predetermined time periods. These maintenance plans often contributed a secondary benefit by facilitating the digital transformation
lead to extra costs due to early replacement of equipment. The case of in the production lines. By implementing IoT sensors on the equipment,
late replacement causes unplanned stops in the production lines that the production process has become digitized such that the collected data
result in heavy loss of capacity and products. Although some firms plan can be used in data analytics for improving the performance of pro­
maintenance based on lifetime of individual equipment recommended duction lines in the future.
by the vendors, there still exists a high risk of late equipment mainte­ For testing the effectiveness and the predictive power of the system,
nance situation due to the fact that this maintenance strategy does not we evaluated multiple machine learning algorithms and investigated
take into account the entire system. various configurations. The results indicated that the models based on
As opposed to traditional methods, the predictive maintenance sys­ three ensemble learning algortims, namely Random Forest, XGBoost and
tem developed in the study considers entire production line based on Gradient Boosting, outperformed individual MLP Regressor and SVR
data collected from IoT sensors in real-time. The machine learning algorithms alone. Our findings were in line with the related studies as
models evaluate changing conditions and estimate the best time for stated in literature that ensemble learning techniques are known to
maintenance of the equipment. Consequently, many potential un­ perform better in general in terms of prediction power than other indi­
planned line stops, discarded products and capacity losses can be vidual machine learning algorithms (Polikar, 2006; Rokach, 2010). The
prevented. AdaBoost algorithm was an exceptional case as it has performed poorly
While there are many approaches to predictive maintenance in in fitting the data. We investigated various hyperparameter tuning op­
literature, it is an active research area and there is no one fits all solution tions. Yet, the models were unable to achieve better results than con­
available. In that context, we think that our approach contributes to the stituent counterparts.
literature in the following ways. Firstly, we provide an end-to-end ML A potential limitation of the current study is that the dataset was
based predictive maintenance approach for manufacturing that in­ collected from one factory only. Thus, the machine learning models
tegrates all components in a real-life factory environment from IoT were based on the production lines from the same factory and may not
sensors to development of ML models and creating alerts. Secondly, to be generalized to all cases in manufacturing. Nevertheless, the equip­
the best of our knowledge, this is the first AI based predictive mainte­ ment is standardized and commonly available in the industry. The
nance system implemented in a real factory in this specific sector. proposed approach is scalable and generalizable. With minor configu­
Moreover, we have shown that the system is effective and scalable to rations in the system, the approach can be applied to similar production

8
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

environments. That being said, the ML models cannot be applied directly Chai, T., & Draxler, R. R. (2014). Root mean square error (rmse) or mean absolute error
(mae)?–arguments against avoiding rmse in the literature. Geoscientific Model
to new production lines as is because the models must be retrained with
Development, 7(3), 1247–1250.
new dataset when different IoT devices are used or different failure types Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). Smote: Synthetic
are targeted. minority over-sampling technique. Journal of Artificial Intelligence Research, 16,
On the other hand, we think that similar approach or intelligent 321–357.
Chen, C., Liu, Y., Wang, S., Sun, X., Di Cairano-Gilfedder, C., Titmus, S., & Syntetos, A. A.
function of the proposed approach can also be applied to product quality (2020). Predictive maintenance using cox proportional hazard deep learning.
prediction with integration of additional sensors and image processing Advanced Engineering Informatics, 44, Article 101054.
functionality for monitoring the final products as the proposed frame­ Draper, N. R., & Smith, H. (1998). Applied regression analysis (Vol. 326). John Wiley &
Sons.
work lay the foundation for model extensions. Edward, K., Dariusz, M., Żabiński, T., Prucnal, S., Se, J. & et al. (2020). Machining sensor
Due to rare occurrences some failure categories in the dataset, we data management for operation-level predictive model. Expert Systems with
only addressed the most frequent failure types. However, we plan to Applications, p. 113600.
Erfani, S. M., Rajasegarar, S., Karunasekera, S., & Leckie, C. (2016). High-dimensional
develop ML models to consider rare failure types as well in the future. and large-scale anomaly detection using a linear one-class svm with deep learning.
Pattern Recognition, 58, 121–134.
Fernandes, M., Canito, A., Corchado, J. M., & Marreiros, G. (2019). Fault detection
7. Conclusion
mechanism of a predictive maintenance system based on autoregressive integrated
moving average models. In International symposium on distributed computing and
In this study, we developed a machine learning based predictive artificial intelligence (pp. 171–180). Springer.
maintenance system for manufacturing environments. We evaluated the Francis, F., & Mohan, M. (2019). Arima model based real time trend analysis for
predictive maintenance. In 2019 3rd International conference on electronics,
effectiveness of the system using real-world manufacturing system IoT communication and aerospace technology (ICECA) (pp. 735–739). IEEE.
data. The output of the model represents the estimated useful time Froger, A., Gendreau, M., Mendoza, J. E., Pinson, E., & Rousseau, L.-M. (2016).
remaining before failure. The evaluations results show that our proposed Maintenance scheduling in the electricity industry: A literature review. European
Journal of Operational Research, 251(3), 695–706.
predictive maintenance system is effective in capturing the signals of Furman, J., & Seamans, R. (2019). Ai and the economy. Innovation Policy and the
machinery failure using real-time sensor data and it can help prevent Economy, 19(1), 161–191.
potential production stops by taking preventive actions suggested by the Gensler, A., Henze, J., Sick, B. & Raabe, N. (2016). Deep learning for solar power
forecasting–an approach using autoencoder and lstm neural networks. In 2016 IEEE
system. Our findings indicate that boosting and bagging ensemble international conference on systems, man, and cybernetics (SMC) (pp.
models perform well in general. For the future work, we plan to apply 002858–002865). IEEE.
the system to other types of production lines in different settings. He, B., & Bai, K.-J. (2020). Digital twin-based sustainable intelligent manufacturing: A
review. Advances in Manufacturing, 1–21.
He, Y., Gu, C., Chen, Z., & Han, X. (2017). Integrated predictive maintenance strategy for
CRediT authorship contribution statement manufacturing systems by combining quality control and mission reliability analysis.
International Journal of Production Research, 55(19), 5841–5862.
Ho, S.-L., Xie, M., & Goh, T. N. (2002). A comparative study of neural network and box-
Serkan Ayvaz: Conceptualization, Data curation, Methodology, jenkins arima modeling in time series prediction. Computers & Industrial Engineering,
Software, Writing - original draft, Writing - review & editing, Visuali­ 42(2–4), 371–375.
zation, Funding acquisition, Formal analysis, Investigation, Resources, Huang, J., Chang, Q., & Arinez, J. (2020). Deep reinforcement learning based preventive
maintenance policy for serial production lines. Expert Systems with Applications, 160,
Validation. Koray Alpay: Conceptualization, Data curation, Project Article 113701.
administration, Writing - review & editing, Resources, Funding acqui­ Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy.
sition, Validation. International Journal of Forecasting, 22(4), 679–688.
Jasiulewicz-Kaczmarek, M., Legutko, S., & Kluk, P. (2020). Maintenance 4.0
technologies–new opportunities for sustainability driven maintenance. Management
and Production Engineering Review, 11.
Declaration of Competing Interest Jena, M. C., Mishra, S. K., & Moharana, H. S. (2020). Application of industry 4.0 to
enhance sustainable manufacturing. Environmental Progress & Sustainable Energy, 39,
The authors declare that they have no known competing financial 13360.
Jeschke, S., Brecher, C., Meisen, T., Özdemir, D., & Eschert, T. (2017). Industrial internet
interests or personal relationships that could have appeared to influence of things and cyber manufacturing systems. In Industrial internet of things (pp. 3–19).
the work reported in this paper. Springer.
Kanawaday, A., & Sane, A. (2017). Machine learning for predictive maintenance of
industrial machines using iot sensor data. In 2017 8th IEEE international conference on
Acknowledgments software engineering and service science (ICSESS) (pp. 87–90). IEEE.
Lee, J., Davari, H., Singh, J., & Pandhare, V. (2018). Industrial artificial intelligence for
industry 4.0-based manufacturing systems. Manufacturing Letters, 18, 20–23.
This work was supported by the Scientific and Technological Lee, J., Kao, H.-A., Yang, S., et al. (2014). Service innovation and smart analytics for
Research Council of Turkey (TÜBÍTAK) Grant in TEYDEB-1501 TUBI­ industry 4.0 and big data environment. Procedia Cirp, 16(1), 3–8.
TAK Industrial R&D Projects under Grant No: 3181393 and the Procter Lee, W. J., Wu, H., Yun, H., Kim, H., Jun, M. B., & Sutherland, J. W. (2019). Predictive
maintenance of machine tool systems using artificial intelligence techniques applied
& Gamble Turkey. Special thanks to Mesut Saban and Ammar Omar
to machine condition data. Procedia CIRP, 80, 506–511.
from the Procter & Gamble Turkey, and Orhan Dengiz from DVM Li, Z., Wang, Y., & Wang, K.-S. (2017). Intelligent predictive maintenance for fault
Technology for their support and constructive feedback throughout the diagnosis and prognosis in machine centers: Industry 4.0 scenario. Advances in
project. Manufacturing, 5(4), 377–387.
Locke, D. (2010). Mq telemetry transport (mqtt) v3. 1 protocol specification. IBM
DeveloperWorks Technical Library, 15.
References Lu, Y. (2017). Industry 4.0: A survey on technologies, applications and open research
issues. Journal of Industrial Information Integration, 6, 1–10.
Lu, Y., Liu, C., Kevin, I., Wang, K., Huang, H., & Xu, X. (2020). Digital twin-driven smart
Al-Kandari, N. M., & Jolliffe, I. T. (2005). Variable selection and interpretation in
manufacturing: Connotation, reference model, applications and research issues.
correlation principal components. Environmetrics: The Official Journal of the
Robotics and Computer-Integrated Manufacturing, 61, Article 101837.
International Environmetrics Society, 16(6), 659–672.
Machado, C. G., Winroth, M. P., & Ribeiro da Silva, E. H. D. (2020). Sustainable
Altman, N., & Krzywinski, M. (2018). The curse (s) of dimensionality. Nature Methods, 15
manufacturing in industry 4.0: An emerging research agenda. International Journal of
(6), 399–400.
Production Research, 58(5), 1462–1484.
Ansari, F., Erol, S., & Sihn, W. (2018). Rethinking human-machine learning in industry
Menze, B. H., Kelm, B. M., Masuch, R., Himmelreich, U., Bachert, P., Petrich, W., &
4.0: How does the paradigm shift treat the role of human learning? Procedia
Hamprecht, F. A. (2009). A comparison of random forest and its gini importance
Manufacturing, 23, 117–122.
with standard chemometric methods for the feature selection and classification of
Bellman, R. (1966). Dynamic programming. Science, 153(3731), 34–37.
spectral data. BMC Bioinformatics, 10(1), 213.
Bolton, C., Machová, V., Kovacova, M., & Valaskova, K. (2018). The power of
Mocanu, E., Nguyen, P. H., Gibescu, M., & Kling, W. L. (2016). Deep learning for
human–machine collaboration: Artificial intelligence, business automation, and the
estimating building energy consumption. Sustainable Energy, Grids and Networks, 6,
smart economy. Economics, Management, and Financial Markets, 13(4), 51–56.
91–99.
Canziani, A., Paszke, A. & Culurciello, E. (2016). An analysis of deep neural network
models for practical applications. arXiv preprint arXiv:1605.07678.

9
S. Ayvaz and K. Alpay Expert Systems With Applications 173 (2021) 114598

Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018). Deep learning for iot Svetnik, V., Liaw, A., Tong, C., Culberson, J. C., Sheridan, R. P., & Feuston, B. P. (2003).
big data and streaming analytics: A survey. IEEE Communications Surveys & Tutorials, Random forest: A classification and regression tool for compound classification and
20(4), 2923–2960. qsar modeling. Journal of Chemical Information and Computer Sciences, 43(6),
Polikar, R. (2006). Ensemble based systems in decision making. IEEE Circuits and Systems 1947–1958.
magazine, 6(3), 21–45. Traini, E., Bruno, G., D’Antonio, G., & Lombardi, F. (2019). Machine learning framework
Powell, W. B. (2007). Approximate dynamic programming: Solving the curses of for predictive maintenance in milling. IFAC-PapersOnLine, 52(13), 177–182.
dimensionality (Vol. 703). John Wiley & Sons. Wall, M. E., Rechtsteiner, A., & Rocha, L. M. (2003). Singular value decomposition and
Rabatel, J., Bringay, S., & Poncelet, P. (2011). Anomaly detection in monitoring sensor principal component analysis. In A practical approach to microarray data analysis (pp.
data for preventive maintenance. Expert Systems with Applications, 38(6), 7003–7015. 91–109). Springer.
Ramos, P., Oliveira, J. M. S., & Silva, P. (2014). Predictive maintenance of production Wamba-Taguimdje, S.-L., Wamba, S. F., Kamdjoug, J. R. K., & Wanko, C. E. T. (2020).
equipment based on neural network autoregression and arima. In 21st International Influence of artificial intelligence (ai) on firm performance: The business value of ai-
EurOMA conference-operations management in an innovation economy. based transformation projects. Business Process Management Journal.
Rieger, T., Regier, S., Stengel, I., & Clarke, N. L. (2019). Fast predictive maintenance in Wang, J., Zhang, L., Duan, L., & Gao, R. X. (2017). A new paradigm of cloud-based
industrial internet of things (iiot) with deep learning (dl): A review. CERC, 69–80. predictive maintenance for intelligent manufacturing. Journal of Intelligent
Rivera, D. L., Scholz, M. R., Bühl, C., Krauss, M., & Schilling, K. (2019). Is big data about Manufacturing, 28(5), 1125–1137.
to retire expert knowledge? A predictive maintenance study. IFAC-PapersOnLine, 52 Wang, S., Wan, J., Zhang, D., Li, D., & Zhang, C. (2016). Towards smart factory for
(24), 1–6. industry 4.0: A self-organized multi-agent system with big data based feedback and
Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review, 33(1–2), coordination. Computer Networks, 101, 158–168.
1–39. Willmott, C. J., & Matsuura, K. (2005). Advantages of the mean absolute error (mae) over
Shaw, P. J. (2009). Multivariate statistics for the environmental sciences. Wiley. the root mean square error (rmse) in assessing average model performance. Climate
Song, X., Kanasugi, H., & Shibasaki, R. (2016). Deeptransport: Prediction and simulation Research, 30(1), 79–82.
of human mobility and transportation mode at a citywide level. IJCAI, 16, Xie, X., Wu, D., Liu, S. & Li, R. (2017). Iot data analytics using deep learning. arXiv
2618–2624. preprint arXiv:1708.03854.
Song, Z., & Moon, Y. (2017). Assessing sustainability benefits of cybermanufacturing Yu, W., Dillon, T., Mostafa, F., Rahayu, W., & Liu, Y. (2019). A global manufacturing big
systems. The International Journal of Advanced Manufacturing Technology, 90(5–8), data ecosystem for fault detection in predictive maintenance. IEEE Transactions on
1365–1382. Industrial Informatics, 16(1), 183–192.
Song, Z., & Moon, Y. (2018). Cybermanufacturing system: A solution for sustainable Zhang, J., Wang, P., Yan, R., & Gao, R. X. (2018). Deep learning for improved system
manufacturing. In ASME international mechanical engineering congress and exposition remaining life prediction. Procedia CIRP, 72, 1033–1038.
(volume 52019)American Society of Mechanical Engineers. page V002T02A068. Zhang, J., Wang, P., Yan, R., & Gao, R. X. (2018). Long short-term memory for machine
Stanford-Clark, A. & Hunkeler, U. (1999). Mq telemetry transport (mqtt). Online]. remaining life prediction. Journal of Manufacturing Systems, 48, 78–86.
http://mqtt. org. Accessed September, 22:2013. Zhao, G., Liu, X., Zhang, B., Zhang, G., Niu, G., & Hu, C. (2017). Bearing health condition
Stock, T., & Seliger, G. (2016). Opportunities of sustainable manufacturing in industry prediction using deep belief network. In Proceedings of the annual conference of
4.0. Procedia Cirp, 40, 536–541. prognostics and health management society, Orlando, FL, USA (pp. 2–5).
Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., & Zeileis, A. (2008). Conditional
variable importance for random forests. BMC Bioinformatics, 9(1), 307.

10