See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/330452351

Machine Learning Approaches to Condition Monitoring

Presentation · January 2019

CITATIONS READS

0 3,885

1 author:

Karl Ezra Salgado Pilario

University of the Philippines
27 PUBLICATIONS   285 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Fault Diagnosis and Prognosis using Kernel Methods in Nonlinear Dynamic Processes View project

Statistical Machine Learning Applications in Chemical Process Operations, Design, and Control View project

All content following this page was uploaded by Karl Ezra Salgado Pilario on 25 July 2019.

The user has requested enhancement of the downloaded file.

Machine Learning Approaches
to Condition Monitoring

Karl Ezra Pilario

PhD Student
Department of Energy and Power

January 2019

www.cranfield.ac.uk
 Outline

• What is Machine Learning?

• What are the types of ML problems?
• Why is ML taking off only now?
• How can ML be applied to Condition Monitoring?
• What are the various ML approaches?
• What is the main challenge in applying ML?

2
 What is Machine Learning?
A field of study concerned with giving computers the
ability to learn without being explicitly programmed.
(Arthur Samuel, 1959)

Arthur Samuel and the IBM 701 Computer

Images from:
https://www.regencychess.co.uk/checkers-game-p-1626.html
https://www.ibm.com/ibm/history/ibm100/us/en/icons/ibm700series/impacts/
3
 What is Machine Learning?

IBM Watson
Jeopardy, 2011

AlexNet
ImageNet Large Scale Visual Recognition
Challenge 2012
Google DeepMind’s Alpha Go
March 2016

Images:
http://www.slate.com/blogs/future_tense/2016/03/09/google_deepmind_s_alphago_ai_beats_champion_lee_sedol_in_go.html
http://time.com/4281476/ibm-artificial-intelligence-watson-2016/
https://www.cc.gatech.edu/~hays/compvision/proj6/
4
 Types of Machine Learning Problems

Supervised Learning Goal: Make a computer Unsupervised Learning Goal: Make a computer
learn from examples learn patterns from data
and non-examples. by itself.

These are dogs. These are cars. Here are some pictures.

Is there a picture that

Now, what is this? does not belong?

5
 Types of Machine Learning Problems

Semi-Supervised Learning Reinforcement Learning

Goal: Make a computer learn from Goal: Make a computer learn by letting
both labelled and unlabelled data. it interact with the environment.

Labelled
Data

Unlabelled
Data

6
 Other applications of Machine Learning

• Filtering emails as spam / not spam

• Handwritten digits recognition
• Speech recognition or Natural Language Processing (NLP)
• Social media (Face recognition, News Feed Ranking, etc.)
• Recommender systems (movies, products, videos, webpages, bookings, etc.)
• Finance (Stock market prediction, customer behaviour, etc.)
• Transportation (Self-driving cars, travel demand modelling, etc.)
• Healthcare (Early diagnostics, hospital demand forecasting, etc.)
• Etc.

7
 Why is ML taking off only now?

BIG DATA
Nowadays… (Industry 4.0)

• Massive amounts of • Data are created at

data are being
generated and stored.
Volume Velocity an accelerating rate,
yet, analysis must still
be timely.

• More and more • Amidst these, some data

various sources and
types of data are
Variety Veracity may still be inaccurate,
uninformative, and laden
becoming available. with uncertainty.

Source: https://www.ibmbigdatahub.com/infographic/four-vs-big-data
8
 Why is ML taking off only now?

Tweets, Likes Traffic data GIS /

So, where are all the data? and Comments Satellite data
Uploaded Weather data
Web clicks
online content Flight data
(video, audio, image, etc.)
Legal case
reports
Search BIG DATA
queries
Government
statistics
Research Sales Consumer data
articles transactions
House bills
DNA & Protein Plant sensors
structures & SCADA
Sports Utilities / Energy
statistics consumption data
Medical records Student
exam scores
Chemical data sheets

9
 Why is ML taking off only now?

So, where are all the data?

http://archive.ics.uci.edu/ml/datasets.html https://www.kaggle.com/competitions

There are also popular

websites that open
datasets and ML
challenges to the public:
(some with prize money)

10
 Outline

• What is Machine Learning?

• What are the types of ML problems?
• Why is ML taking off only now?
• How can ML be applied to Condition Monitoring?
• What are the various ML approaches?
• What is the main challenge in applying ML?

11
General Condition Monitoring Methodology

Example:

Good Process Data Train a model

Working Simulate the Process at Normal (Supervised /
Condition Condition Unsupervised)

Process Data +
Fault
Cases
Simulate the Process at Faulty
Condition
-
1. Air Line Blockage
2. Water Line Blockage
3. Top Separator Blockage
4. Open Direct Bypass
5. Slugging Conditions in Riser
Fault Detection,
6. And so on… Diagnosis, and
Prognosis
12
 General Condition Experience of
Plant Operators
Physics & Chemistry
Monitoring Principles
Methodology Knowledge-
System of Experts
Differential
Equations
Process using based Process
Modelling Simulation Historical
Software Fault Data
Physics- Rules of Thumb

based Expert Expert

Systems Knowledge
Train a model Grey-box
Mass & Energy models
and/or A.I.
Balances
Machine
Learning
Black-box
models
Data-based

Multivariate
Statistics
Ge et al. (2013) Review of Recent Research on Data-based Process Monitoring. Industrial and Engineering Chemistry Research, 52.
13
Chiang et al. (2001). Fault Detection and Diagnosis in Industrial Systems. Springer-Verlag London.
 Machine Learning for Condition Monitoring

Consider a machine from PV 1

PV 2
where sensor data is available MACHINE
for 2 process variables.

Supervised Learning Unsupervised Learning

Process Variable 2

Process Variable 2
? Faulty ?
Data

Classification
Normal Clustering
Data

Process Variable 1 Process Variable 1

14
 Machine Learning for Condition Monitoring

Consider a machine from PV 1

PV 2
where sensor data is available MACHINE
for 2 process variables.

Supervised Learning Unsupervised Learning

? Dimensionality
Regression Reduction

Process Variable 2
Fault magnitude

?
Faulty Data
Density
Estimation
Normal Data

Time Process Variable 1

15
 Machine Learning Algorithms

Supervised Learning Unsupervised Learning

Manifold Learning
Decision Trees Random Principal Autoencoders
Forests Components Canonical Variate
Logistic Analysis Analysis
Regression Support Vector Dimensionality
Classification Machines
Independent Reduction
Linear
Discriminant
Components
Naïve Bayes Analysis
Analysis
Partial Least Self-organizing
Artificial Neural Squares
maps
Support Vector
Networks
Bayesian Regression Expectation
Boosting Networks Hierarchical
Density Clustering Maximization
K-Nearest
Ensemble Regression Neighbors Estimation
Methods
Gaussian
Process
Clustering
Regression Ordinary Least- Kernel density
squares estimation K-means Gaussian
Bagging
Ridge Regression Clustering Mixture Model
16
 Artificial Neural Networks (ANN)
for classification
Model: 𝑦 = 𝜎(𝑓 𝒙 )
Given: Decision
PV 1 PV 2 𝒚 Boundary where:
1.25 0.85 Faulty
PV 1
𝒙=

Process Variable 2
1.33 2.14 Normal Input data
PV 2
1.01 2.01 Normal
… … …
𝑓 𝒙 = the neural network itself
− output for faulty
In ANN, the aim is to find a = (+)
output for normal
decision boundary so that
the classification error is a
minimum. Process Variable 1 𝜎 ⋅ = 0 to 1, probability that
𝒙 is normal.
Faulty Data
Normal Data *The decision boundary consists of
point locations where 𝜎 ⋅ = 0.5.

17
 Artificial Neural Networks (ANN)
for classification
Model: 𝑦 = 𝜎(𝑓 𝒙 )

where:

𝑥1 PV 1
𝒙= Input data
𝑓 𝒙 PV 2

𝑓 𝒙 = the neural network itself

𝑥2 − output for faulty
= (+)
output for normal

Input layer Hidden layers Output layer 𝜎 ⋅ = 0 to 1, probability that

𝒙 is normal.
𝑥1′
Each neuron is a function,
𝑥2′ 𝑔 𝒙′ *The decision boundary consists of
𝑔 𝒙′ = 𝜎(𝒘𝑇 𝒙′ + 𝑏) 𝑥3′ point locations where 𝜎 ⋅ = 0.5.

18
 Support Vector Machines (SVM)
for classification
Model: 𝑦 = 𝜎(𝒘𝑇 𝒙 + 𝑏)
Given: Support
Margin Vector
PV 1 PV 2 𝒚 where:
1.25 0.85 Faulty
PV 1
𝒙=

Process Variable 2
1.33 2.14 Normal Input data
1.01 2.01 Normal
PV 2

𝑦= −
… … … output for faulty
(+) output for normal
In SVM, the aim is to find a
decision boundary that 𝜎 ⋅ = 0 to 1, probability that
maximizes the margin of 𝒙 is normal.
separation between the Process Variable 1
two classes of data. *The decision boundary consists of
Decision point locations where 𝜎 ⋅ = 0.5.
Faulty Data
Boundary
Normal Data *Nonlinear boundaries are
achievable using kernels.
19
 Solution by Optimization

NOTICE: In ANN, the aim is to find a In SVM, the aim is to find a decision
decision boundary so that the boundary that maximizes the margin of
classification error is a minimum. separation between the two classes of data.

Optimization Problem: Error = 1.2

0.8
In ML, training means finding an optimum solution. 0.5 Initial
0.3
guess
Genetic 0.2
Backpropagation Algorithm
Parameter 1
Sequential
Quadratic
Training Particle Swarm
Programming Algorithms Optimization

Differential
Stochastic Evolution
Gradient Descent
Parameter 2
20
 Outline

• What is Machine Learning?

• What are the types of ML problems?
• Why is ML taking off only now?
• How can ML be applied to Condition Monitoring?
• What are the various ML approaches?
• What is the main challenge in applying ML?

21
 Nature of Plant Data

1. Plant Data is Noisy

Gaussian
Non-Gaussian

Multivariate
2. Plant Data is High-dimensional
(~100-1000 variables)

Autocorrelation due to Plant Dynamics

3. Plant Data are Correlated

Cross-correlation

4. Plant Data exhibits Nonlinear Behavior

And others… Time-varying /
5. Have missing data samples
6. Have prevalent outliers Non-stationary
7. Consist of process data collected at inconsistent sampling rates
8. Come either by batch or continuous modes
9. Be sourced heterogeneously (alarm history, image, video, SCADA) 22
 Challenge for unsupervised learning

Throughout the years (since the 90s), various

techniques for unsupervised learning were
developed to address the challenges…

1920s-80s 1990s 2000s 2010s Present

Noisy & High- Non-Gaussian Dynamic Nonlinear Time-varying

Univariate Dimensional
• Independent • Canonical • Kernel PCA • Recursive
• (Shewhart) • Principal Components Variate • Kernel PLS PCA
Control charts Components Analysis (ICA) Analysis • Kernel CVA • Recursive
• S-charts Analysis (PCA) • Gaussian (CVA)
• Nonlinear PLS
• CUSUM • Partial-Least Mixture • Dynamic PCA Embedding • Recursive
charts Squares (PLS) Models • Dynamic PLS • Bayesian CVA
• EWMA charts • Linear • Kernel Density • Dynamic ICA methods
Discriminant Estimation
Analysis (LDA)

23 Ge et al. (2013) Review of Recent Research on Data-based Process Monitoring. Industrial and Engineering Chemistry Research, 52.
 Challenge for supervised learning

Underfitting Good fit Overfitting

• Performance is too low on the • The pattern was learned • Performance is too good to be true
training data set • Bias and variance are balanced on the training data set
• Patterns were not learned at all • Model is not too complex nor simple • Patterns “learned” from noise
• The model is too simple • The model may generalize well to • The model is too complex
(high bias, low variance) unseen test data (low bias, high variance)
• The model cannot generalize to • The model cannot generalize to
unseen test data unseen test data
24
 Challenge for supervised learning

Underfitting Good fit Overfitting

• Performance is too low on the • The pattern was learned • Performance is too good to be true
training data set • Bias and variance are balanced on the training data set
• Patterns were not learned at all • Model is not too complex nor simple • Patterns “learned” from noise
• The model is too simple • The model may generalize well to • The model is too complex
(high bias, low variance) unseen test data (low bias, high variance)
• The model cannot generalize to • The model cannot generalize to
unseen test data unseen test data
25
 Challenge for supervised learning

Main Challenge: Generalization

Underfitting Good fit Overfitting
In any ML project, you must have both
Training data and Test data, to check
how well the model can generalize. Performance on
Test Data

Classification
Error

Even so, there is no guarantee for Performance on

Training Data
good generalization.

Model complexity

26
 Further reading / studying

• Christopher Bishop (2006). Pattern Recognition and Machine Learning. Springer.

• Hastie et al. (2008). The Elements of Statistical Learning. Springer.
• Coursera: Machine Learning by Andrew Ng.
• deeplearning.ai (Youtube channel)
• Cristianini & Shawe-Taylor (2000). Support Vector Machines and other kernel-based learning
methods. Cambridge University Press.
• Chiang et al. (2005). Fault Detection and Diagnosis in Industrial Systems. Springer

27
View publication stats