Preprints of the 7th IFAC Symposium on

Fault Detection, Supervision and Safety of Technical Processes

Barcelona, Spain, June 30 - July 3, 2009

Distributed Fault Detection and Isolation Algorithm Using Modified Gath-Geva

Clustering Technique
S.Bahrampour*, K. Salahshour**, B. Moshiri***

* Control and Intelligence Processing Center of Excellence, School of ECE, University of Tehran, Iran
(phone: +98 912 246 23 27; e-mail: [email protected])..
** Petroleum University of Technology, Iran (e-mail: [email protected]).
*** Control and Intelligence Processing Center of Excellence, School of ECE,
University of Tehran, Iran (e-mail: [email protected]).
Abstract: This paper presents a set of new fault detection and isolation (FDI) approaches based on a
Modified Gath-Geva (MGG) Clustering approach. The proposed approaches are formulated in the forms
of a combined principal component analysis (PCA)-MGG or nonlinear PCA (NPCA)-MGG schemes.
The PCA and NPCA perform as a data preprocessing step on the measured industrial multivariate time
series data to enhance their informative richness, capsulated in a more compact form. Then, the MGG
clustering approach is used to detect and isolate the faults by organizing the PCA/NPCA-transformed
data in different clusters. A distributed-MGG scheme is also proposed in this paper as a new FDI
approach which is based on a distributed monitoring configuration. The main idea is to divide the overall
monitoring task into a series of local distributed monitoring sub-tasks so as to track and capture the
process faults locally. The diagnostic performances of the proposed FDI approaches are evaluated by a
set of comparative test studies on the Tennessee Eastman process plant as a large-scale benchmark
problem.

determine the required number of segments, while the

1. INTRODUCTION
required numbers of principal components are determined by
Large amounts of data are collected in many industrial the screeplots of the eigenvalues of the fuzzy covariance
processes for monitoring and control purposes. The most matrices.
important parts of condition monitoring are fault detection
The major contribution of this paper is to modify the original
and isolation (FDI). Fault detection is used to determine
method in (Abonyi et al., 2005) so that it can be applied more
when abnormal process behavior has occurred whereas in
conveniently for fault detection and isolation purposes in
fault isolation (FI), one distinguishes between faults.
large-scale process plants. Practical observations on this
Data driven methods have attracted much attentions in the method indicate that its straight application to industrial
recent research works. These methods can be divided into problems with huge raw data sets is non-efficient. Although
statistical or non-statistical ones. Principal component this algorithm is a powerful technique for time series
analysis (PCA), fisher discriminate analysis (FDA), partial segmentation, it fails in faults isolations because of the
least squares (PLS) and statistical pattern classifiers form a dimensionality curse of the large data sets. Our experimental
major classes of statistical feature extraction methods observation indicates that employing different data reduction
(Chiang, et al., 2000). Neural network and support vector methods such as PCA and nonlinear PCA can help the
machines are the most powerful methods of non-statistical original algorithm to isolate faults. In a typical industrial
methods. One limiting disadvantage of all these methods, process, hundreds of process variables are stored. However, a
however, is their needs to a priori knowledge of the complete few variables are usually responsible for a process fault and
fault data set including training and testing data. Furthermore, also different variables are responsible for different faults. A
they have difficulty to recognize novel faults. variable selection scheme is presented to select key variables
for use in clustering for better identification of the faults in
To tackle these problems, a clustering-based approach large scale processes. This can effectively enhance the
reported in (Abonyi et al., 2005) has been adopted in this original time-series segmentation method, yielding into
research work. Fault isolation can be view as a time-series proper FDI technique.
segmentation algorithm while values of the process variables
in different time samples provide the time-series. This Tennessee Eastman (TE) process is tested to illustrate the
approach uses the simultaneous identification of local effectiveness of the proposed FDI approach. TE process was
probabilistic principal component analysis (PPCA) models to created by the Eastman Chemical to provide a realistic
measure the homogeneity in time and is able to detect industrial process for evaluating process control and
changes in the hidden structure of multivariate time-series. In monitoring methods (Downs et al., 2002). The simulation
this method, a fuzzy decision making algorithm based on results demonstrate the capability of the proposed approach
compatibility criteria of the clusters has been worked out to to detect and isolate the TE faults.

858
This paper is organized as follows. In the next section, the where µ i,k represents the degree of membership of the
proposed FDI approaches are introduced. Section III observation, Z k is the ith cluster (i=1,…,c) and m ∈ [1, ∞ ) is
demonstrates the experimental simulation test study to
provide a comparative performance evaluation of the a weighting exponent that determines the fuzziness of the
proposed FDI approaches on the TE process plant as a large- resulting clusters, usually chosen as m=2. The data are
scale benchmark problem. Finally, some concluding remarks assumed to be normally distributed random variables with
are summarized in the last section. expected value Vi and covariance matrix Fi . The GG
clustering algorithm is equivalent to the identification of a
2. DEVELOPMENT OF THE PROPOSED FDI mixture of Gaussians that represents the P (Z k η i )
APPROACHES probability density function expanded in a sum over the c
clusters,
In this section, first a powerful method for time-series
segmentation, introduced by Abonyi et al., 2005, is briefly
c
reviewed and then different modifications of the algorithm
are proposed for an enhanced FDI performance. P (Z k η ) = ∑ P(Z k η i )P(η i ), (2)
i =1

2.1 MGG time-series segmentation review

where p(ηi ) is the unconditional cluster probability and
A time-series T = { X k 1 ≤ k ≤ N } is a finite set of N samples
p(Z η i ) distribution generated by the ith cluster is
labeled by time points t1 ,..., t N , where
T
represented by the Gaussian function. Since the time variable
X k = [ x1, k , x 2,k ,..., x n,k ] . A segment of T is a set of is independent from the X K variables, the distribution can be
consecutive time points S (a, b) = {a ≤ k ≤ b} , generated based on the following D (Z k , η i ) distance
2

X a , X a +1 ,..., X b . The c-segmentation of T is a partition of T measure:

to c non-overlapping segments S Tc = {S i (a i , bi 1 ≤ i ≤ c)} ,
such that a1 = 1 , bc = N , and a i = bi −1 + 1 . Time-series
p (z k η i ) =
1
= α
1 ⎛ 1 t − vt 2 ⎞
exp⎜⎜ −
k i ⎟ ( )
segmentation may be considered as clustering with a time- D 2 ( z k ,η i )
{i 2
p (η i ) 2πσ i , t ⎜ 2 σ i2,t ⎟⎟
ordered structure. The clustering is based on a distance 144444⎝244444 3⎠ (3)
p (t k η i )
measure which consists of two terms: the first distance term
is based on how the data are in the given segment defined by × r
1 ⎛ 1
⎝ 2
T
( ⎞
exp⎜ − X k − Vi x Ai−1 X k − Vi x ⎟
⎠
) ( )
the Gaussian fuzzy sets defined in the time domain, while the (2π ) det( Ai )4444
144444
2
424444444444 3
p (X k η i )
second term measures how far the data are from the
hyperplane of the PCA model of the segments.
The key problem of the application of fuzzy clustering for which consists of three terms. The first α i term represents
time-series segmentation is the selection of the number of the a priori probability of the cluster, while the second
segments for the clustering process. Obviously, this is a represents the distance between the kth data point and the v it
standard problem also in the classical c-means clustering. In center of the ith segment in time. The third term represents
the context of time-series, however, it appears to be even the distance between the cluster prototype and the data in the
more severe. For this purpose, a bottom-up algorithm has
been worked out where the clusters are merged during a feature space where vix means the coordinate of the ith cluster
recursive process. The cluster merging is coordinated by a center in the feature space and r is the rank of Ai distance
fuzzy decision-making algorithm which utilizes a norm corresponding to the ith cluster. The Ai distance norm
compatibility criterion of the clusters, where this criterion is is calculated based on the PPCA models of the clusters. One
calculated by the similarity of the principal component of the most important advantages of PPCA models is that it
models of the clusters (Abonyi et al., 2005). allows their combination into mixture of models (Lin et al.,
2000). The optimal parameters of the
The original algorithm assumes that the data can be
effectively modeled as a mixture of multivariate Gaussian η i = {Vi x , Ai , v it , σ i , x , α i } cluster prototypes are determined
distribution, so it minimizes the sum of the weighted squared by the minimization of (1) subjected to:
distances between the Z k = [t k , X kT ]T data points and the η i
cluster prototypes c

∑µ
i =1
i,k =1 (4)
c N
J= ∑∑ (µ
i =1 k =1
i,k )m D 2 (Z k , ηi ), (1)
to get a fuzzy partitioning space. Detailed description of the
optimization approach is out of the scope of this paper and
interested readers are referred to (Abonyi et al., 2005) for the

859
recursive formulation of solving minimization problem as 2.3 Variable selections in a distributed FDI approach
well as the detail description of the bottom-up algorithm for
merging similar clusters. When dealing with high-dimensional datasets, the cluster
structure of the observation is often confined to a small
2.2 PCA and nonlinear PCA application in FDI algorithm subset of variables. As pointed out by several authors (Ding
PCA is used wherever large and complicated multivariate et al., 2002 and Raftery et al., 2006), the inclusion of
data sets have to be reduced to a simpler form. NPCA, like unrelated variables could complicate or even mask the
PCA, is used to identify and remove correlations among recovery of the clusters. Common approaches for mitigating
problem variables as an aid to dimensionality reduction, the effect of noisy variables or identifying those that define
visualization, and exploratory data analysis. However, NPCA true cluster structure involve differentially weighting the
uncovers both linear and nonlinear correlations. It works as a covariates or selecting the discriminating ones.
feature extraction algorithm, producing features that retain We propose the concept of distributed FDI for proper
the maximum possible amount of information from the clustering of various faults. Since different variables are
original data set, for a given degree of data compression. In responsible for different faults, using all process variables for
this paper, we investigate using PCA and NPCA as data identification of all faults is time-consuming and non-
reduction techniques and a preprocessing step for the efficient. In the proposed method, process is divided into
application of FDI in large scale processes. The main goal of subsystems and only the related variables of each subsystem
this paper in this section is to incorporate statistical PCA and are considered for fault identification purposes corresponding
NPCA data preprocessing to improving FDI performances. to that subsystem. This idea can be very helpful for proper
For the sake of brevity, implementations steps of these FDI of industrial large scale processes.
algorithms are briefly summarized. Interested readers are
referred to the relevant references. Fig. 1 shows the different approaches employed in this paper
for solving FDI problem in large scale processes. These
PCA or NPCA seeks for a linear or nonlinear transformation approaches are implemented and compared in the next
k ×m
G ∈ℜ such that: section for FDI of the TE process plant as an industrial
benchmark problem.
T = G (Y ) (5)

where Y ∈ ℜ m×n is the original data with m process variables

in n samples of time and T ∈ ℜk ×n is the transformed data,
called as scores with k features. This features or new
variables which are used for fault isolations are uncorrelated.
The inverse transformation, restoring the original
dimensionality of the data is implemented by a similar linear
or nonlinear function H ∈ ℜ m×k , such that:

Fig. 1. Different methods employed in this paper to solve FDI

Y ′ = H (T ) (6)
problem in large scale systems.

G and H are selected to minimize loss of information 3. EXPERIMENTAL SIMULATIONS OF THE PROPOSED
E where FDI APPROACHES ON THE TE PROCESS PLANT

3.1. Tennessee Eastman Industrial Benchmark

∑∑ (Y )
n m
1 i The process simulator for the Tennessee Eastman (TE)
E= j − Y j′i (7)
mn i =1 j =1 industrial Challenge Problem was created by the Eastman
Chemical Company to provide a realistic industrial process
for evaluating process control and monitoring methods
PCA can be implemented by singular value decomposition of (Downs et al., 2002). The TE process simulator has been
the covariance matrix of the data or by a three layer neural widely used by the process monitoring community as a
network (Baldi et al., 1989). A five layer neural network, as source of data for comparing various approaches (Chiang, et
showed by Kramer (1991), can be employed for al., 2001 and Lin et al., 2000). The control structure is shown
implementing NPCA. Here, we employ technique introduced schematically in Fig. 2. There are five major unit operations
by Scholz et al (2002) to enforce hierarchical order of the in the process: a reactor, a condenser, a recycle compressor, a
principal components which yields essentially uncorrelated separator, and a stripper. The four reactants A, C, D, and E
features. and the inert B are fed to the reactor where the products G
and H are formed and a by-product F is also produced. The
process has 22 continuous process measurements, 12

860
Fig. 2. Different Process layout of the Tennessee Eastman process.

manipulated variables, and 19 composition measurements The proposed FDI approaches, including PCA-based, NPCA-
sampled less frequently. Details on the process description base and distributed-based techniques, are evaluated in this
are well explained by (Chiange et al., 2001). A total of 33 section to demonstrate their diagnostic capabilities. The
variables, listed in Table 1, are used for monitoring in the obtained results will be compared with the original MGG
simulation studies. We excluded all composition clustering approach. Two sets of simulation experiments
measurements because they are unreliable. A sampling were conducted to investigate the comparative performances.
interval of 3 min was used to collect the simulated data for 1) Simulation Experiment 1
the training and testing sets. Some of the preprogrammed TE In this experiment, the faults corresponding to the TE Reactor
faults which are used in this section for FDI purpose have cooling water process, i.e. faults 4, 9 and 11, were
been listed in Table 2. A set of 100 data samples considered. To implement the experiment, a time series data
corresponding to each fault training data sets has been used in with a length of 300 data samples was constructed by
test study due to the unsupervised nature of the classification cascading a set of 100 data samples for each of the mentioned
technique employed in the proposed FDI approaches.. The TE Faults, comprising of all the 33 variables listed in Table
data have been generated and can be downloaded from 1. Fig. 3(a) illustrates the diagnostic performance of the
http://brahms.scs.uiuc.edu. original MGG clustering approach to detect and isolate the
3.2. Simulation Results and discussion

Table 1. Monitored Variables in the Tennessee Eastman Process

NO. Process Measurements NO. Process Measurements
1 A feed (stream 1) 18 Stripper temperature
2 D feed (stream 2) 19 Stripper steam Flow
3 E feed (stream 3) 20 Compressor work
4 Total feed (stream 4) 21 Reactor cooling water outlet temperature
5 Recycle flow (stream 8) 22 Condenser cooling water outlet temperature
6 Reactor feed rate (stream 6) 23 D feed flow valve (stream 2)
7 Reactor pressure 24 E feed flow valve (stream 3)
8 Reactor level 25 A feed flow valve (stream 3)
9 Reactor temperature 26 Total feed flow valve (stream 4)
10 Purge rate (stream 9) 27 Compressor recycle valve
11 Product separator temperature 28 Purge valve (stream 9)
12 Product separator level 29 Separator pot liquid flow valve (stream 10)
13 Product separator pressure 30 Stripper liquid product flow valve (stream 11)
14 Product separator underflow (stream 10) 31 Stripper steam valve
15 Stripper level 32 Reactor cooling water flow
16 Stripper pressure 33 Condenser cooling water flow
17 Stripper underflow (stream 11)

861
 Table 2. Process Faults For The Tennessee
Eastman Process
NO. Description Type
4 Reactor cooling water inlet temperature Step
5 Condenser cooling water inlet temperature Step
9 D feed temperature (stream 2) Random variation
11 Reactor cooling water inlet temperature Random variation
12 Condenser cooling water inlet temperature Random variation
15 Condenser cooling water valve Sticking

a)

a)
b)

b)
c)

Fig. 4. Scatter plots of faults 4, 9 and 11. a) PCA-based

features. b) NPCA-based features. c) Distributed features.

To implement the combined PCA-MGG approach, the

dominant numbers of principal components (PCs) should first
be determined. A study on the screeplot of the decomposed
eigenvalues of the data covariance matrix represented that 3
PCs are enough to capture more than 90% of the covariance
c) of the TE fault data. Fig. 3(b) illustrates the resulting
diagnostic performance due to application of the combined
PCA-MGG approach. As shown, the proposed approach
performs better than the original MGG approach. However, it
cannot discriminate the induced faults properly. The scatter
plot of the TE fault data set in Fig. 4(a) shows that the
introduced faults have overlaps and hence their isolations is a
difficult task. Fig. 3(c) depicts the diagnostic performance of
the combined NPCA-MGG approach. As shown, the FDI
performance has been improved further. However, the faults
have not been isolated yet. The corresponding scatter plot in
d) Fig 4.(b) confirms this improvement.
Fig. 3. FDI results for faults 4, 9 and 11. The above experimental observations gave us the important
idea that all the TE process variables or their
TE faults 4, 9 and 11. As shown, this approach fails to linearly/nonlinearly reduced versions cannot solve the FDI
perform the diagnostic task due to inclusion of unrelated problem by the employed clustering technique.
variables.

862
 distributed-MGG, have been illustrated in Figs .5(a, b, c, d),
respectively. The experimental observations are quite similar
to those recorded in section 1. For the sake of brevity, the
supporting scatter plots have not been presented.

4. CONCLUSION
A number of new monitoring approaches were presented in
this paper based on the MGG clustering approach. It was
shown that the original MGG is unable to perform FDI task
a)
in the TE process plant as a large-scale benchmark problem.
It was shown that the proposed PCA-MGG and NPCA-MGG
approaches could improve the diagnostic monitoring
performances with respect to the original MGG approach.
However, they were unable to exactly perform the FDI task.
The proposed distributed-MGG approach, based on the
monitoring of local subsystems, proved to be a promising
diagnostic approach with superb FDI performances to
discriminate the TE faults 4, 9 ,11 5, 12 and 15. It is noted
b) that the approach could manage to diagnose the TE faults 9
and 15 which are known as the TE most difficult faults. To
the best of our knowledge, these faults have not been
diagnosed yet.

REFERENCES
Abonyi, J., B. Feil, S. Nemeth and P. Arva (2005), Modified
Gath–Geva clustering for fuzzy segmentation of
multivariate time-series. Fuzzy sets and systems J., vol.
c) 140, pp. 39–56.
Baldi, P., and K. Hornic (1989), Neural network and
principal component analysis: learning from example
without Local Minima. Neural network, vol. 2, pp. 53.
Chiang, L. H., E. L. Russell and R. D. Braatz (2000), Fault
diagnosis in chemical processes using Fisher
discriminant analysis, partial least squares, and principal
component analysis. Chemometrics and Intelligent
Laboratory Systems J., vol. 50, pp. 243–252.
Chiang, L.H., E.L. Russell and R.D. Braatz (2001), Fault
d) Detetction and Diagnosis in industrial systems. Springer
Verlag, London.
Fig. 5. FDI results for faults 5, 12 and 15. Ding, C., X. He, H. Zha and H. D. Simon (2002), Adaptive
dimension reduction for clustering high-dimensional
As a consequence, the proposed distributed-MGG monitoring data. Proceeding of the IEEE International conference
approach, based on the locally selected process variables, was on Data Mining, pp. 147–154.
tested. The obtained result shown in Fig. 3(d) demonstrates Downs, J. J., E.F. Vogel (1993), A plant-wide industrial-
diagnostic performances of the proposed approach. In this process control problem. Computers & Chemical
distributed monitoring scheme, the local process variables Engineering J., vol. 17, pp. 245–255.
corresponding to the Reactor, i.e. variables 9, 21 and 32 in Kramer, M. (1991), Nonlinear principal component analysis
Table I, have been employed as the input fault data set. The using auto-associative neural networks. AIChE Journal,
scatter plot illustrated in Fig. 4(c) verifies the validity of this vol. 37, pp. 233–243.
approach. As shown, the three TE Reactor faults have Lin, W., Y. Qian and X. Li (2000), Nonlinear dynamic
properly been separated, making their clustering an easy task principal component analysis for on-line process
to do. monitoring and diagnosis. Computers & Chemical
Engineering J., vol. 24, pp. 423–429.
2) Simulation Experiment 2 Raftery, A. E. and N. Dean (2006), Variable selection for
The same experimental scenario was repeated for the faults model-based clustering. Journal of the American
corresponding to the Condenser cooling water process, i.e. Statistical Association, vol. 101, No. 473, pp. 168–178.
faults 5, 12 and 15. The obtained results for the different Scholz, M., and R.Vigario, (2002), Nonlinear PCA: A new
monitoring approaches, i.e. the original MGG clustering, the hierarchical approach. Proc.European Symposium on
combined PCA-MGG, the combined NPCA-MGG and the Artificial Neural Networks, pp. 439–444.

863