2004 IEEE International Conference on Systems, Man and Cybernetics

A Fault Diagnosis Model Through G-K Fuzzy Clustering

Ning Lv Xiaoyang Yu Department of Automation Department of Measurement-control Harbin University of Science and and Instrument, Harbin University of Technology, Harbin, China Sci. and Tech, Harbin, China. [email protected] [email protected]
Abstract A new method for fast building the knondedgebased fault diagnosis model by means offuzzy clustering is proposed. The scheme is contrived by Gustafon-Kessel (GK) algorithm, which is of many goodproperties. In this paper, it is jirst investigated how to integrate the properlies of fault diagnosis systems into the GK clustering algorithm in the product space of input and output voriables. Then the way to convert thefizzy clusters to the fault diagnosis model i suggested. Hence, an s efficient olgorithm to acquire the knowledge-based fault diagnosis model fiom observations is worked out. As a result, the obtained fault diagnosis model can ident% foult patterns of different shape and orientotion in one data set. Moreover, by introducing the concept of thejiury degree o faultiness (DaF), the proposed approach seems to be f much moreflexible and with more power@ ability to deal with data contominafed by noise compared with the traditional fault diognasis method. Finolly, An experiment of the fault diagnosis o a satellite power supply subsystem f demonstrates the effectiveness of the proposed fmlt diagnosis model. Junfeng Wu Department of Automation Harbin University of Science and Technology, Harbin, China [email protected]

Keywords: Fault diagnosis, knowledge-based system, fuzzyclustering, degree of faultiness (DoF)

1 Introduction
Due to the increasing complexity and riskiness of modem control systems, fault diagnosis has become an important issue in modem automatic control theory, and during the last two and a half decades, an immense amount of research has been done in this field resulting in a great variety of different methods with increasing acceptance in practice [1,2]. The core of the fault-diagnostic methodology is the so-called model-based approach, where either the analytical or knowledge-based models, or combinations of bath, are used in combination with analytical or heuristic reasoning [l]. However, in the case of fault diagnosis in complex systems, one is faced with the problem that no, or no sufficiently accurate mathematical models are available. The use of knowledge-based models in the kamework of diagnosis expert systems is then the only feasible way. Nevertheless, the major difficulty in howledge-model-hased techniques is the knowledge acquisition, which is known as an extremely difficult task. To copy with this difficulty, some
* 07803-8566-7104/$20.00 0 2004 IEEE.

soft computing techniques, such as fuzzy logic, neural networks, and rough set theory are applied (3-51. The fuzzy expert system combines the expertise and experience of the diagnosis experts with the diagnosis system using fuzzy $then rules so that it can tackle the uncertainties in fault diagnosis. Some people use the general fuzzy clustering approach to build the fuzzy relational models from observations [6,7], but few further researches that integrate fuzzy clustering into fault diagnosis models have been done so far despite of some work for rough data models done by Huang [8]. For the fault diagnosis models, the input variables are usually continuous measures, while the output variable always takes discrete values, e.g. some integers like 1,2, ,,., that represent the corresponding fault types, e.g. fault I, fault 2 ,,. , Based on these properties, in this paper, a novel approach to knowledge-based fault diagnosis models is suggested through Gustafson-Kessel (GK) fuzzy clustering [9] in the product space of input and output variables. The resulting fault diagnosis model not only can identify fault pattems of different shape and orientation in one data set, but also has a strong ability to deal with a variety of noises. An experiment of the fault diagnosis of a satellite power supply subsystem demonstrates the effectiveness of the proposed fault diagnosis mode. The rest of the paper is organized as follows: Section 2 presents the main idea of integration fuzzy clustering into fault diagnosis models after a brief review of GustafsonKessel (GK) fuzzy clustering algorithm. Section 3 describes the methodology of the proposed approach to building the fuzzy fault diagnosis model. Experimental results of the fault diagnosis of a satellite power supply subsystem are shown in Section 4, and final conclusions are discussed in Section 5 .

2 Adaptive Fuzzy Clustering in Combination with Fault Diagnosis Model

2.1

Gustafson-Kessel (GK) algorithm

Fuzzy clustering is an important tool to identify the structure in data. The Gustafson-Kessel (GK) algorithm [7], which is the fuzzy generalization of the Adaptive Distance Dynamic Clusters algorithm [7], searches for ellipsoidal

5114

clusters. It can be used for linear or planar clusters because this type of cluster can be viewed as a special case of ellipsoids for which one or more radii are zero. In the GK algorithm, the distance from a point xk to a cluster prototype (center) vi is a squared inner-product distance norm,
2 D h j = l l X k - v i / l ~= ( X k - v i ) T h f i ( X k - v ; ) z W ~

be used to compute optimal local linear models h m the covariance matrix.

(1)

where M i = det(Fi)'/" F;' is a positive-definite symmetric matrix related to the covariance matrix Fi of the: ih t prototype, and n is the dimension of the input-output product space. A partition of data set {XI, ... , x N } into c fuzzy clusters is performed by minimizing the objective function

Fig. 1 Equation ( ~ - v ) ~ F - ' ( x - v ) = defines a I hyperellipsoid. The length of the jth axis of this hyperellipsoid is given by and its direction is spanned by@j, wherel, andq+j are the jth eigenvalue and the corresponding eigenvector of F, respectively. In sum, the GK algorithm may be formulated as follows: given a set of data { x k I k = 1,2,...,N) , choose the number of clusters I W N , the weighting exponent m>l and the termination tolerance E>O. Initialize the fuzzy partition matrix randomly, such that (3) holds. Repeat for /=1,2, ... Step 1: compute the cluster centers

where U=[uik] a fuzzy partition matrix, and is

Zuik=l, l<k<N
i=l

(3)

uik ~ [ 0 , 1 ] me[l, m ] is a weighting exponent which ; determines the fuzziness of the resulting clusters, (for a crisp model m=l, fuzzy model m>l, but mostly m=2). The stationary points of the objective function (2) can be hund by adjoining the constraint (3) by means of Lagrange multioliers:

(4) and by setting the gradients of 7 with respect to U,V and h to zero. If D& > O , V i , k and m>l, then (U,V) may
minimize (2) only if

Step 2: compute the cluster covariance matrices

N
C(U":-")yXt

- v f ) ) ( x i - v i(4) T

F , = *=I
N
E(Up")" k=I

,l<i<c

(5)
and

Step 3: compute the distances Mi =det(Fi)""F,-I, D 2 g = ( x k - V : ' ) ) ~ M ~-"!'I), W (X~

l<i<c, l < k < N Step 4: Update the partition matrix 1 U?) =
tk

An advantage of the GK algorithm over FCM is that GK can detect clusters of different shape and orientation in one data set. This is due to the fact that the eigenstructure of the cluster covariance matrix provides information about the shape and orientation of the cluster. The ratio of the lengths of the cluster's hyperellipsoid axes is given by the ratio of the square roots of the eigenvalues of F, . The directions of the axes are given by the eigenvectors of F;,as shown in Figure 1. Linear subspaces of the data space are represented by flat hyperellipsoids, which can be seen as hyperplanes. The eigenvector Corresponding to the smallest eigenvalue determines the normal to the hyperplane, and can

( D i w g/ D j w , ) Z / ( n - ' ) '

until

I u(') -U('- )I

1 5 i <c, 1 5 k S N

<e ,

2.2

Fuzzy clustering in fault diagnosis models

Let's take a look at the properties of clustering results corresponding to fault diagnosis models of different qualities in the product space of input and output,

5115

Assume that x E R" ,is the input data vector, y E R ,is the category data, i.e., the output data, which are some T integers. Denote Z, = [X, ,y, IT, k indicates the kth data point, and defme the type of fuzzy cluster Ci as the component of the corresponding cluster center
V, = [Vi, .vi2 Vi."+lI . Proposition I The fault diagnosis model of high quality means that the classification accuracy of every cluster is high. This is exhibited intuitively in clustering results that almost all the category values of the data points of cluster Ci are equal, and they nearly equal to the type component of the cluster center
I...,

This implies that (a) the covariances of fault category variable y and the other data point components of the same cluster Ci are usually some nonzero values, i.e. cov(x,y)>o, (b) The variance of fault category variable y of a cluster Ci is a large value, i.e., D(~,,+~)=cov(y,y)=(n+l,n+l)>O Fi (9)
MembershiD fundion dots

Vi =[vil,viz ,..., v ~ , , ~ i.e., ~ , ,]

= y e . This leads to the

v1.n+1

fuzzy covariance matrix Fi of cluster Ci with the following form ... * 01

v2,n+1

. .. vc,n+1

r.

Fig. 3 Membership function of C , in the fype of fault diagnosis model of low quality in v2,,,+, Therefore, the Gaussian membership function of clusters corresponding to fault diagnosis model of low quality has some flat curves whose centers diverge from their true category values (Fig.3). AAer clustering with GK algorithm in the product space, we in fact obtain a set of fuzzy clusters Ci , i=1,2,._., c. In our approach, we set a tolerance vector, rolSig220, E' where c is the number of the clusters, for the variance R, D(v.+J of the c cluster centers in the product-space. Only the clustering results, in which all the variances of the cluster centers satisfy D(~,,~~)<fo/SigZ(i), ..., i=l, c are accepted for building the fault diagnosis model; otherwise, we should increase the number of clusters c and perform the fuzzy clustering algorithm once more. So, this fuzzy clustering in product space of input and output variables under the supervision of the resulting fault diagnosis model quality is here referred to as the supervised fuzzy clustering.

F. =

(7)

where the last row and column of F, correspond to the type of c,. It shows that: (a) the covariances of fault category variable y and the other data point components are nearly zeros, i.e., CO~(X,J,) = 0; (b) The variance of fault category variabley of clusterCi is nearly zero, i.e., D ( ~ ~ ~ , ~ ) = c o v ( y , y ) = F ~ ( n + l , n + l ) (8) oO
Membership
funclion plots

.:mi
0
v1,n+1

vz. n*1

v c , n+1

Fig. 2 Membership function of C j in the fype vLmlof fault diagnosis model of high quality (i=l,2, c) ..., Thus, the Gaussian membership b c t i o n of clusters Ci (1 5 i 5 c) is some narrow-pulses with their centers equal to of cluster centers (Fig.2). the type components

2.3

Strategy of searching for the optimal partition of product space

Proposition 2 The fault diagnosis model of low quality means that the classification accuracies of the most clusters are low. This is exhibited in clustering results that most fault category values of the data points of cluster Ci are of much difference, and they diverge from the type component v ~ ,of the cluster center Vi = [ v i l viz ,..., ~ + ~ , I T greatly.

In our approach, GK fuzzy clustering algorithm is fmt employed to perform the parlition of input and output product space. The result clusters are used to build the corresponding fault diagnosis model, and then some performance indices such as the cumulative fault identifying accuracy etc. are computed. This procedure can start fiom a small pre-specified number of clusters, c. If the performance is not satisfied, then increase the number of clusters c and repeat to use GK algorithm, until achieving the "best" performance. This is due to the obvious fact that as the cluster number c increases h m a small number, the corresponding fault diagnosis model's accuracy will be

5116

improved increasingly. When the number of clusteis is chosen equal to the number of fault pattem groups that actually exist in the data, it can be expected that the clustering algorithm will identify them correctly, and the accuracy of the fault diagnosis model becomes the best.

Cuse I: There exist one or more DoF,(Dafo(i)),such

that DoF,(Dufu(i)) >THG),j=I,Z

,...,c.
, and

In this case, if the consequent values (i.e.,

u & + ~ the cluster centers) of all these cluster rules are of

3 Convert Fuzzy Clusters to Fault Diagnosis Model

Number section and subsection headings consecutively in Arabic numbers and type them in bold. Use point size 14 for section beadings and 12 for subsection beadings. Avoid using too many capital letters. Keep section and subsection headings always flushed left. If any further subdivision of a subsection is needed the titles should be 12 point. The fuzzy cluster C, can be characterized by its center vector
[v,], ,v, ] and variance vector [ U : , .,a i ] in the input .

the same or approximately equal to each other, then take the cluster with the maximumDoF,(Dafa(i)) value as the cluster to which Dafu(i) belongs; otherwise, Do/u(i) is taken as the data point which can not be identified. Case 2: There exist no DoF, (Dofo(i)), such that
DoF,(Dafu(i))L T H ( J ) ,j=1,2 ,...,c

This means that no faults have been detected, and the system may operate in normal state.

Experiments Study

space. If we assign the Gaussian type membership funi:tion to each cluster component, A , ( x y )
= exp

i=l,__., j = l , ...,n, k , N, X=[xi,... ,xJ, where these c, l ..., membership functions can be obtained by projecting cluster C, to its every dimension], then we will have the fiuzzy fault diagnosis rules in which each antecedent proposition is expressed as a logical combination of propositions with univariate ficzzy sets defined for the individual comporients ofX, and usually in the following conjunctive form: R,: If xI is A,,(xl) and ... and x. is A,(x,), then fau/f-ppe(C,)=c,. In this case, the degree of faultiness of a data X kwith respect to the fault cluster C, , DoF8(X k ) , can be defined as the product of the individual membership degrees in the projecting space
DoF, (X, = )
]=I

We consider the fault diagnosis of a satellite power supply subsystem. Table 1 gives a measure data set generated by the fault simulator stand of the satellite power supply subsystem. 15 samples are given, and the last column is the fault type: 1-faults in power attenuation, 2 - faults in linear shunt current controller, 3 -no faults. Table 1 Data set for fault diagnosis of a satellite power suu~lv _. subsvstem Isc Fault Vdl ICM Icnb No 1 0.32 30.5 0.18 0.23. 1 1 0.21 0.32 33.6 0.17 2 0.32 2 2.81 31.6 2.97 3 1.86 30.8 0.32 3 4 2.15 0.19 1 34.3 0.32 0.16 5 0.18 0.22 29.7 0.29 3 6 2.97 33.9 0.32 2 I 2.66 2.69 2.84 34.6 0.28 3 8 2.78 30.8 0.30 3 9 2.93 2.85 2.68 33.6 0.32 2 IO 0.22 0.19 32.5 0.32 1 11 2.76 2.83 34.2 0.32 2 12 0.18 32.1 0.32 1 13 0.22 3 2.85 0.22 28.6 0.32 14 2.78 2.67 34.7 0.32 2 15
I

4,

(XI )

(10)

While in the Cartesian product-space, DoFX X k) is simply defined as the membership degree of the multidimensional fuzzy set C,

j=l

where D;wj is the distance between xk and the center of cluster Ci in GK algorithm (Ref. to Eq.(2)). Now, we can identify the type of a fault sample using DoF by predefining a constant threshold vector TH. For any data Dufu(9, if DoF,(Dufo(i)) -THO, j=1,2 ,...,c, then
Dufu(i)E C, ;otherwise Datu(i) e C, . The fml type o:fthe

data sample Do~o(J] synthesized according to the is DoF, (Duta(i)),j=1,2,. ..,c as follows.

Applying our approach on the data set, feature Icna and lcnb are selected to build the fault diagnosis model, and seven fuzzy fault pattem clusters are produced as shown in Table 2. So, according to Table2, the fault diagnosis rules for the satellite power supply subsystem are obtained as follows: IfXECl orXEG, Thenfaulf-Qpe=l; IfXEC5, Thenfmm_tJpe=2; IfXECzorXEC~orXECsorXEC7, Thenfault-1p~3; where, X=[Icna, Icnb]. When setting foN=O.Ol and to/Sig2=0.01, the identifymg accuracy of the above fuzzy fault diagnosis model on the training data (Table 1) reaches 100%. Another 15 data samples are generated by the fault

5117

simulator stand as a test data set. The valid identifjmg accuracy of the obtained fault diagnosis model is 93.3%. The experimental results show that our method is effective. Table 2 Fuzzv clusters for fault diarmosis model Variance Vector Fault Sig2: (Icna Icnb) BP5 14.4439e-09 7.1 102e[6.5718e-16 3.3270e[0.1800 3 [6.2500e-40 6.2500e[0.2200 1 [2.4810e-05 2.6913e[2.1885 3 [1.1202e-04 1.482% [2.8040 2 [2.0859e-04 8.1480e[2.8190 11.2969e-08 3.9074e3 ,
I

_ .

5 Conclusions
In the field of fault diagnosis systems, there is a rapid development 60m the well-established but limited efficiency traditional methods of signal-based fault diagnosis, towards model-based approaches, using analytical andor knowledgebased models. In this paper, a novel knowledge-model-based approach through Gustafson-Kessel (GK) fuzzy clustering algorithm is developed, which integrates the properties of both fault diagnosis systems and GK algorithm. The proposed fault diagnosis model consists of a set of fuzzy rules, and as a result, it can detect fault patterns of different shape and orientation in a data set. Moreover, by introducing the concept of the fuzzy degree offmlrirress (DoF), the approach seems to be much flexible and with more powerful ability to handle the noise data compared with the traditional fault diagnosis models. The proposed approach is really a softer technique to build the fault diagnosis model in a sense.

[3] Plamen P. Angelov, An evolutionary approach to fuzzy rule-based model synthesis using indices for rules. Fuzzy Sets and Systems, Val: 137, Issue: 3, pp. 325338,2003 141 . Jakubek and T. Strasser, Neural networks applied to _ S. automatic fault detection. The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002. vol.l,639-642,2002 [5] E. L. Bonaldi, L. E. B. da Silva, G. Lambert-Torres, L. E. L. Oliveira, and F. 0. Assunco, Using rough sets techniques as a fault diagnosis classifier for induction motors. IEEE 2002 28th Annual Conference of the Industrial Electronics Society, IECON 02. vo1.4, pp.3383- 3388,2002 [6] P. A m , J. M. Perronne, G. L. Gissinger, and P. M. Frank, Identification of fuzzy relational models for fault detection. Control Engineering Practice. Val: 9, Issue: 5, pp. 555-562,2001 [7] Jinjie Huang, Sbiyong Li, Chuntao Man. A T-S Type of Rough Fuzzv Controller Based on Process Inoutoutput Data. Proceedings of the 42nd IEEE Conference on Decision and Control. Maui, Hawaii USA, pp. 4729-4734.2003 [SI Jinjie Huang, Shiyong Li. A GA-based Approach to Rough Data Model. The 5th World Congress on Intelligent Control and Automation (WCICA2004), Hangzhoy China, pp. 1880-1884,2004 191 D. E. Gustafson, and W. C. Kessel, Fuzzy clustering with a fuzzy covariance matrix. In: Proc. IEEE CDC, San Diego, CA, USA, pp. 761-766,1979

.,

Acknowledgement
The authors thank to the suppolts from National 973 Science Research Plans of China (No. 2002cb312200-01-1) and the Provincial Natural Science Foundation of Heilongjiang, China (F0202).

References
[l] Paul M Frank, Birgit, Koppen-Seliger. New Developments Using AI in Fault Diagnosis. Engineering Applications of Artificial Intelligence, Val 10, Issue: 1, pp. 3-14, 1997 [2] A. Siddique, G. S. Yadava, and B. Singh, Applications of artificial intelligence techniques for induction machine stator fault diagnostics: review. 4th IEEE Intemational Symposium on Diagnostics for Electric Machines, Power Electronics and Drives, 2003. SDEMPED, pp. 29- 34,2003.
5118