Fuzzy Min-Max Neural Networks—Part 2: Clustering Patrick K. Simpson, Member, IEEE Absiract—tn an earlier companion paper [56] a supervised learning neural network pattern clasifer called the fuzzy min- ‘max claslfction neural network was described. In this sequel, the unsupervised learning pattern clustering sibling called the {azzy min-max clustering neural network Is presented. Pattera clusters are implemented here a fuzy sls using a membership Tanetion with a hyperbox core that & constructed from a ein Point anda max point. The min-max points are determined wsing {he fuzzy min-mas learning algorithm, an expansion-contration proces that refines the author's earlier Fuzzy Adaptive Reso- hance Theory neural network {50}. The fuzzy min-max clustering neural network stabilize into patter casters in only few pases {through a dataset; it can be reduced to hard cluster boundaries that are easly examined without sacrificing the fuzzy boundaries, 1 provides the ability to Incorporate new data and add new clusters without retraining; and it inherently provides degre af ‘membership Information that Is extremely useful in higher level ‘eeision making and information processing. "This paper will provide some’ background concerning the evelopment of the fuzzy ‘clustering. neural network ‘and provide » comparison with similar work tha has recently fmerged. A brief description of fuzzy sets, pattern clustering, ‘and their synergistic combination is presented. The fuzzy min. ‘max chstering neural network wil be explained in detail amd ‘examples of is clustering performance wil be given. The paper ‘ill conclude with a description of problems that need to’ be ‘addressed and a list of some potential applications. 1. Inrropuction 'NLIKE patter classification, which provides class labels with patiem exemplars and seeks to find the decision boundary between classes that minimizes misclassification, pattem clustering has unlabeled pattem data and attempts to Find natural groupings amongst the exemplars. The pattern clusters are formed according to some predefined metric or criterion [16, 19, 23]. As Bezdck [5] points out in his seminal book Pattern Recognition With Fussy Objective Functions, there are many possible clustering criteria that ean be use including distance, angle, curvature, symmetry, connectivity, nd intensity ‘The choice ofthe proper grouping metric is only one aspect of the clustering problem. The fundamental question of how many clusters also remains. To illustrate this point, Figure I shows a set of twordimensional pattems (data points) that have no class labels. In this four small clusters ax shown in Fig. (a)? Is itt large clusters like that shown in Fig. 100)? (Oris there another clustering scheme that is more appropriate Mom end eo 1 ev September 1B, ce © o Fig. 1. An lesntion ofthe deen eps of cers a a poss Pe) shown the dat roped tt four csr, panel) ows Sine dts pouped ina two cumers Which one comet depends ret ‘al onthe data he span, “The answer isnot clear at all rather itis fuzzy. Continuing the ‘example, assume the clusters found in Fig. (a) were formed by some clustering technique. There are some datapoints that ‘completely belong to one cluster, but can also panally belong to anosher. Consider Ay and Az: it seems reasonable that A should have a high degree of membership in the cluster that Ay is associated with, and vice versa. ‘The fuzzy min-max clustering neural network addresses these concerns. ‘This paper describes the fuzzy min-max clustering neural network, a neural network that ereates and refines patter clus ters in a fashion similar to adaptive resonance theory (ART) {9}, (10) and the leader-custer algorithm (19). The fuzzy min- max clustering neural network makes the natural connection between clusters and fuzzy sets, This work is based upon an earlier neural network entitled fuzzy ART that was introduced by the author {S0]- Since the introduction of fuzzy ART, another fuzzy ART neural network has been introduced into the literature [11], [12] To eliminate any potential confusion and allow mutual recognition withthe sibling classification network, the ttle fuzzy min-max clustering. neural network has been selected “The fuzzy min-max clustering neural network has several appealing aributes: 1) The fuzzy min-max clustering neural network does notbound the number of clusters, rather the number grows to meet the demands of the problem, 2) The operations used by the fuzzy min-max clustering ‘neural network are very simple, requiting only comple ‘ent, compare, and accumulate operations which makes the implementation of this algorithm in fxed point hardware both feasible and efficient tis possible 19 reduce the fuzzy min-max clustering neural network to a set of hard cluster boundaries, ‘This allows the verification and validation process 10 be performed quickly and easily 4) There are only two parameters that need 19 be adjusted. ‘One governs the maximum size ofthe hyperbox, and the ‘other controls the fuzziness ofthe cluster membership. ‘The fuzzy min-max clustering neural network is constructed using. hyperbox fuzzy sets. A hyperbox defines a region of the n-dimensional patter space, and all pattems. con- tained within the hyperbox have full cluster membership A hyperbox is completely defined by its min point and is max point. The combination of the min-max points and the hhyperbox membership function defines a fuzzy sot (cluster) ‘The resulting byperbox fuzzy set fits naturally into @ neural network framework (hence its name). Learning in the fuzzy min-max clustering neural network consists of creating and adjusting hyperboxes in pattern space as they are received, Once the fuzzy min-max clustering neural network is vained, it is operated by presenting a pattern and computing the membership value that patern has in each of the currently ceistng fuzzy sets ‘The remainder of this paper is organized a follows. Section I provides some background conceming the development of the fuzzy min-max clustering neural network and presents a brief overview of fuzzy sets. clustering techniques, and the motivation for their synergistic combination. Section I de- seribes the fuzzy min-max clustering neural network in deta including learning, recall, is neural nenwork implementation, possible applications, and examples of its operation using different size hyperboxes. Section IV compare the fuzzy mine max clustering neural network with other fuzzy clustering neural newworks, including the Fuzzy c-means neural nework, the Carpenter-Grossberg-Rosen fuzzy ART and the adaptive fuzzy leader cluster neural network. Section V presents some future issues that need to be addressed 3 H, BackcRoUND A, The Original Fussy ART ‘The development of the fuzzy min-max clustering neural network his been an evolutionary process that had as its goal the development of an unsupervised leaming clustering neural network that is fast. efficient, reliable, and stil capable of revealing the structure ofthe partitions being formed. These developments led 1 a neural network paradigm that combined fuzzy logic with the ART newworks of Carpenter and Grossberg {9} into an analog pattern clustering system called fuzzy adaptive resonance theory (fuz7y ART) [50]. This original ‘development had several laws (none of which were crippling) 1) The relationship to fuzzy sets was not appropriately iden tified. Inthe original fuzzy ART description, the fuzzy portion of the neural network clusterer was attributed to the use of min and max operations for adjusting the ‘min and max points and the use of data points that were rescaled to the range (0, 1). In reality, the use of min and max operations in this context hed nothing to do with fuzzy sets. Similarly, just hecause data points were ‘constrained to the range [0,1] did oc make the system fuzzy. There was, however, one important element of the original fuzzy ART that was properly described as fuz7y-—the membership function, ‘The orginal fuzzy ART had a membership function that uilized the so-called fuzzy subsethood measure, The fuzzy subsethood measure is based upon an interpre- {ation of fuzzy sets as points in the unit hypercube [38], and was not appropriate for this setting. This measure has been since replaced witha more appropriate membership function ‘The hyperboxes could overlap, which allowed one data point to have full membership in more than one set. In ‘retrospect, this significantly weakens the use of fuzzy sets. It is more plausible 10 utilize non-overtapping clusters which guarantees that a data point has che full ‘membership in only one luster unless itis & point along, the boundary between two abutting hyperbores, 4) The original fuzzy ART utilized a maximum (hyper: box) cluster size parameter. Although this was the only parameter in the entire clustering system, it is not easonable to assume tha one parameter is sufficient for the entire system (rather it would be a function of each cluster being formed) In reality, the clustering operation should be conducted in two phases. In the first phase low-level clusters would be found and in the second phase they would be hierarchically grouped. This paper ‘will not examine the hierarchical grouping it will only focas on low-level clustering. Anextremely desirable tribute of any clustering system (neural, fuzzy, oF otherwise) is (0 process new data ‘immediately without having to retrain or refer to any of the previous taining data. This anribute, refered t0 here as on-line learning, is preseat in the fuzzy min= ‘max clustering neural network, Unfortunately, on-line learning typically results in a system that becomes order dependant during training, meaning if the same data Points are processed in a different order during taining, is possible (and acwally common) to have different pattem clusters emerge. The furzy min-max clustering rewral network also shares this property. Uslizing the previously mentioned hierarchical grouping of low-level ‘clusters could alleviate this problem. This is the subject fof future research, Despite these shortcomings, the original fuzzy ART demon strated excellent performance on several data sets (50). This oper describes improvements to the original fuz7y ART system that eliminate all but the lst Wo of these flaws, These issues are by far the most dificult to tackle for any clustering technique and will remain the subject of future esearch 2 3 5