Applied Soft Computing 67 (2018) 299–308

Contents lists available at ScienceDirect

Applied Soft Computing

journal homepage: www.elsevier.com/locate/asoc

A hybrid metaheuristic and kernel intuitionistic fuzzy c-means

algorithm for cluster analysis
R.J. Kuo a,∗ , T.C. Lin b , F.E. Zulvia c , C.Y. Tsai d
a
Department of Industrial Management, National Taiwan University of Science and Technology, No. 43, Section 4, Kee-Lung Road, Taipei 10607, Taiwan
b
Win Semiconductors, No. 35, Keji 7th Road, Hwaya Technology Park, Guishan District, Taoyuan City 33383, Taiwan
c
Department of Logistics Engineering, Universitas Pertamina, Teuku Nyak Arief Road, Simprug, Kebayoran Lama, Jakarta 12220, Indonesia
d
Department of Industrial Engineering and Management, Yuan Ze University, No. 135, Yuan-Tung Road, Chungli, Taoyuan 32003, Taiwan

a r t i c l e i n f o a b s t r a c t

Article history: Cluster analysis is a very useful data mining approach. Although many clustering algorithms have been
Received 25 October 2016 proposed, it is very difficult to find a clustering method which is suitable for all types of datasets. This
Received in revised form 2 January 2018 study proposes an evolutionary-based clustering algorithm which combines a metaheuristic with a ker-
Accepted 21 February 2018
nel intuitionistic fuzzy c-means (KIFCM) algorithm. The KIFCM algorithm improves the fuzzy c-means
Available online 9 March 2018
(FCM) algorithm by employing an intuitionistic fuzzy set and a kernel function. According to previous
studies, the KIFCM algorithm is a promising algorithm. However, it still has a weakness due to its high
Keywords:
sensitivity to initial centroids. Thus, this study overcomes this problem by using a metaheuristic algo-
Cluster analysis
Metaheuristics
rithm to improve the KIFCM result. The metaheuristic can provide better initial centroids for the KIFCM
Particle swarm optimization algorithm. This study applies three metaheuristics, particle swarm optimization (PSO), genetic algorithm
Genetic algorithm (GA) and artificial bee colony (ABC) algorithms. Though the hybrid method is not new, this is the first
Artificial bee colony algorithm paper to combine metaheuristics and KIFCM. The proposed algorithms, PSO-KIFCM, GA-KIFCM and ABC-
Intuitionistic fuzzy set KIFCM algorithms are evaluated using six benchmark datasets. The results are compared with some
Kernel function other clustering algorithms, namely K-means, FCM, Kernel fuzzy c-means (KFCM) and KIFCM algorithms.
Fuzzy c-means The results prove that the proposed algorithms achieve better accuracy. Furthermore, the proposed algo-
rithms are applied to solve a case study on customer segmentation. This case study is taken from franchise
stores selling women’s clothing in Taiwan. For this case study, the proposed algorithms also exhibit better
cluster construction than other tested algorithms.
© 2018 Elsevier B.V. All rights reserved.

1. Introduction achieve better mapping for nonlinear separable datasets. Accord-

ing to the study conducted by Zhang and Chen [10], the KFCM
Cluster analysis has been applied in many areas including image algorithm performs better than the FCM algorithm. Furthermore,
processing, business, medicine, engineering and industrial engi- Chaira [11] applied intuitionistic fuzzy set to the FCM algorithm
neering [1–7]. Cluster analysis aims to assign similar data points (IFCM). Intuitionistic fuzzy set was first proposed by Atanassov
to one cluster. This becomes easier to analyze data characteristics. [12]. While fuzzy c-means only considers the membership values,
Many clustering methods have been proposed, such as the K-means the IFCM algorithm not only considers the membership values of a
[8] and fuzzy c-means (FCM) algorithms [1]. The K-means algo- data point in each cluster, but also considers the non-membership
rithm is a well-known clustering algorithm [9]. It has also improved values in each cluster. Compared with the FCM algorithm, the IFCM
by applying fuzzy theory, referred to as the fuzzy c-means (FCM) algorithm can obtain better result. Therefore, in order to obtain a
algorithm. Recently, a kernel-based fuzzy c-means (KFCM) algo- better clustering result, this study combines the Kernel function
rithm was proposed by Zhang and Chen [10]. The KFCM algorithm with intuitionistic fuzzy set in the FCM algorithm. The idea of com-
replaces the Euclidean distance metric used in previous algorithms bining Kernel function with intuitionistic fuzzy set for clustering
with a Kernel metric. The Kernel function is applied in order to algorithms was first suggested by Lin [13] as the Kernel intuitionis-
tic fuzzy c-means algorithm (KIFCM). Basically, the KIFCM replaces
the Euclidean metric in FCM with a Kernel metric, while the fuzzy
set is replaced by intuitionistic fuzzy set. In the previous paper,
∗ Corresponding author. KIFCM has shown promising results. However, the results are not
E-mail address: [email protected] (R.J. Kuo).

https://doi.org/10.1016/j.asoc.2018.02.039
1568-4946/© 2018 Elsevier B.V. All rights reserved.
300 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308

stable. The reason is because the KIFCM algorithm was developed Step 1: Let T = 0. Set up cluster number c, membership degree
from a fuzzy c-means algorithm. This algorithm is highly sensi- m, tolerance rate ε and generate the initial membership value U0
tive to the initial membership value which commonly initialize randomly.
randomly. Therefore, the drawback of FCM still remains in KIFCM Step 2: Calculate the cluster centers v1 T+1 , . . ., vc T+1 using Eq.
algorithm. This study aims to overcome this drawback by applying (4):
metaheuristic. The metaheuristic can provide better initial cen- n  m
troids for the KIFCM algorithm. Thus, it will improve the KIFCM i=1
uTij xi
T +1
result. Since this study employs three metaheuristics, PSO, GA, and vj =
n  m , ∀j = 1, 2, . . ., c. (4)
ABC, there are three proposed algorithms in this study. They are the i=1
uTij
PSO-KIFCM, GA-KIFCM, and ABC-KIFCM algorithms.
After evaluating the three algorithms’ performances using Step 3: Update membership function UT+1 using Eq. (5):
benchmark datasets, a case study is presented. This study applies 1
the proposed algorithms to a customer segmentation problem. In uij (T +1) = (5)
1⁄m − 1
marketing, designing accurate marketing strategy for all customers c 2  
T +1 2
is very difficult, since every individual has a unique customer k=1
dij xi , vTj +1 /dij xi , vk
behavior. Customer segmentation improves understanding of cus-
tomer behavior. This study presents an application for customer Step 4: Update objective function JT+1 using Eq. (6):
segmentation for franchise stores selling women’s clothing in c 

n 
m 2
Taiwan. This company has many stores all over Taiwan. In order J T +1 = uTij +1 × dij xi , vTj +1 (6)
to increase their profit, the company must identify potential cus-
i=1 j=1
tomers before designing their marketing strategies. This study
therefore applies the proposed algorithms to solve this problem. Step 5: If |J (T +1) − J (T ) | ≤ ε, then stop. Otherwise, increase T by
The remainder of this study is organized as follows. Section 2 one and return to Step 2.
presents a survey of related literature for this study. The proposed In the intuitionistic fuzzy set, an object x ∈ Ã consists of mem-
methods are presented in Section 3. Section 4 discusses the method bership degree A (x) and non-membership degree vA (x), where
validation, while Section 5 presents the case study in customer 0 ≤ A (x) + vA (x) ≤ 1. If A (x) = 1 − vA (x) , ∀x ∈ Ã, then à is an
segmentation. Finally, concluding remakes are made in Section 6. intuitionistic fuzzy set (IFs). In IFs, there is a hesitation degree A (x),
defined in Eq. (7):
2. Literature review A (x) = 1 − A (x) − vA (x) (7)

This section presents the related background necessary for this Therefore, in the IFCM algorithm, the membership degree is
study. This includes cluster analysis using kernel intuitionistic calculated by Eq. (8):
fuzzy c-means algorithm, and metaheuristics algorithms. u∗ij = uij + ij , (8)

2.1. Cluster analysis using kernel intuitionistic fuzzy c-means where

algorithm  1⁄˛
ij = 1 − uij − 1 − u˛
ij
,˛ < 0 (9)
Cluster analysis is an exploratory data analysis method which ˛ is a parameter in the Yager-generating function. Further
divides instances into several groups based on their similarity [9]. improvement is achieved by adding a Kernel function. In the Ker-
Due to its importance, many studies have proposed a large variety
of clustering algorithms. One clustering method suggested is fuzzy  
nel intuitionistic fuzzy c-means (KIFCM) algorithm, the data point
xi and centroid vj are mapped in the Kernel space as  (xi ) and vj ,
clustering. In fuzzy clustering, each data point can be assigned to respectively. The objective function is defined in Eq. (10)[13]:
more than one cluster with respect to a degree of membership. The
degree of membership indicates how strongly an object is associ- 
n

c
 m  2
ated with a particular cluster. One of the most popular algorithms J= ij ×  (xi ) ,  vj  (10)

  This algorithm involves

for fuzzy clustering is the FCM algorithm. i=1 j=1
a membership value matrix U = ij n×c , where n is number of  2
where  (xi ) ,  vj  is the Euclidean distance between  (xi ) and
data points, c is number of clusters, and ij is membership degree  
of data point i to cluster j. The objective of the FCM algorithm is to  vj , calculated based on Eq. (11):
minimize the objective function as defined in Eq. (1):  2    
 (xi ) ,  vj  = K (xi , xi ) + K vj , vj − 2K xi , vj , (11)

n c
m  2
J= ij × dij xi , vj (1) where K (x, y) is the Kernel function. This study applies the Gaus-
i=1 j=1
sian Kernel as defined in Eq. (12):
 
with respect to K (x, y) = exp −x − y2 / 2 , (12)
where  is a parameter determining the width of the Gaussian
0 ≤ ij ≤ 1, ∀i = 1, 2, . . ., n; j = 1, 2, . . ., c (2)
Kernel. Finally, the membership degree matrix U and centroid v∗j in

C KIFCM are defined in Eqs. (13) and (14):
ij = 1, ∀i = 1, . . ., c (3) 
1
m−1
c
    m−1
1
j=1 uij (T +1) = 1/K xi , vTj +1 / 1/ 1 − K xi , vTk +1 (13)
k=1
where m > 1 is the membershipgrade.  According to previous  
n
studies, m is set as 2 [14,15]. The dij xi , vj is the distance between u∗ K xi , vj xi
the data point i, xi , and centroid of cluster j, vj . v∗j = 
i=1 ij
n   , ∀j = 1, . . ., c (14)
u∗ K xi , vj
The procedure of the FCM algorithm is as follows: i=1 ij
 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308 301

Fig. 1. Solution representation.

2.2. Metaheuristic methods Table 1

Dataset characteristics.

Due to its complexity, solving an optimization problem using Datasets Iris Wine Tae Flame Glass Wbc
an exact method is very difficult. For this reason, metaheuristic
Parameters
methods have become popular means of solving such problems.
Number of attributes 4 13 5 2 9 9
Metaheuristic algorithms such as particle swarm optimization
Number of instances 150 178 151 240 214 683
(PSO), genetic algorithm (GA), artificial bee colony (ABC), differ- Number of clusters 3 3 3 2 6 2
ential evolution (DE), ant colony optimization (ACO), gradient
evolution (GE), to name a few, have been employed to solve many
optimization problems [16–22]. In order to improve the KIFCM GA algorithm proposed by Maulik and Bandyopadhyay [25], and
algorithm, this study employs PSO, GA and ABC algorithms. These the ABC algorithm for clustering introduced by Krishnamoorthi and
algorithms are chosen since they have shown good performance Natarajan [26]. Figs. 2–4 illustrate the PSO-KIFCM, GA-KIFCM and
in solving many problems [23]. The GA algorithm is quite an old ABC-KIFCM algorithms, respectively.
metaheuristic algorithm. It explores the search space using three In this study, the distance between two data points is calcu-
operators: selection, crossover and mutation. The PSO algorithm lated using Euclidean distance. The reason for this is that it is the
was inspired the flocking or schooling behavior of birds or fish. In simplest and most used similarity measurement. Another reason
exploring the search space, particles move according to their indi- is that the base of this study is the KIFCM algorithm. In FCM, it
vidual best and social best positions. On the other hand, the ABC is more appropriate to use Euclidean distance. For other types of
algorithm mimics the behavior of a bee colony, consisting of scout, data, other similarity measurements could be applied. However,
worker and onlooker bees, each with a different purpose. Scout bees in order to minimize computational time, this study only applies
are the pioneers, and search for potential food locations. The search- the Euclidean distance, since the KIFCM algorithm itself requires
ing process is then continued by worker bees, who measure the significant computation.
amount of food in each source. Finally, onlooker bees will explore
the chosen food sources. 4. Experiment results

3. Methodology In order to evaluate the performance of the proposed algorithms,

this study implemented applied them to six benchmark cluster-
The KIFCM algorithm has been shown to have good performance ing datasets. The datasets were downloaded from the UCI machine
by previous studies [13]. However, its results are not stable since it learning repository, with the exception of the flame dataset, which
is highly dependent on the initial membership values and centroids. was downloaded from Fu and Medico [27].
Therefore, this study is the first study which intends to improve this Table 1 gives a brief overview of these datasets. These datasets
algorithm by employing metaheuristic algorithms. Each dataset has were cases proposed in some previous studies from various fields
unique characteristics, and most clustering algorithms are limited including medical, criminology, social, and agriculture.
for some particular data types. Thus, combining the KIFCM algo-
rithm with a metaheuristic algorithm will deliver a more general • Iris dataset was introduced by Fisher [28] as a dataset about three
and robust algorithm. species of iris flower named iris Sentosa, virginica, and versicolor.
In the proposed improvement, the KIFCM algorithm result is Fisher [28] distinguished the species from four features included
further improved by the application of a metaheuristic algorithm. in the dataset. They are the length and the width of the sepals and
Basically, the searching ability of the KIFCM algorithm is a local petals.
search procedure. Although it iteratively improves the random ini- • Wine dataset contains chemical analysis of wine derived from
tial centroids, the final centroids are relatively close to the initial three different cultivars grown in the same region. The wine
centroids. If the set of initial centroids is significantly far from the types are distinguished from 13 features namely alcohol, malic
optimal centroids, the clustering result will be poor. On the other acid, ash, alcalinity of ash, magnesium, total phenols, flavanoids,
hand, a metaheuristic algorithm has both local and global search nonflavanoid phenols, proanthocyanins, color intensity, hue,
procedures. By combining the KIFCM algorithm with a metaheuris- OD280/OD315 of diluted wines, proline [29].
tic algorithm, the clustering result will be improved. • Tae dataset is an evaluation of teaching performance of 151
This study applies three different metaheuristic algorithms, PSO, teaching assistants at the Statistics Department of the Univer-
GA and ABC. Basically, these metaheuristic algorithms aim to find sity of Wisconsin-Madison. The performance is measured based
the optimal centroids. Thus, as illustrated in Fig. 1, the population on the class size, type of semester, course, course instructor, and
represents the centroid of each cluster. Here, one solution in the whether or not the teaching assistant is a native English speaker.
initial population is taken from the KIFCM result. The other solu- The performance is categorized in three classes, “low,” “medium,”
tions in the initial population are randomly chosen from the data and “high” [30].
points. This initial population is then updated using the PSO, GA and • Flame is a dataset used to analyze DNA microarrays using a fuzzy
ABC algorithms. The fitness of each solution is the sum-of-square clustering by local approximation of membership (FLAME) [30].
distance within the cluster. This study applies the PSO algorithm • Glass dataset was applied in a study about criminological inves-
with self-adaptive inertia weight as applied in [24], the real-coded tigation. In order to use the glass left in the criminal scene as the
302 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308

Fig. 2. PSO-KIFCM flowchart.

evidence, the investigator should identify it correctly. The dataset cluster. Thus, the final cluster label for each data point is defined as
comprises of six types of glasses. They are building windows the cluster which has the highest membership value.
float processed, building windows non-float processed, vehicle
windows float processed, vehicle windows non-float processed, 4.1. Analysis of parameter settings
containers, tableware, and headlamps. In this study, the inves-
tigator should be able to distinguish the glass type based on the Besides comparisons with other algorithms, this study also con-
composition of the glasses which are the refractive index, sodium, ducted experiments to evaluate the effect of parameter settings
magnesium, aluminum, silicon, potassium, calcium, barium, and on the results. This is because metaheuristic algorithms require
iron [31]. predefined parameters, which control the scope of the exploration
• Wisconsin-Breast Cancer (Diagnostics) dataset (WBC) consists and exploitation performed by the algorithm. In some cases, these
of the measurements for breast cancer cases. This dataset was parameters can significantly influence the results. Therefore, this
obtained from the University of Wisconsin Hospitals, Madison study conducted experiments to analyze the effect of parameter
from Dr. Wolberg. There are two types of breast cancer cases, settings on the results obtained by the PSO-KIFCM, GA-KIFCM and
benign and malignant. In this dataset, the type of cancer is iden- ABC-KIFCM algorithms. Table 2 lists the tested parameter settings
tified based on clump thickness, uniformity of cell size and shape, in this study. These values are taken from previous studies [33–36].
marginal adhesion, single epithelial cell size, bare nuclei, bland 30 independent runs are conducted for each combination of param-
chromatin, normal nucleoli, and mitoses [32]. eter settings.
The experiment results reveal that for the PSO-KIFCM algorithm,
In the experiments, the proposed PSO-KIFCM, GA-KIFCM and learning rate 2, which control social exploration, has a significant
ABC-KIFCM algorithms were compared with the K-means, FCM, influence on the results for most datasets. Learning rate 1, how-
IFCM, KFCM, KIFCM, PSO-IFCM, GA-IFCM and ABC-IFCM algorithms. ever, appears to only have a significant effect on some datasets.
Since the tested datasets include cluster labels, the comparisons For the GA-KIFCM algorithm, crossover rate has a greater effect
conducted based on the accuracy achieved by each algorithm. In on the results than does mutation rate. According to these results,
fuzzy clustering, each data point has a membership value for each a higher crossover rate will yield a better result. On the other
R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308 303

Fig. 3. GA-KIFCM.

Fig. 4. ABC-KIFCM.
304 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308

Table 2 Table 3
Parameter settings for all clustering algorithms. Best parameter settings.

Method Factors Level 1 Level 2 Level 3 Dataset Iris Wine Tae Flame Glass Wbc

PSO-KIFCM Number of particles 50 80 100 PSO-KIFCM

C1 0.5 1.495 2 Number of particles 100 100 100 80 100 100
C2 0.5 1.495 2 learning rates 1 1.495 1.495 1.495 1.495 1.495 1.495
GA-KIFCM Number of chromosomes 50 80 100 learning rates 2 1.495 1.495 1.495 1.495 1.495 1.495
Crossover rate 0.85 0.9 0.95 GA-KIFCM
Mutation rate 0.001 0.01 0.1 Number of chromosomes 100 80 80 80 100 80
ABC-KIFCM Number of sources 50 80 100 Crossover rate 0.95 0.95 0.95 0.95 0.95 0.95
Limit of search 5 10 20 Mutation rate 0.01 0.1 0.1 0.01 0.1 0.01
Limit of scouts 5 10 20 ABC-KIFCM
Number of sources 100 80 80 80 100 100
Limit of search 10 10 20 10 20 10
hand, both limit and search, and scouts in ABC-KIFCM algorithm Limit of scouts 20 20 20 20 20 20

have significant effects on the results. These results imply that the
metaheuristic-based KIFCM algorithm requires wider exploration
in order to upgrade the local search conducted by the KIFCM algo-
rithm. low mutation rate are suggested, while for the ABC-KIFCM, the limit
Finally, the best parameter settings for each algorithm are of scouts should be around 20% of the number of food sources, and
summarized in Table 3. According to these results, this study the search limit should be around 10% to 20% of the number of food
recommends parameter settings for PSO-KIFCM to be 1.495 for sources. Population sizes of 80–100 are recommended. However,
learning rates 1 and 2. For GA-KIFCM, a higher crossover rate and for datasets with high noise, larger populations should be tried.

Table 4
Accuracy obtained by each algorithm.

Methods Iris Wine Tae Flame Glass Wbc

K-means Average(%) 79.246 91.195 46.004 83.944 39.034 96.066

Best(%) 88.670 94.940 56.291 85.833 45.327 96.193
Worst(%) 57.330 67.416 38.411 83.750 33.178 96.047
SD 0.146 0.095 0.051 0.006 0.038 0.001
FCM Average(%) 89.330 94.940 49.669 85.000 42.056 95.608
Best(%) 89.330 94.940 49.669 85.000 42.056 95.608
Worst(%) 89.330 94.940 49.669 85.000 42.056 95.608
SD 0.000 0.000 0.000 0.000 0.000 0.000
IFCM Average(%) 91.333 71.178 46.203 84.167 44.860 96.340
Best(%) 91.333 75.280 47.682 84.167 44.860 96.340
Worst(%) 91.333 65.170 42.384 84.167 44.860 96.340
SD 0.000 0.035 0.013 0.000 0.000 0.000
KFCM Average(%) 89.467 94.233 50.177 85.292 46.012 96.003
Best(%) 96.000 94.944 51.656 87.083 48.598 96.047
Worst(%) 84.667 92.700 42.384 85.000 43.925 95.754
SD 0.019 0.005 0.015 0.005 0.016 0.001
KIFCM Average(%) 91.556 93.205 51.832 85.319 46.573 96.389
Best(%) 96.000 94.944 56.954 87.083 49.065 97.365
Worst(%) 89.333 92.700 50.331 85.000 43.925 96.193
SD 0.019 0.009 0.023 0.006 0.016 0.003
PSO-IFCM Average(%) 80.667 65.506 44.768 85.583 41.869 95.666
Best(%) 90.000 84.831 51.656 88.333 48.598 96.633
Worst(%) 60.000 52.809 34.437 84.167 37.383 94.290
SD 12.009 14.605 7.646 1.990 4.341 0.980
GA-IFCM Average(%) 84.933 65.169 44.238 85.000 39.346 95.959
Best(%) 97.333 80.899 45.695 87.500 50.000 97.218
Worst(%) 74.000 41.011 42.384 84.167 28.972 94.583
SD 9.691 17.465 1.510 1.413 7.912 1.094
ABC-IFCM Average(%) 68.933 64.157 44.768 84.667 46.729 97.482
Best(%) 78.667 71.348 47.682 85.833 51.869 97.804
Worst(%) 61.333 60.112 42.384 81.250 40.187 97.218
SD 6.229 4.304 2.369 1.941 4.494 0.217
PSO-KIFCM Average(%) 91.822 95.206 52.958* 85.167 47.118 96.960
Best(%) 94.667 96.629 56.954 87.083 49.065 97.365
Worst(%) 90.000 94.382 50.331 84.167 45.327 96.340
SD 0.014 0.008 0.025 0.007 0.014 0.005
GA-KIFCM Average(%) 91.844* 95.243* 52.936 85.486* 47.103 97.057*
Best(%) 96.000 97.191 55.629 87.083 49.065 97.365
Worst(%) 90.000 94.382 51.656 85.000 46.729 96.633
SD 0.013 0.007 0.009 0.007 0.005 0.002
ABC-KIFCM Average(%) 91.733 95.187 52.848 85.292 47.134* 96.911
Best(%) 96.000 96.067 55.629 86.667 48.131 97.365
Worst(%) 90.000 94.382 52.318 85.000 46.262 96.633
SD 0.015 0.005 0.008 0.003 0.004 0.002
Other algorithms [37] Min 66 38.76 42.38 51.67 NA NA
Max 92.91 96.7 55.3 97.75 NA NA

SD: standard deviation.

*
Best result.
 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308 305

Table 5
Statistic testing results.

Comparison Iris Wine Tae Flame Glass Wbc

*** ** *** *** *** ***
PSO-KIFCM vs K-means 0.000 0.004 0.000 0.000 0.000 0.000
*** * *** * *** ***
PSO-KIFCM vs FCM 0.000 0.035 0.000 0.033 0.000 0.000
* *** *** *** *** ***
PSO-KIFCM vs IFCM 0.033 0.000 0.000 0.000 0.000 0.000
*** *** *** ** ***
PSO-KIFCM vs KFCM 0.000 0.000 0.000 0.791 0.003 0.000
*** * * ***
PSO-KIFCM vs KIFCM 0.27 0.000 0.017 0.83 0.035 0.000
PSO-KIFCM vs GAKIFCM 0.525 0.525 0.482 0.967 0.477 0.85
PSO-KIFCM vs ABCKIFCM 0.408 0.456 0.409 0.89 0.524 0.295
*** ** *** *** *** ***
GA-KIFCM vs K-means 0.000 0.004 0.000 0.000 0.000 0.000
*** * *** *** *** ***
GA-KIFCM vs FCM 0.000 0.01 0.000 0.000 0.000 0.000
* *** *** *** *** ***
GA-KIFCM vs IFCM 0.019 0.000 0.000 0.000 0.000 0.000
*** *** *** ** ***
GA-KIFCM vs KFCM 0.000 0.000 0.000 0.108 0.001 0.000
*** * * ***
GA-KIFCM vs KIFCM 0.248 0.000 0.01 0.152 0.048 0.000
*
GA-KIFCM vs PSOKIFCM 0.475 0.421 0.518 0.033 0.523 0.15
**
GA-KIFCM vs ABCKIFCM 0.381 0.356 0.344 0.134 0.601 0.004
*** ** *** *** *** ***
ABC-KIFCM vs K-means 0.000 0.003 0.000 0.000 0.000 0.000
*** ** *** *** *** ***
ABC-KIFCM vs FCM 0.000 0.004 0.000 0.000 0.000 0.000
* *** *** *** *** ***
ABC-KIFCM vs IFCM 0.042 0.000 0.000 0.000 0.000 0.000
*** *** *** *** *** ***
ABC-KIFCM vs KFCM 0.000 0.000 0.000 0.000 0.000 0.000
*** * * ***
ABC-KIFCM vs KIFCM 0.346 0.000 0.015 0.455 0.037 0.000
ABC-KIFCM vs PSOKIFCM 0.592 0.544 0.591 0.11 0.476 0.705
ABC-KIFCM vs GAKIFCM 0.619 0.644 0.656 0.866 0.399 0.996
*
p-value<0.05.
**
p-value<0.01.
***
p-value<0.001.

4.2. Comparison with other algorithms the time spent by metaheuristic algorithm is still acceptable. For
instance, the longest time for PSOKIFCM is around 18 s, while the
The comparisons with other algorithms are conducted using GAKIFCM spends around 8 min for the biggest dataset. ABCKIFCM
the best parameter settings, as listed in Table 3. Each algorithm needs more time, since it has more computation with three differ-
is executed 30 times. Table 4 summarizes the results. ent kinds of bees, scout, employee, and onlooker bees. According
According to the result in Tables 4 and 5, the proposed to these results, GAKIFCM shows a better performance within a
metaheuristic-based KIFCM algorithms perform significantly bet- relatively short time.
ter than the other algorithms. Table 4 shows the comparison with a
previous paper which used similar datasets. In the previous paper, 5. Case study
Kuo et al. [37] showed the comparison with 13 clustering algo-
rithms. Table 4 shows the minimum and maximum accurate rates Clustering algorithms can be used in many applications, includ-
in that paper. It shows that the results obtained by the proposed ing marketing. In this study, the proposed algorithms are employed
algorithm are close to the best results obtained by the previous to analyze the customer data collected by a franchise store in
algorithms. For Tae and Flame datasets, the proposed algorithm Taiwan. This company sells women’s clothing, and has many stores
obtained the better results. The global search procedure in these all over Taiwan. In order to increase the company’s profit, it must
algorithms is able to improve on KIFCM results. Where the KIFCM design an accurate marketing strategy. However, in order to do
algorithm is only able to search within a small area, and is prone this, analysis of customer data is very important. To this end, cus-
to becoming trapped in local optima, the metaheuristic algorithms tomer segmentation can be applied to help the company better
can dramatically move the centroids to a different area in order understand its customers.
to obtain a better solution. The standard deviations of the results The collected customer data includes purchasing data, in addi-
obtained by the metaheuristic-based KIFCM algorithms are also tion to the products bought and the transaction times. In this study,
smaller than that of the KIFCM algorithm. This shows that the clus- this raw data is preprocessed using the recency, frequency and
tering results obtained by the proposed algorithms are more stable. monetary (RFM) method [38]. There are 1786 customers included
In other words, the cluster centroids obtained by the proposed in the dataset. Before applying the proposed algorithms, a prelim-
algorithms have converged to the optimal locations, and the algo- inary study is conducted to determine the number of clusters. This
rithms therefore achieve higher accuracies and more stable results. step is conducted by dividing the dataset into several clusters and
Of the PSO-KIFCM, GA-KIFCM and ABC-KIFCM algorithms, the GA- calculating the DB index for each cluster. The DB index is a ratio
KIFCM algorithm achieves better results for four out of six datasets, of intra-cluster and inter-cluster distances [39]. Let a dataset be
although the differences are not significant. This is because the divided into k clusters. The DB value is calculated using Eq. (15):
mutation in GA makes it better able to avoid becoming trapped
1
k
in local optima than the PSO and ABC algorithms. These results
demonstrate that the KIFCM algorithm needs an improvement to DB (k) = Di,j , (15)
k
enhance its searching radius in order to find better centroids. This i=1
requirement can be successfully met by metaheuristics. In this where Di,j is defined in Eq. (16):
study, GA with its mutation operator achieves better results than
 
the PSO and ABC algorithms. d̄i + d̄j
In terms of computational time as shown in Table 6, PSOKIFCM Di,j = max . (16)
i=
/ j di,j
algorithm works faster than GAKIFCM and ABCKIFCM. The reason
is that PSO algorithm has the simpler algorithm than GA and ABC. d̄i is the average distance between each point in cluster i to the
The computational time of PSOKIFCM is about three times longer centroid of cluster i, while di,j is the distance between centroids i
than KIFCM algorithm. However, although they require more time, and j.
306 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308

Fig. 5. DB value for different number of clusters.

Table 6
Computational Time.
Fig. 6. Clustering result obtained by FCM algorithm.
Iris Wine Tae Flame Glass Wbc

K-means 0.02 0.05 0.03 0.02 0.02 0.05

FCM 0.80 2.62 0.97 0.47 4.68 4.56
IFCM 0.62 2.09 0.80 0.44 3.84 3.73
KFCM 1.19 4.09 1.47 0.70 7.14 7.27
KIFCM 1.14 3.67 1.37 0.73 6.52 7.06
PSOKIFCM 3.51* 10.72* 3.37* 1.72* 18.64* 17.75*
GAKIFCM 89.07 204.06 55.11 61.40 531.21 528.03
ABCKIFCM 856.34 1511.79 756.26 158.34 1219.68 1410.51
*
Fastest among metaheuristic-based KIFCM algorithms.

This experiment focuses on evaluating the DB value for clusters

2–15. It applies K-means to cluster the data. Since the K-means
algorithm begins with a random initial centroid, the results might
be different for each trial. Therefore, 30 replications are conducted
for each evaluation. The result shows that the smallest DB value is
obtained with five clusters (Fig. 5).
Based on this result, the proposed algorithms are applied to Fig. 7. Clustering result obtained by KIFCM algorithm.
obtain better clustering results. The results obtained by the algo-
rithms are compared based on the average distance between each
data point and its centroid. Table 7 summarizes the results. It shows
that the clusters constructed by the proposed metaheuristic-based
KIFCM algorithms are more compact since they have a smaller aver-
age distance to their centroids. Statistic T-test results, summarized
in Table 8, also show that the differences are significant. Of the
proposed algorithms, the PSO-KIFCM algorithm obtained the best
average and result. Although the most stable result is given by the
ABC-KIFCM algorithm, the standard deviation of the PSO-KIFCM
algorithm also small.
Compared with the results using benchmark datasets, in this
study case, the performance of GA-KIFCM is no better than the
PSO-KIFCM and ABC-KIFCM algorithms. This is because the data
in this study case are relatively concentrated within a small area.
In this case, the metaheuristic algorithms should be more focused
on deep exploitation than on wide exploration. The PSO-KIFCM
and ABC-KIFCM algorithms therefore perform better than the GA-
KIFCM algorithm. Thus, for further application, it is recommended
that a preliminary analysis be conducted in order to evaluate the Fig. 8. Clustering result obtained by PSO-KIFCM algorithm.
data characteristics before choosing a clustering algorithm.
Figs. 6–10 show the clustering results of the FCM, KIFCM, PSO- 6. Conclusion
KIFCM, GA-KIFCM and ABC-KIFCM algorithms. These figures show
that the clusters constructed by metaheuristic-based KIFCM algo- Clustering methods have been proposed for various purposes
rithms are clearer, and that there is no overlapping class. This result with various approaches, including fuzzy clustering using the
can be used by the company to better understand its customers, and KIFCM algorithm. The KIFCM algorithm is an improvement of a
thus design the best marketing strategy for each type of customer. well-known fuzzy clustering method called the FCM algorithm,
 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308 307

Table 7
Case study result.

K-means FCM IFCM KFCM KIFCM PSOKIFCM GAKIFCM ABCKIFCM

Average 5.150E-02 1.178E-01 9.751E-02 1.287E-01 6.430E-02 4.910E−02* 5.031E-02 4.984E-02

SD 2.562E-03 5.314E-03 5.007E-03 5.646E-17 4.235E-17 1.241E-05 1.052E-03 2.823E−17*
Best 4.985E-02 1.119E-01 8.014E-02 1.287E-01 6.430E-02 4.906E−02* 4.974E-02 4.984E-02
Worst 5.675E-02 1.287E-01 1.002E-01 1.287E-01 6.430E-02 4.911E-02 5.219E-02 4.984E-02

SD: standard deviation.

*
Best result.

Table 8
Statistic test for case study (p-value).

vs K-means FCM IFCM KFCM KIFCM PSOKIFCM GAKIFCM ABCKIFCM

K-means – 0.0000* 0.0000* 0.0000* 0.0000* 0.0000* 0.0220* 0.0010*

FCM – 0.0000* 0.0000* 0.0000* 0.0000* 0.0000* 0.0000*
IFCM – 0.0000* 0.0000* 0.0000* 0.0000* 0.0000*
KFCM – 0.0000* 0.0000* 0.0000* 0.0000*
KIFCM – 0.0000* 0.0000* 0.0000*
PSOKIFCM – 0.0000* 0.0000*
GAKIFCM – 0.0170*
*
Significantly different.

study addresses this problem by using a metaheuristic algorithm to

provide a better initial centroid and improve the KIFCM result. Since
the KIFCM algorithm can only perform a local search, the meta-
heuristic algorithm performs a global search to enhance its search
scope. The metaheuristic algorithm also helps the KIFCM algorithm
to avoid local optima. This study applies three metaheuristic algo-
rithms, namely the PSO, GA and ABC algorithms.
Three metaheuristic-based KIFCM algorithms are proposed in
this study, specifically the PSO-KIFCM, GA-KIFCM and ABC-KIFCM
algorithms. These algorithms aim to find the optimal cluster cen-
troids in their populations. For the PSO-KIFCM and GA-KIFCM
algorithms, the KIFCM algorithm is executed once. The final clus-
ters obtained by the KIFCM algorithm are used by the PSO and GA
algorithms as one of their initial solutions, while the other initial
solutions are randomly chosen from the dataset. On the other hand,
in the ABC-KIFCM algorithm, each scout bee improves its position
by using the KIFCM algorithm. To do this, the current position of
the scout bee is used by KIFCM algorithm as the initial centroid,
Fig. 9. Clustering result obtained by GA-KIFCM algorithm. and the new scout bee position is the final centroid given by the
KIFCM algorithm.
The proposed algorithms are evaluated using six benchmark
clustering datasets. The experiment results show that the proposed
metaheuristic-based KIFCM algorithms perform significantly bet-
ter than the K-means, FCM, IFCM, KFCM and KIFCM algorithms.
These results show that global search is needed in order to improve
the KIFCM algorithm. The GA-KIFCM algorithm achieves better
results than the PSO-KIFCM and ABC-KIFCM algorithms because it
has a mutation operator. The mutation in the GA algorithm prevents
the algorithm from becoming trapped in local optima, and helps
to expand its search scope. In addition, the proposed algorithms
are also applied to solve a customer segmentation problem. The
clusters constructed by the PSO-KIFCM, GA-KIFCM and ABC-KIFCM
algorithms are more compact and clearer than those obtained by
the KIFCM algorithm.
According to these results, further study should focus on
improving the algorithm performance. In the current algorithms,
user is still required to determine some parameters which control
the trade-off between local and global search. If the data points
Fig. 10. Clustering result obtained by ABC-KIFCM algorithm. are distributed in a large area, wide exploration is required. On
the other hand, if the data points are located within a relatively
embedding intuitionistic fuzzy set and a Kernel function in the small area, deep exploitation is needed. Therefore, the improved
original FCM algorithm. According to previous studies, the KIFCM algorithm should be able to autonomously evaluate the scenario
algorithm is a promising clustering algorithm. However, it still has and determine whether wide exploration or deep exploitation is
a weakness due to its high sensitivity towards initial centroids. This
308 R.J. Kuo et al. / Applied Soft Computing 67 (2018) 299–308

required. In addition, new metaheuristics, like imperialist compet- [20] M. Dorigo, C. Blum, Ant colony optimization theory: a survey, Theoretical
itive algorithm [33], can be applied to clustering. Comput. Sci. 344 (2005) 243–278.
[21] R.J. Kuo, F.E. Zulvia, The gradient evolution algorithm: a new metaheuristic,
Inform. Sci. 316 (2015) 246–265.
References [22] S.J. Nanda, G. Panda, A survey on nature inspired metaheuristic algorithms for
partitional clustering, Swarm Evol. Comput. 16 (2014) 1–18.
[1] J.C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, [23] X.S. Yang, Engineering Optimization: An Introduction with Metaheuristic
Kluwer Academic Publishers, 1981. Applications, John Wiley & Sons, Inc, New Jersey, 2010.
[2] M.-S. Yang, A survey of fuzzy clustering, Math. Comput. Modell. 18 (1993) [24] C. Dong, G. Wang, Z. Chen, Z. Yu, A method of self-adaptive inertia weight for
1–16. PSO, 2008 International Conference on Computer Science and Software
[3] X. Liu, L. Wang, Computing the maximum similarity bi-clusters of gene Engineering, IEEE (2008) 1195–1198.
expression data, Bioinformatics 23 (2007) 50–56. [25] U. Maulik, S. Bandyopadhyay, Genetic algorithm-based clustering technique,
[4] T. Kohonen, Self-Organizing Maps, Springer Science & Business Media, Berlin, Pattern Recogn. 33 (2000) 1455–1465.
Germany, 2001. [26] M. Krishnamoorthi, A. Natarajan, A comparative analysis of enhanced Artificial
[5] K. Honda, H. Ichihashi, Linear fuzzy clustering techniques with missing values Bee Colony algorithms for data clustering, 2013 International Conference on
and their application to local principal component analysis, IEEE Trans. Fuzzy Computer Communication and Informatics (ICCCI), IEEE (2013) 1–6.
Syst. 12 (2004) 183–193. [27] L. Fu, E. Medico, FLAME a novel fuzzy clustering method for the analysis of
[6] L. An, X. Gao, X. Li, D. Tao, C. Deng, J. Li, Robust reversible watermarking via DNA microarray data, BMC Bioinf. 8 (2007) 3.
clustering and enhanced pixel-wise masking, IEEE Trans. Image Process. 21 [28] R.A. Fisher, The use of multiple measurements in taxonomic problems, Ann.
(2012) 3598–3611. Eug. 7 (1936) 179–188.
[7] L. An, X. Gao, Y. Yuan, D. Tao, Robust lossless data hiding using clustering and [29] M. Forina, PARVUS – an Extendible Package for Data Exploration Classification
statistical quantity histogram, Neurocomputing 77 (2012) 1–11. and Correlation.
[8] J. MacQueen, Some methods for classification and analysis of multivariate [30] W.-Y. Loh, Y.-S. Shih, Split selection methods for classification trees, Stat. Sin.
observations, in: Proceedings of the Fifth Berkeley Symposium on (1997) 815–840.
Mathematical Statistics and Probability, California, USA, 1967, pp. 281–297. [31] I.W. Evett, J.S. Ernest, Rule Induction in Forensic Science. Central Research
[9] P.-N. Tan, M. Steinbach, V. Kumar, Introduction to Data Mining, Pearson Establishment, Home Office Forensic Science Service, Aldermaston, Reading,
Education, Inc, USA, 2006. Berkshire RG7 4PN, 1987.
[10] D.-Q. Zhang, S.-C. Chen, Clustering incomplete data using kernel-based fuzzy [32] W.H. Wolberg, O.L. Mangasarian, Multisurface method of pattern separation
c-means algorithm, Neural Process. Lett. 18 (2003) 155–162. for medical diagnosis applied to breast cytology, Proc. Natl. Acad. Sci. 87
[11] T. Chaira, A novel intuitionistic fuzzy C means clustering algorithm and its (1990) 9193–9196.
application to medical images, Appl. Soft Comput. 11 (2011) 1711–1717. [33] R.C. Eberhart, S. Yuhui, Particle swarm optimization: developments,
[12] K.T. Atanassov, Intuitionistic fuzzy sets, Fuzzy Sets Syst. 20 (1986) 87–96. applications and resources, Proceedings of the 2001 Congress on Evolutionary
[13] K.-P. Lin, A novel evolutionary kernel intuitionistic fuzzy-means clustering Computation, Seoul Korea vol. 81 (2001) 81–86.
algorithm, IEEE Trans. Fuzzy Syst. 22 (2014) 1074–1087. [34] R. Kuo, C.-F. Wang, Z.-Y. Chen, Integration of growing self-organizing map and
[14] W. Pedrycz, P. Rai, Collaborative clustering with the use of Fuzzy C-Means and continuous genetic algorithm for grading lithium-ion battery cells, Appl. Soft
its quantification, Fuzzy Sets Syst. 159 (2008) 2399–2427. Comput. 12 (2012) 2012–2022.
[15] J. Fan, M. Han, J. Wang, Single point iterative weighted fuzzy C-means [35] E. Michielssen, S. Ranjithan, R. Mittra, Optimal multilayer filter design using
clustering algorithm for remote sensing image segmentation, Pattern Recogn. real coded genetic algorithms, IEE Proc. J. (Optoelectronics) 139 (1992)
42 (2009) 2527–2540. 413–420.
[16] J. Kennedy, R. Eberhart, Particle swarm optimization, IEEE International [36] B. Akay, D. Karaboga, A modified artificial bee colony algorithm for
Conference on Neural Networks, 1995 Proceedings, IEEE (1995) 1942–1948. real-parameter optimization, Inform. Sci. 192 (2012) 120–142.
[17] C.A. Murthy, N. Chowdhury, In search of optimal clusters using genetic [37] R.J. Kuo, C.H. Mei, F.E. Zulvia, C.Y. Tsai, An application of a metaheuristic
algorithms, Pattern Recognit. Lett. 17 (1996) 825–832. algorithm-based clustering ensemble method to APP customer segmentation,
[18] R. Storn, K. Price, Differential evolution – a simple and efficient heuristic for Neurocomputing 205 (2016) 116–129.
global optimization over continuous spaces, J. Global Optim. 11 (1997) [38] J.R. Bult, T. Wansbeek, Optimal selection for direct mail, Market. Sci. 14 (1995)
341–359. 378–394.
[19] D. Karaboga, B. Basturk, A powerful and efficient algorithm for numerical [39] D.L. Davies, D.W. Bouldin, A cluster separation measure, IEEE Trans. Pattern
function optimization: artificial bee colony (ABC) algorithm, J. Global Optim. Anal. Mach. Intell. PAMI-1 (1979) 224–227.
39 (2007) 459–471.