Barebones Particle Swarm Methods for

Unsupervised Image Classification

M. Omran and S. Al-Sharhan

Abstract— A clustering method that is based on barebones on the initial conditions, which may cause the algorithm to
Particle Swarm (BB) is developed in this paper. BB is a variant converge to suboptimal solutions; and ” the user needs to spec-
of Particle Swarm Optimization (PSO) where parameter tuning ify the number of classes in advance. Clustering approaches
is not required. The proposed algorithm finds the centroids
of a user specified number of clusters, where each cluster inspired by the collective behaviors of ants have been proposed
groups together similar patterns. The application of the proposed by [5], [6], [7]. The main idea of these approaches is that
clustering algorithm to the problem of unsupervised classification artificial ants are used to pick up items and drop them near
and segmentation of images is investigated. To illustrate its similar items resulting in the formation of clusters. Omran
wide applicability, the proposed algorithms are then applied et al. [8] proposed the first clustering algorithm based on
to synthetic, MRI and satellite images. Experimental results
show that the BB-based clustering algorithm performs very well Particle Swarm Optimization (PSO) [9]. The results of [8], [9],
compared to other state-of-the-art clustering algorithms in all [10] show that PSO outperformed K-means and other state-of-
measured criteria. the-art clustering algorithms. The same algorithm of [8] was
Index Terms— Particle swam optimization, Barebones particle used by [11] to cluster general data sets. It was applied on a
swarm, clustering, unsupervised image classification set of multi-dimensional data (e.g. the Iris plant data base).
In general, the results show that the PSO-based clustering
algorithm performs better than the K-means algorithm, which
I. I NTRODUCTION
verify the results of [8]. Furthermore, the authors in [12] used

I MAGE clustering is the process of identifying groups of

similar image primitives [1]. These image primitives can
be pixels, regions, line elements and so on, depending on
PSO for gene clustering. Paterlini and Krink [13] compared
the performance of K-means, Genetic Algorithms (GAs) [14],
PSO and Differential Evolution [15] for a representative point
the problem encountered. Many basic image processing tech- evaluation approach to partitional clustering. The results show
niques such as quantization, segmentation and coarsening can that GAs, PSO and DE outperformed the K-means algorithm.
be viewed as different instances of the clustering problem [1]. The results also show that DE performed better than GAs and
The clustering algorithms can be categorized into two PSO. More recently, Omran et al. [16] proposed a DE-based
groups: hierarchical and partitional [2], [3]. In hierarchical clustering algorithm for unsupervised image classification with
clustering, the output is ”a tree showing a sequence of encouraging results. Recently, Kennedy [17] removes the PSO
clustering with each clustering being a partition of the data velocity equation and samples from a normal distribution
set” [3]. On the other hand, partitional clustering algorithms with promising results. The proposed approach, known as
partition the data set into a specified number of clusters. These barebones Particle Swarm (BB), requires no parameter tuning.
algorithms try to minimize certain criteria (e.g. a square error Furthermore, in [17] the author proposes another version of BB
function); therefore, they can be treated as an optimization where approximately half the time the vector element derives
problem. from the Gaussian distribution and the other half of the time
The most widely used partitional algorithm is the iterative the vector element derives from the particle’s personal best
K-means approach. The K-means algorithm starts with K experience. This version is called the exploiting barebones
cluster centers or centroids. Cluster centroids can be initialized (BBExp). In this paper, unsupervised image classification
to random values or can be derived from a priori information. approach is developed which uses BB (or BBExp). The paper
Each pixel in the image is then assigned to the closest cluster shows that the barebones particle swarm approaches have
(i.e. closest centroid). Finally, the centriods are recalculated promise in unsupervised image classification. The rest of the
according to the associated pixels. This process is repeated paper is organized as follows: An overview of PSO is given in
until convergence [4]. The K-means algorithm suffers from the section II. BB and BBExp are summarized in section III. The
following drawbacks: ” it is a greedy algorithm that depends new unsupervised image classification algorithm is presented
in section IV. Section V presents experimental results to
The authors are with the Computer Science Department at the Gulf illustrate the efficiency of the algorithm. Section VI concludes
University for Science and Technology, P.O. Box 7207, Hawally 32093,
Kuwait. http://www.gust.edu.kw, E-mail Address: [email protected] and the paper.
[email protected]

3247
1-4244-1340-0/07$25.00 2007
c IEEE
 II. PARTICLE SWARM OPTIMIZATION The position of particle i, xi , is then updated using the
following equation:
In PSO, each particle in the swarm is represented by the
following characteristics: xi (t + 1) = xi (t) + vi (t + 1) (4)
xi : The current position of the particle; The reader is referred to [18] for a study of the relationship
between the inertia weight and acceleration constants, in
vi : The current velocity of the particle; order to select values which will ensure convergent behavior.
yi : The personal best position of the particle. Velocity updates can also be clamped through a user defined
The personal best position of particle i is the best position maximum velocity, Vmax , which would prevent them from
(i.e. one resulting in the best fitness value) visited by particle exploding, thereby causing premature convergence. The PSO
i so far. Let f denote the objective function. Then the personal algorithm performs the update equations above, repeatedly,
best of a particle at time step t is updated as until a specified number of iterations have been exceeded, or
velocity updates are close to zero. The quality of particles is
⎧ measured using a fitness function which reflects the optimality
⎪
⎨ yi (t), if f (xi (t + 1)) ≥ f (yi (t)); of a particular solution.
yi (t + 1) =
⎪
⎩ xi (t + 1), if f (xi (t + 1)) < f (yi (t)).
III. BAREBONES PARTICLE SWARM
(1)
If the position of the global best particle is denoted by the BB replaces (3) and (4) with the following equation
 
vector ŷ, then yij (t) + ŷj (t)
xij (t + 1) = N , |yij (t) − ŷj (t)| (5)
2
and for BBExp, (3) and (4) are replaced with equation 6.
ŷ(t) ∈ {y0 (t), y1 (t), · · · , ys (t)} =
(2) ⎧ 
min{f (y0 (t)), f (y1 (t)), .., f (ys (t))} ⎨ N yij (t)+ŷj (t) , |yij (t) − ŷj (t)| , U (0, 1) > 0.5
2
xij (t+1) =
where s denotes the size of the swarm. ⎩ y (t), otherwise.
ij
For each iteration of a PSO algorithm, the velocity vi update (6)
step is specified for each dimension j ∈ 1, ..., Nd , where Nd
According to [17], BBExp generally outperformed other versions
is the dimension of the problem. Hence, vij represents the j th
of PSO in the benchmark functions.
element of the velocity vector of the ith particle. Thus the
velocity of particle i is updated using the following equation: IV. BB-BASED CLUSTERING ALGORITHM
This section defines the terminology used throughout the rest
of the paper. A measure is given to quantify the quality of a
vij (t + 1) = wvij (t) + c1 r1j (t)(yij (t) − xij (t)) +
clustering algorithm, after which the BB-based clustering algorithm
c2 r2j (t)(ŷ(t) − xij (t)) (3) is introduced.
Define the following symbols:
where w is the inertia weight, c1 and c2 are the acceleration • Nb denotes the number of spectral bands of the image set
constants and r1j , r2j ∼ U (0, 1) . Equation 3 consists of three • Np denotes the number of image pixels
components, namely: • Nc denotes the number of spectral classes (as provided by the
user)
• The inertia weight term, w which serves as a memory
• zp denotes the Nb components of pixel p
of previous velocities. The inertia weight controls the • mj denotes the centroid (mean) of cluster j
impact of the previous velocity: a large inertia weight
favors exploration, while a small inertia weight favors
A. Measure of Quality
exploitation.
Different measures can be used to express the quality of a
clustering algorithm. The most general measure of performance is
• The cognitive component, yi − xi which represents the the quantization error, defined as
particle’s own experience as to where the best solution is.
K 

k=1 ∀zp ∈Ck
d(zp , mk ) /nk
Je = (7)
• The social component, ŷ − xi which represents the belief K
th
of the entire swarm as to where the best solution is. where Ck is the k cluster, and nk is the number of pixels in Ck .

3248 2007 IEEE Congress on Evolutionary Computation (CEC 2007)

 Algorithm 1: The BB clustering algorithm
begin
Initialize each individual to contain K randomly selected cluster centroids
for t = 1 to tmax do
for each individual i do
for For each pattern zp do
Calculate d(zp , mi,k ) for all clusters Ci,k
Assign zp to Ci,k where
d(zp , mi,k ) = min {d(zp , mi,k )}
∀k=1,···,K

Calculate the fitness, f (xi , Zi )

Find the global best solution ŷ(t)
Update the cluster centroids using (5) (or using (6) for BBExp)
end

B. BB-Based Clustering Algorithm (79 m GSD) images. These data sets have been selected to test the
In the context of data clustering, a single individual represents algorithms, and to compare them with other algorithms, on a wide
the K cluster centroids. That is, each individual xi is constructed range of problem types, as listed below:
as xi = (mi,1 , · · · , mi,k , · · · , mi,k ) where mi,k refers to the kth • Synthetic Image: Fig. 1(a) shows a 100 × 100 8-bit gray

cluster centroid vector of the ith individual. Therefore, a population scale image created to specifically show that the BB (and
represents a number of candidate data clusterings. The quality of each consequently BBExp) algorithm does not get trapped in the local
individual is measured using: minimum. The image was created using two types of brushes,
one brighter than the other.
d¯max (Zi , xi ) + Je,i • MRI Image: Fig. 2(a) shows a 300×300 8-bit gray scale image
f (xi , Zi ) = (8)
dmin (Zi , xi ) of a human brain, intentionally chosen for its importance in
where Zi is a matrix representing the assignment of patterns to the medical image processing.
clusters of individual i. Each element zi,k,p indicates if pattern zp • Remotely Sensed Imagery Data: Fig. 3(a) shows band 4 of

belongs to cluster Ck of individual i. Also, the four-channel multispectral test image set of the Lake Tahoe
⎧ ⎫ region in the US. Each channel is comprised of a 300 × 300, 8-
⎨ ⎬ bit per pixel (remapped from the original 6 bit) image. The test
d¯max (Zi , xi ) = max d(zp , mi,k )/ni,k (9) data are one of the North American Landscape Characterization
k=1,···,K ⎩ ⎭
∀zp ∈Ci,k (NALC) Landsat multispectral scanner data sets obtained from
the U.S. Geological Survey (USGS).
is the maximum average Euclidean distance of individuals to their
associated clusters, and
A. BB and BBExp versus state-of-the-art clustering algorithms
dmin (xi ) = min d(mi,k , mi,kk ) (10)
∀k,kk,k=kk This section compares the performance of the BB and BBExp with
is the minimum Euclidean distance between any pair of clusters. In K-means, FCM [19], KHM [20], a GA clustering algorithm, a PSO
the above, ni,k is the number of patterns that belong to cluster Ci,k clustering algorithm [10] and a DE clustering algorithm [16]. The
of individual i. fitness function for GA, PSO, DE, BB and BBExp is defined in (8).
The fitness function in (8) has as objective to simultaneously In all cases, for BB, BBExp, DE, PSO and GA, 50 individuals were
minimize the intra-cluster distance between patterns and their cluster trained for 100 iterations; for the other algorithms 5000 iterations
centroids, as quantified by d¯max (Zi , xi ) and the quantization error, as were used (i.e. all algorithms have performed 5000 function evalua-
quantified by Je , and to maximize the inter-cluster distance between tions). For K-means, FCM, KHM, GA and PSO, the parameters were
any pair of clusters, as quantified by, dmin (xi ). set as in [22]. For DE, the parameters were set as in [16]. BB and
According to the definition of the fitness function, a small value BBExp require no parameter tuning. The results are summarized in
of f (xi , Zi ) suggests compact and well-separated clusters (i.e., Tables I,II, and III. These results are averages and standard deviations
good clustering). The BB clustering algorithm is summarized in over 20 simulation runs.
Algorithm 1. Table I shows the results of the experiments for the synthetic
An advantage of using BB is that a parallel search for an opti- image. The results show that BB and BBExp outperformed K-means,
mal clustering is performed. This population-based search approach FCM and KHM with respect to all the measured criteria (i.e. Je ,
reduces the effect of the initial conditions, compared to K-means, dmin and d¯max ). On the other hand, BB, BBExp, DE, PSO and
especially for relatively large population sizes. In addition, both BB GA showed similar performance, with no significant difference. The
and BBExp do not require any parameter tuning (except for the swarm segmented synthetic images resulting from both BB- and BBExp-
size). based clustering algorithms are shown in Fig. 1. The figure shows
that the segmented images generated by the two approaches are very
similar and that both approaches did not get trapped in the local
V. EXPERIMENTAL RESULTS optimum.
The BB-based clustering algorithms has been applied to three Table II shows the results for the MRI image. The results show
types of imagery data, namely synthetic, MRI and LANDSAT 5 MSS that BB and BBExp generally outperformed the other approaches in

2007 IEEE Congress on Evolutionary Computation (CEC 2007) 3249

 TABLE I
COMPARISON BETWEEN K-MEANS, FCM, KHM, GA, PSO, DE, BB AND BBE XP FOR FITNESS FUNCTION DEFINED IN (8)

Synthetic Image

Algorithm Je d¯max dmin

1 K-means 20.21225 (0.937836) 28.04049 (2.7779388) 78.4975 (7.0628718)

2 FCM 20.731920 (0.650023) 28.559214 (2.221067) 82.434116 (4.404686)

3 KHM 20.168574 (0.00) 23.362418 (0.00) 86.307593 (0.000008)

4 GA 17.342428 (0.039557) 22.25894 (0.180983) 89.771898 (0.30411)

5 PSO 17.284 (0.09) 22.457 (0.414) 90.06 (0.712)

6 DE 17.349039 (0.024415) 22.208008 (0.045002) 89.674503 (0.071472)

7 BB 17.324602 (0.026192) 22.229277 (0.036073) 89.706956 (0.029875)

8 BBExp 17.337303 (0.024775) 22.211746 (0.035281) 89.691711 (0.029087)

(a) (b) (c)

Fig. 1. (a) Synthetic image, (b) The BB Segmented Synthetic Image, and (c) The BBExp Segmented Synthetic Image

dmin and d¯max , while performing comparably with respect to Je . In R EFERENCES

addition, Table II shows that BB outperformed BBExp. Thus, for the
MRI image, BB is the best approach. The segmented MRI images [1] J. Puzicha, T. Hofmann, and J. M. Buhmann, “Histogram clustering
resulting from both BB- and BBExp-based clustering algorithms are for unsupervised image segmentation,” in IEEE Proceedings of the
Computer Vision and Pattern Recognition, 2000, vol. 2, pp. 602–608.
shown in Fig. 2.
[2] H. Frigui and R. Krishnapuram, “A robust competitive clustering
Finally, Table III shows the results for the Tahoe image. The results algorithm with applications in computer vision,” IEEE Transactions on
show that BB and BBExp generally performed better than the other Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 450–465,
approaches in dmin , while performing comparably with respect to 1999.
Je and d¯max . The segmented Tahoe images resulting from both BB- [3] J. Zhang Y. Leung and Z. Xu, “Clustering by space-space filtering,”
and BBExp-based clustering algorithms are shown in Fig. 3. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
22, no. 12, pp. 1396–1410, 2000.
Thus, BB and BBExp are the best choices to use. These results
[4] E. Forgy, “Cluster analysis of multivariate data: efficiency versus
show that the BB-based clustering algorithms are viable alternatives interpretability of classification,” Biometrics, vol. 21, pp. 768–769, 1965.
that merit further investigation. [5] E. Lumer and B. Faieta, “Diversity and adaptation in populations of
clustering ants,” in In Proceedings of the Third International Conference
on Simulation and Adaptive Behavior, 1994, pp. 501–508.
[6] B. Wu and Z. Shi, “A clustering algorithm based on swarm intelligence,”
VI. CONCLUSIONS in In Proceedings of the International Conference on Info-tech and Info-
net, 2001, pp. 58–66.
This paper presented a clustering approach using BB and BBExp. [7] N. Monmarche N. Labroche and G. Venturini, “Visual clustering based
on the chemical recognition system on ants,” in In Proceedings of the
The proposed clustering algorithm has as objective to simultaneously
European Conference on Artificial Intelligence, Lyons, France, 2002.
minimize the quantization error and intra-cluster distances, and to
[8] A. Salman M. Omran and A. Engelbrecht, “Image classification using
maximize the inter-cluster distances. The BB clustering algorithms particle swarm optimization,” in Conference on Simulated Evolution and
were compared against K-means, FCM, KHM, GA, PSO and DE. Learning, Singapore, 2002, vol. 1, pp. 370–374.
In general, the BB and BBExp algorithms performed very well [9] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in In
with reference to inter- and intra-cluster distances, while having Proceedings of IEEE International Conference on Neural Networks,
quantization errors comparable to the other algorithms. Perth, Australia, 1995, vol. 4, pp. 1942–1948.

3250 2007 IEEE Congress on Evolutionary Computation (CEC 2007)

 TABLE II
COMPARISON BETWEEN K-MEANS, FCM, KHM, GA, PSO, DE, BB AND BBE XP FOR FITNESS FUNCTION DEFINED IN (8)

MRI Image

Algorithm Je d¯max dmin

1 K-means 7.3703 (0.042809) 13.214369 (0.761599) 9.93435 (7.308529)

2 FCM 7.205987 (0.166418) 10.851742 (0.960273) 19.517755 (2.014138)

3 KHM 7.53071 (0.129073) 10.655988 (0.295526) 24.270841 (2.04944)

4 GA 7.98903 (0.33169) 10.528858 (1.060477) 27.017189 (2.69582)

5 PSO 7.839 (0.238) 9.197 (0.56) 29.45 (1.481)

6 DE 8.489362 (0.518571) 11.193335 (0.620451) 26.561583 (1.339439)

7 BB 7.722666 (0.091995) 8.940551 (0.563716) 29.969806 (0.969355)

8 BBExp 7.813303 (0.165253) 9.283251 (0.489863) 29.309536 (0.833855)

(a) (b) (c)

Fig. 2. (a) MRI Image, (b) The BB Segmented MRI Imag, and (c) The BBExp Segmented MRI Image

TABLE III
COMPARISON BETWEEN K-MEANS, FCM, KHM, GA, PSO, DE, BB AND BBE XP FOR FITNESS FUNCTION DEFINED IN (8)

Lake Tahoe Image

Algorithm Je d¯max dmin

1 K-means 3.280730(0.095188) 5.234911 (0.312988) 9.402616 (2.823284)

2 FCM 3.164670 (0.000004) 4.999294 (0.000009) 10.970607 (0.000015)

3 KHM 3.830761 (0.000001) 6.141770 (0.0) 13.768387 (0.000002)

4 GA 3.869511 (0.193642) 4.936353 (0.295057) 15.965691 (1.073189)

5 PSO 3.882 (0.274) 5.036 (0.368) 16.410 (1.231)

6 DE 4.190698 (4.190698) 5.216843 (0.321865) 16.906206 (1.089620)

7 BB 3.980572(0.192029) 5.131814 (0.266169) 17.037257 (0.861458)

8 BBExp 4.051751 (0.120513) 5.210299 (0.18998) 17.31936 (0.579856)

2007 IEEE Congress on Evolutionary Computation (CEC 2007) 3251

 (a) (b) (c)

Fig. 3. (a) Band 4 of the Landsat MSS test image of Lake Tahoe, (b) The BB Segmented of the Image, and (c) The BBExp Segmented of the Image

[10] A. Engelbrecht M. Omran and A. Salman, “Particle swarm optimization cient adaptive scheme for global optimization over continuous spaces,”
method for image clustering,” International Journal of Pattern Recog- 1995.
nition and Artificial Intelligence, vol. 19, no. 3, pp. 297–322, 2005. [16] A. Engelbrecht M. Omran and A. Salman, “Differential evolution
[11] D. Van der Merwe and A. Engelbrecht, “Data clustering using particle methods for unsupervised image classification,” in In Proceedings of the
swarm optimization,” in In Proceedings of the IEEE Congress on IEEE Congress on Evolutionary Computation (CEC2005), 2005, vol. 2,
Evolutionary Computation (CEC2003), 2003, vol. 4, pp. 215–222. pp. 966–973.
[12] R. Eberhart Z. Ben Miled X. Xiao, E. Dow and R. Oppelt, “Gene [17] J. Kennedy, “Bare bones particle swarm,” in IEEE Swarm Intelligence
clustering using self-organizing maps and particle swarm optimization,” Symposium, 2003, pp. 80–87.
in In Proceeding of Second IEEE International Workshop on High [18] F. Van den Bergh, “An analysis of particle swarm optimizers,” 2002.
Performance Computational Biology, Nice, France, 2003. [19] J. Bezdek, “A convergence theorem for the fuzzy isodata clustering
[13] S. Paterlini and T. Krink, “Differential evolution and particle swarm algorithms,” IEEE Transactions on Pattern Analysis and Machine
optimization in partitional clustering,” Computational Statistics and Intelligence, vol. 2, pp. 1–8, 1980.
Data Analysis, vol. 50, no. 5, pp. 1220–1247, 2005. [20] B. Zhang, “Generalized k-harmonic means - boosting in unsupervised
[14] D. Goldberg, Genetic algorithms in search, optimization and machine learning,” 2000.
learning, Addison-Wesley, 1989.
[15] R. Storn. and P. Kenneth P, “Differential evolution - a simple and effi-

3252 2007 IEEE Congress on Evolutionary Computation (CEC 2007)