Fast Image Segmentation Based on K-Means

Clustering with Histograms in HSV Color Space

Tse-Wei Chen 1 , Yi-Ling Chen 2 , Shao-Yi Chien 3
Media IC and System Lab
Graduate Institute of Electronics Engineering and Department of Electrical Engineering
National Taiwan University
BL-421, No. 1, Sec. 4, Roosevelt Rd., Taipei 106, Taiwan
{1 variant, 2
yipaul, 3
sychien}@media.ee.ntu.edu.tw

Input Color Image

Abstract— A fast and efficient approach for color image seg- (RGB)
K-Means Clustering
mentation is proposed. In this work, a new quantization technique in HSV Color Space
for HSV color space is implemented to generate a color histogram
and a gray histogram for K-Means clustering, which operates Color Space Transform Maximin Initialization
across different dimensions in HSV color space. Compared with
the traditional K-Means clustering, the initialization of centroids
and the number of cluster are automatically estimated in the Histogram Generation Parameter Estimation
proposed method. In addition, a filter for post-processing is
introduced to effectively eliminate small spatial regions. Experi- Post-Processing of Iteration of
ments show that the proposed segmentation algorithm achieves Spatial Regions K-Means Clustering
high computational speed, and salient regions of images can be
effectively extracted. Moreover, the segmentation results are close Output Segmentation
Pixel Labeling
to human perceptions. Result

I. I NTRODUCTION
Fig. 1. Overview of the proposed algorithm.
Color image segmentation [1], whose purpose is to decom-
pose an image into meaningful partitions, is widely applied
in multimedia analysis. Depending on different applications,
various kinds of techniques, including feature-space methods coding standards, they are not close to human perceptions.
and spatial-domain methods, are used for color image seg- Besides, CIE color spaces are perceptually uniform but in-
mentation. Feature-space methods, such as clustering, intend to convenient since they require complicated computations. HSV
classify pixels to different groups in a pre-defined color space, (Hue, Saturation, Value) [6], which is shown to have better
whereas spatial-domain methods, such as regions growing, ma- results for image segmentation than RGB color space [5],
nipulate pixels to form connected regions. Since feature-based [7], is capable of emphasizing human visual perception in
methods and spatial-domain methods both have their strengths, hues and has an easily invertible transform from RGB [8].
algorithms which combine two methods are developed [2], Nevertheless, to our best knowledge, clustering in HSV color
[3]. Among them, K-Means clustering is often applied as an space is still an open question since there is no appropriate
essential step in the segmentation process. criteria for separation or distance measurement of gray and
As a traditional clustering algorithm, K-Means is popular color pixels. Based on these observations, a new method that
for its simplicity for implementation, and it is commonly combines gray and color histogram bins for clustering in HSV
applied for grouping pixels in images or video sequences. color space is developed in this paper.
However, the quality of K-Means suffers from being confined In brief, the proposed technique has three main advantages
to run with a fixed value of K rather than dynamically for the segmentation process: one is to reduce the computation
adjusted value of K [4]. There are solutions that run K- complexity of K-Means by using histogram quantization in
Means many times to find a suitable number of K [2], [5], HSV color space, another is to efficiently estimate the number
but it is time-consuming. In addition, random initialization of cluster of K-Means without testing each value of K; the
of centroids (cluster centers) makes the results different each other is to consider gray and color histogram bins together for
time. Therefore, a fast and efficient algorithm for K-Means the clustering procedure. Furthermore, the algorithm integrates
image segmentation is proposed to handle these problems. a filter to efficiently alleviate over-segmentation, and the
On the other hand, choices of color space may have sig- results of segmentation are close to human perception. Due
nificant influences on the result of image segmentation. There to these advantages, it is not only useful for processing large
are many kinds of color space, including RGB, YCbCr, YUV, amount of image data but also suitable for embedded systems
HSV, CIE L*a*b*, and CIE L*u*v*. Although RGB, YCbCr, which have few computational resources.
and YUV color spaces are commonly used in raw data and The paper is organized as follows. The proposed algorithm

978-1-4244-2295-1/08/$25.00 © 2008 IEEE 322 MMSP 2008

 TABLE I
H
C ORRESPONDENCE OF HISTOGRAM BINS AND HSV INDICES .
Color Histogram
(1470 Bins)
Histogram bin Range of parameter Correspondence of index
G(v) v = 0 v = v S Histogram Generation
or (Quantization)
Gray Histogram
v ∈ [1, 7] and s = 0 (8 Bins)
B(h, s, v) h ∈ [0, 29] h = h V

s ∈ [1, 7] s = s
v ∈ [1, 7] v = v
Fig. 2. Histogram generation process in HSV color space.

is first described in Sec. II. Next, in Sec. III, the experimental briefly stated as follows:
results will be shown. Finally, a short conclusion is given in Step A: From the color histogram bins and gray histogram
Sec. IV. bins, find the bin which has the maximum number of pixels
to be the first centroid.
II. P ROPOSED A LGORITHM Step B: For each remaining histogram bin, calculate the min
An overview of the proposed algorithm can be illustrated in distance, which is the distance between it and its nearest
Fig. 1, the steps in which will be explained in the following centroid. Then the bin which has the maximum value of min
sections. distance is chosen as the next centroid.
Step C: Repeat the process until the number of centroid equals
A. Color Space Transform and Histogram Generation
to KMax or the maximum value of the distance in Step B
HSV color space is chosen for the proposed algorithm, and is smaller than a predefined threshold T hM . The threshold
the transform from RGB to HSV in [8] is adopted. Since the KMax = 10 is set based on the assumption that there should
ranges of three dimensions in HSV color space are not the be no more than 10 dominant color in one image for high-
same (Hue: [0, 360◦], saturation: [0, 1], and value: [0, 1]), a level image segmentation, and T hM = 25 is set empirically
quantization process is performed to normalize the values in according to the human perceptions of different color in HSV
each dimension: color space. The distance measurements of histogram bins and
h = Hue/hQ  , (1) centroid vectors will be explained in Sec. II-C.
s = Saturation/sQ  , (2) C. K-Means Clustering in HSV Color Space

v = Value/vQ  , (3) The proposed K-Means clustering in HSV color space
   includes five steps, which are listed as follows:
where (h , s , v ) denotes the quantization index, and the
Step 1: Estimate the parameters of K-Means, including suit-
quantization parameters hQ = 12◦ , sQ = 0.125, vQ = 0.125
able value of K and the position of K initial centroids from
are set empirically to accentuate the importance of hue and
the Maximin algorithm in Sec. II-B.
to save computational costs. Therefore, the HSV color space
Step 2: Two kinds of histogram bins will be clustered together
are divided into 30 × 8 × 8 = 1920 partitions. However, the
in this step. For color histogram bins, since the hue dimension
hue of pixels with low saturation is often meaningless since
is circular (e.g. 0◦ = 360◦ ), the numerical boundary should
their color is close to gray, and it is suggested that color
be considered in the distance measurement and the process
histogram bins and gray histogram bins should be separated
of centroid calculations. The distance measurement between a
for better color representation [5], [9]. G(v) represents the
histogram bin vector Bi = (hi , si , vi ) and a cluster centroid
number of pixel in the gray histogram bin with parameter v, (t) (t) (t) (t)
vector Cj = (hj , sj , vj ) in the current iteration t is
whereas B(h, s, v) represents the number of pixel in the color
defined in the form of the Euclidean distance (2-Norm):
histogram bin with parameters (h, s, v). The correspondence
(t) (t) (t) (t)
of these parameters and the quantization indices (h , s , v  ) are D2 (Bi , Cj ) = Dh2 (hi , hj ) + (si − sj )2 + (vi − vj )2 ,
summarized in Table I. Totally NG gray histogram bins and (4)
NB color histogram bins are generated, where NG = 8 and
where
NB = 30×7×7 = 1470. The process of histogram generation  ◦ (t) (t) 180◦
and quantization is illustrated in Fig. 2. (t) ( 360 2
hQ − |hi − hj |) , if |hi − hj | >
Dh2 (hi , hj ) = (t)
hQ
,
B. Maximin Initialization and Parameter Estimation (hi − hj )2 , otherwise.
(5)
In traditional K-Means algorithm, cluster number K is often
specified by users, and the initial centroid positions are chosen Next, classify each color histogram bin to its nearest cluster
randomly. In the proposed method, the parameters, including centroid by the distance measurement. Thus the membership
cluster number and the initial positions of centroids, are all function of histogram bin Bi is defined by
 (t)
estimated though the Maximin algorithm [3] in a systematic 1, if j = arg min D2 (Bi , Ck )
(t)
approach. Three steps of the proposed Maximin algorithm, φ(j|Bi ) = k . (6)
which are used in the preliminary stage for K-Means, are 0, otherwise.

323
On the other hand, for gray histogram bins, there is no hue
information inside. Thus the distance measurement between a
gray histogram bin Gi = (vi ) and a cluster centroid vector
(t) (t) (t) (t)
Cj = (hj , sj , vj ) is different from (4):
(t) (t) (t)
(a) (b) (c)
D2 (Gi , Cj ) = (sj )2 + (vi − vj )2 , (7)
Fig. 3. Image kodim23: (a) labeled image, (b) filtered image, and (c)
which means that the saturation values of gray histogram bins segmentation result.
are all considered as zero, and the hue values can be arbitrary.
Besides, the membership function of the gray histogram bin
Gi is of the same form as (6), where Bi is replaced by Gi .
Step 3: Recalculate and update K cluster centroids. Again,
since the hue dimension is circular, the indices in the hue
dimension should be considered not absolutely but relatively. (a) (b) (c)
An efficient method is introduced to calculate the relative hue
(t)
index of the original hue index hi to the centroid Cj = Fig. 4. Image kodim06: (a) the original image, (b) the clustering result of
(t) (t) (t) the proposed method , and (c) the clustering result in [5], where color and
(hj , sj , vj ): gray pixels are separated deterministically and marked.
⎧ ◦ (t) ◦ (t) ◦
⎪ 360 180 180
⎨ hi − hQ , if |hi − hj | > hQ and hj < hQ
(t) ◦ (t) 180◦ (t) 180◦ ,
h̃i,j = hi + 360
⎪ hQ , if |hi − hj | > hQ and hj > hQ When the difference of total distortion measurement |Δ(t+1) −
⎩
hi , otherwise. Δ(t) | is smaller than a predefined threshold or when the max-
(8) imum iteration number is reached, the iteration is terminated.
and the values in each dimension of all centroid vectors for the Otherwise, return to Step 2 and t is incremented.
(t+1)
next iteration Cj are updated according to the following Step 5: Image pixels are labeled with the index of nearest
equations: centroid of their corresponding histogram bins. A labeled
image l(x, y) is obtained in this step, and K-Means clustering

N B
(t) (t) is finished. An example is shown in Fig. 3(a).
h̃ φ B(Bi )i,j (j|Bi )
(t+1) i=1
hj = , (9) D. Post-Processing of Spatial Regions

N B
(t)
φ(j|Bi ) B(Bi ) To eliminate noise and unnecessary details of labeled
i=1
images, an efficient statistical filter is introduced. A local

N B
(t) histogram for a pixel on the position (x, y) in the labeled
si φ(j|Bi ) B(Bi ) image is defined as
(t+1) i=1
sj = , (10) 
B
N
(t) G
N
(t) H(z|x, y) = 1, (13)
φ(j|Bi ) B(Bi ) + φ(j|Gi ) G(Gi )
i=1 i=1 l(x ,y  )∈W(x,y)
l(x ,y  )=z

N B
(t) G
N
(t)
vi φ(j|Bi ) B(Bi ) + vi φ(j|Gi ) G(Gi ) where W(x, y) is an N × N window centered in the spatial
(t+1)
vj = i=1 i=1
, (11) coordinate (x, y). Then the processed labeled image ˆ
l(x, y) is

N B
(t) G
N
(t) obtained by using the following equation:
φ(j|Bi ) B(Bi ) + φ(j|Gi ) G(Gi )
i=1 i=1 ˆl(x, y) = arg max H(z|x, y). (14)
z
where B(Bi ) denotes the number of pixels in the color
histogram bin with histogram bin vector Bi , and G(Gi ) The purpose of this filter is to replace the pixel in the labeled
denotes the number of pixels in the color histogram bin with image with the label with maximum number in a window.
histogram bin vector Gi . Note that the range of hue value Afterwards, the spatial regions (8-connected) whose area is
of the new centroid should be normalized in the range of smaller than T hA are merged to the neighboring region with
[0, 360◦/hQ ). the maximum area to avoid over-segmentation. T hA is set to
Step 4: Check if the clustering process is converged according be 0.05% of the total number of pixels. Two examples in this
to the total distortion measurement, which is the summation of stage are shown in Fig. 3(b)(c).
distances between each histogram bin and its nearest cluster
centroid: III. E XPERIMENTAL R ESULTS

N B K The experiments, which contain four parts, are performed
(t) (t)
Δ(t) = φ(j|Bi ) D2 (Bi , Cj )B(Bi ) in Pentium-D 2.66GHz computer with 2GB memory. The first
i=1 j=1
. (12) part is algorithm comparison. For 25 images of size 768×512,
G 
N K
(t) 2 (t)
+ φ(j|Gi ) D (Gi , Cj )G(Gi ) the average execution time of the proposed method is 0.29
i=1 j=1 second, and the method in [5] requires 1.20 second. Also, the

324
 200 10000

Number of Region
Distortion (x10 )
4
Random kodim01
kodim03
Autonomous kodim07
1000 kodim22
100
100

0 10
1 6 11 16 21 26 0 2 4 6 8 10
Iteration Filter Window Size (a) (b) (c)
(a) (b)

Fig. 5. (a) Total distortion v.s. iteration of K-Means with image kodim03.
(b) Region number v.s. filter window size with four images.

(d) (e) (f)

clustering result of these two methods are shown in Fig. 4,
where the method in [5] separates color and gray pixels in
HSV color space deterministically, and the proposed method
clusters color and gray pixels together. It can be seen that the
proposed method obtains better results than the method in [5] (g) (h) (i)
in terms of speed and robustness.
The second part is the functionality testing of the proposed
K-Means clustering. To ensure K-Means operates properly in
HSV color space with gray histogram bins and color histogram
bins, the total distortion measurement in (12) is plotted with
(j) (k) (l)
respect to iteration times. It is found that for all testing images,
the distortion declines as the iteration time increases. Also, Fig. 6. Original images: (a) kodim01, (b) kodim02, (c) kodim03, (g) kodim07,
the Maximin initialization can obtain distortions similar to the (h) kodim22, (i) kodim23. Segmentation results: (d) kodim01 (K = 2), (e)
random initialization: A plot of image kodim03 is shown in kodim02 (K = 2), (f) kodim03 (K = 7), (j) kodim07 (K = 4), (k) kodim22
(K = 6), (l) kodim23 (K = 7).
Fig. 5(a), where the distortions of random initialization are
average values in ten times.
The third part is the performance analysis for post- texture information will be integrated into pixel features for
processing. To test if the proposed filter can effectively reduce clustering, and the segmented images will be used for region-
the region number of the labeled image, the relation of region based image retrieval.
(8-connected) number and filter window size with four testing
images are plotted in Fig. 5(b). It can be seen that the region R EFERENCES
number declines dramatically as the window size N increases. [1] L. Lucchese and S. Mitray, “Color image segmentation: A state-of-the-
The final part focuses on segmentation results and human art survey,” in Proceedings of the Indian National Science Academy, Mar.
2001, pp. 207–221, (Invited Paper).
perceptions. Six testing images are shown in Fig. 6, where the [2] M. Luo, Y.-F. Ma, and H.-J. Zhang, “A spatial constrained K-Means
cluster number K, automatically selected for each image, is approach to image segmentation,” in Proceedings of the Joint Conference
listed below. Additionally, the window size of filter for post- of International Conference on Information, Communications and Signal
Processing, and Pacific Rim Conference on Multimedia, vol. 2, Dec. 2003,
processing is set to N = 7. In Fig. 6(a)(d), the doors and pp. 738–742.
the wall are segmented well; in Fig. 6(b)(e), the door knob [3] V. Mezaris, I. Kompatsiaris, and M. G. Strintzis, “Still image segmen-
and the background are distinguished; in Fig. 6(c)(f), five hats tation tools for content-based multimedia applications,” International
Journal of pattern recognition and artificial intelligence, vol. 18, no. 4,
with different color are extracted; in Fig. 6(g)(j), the gray wall pp. 701–725, Jun. 2004.
can be separated from the gray window; in Fig. 6(h)(k), the [4] T. Elomaa and H. Koivistoinen, “On autonomous K-Means clustering,”
boundary of the sky and the house is clear; in Fig. 6(j)(l), in Proceedings of 15th International Symposium on Methodologies for
Intelligent Systems, May 2005, pp. 228–236.
colorful segments of parrots are partitioned. It can be seen that [5] S. Sural, G. Qian, and S. Pramanik, “Segmentation and histogram
the segmentation results are similar to high-level perceptions generation using the HSV color space for image retrieval,” in Proceedings
of human visions, and the number of cluster is suitable for of IEEE International Conference on Image Processing, Sep. 2002, pp.
589–592.
color representation in these testing images. [6] J. R. Smith, “Color for image retrieval,” in Image Databases. John Wiley
& Sons, Inc., 2002, ch. 11, pp. 285–311.
IV. C ONCLUSIONS AND F UTURE W ORK [7] Z.-K. Huang and D.-H. Liu, “Segmentation of color image using EM
The characteristics of the proposed image segmentation algorithm in HSV color space,” in Proceedings of IEEE International
Conference on Information Acquisition, Jul. 2007, pp. 316–319.
algorithm include a new quantization method to generate gray [8] W. Chen, Y. Q. Shi, and G. Xuan, “Identifying computer grahics using
and color histograms in HSV color space for K-Means cluster- HSV color model and statistical moments of characteristic functions,” in
ing, where the cluster number is automatically estimated based Proceedings of IEEE International Conference on Multimedia and Expo,
Jul. 2007, pp. 1123–1126.
on Maximin initialization. Moreover, the proposed method [9] J. Smith and S.-F. Chang, “Single color extraction and image query,” in
is both fast and efficient to extract regions with different Proceedings of IEEE International Conference on Image Processing, Oct.
colors in images, and the segmentation results are close 1995, pp. 528–531.
to human perceptions. For future developments, spatial and

325