Fifth International Conference on Fuzzy Systems and Knowledge Discovery

A Region-based Image Segmentation Method with Mean-Shift Clustering

Algorithm

Yong-mei Zhou1, 2 Sheng-yi Jiang1 Mei-lin Yin3

1
School of Informatics, Guangdong University of Foreign Studies, Guangzhou 510006
2
School Of Computer Science&Engineering, South China University of Technology,
Guangzhou, 510641
3
Educational Technology Center, Guangdong University of Foreign Studies, Guangzhou 510006
E-mail: [email protected]

Abstract features from each pixel to form feature vector by

selecting suitable color space. Then, these feature
A method of region-based image segmentation with vectors are clustering with mean-shift clustering
mean-shift clustering algorithm is introduced. This algorithm and the window parameter r is decided by
method first extracts color, texture, and location the proposed method of selecting optimal clustering
features from each pixel to form feature vector by amount, so the numbers and the centers of clusters are
selecting suitable color space. Then, these feature also selected, and each pixel is grouped and labeled.
vectors are clustering with mean-shift Clustering Finally, the regions with the same label are segmented
algorithm and the window parameter r is decided by again according to the neighbor connection theory for
the proposed method of selecting optimal clustering pixels and a lot of the features which describe the
amount, so the numbers and the centers of clusters are regions are provided. The paper also gives the
also selected, and each pixel is grouped and labeled. experiment results analysis. Figure 1 is the flow chart
Finally, the regions with the same label are segmented of region-based image segmentation.
again according to the neighbor connection theory for o rig ina l im a ge

pixels and a lot of the features which describe the ex tract p ixe l
regions are provided. Experiment results show this featu res

method can segment images quickly and has good c luster b ased o n
p ix el featu res
segmentation results.
lab el im a ge
reg ion s
1. Introduction
d esc ribe re gio n s
Image segmentation is the chief problem of image Figure 1 The Process chart of Image
analysis and recognition, and one of the classical Segmentation
difficult problems for image process, which decides the 2 Feature extraction for image pixels
discrimination effects of the final analysis and pattern
recognition. Image segmentation divides images into
Image texture has local-neighborhood and space
different regions with special meanings and makes
property,which replies on the neighborhood scale in
them not to overlap, moreover, each region should reality.Therefore, the scale selection is related with the
satisfy the coherence with given regions. Once
description precision for image texture.Texture
segmentation, we can do CBIR(Content-Based Image
features extracted in this paper is contrast, polarity,
Retrieval), classification and recognition and so on. So,
anisotropy for each pixel, using the method in
image segmentation is an important field for image literature[5]. According to the L component in L*a*b*
process and pattern recognition[1,2,3].
space, we define the different scales to be the width of
This paper introduces a method of region-based
the Gaussian window to analyze polarity for each
image segmentation with mean-shift clustering
pixel, and get the optimal window scale parameters,
algorithm, using mean shift theory[4]. The basic idea of then compute contrast and anisotropy for each pixel
this method is: first extract color, texture, and location
with them.

978-0-7695-3305-6/08 $25.00 © 2008 IEEE 366

DOI 10.1109/FSKD.2008.363
 In regions with 2D texture or 1D flow, pσ decays
2.1. Extracting texture features for pixels with σ: as the window size increases, pixels with
gradients in multiple directions are included in the
The polarity is a measure of the extent to which the window, so the dominance of any one orientation
gradient vectors in a certain neighborhood all point in decreases.
the same direction, whose definition is as Definition 1. The region is uniform and the lighteness of it is also
Definition 1 We define polarity pσ is as Eq.(1). invariable by pσ holding an invariable value for all σ.
C − C− Via the above computation, we decide the
pσ = + (1) window parameter σh and compute polarity for each
C + + C−
pixel. The following is to confirm σh and compute
The definitions of C+，C- are as Eq.(2). contrast(denoted by con), anisotropy(denoted by ani),
G G ⎡G x ⎤
C+ = ∑ wσ ( x , y ) ⎡⎣ ∇ G ⋅ n ⎤⎦ , C− = ∑ wσ ( x , y ) ⎡⎣ ∇ G ⋅ n ⎤⎦ , ∇G = ⎢G ⎥
which are computed by the second moment
⎣ y⎦ matrix(SMM) [5]. SMM for a pixel of an image is as
+ −
x , y∈ Ω x , y ∈Ω

(2) Eq.(5).
Where, Gx,Gy are the gradient of L component in T

direction x and y in the L*a*b* space, respectively. SMM σ（' x, y ) = wσ ' ( x, y ) * (∇G )(∇G ) (5)
G Where, SMMσ'(x,y) represents the second moment
n is a unit vector perpendicular to φ, φ ranges from 0 matrix for pixel location (x,y), and it’s a symmetric
to 180°, for computation conveniently φ is 22.5°*i(i=0 positive semidefinite matrix. Other signs are the same
，1,…,7), and φ is corresponding to the gradient vector with that in Eq.(2). Let λ1 and λ2(λ1≥λ2) denote the
for each pixel. [·]+,[·]- are the rectified positive and values of the principal eigenvector of SMMσ'(x,y) by
negative parts of their argument. Ω is the gradient Eq.(6) and Eq.(7).
vector(also called gradient pool) corresponding to
Gaussian window(the width of Gaussian function is σ), con = 2 λ1 + λ2 (6) ani = 1 − λ2 λ1 (7)
wσ(x,y) is the value of Gaussian function when each
gradient vector in gradient pool with width σ. C+,C- 2.2. Color and space features extraction
can be thought of as measures of how many gradient
vectors in the window wσ (x,y) are on the “positive Let the number of pixels be m× n, pixel ponit
side” and “negative side” of the dominant orientation, (x,y) location feature X,Y can be represented as:
respectively. pσ∈[0,1] changes with σ. X=wxy*(x/max(m,n)),Y= wxy*(y/max(m,n)), where,
According to Definition 1, we must decide max(.) is the maximum, wxy ranges from 0 to 1, the
Gaussian parameter σ before computing pσ, and here weight of the space location feature, in this paper
we use the following method. wxy=1. Color features are labL,labA,lab as Eq.(8).
(1)Gaussian window parameter generation with the labL=L/100,labA=(a+50)/100,labB=(b+50)/100
following way (8)
Window size=[h2+1]×[h2+1], h=0,1,…,H, where H Where, L,a,b are the original components in
is decided by Eq.(3). L*a*b* space. We apply Gaussian smoothing to the
three features to get pixel features. For the above
H = ⎢⎣ min(m, n) 2 − 1 ⎥⎦ , and H ≤ 10 (3) analysis and computation, pixel feature vector of an
image is denoted as Ii(i=1,…,n*m), shown by formula
Where, ⎣⋅⎦ denotes the floor integer, m×n denotes the (9).
number of image pixels. While, assign σ=σh=h/2,
Ii (labL,labA,labB,pσ,con,ani,X,Y) (9)
h=0,1,…,H, then, Gaussian window parameters are
In formula(9), each component of Ii is [0,1].
got.
(2) According to the computation Equation of the
polarity pσ in Definition 1, compute p'σh(x,y) for each 3.Region-based image segmentation with
pixel(x,y) when σ=2*σh, h=0,1,…,H. mean-shift clustering algorithm
(3) Select σ according to p'σh(x,y), and the selection
rule is: for h=0,1,..,H, Eq.(4) must be satisfied at first. In section 2, pixel point feature vector is {Ii}i=1…N
p'σ(h+1)-p'σh≤2% (4) (N=n*m). For region-based image segmentation,
Then window parameter σ=h/2. firstly, group image based on the pixel features, using
Selection rule is determined according to the mean shift cluster algorithm, then, label the regions.
characteristics of pσ and σ: During the clustering, the method with the average
shattering(AS) in clusters and the average
The presence of an edge is signaled by pσ holding
division(AD) among clusters is proposed to decide the
values close to 1 for all σ.

367
optimal cluster amount. So, the image segmentation Input: pixel feature vector set {Ii}i=1…N(Ii Rd),
method in this paper is called MSC-SD method. window parameter r, ISna={Ii}i=1…n, ISa=φ,ε(0<ε<1),
K=0.
3.1.Pixel feature-based with mean shift Output: the number of clusters K, cluster centers
clustering algorithm CCk(k=1,…,K) and cluster label for pixel points.
Algorithm:
Let {Ii}i=1…N be the feature vector set for an image, Step1: Select a point Is as the start point for the first
with N pixels, d(Ii ∈ Rd) denotes the feature vectors’ clustering from ISna at random.
dimension in Euclidean space(d=8). In literature[4], Step2: Mean Shift
compute kernel density estimate for pixel point I(I∈ Step2-1: Construct hypersphere Sr(Is) of radius r,
Rd) as Eq.(10). centered on Is, compute the data points in it, and vote
for these points to put them into Isa.
∑K( )
N

fˆ ( I ) = Step2-2: Compute Mr(Is) and || Mr(Is)|| according to

I − Ii
1
d r
(10)
Nr
i =1 Eq.(15).
Step2-3: If || Mr(Is)||< ε, cluster shift is convergent, go
In Eq.(10) fˆ ( I ) denotes density estimate for
to Step3, otherwise, Is=Is+ Mr(Is) and go to Step2-1.
pixel I, r denotes window radius, K(I) is kernel Step3: Construct and Unite Clusters
function and here as Eq.(11). Step3-1: Construct a new cluster when one of the
K (I ) = { 1 v −1 ( d + 2)(1− I T I )
2 d
0
if IT I <1
others (11) following conditions is satisfied.
(1) K=0(the first cluter).
Where, v is the volume of unit hypersphere in the (2) K>0,there is no CCk(k=1,…,K) to make ||Is-
d-dimensional space. CCk||<r/2 come to existence.
Eq.(12) is to compute gradient of kernel density If satisfy the conditions, then do the following:
estimate according to Eq.(10). K=K+1,CCK=Is, record the ballot for pixels
N during window shift.
ˆ f ( I ) ≡ ∇fˆ ( I ) =
∑ ∇K (
I − Ii
∇ 1
d r
) Step3-2: K>0,there is CCg, g=arg min||Is-CCk||
Nr
i =1 (k=1,…, K) to make ||Is-CCg||<r/2 come to existence,
= 1
d
N ( r vd )
d +2
r
2 ∑ (Ii − I ) (12) and unite with the g-th cluster, then do the following:
CCg=(CCg+Is), accumulate the ballot of pixels.
I i ∈S r ( I i )

⎛1 ⎞ Step4: ISna=ISna-ISa, if ISna≠φ, go to Step1,

⎜ N ∑ (Ii − I ) ⎟
NI
= d
d +2
2 otherwise, go to Step5.
⎝ I ∈S ( I ) ⎠
N ( r vd ) r I

i r i Step5: Put pixels into the cluster with most ballots.

Where the region S r ( I ) is a hypersphere of radius Step6:Output K,CCk(k=1,…,K) and the sequence
number of each cluster.The algorithm ends.
r centered on I in the d-dimensional space, containing
NI data points. Define sample mean shift vector Mr(I)
3.2. Selection the optimal cluster number
as Eq.(13).
M r ( I ) = N1 ∑ ( I i − I ) = N1 ∑ I i − I (13) Section 3.1 introduces the cluster algorithm with
I I
I i ∈S r ( I ) I i ∈S r ( I )
mean shift under fixed r to cluster for obtaining cluster
is kernel density estimate fˆ ( I ) parameters.In this paper, the method for deciding r in
NI
Where,
N ( r d vd ) region-based image segmentation is proposed as
computed with the hypersphere Sr ( I ) , and thus, we follows. At first, determine the possible range for r,
denoted as [a,b] (0<a<b), select a step parameter l,
rewrite Eq.(13) as Eq.(14).
which are in effect under these conditions, such as
ˆ f ( I ) = fˆ ( I ) d + 2 M ( I )
∇ (14) a 0.2,b 0.6,l=0.1 to 0.2.In our research,a=0.2,b=0.6,
r r 2

Which yields Eq.(15). l=0.1. Therefore, r has 5 value from 0.2 to 0.6 during
2 ˆ f (I )
∇
one clustering process. Then, all the clustering
M r (I ) = r
d + 2 fˆ ( I )
(15) parameters for r can be obtained by mean shift cluster
The following is the clustering algorithm with const algorithm. Finally, determine the cluster number K
r. with the optimal r determination method, which uses
Assume the signs: ISa and ISna are the begirt set and average shattering(AS) in clusters and average
unbegirt set for pixels, respectively. K is the clustering division(AD) among clusters, where AS and AD are
number, CCk(k=1,..,K) is the cluster centers.The mean defined as Eq.(16) and Eq.(17).
shift clustering algorithm is as follow.

368
 NW
1 1 Nk
1 NW
1
∑
K

AS =
K
∑
Nk
∑ I i − CC k (16) con =
nW
∑ I i (con) , ani =
NW
I i ( ani )
k =1 i =1 i =1, I i ∈W i =1, I i ∈W

1 K
1 K (19)
AD = ∑ ∑ CCk − CCk ' (17) Where, NW is the number of pixel points for region
K k =1 K − 1 k ', k ' ≠ k W, Ii(con),Ii(ani) are contrast and anisotropy for the i-
Where, Nk is the pixel point number of the k-th th pixel feature components. Texture features are
cluster. The average shattering in clusters is the quantized by Eq.(20), called bins(texture bins) [5],
difference among pixels, the smaller the shattering, the denoted by texbin.
more similar the pixels. While the average division is
texbin = (cbin − 1) * m0 + abin
the distance among cluster centers, the larger the
division, the more different the clusters[6]. According to cbin = ⎡⎢ con * m1 ⎤⎥ (if (con = 0) cbin = 1)
k
the above description, the rule of K selection is the 1

small S and the large D. So, we use RK=SK/DK to select abin = ⎡⎢ * m2 ⎤⎥ (if (ani = 0) abin = 1)
ani
k2
(20)
K, that is, select the smallest R for K to determine r as
Eq.(18). Where, ⎡⋅⎤ is the ceiling of the integer, here
(18) m0=3,m1=5, m2=3,k1=0.5,k2=0.7.
K = arg min SK D
K
K (2)Space Location Feature of Region
4.Description for Image Region In 2D vision space, the row direction( X ) and the
col direction( Y ) of region pixel location are computed
Pixel points are initially grouped by mean shift as Eq.(21).
NW NW
algorithm, and grouping is to label regions in pixel
feature space, but the regions with the same label may
X = NW
1
∑ I i ( X ), Y = 1
NW ∑ I (Y ) i
(21)
i , Ii i , Ii
be in different neighborhood in 2D, as Figure 2(b)
shows the region 2.So, it’s necessary to relabel the Where, the symbols are the same with that in
regions as Figure 3(c). In addition, during image Eq.(19). The two feature vectors are quantized as one
segmentation, the boundaries of regions are disposed vector by Eq.(22), called location bins, denoted by
by morphologic method(dilate and erode), and Figure locbin.
2(d) is segmentation results for an image. locbin = ( ybin − 1) * m0 + xbin
xbin = ⎡⎢ X * m1 ⎤⎥ (if ( X ==0) xbin = 1)
ybin = ⎡⎢Y * m2 ⎤⎥ (if ( Y ==0) ybin = 1) (22)
In formula (22),m0=m1=m2=3.
(3)Color Feature for Region
Color features for each region are the average color
for each pixel in L*a*b space, and computed as
Eq.(23).
1 N
∑ I i (labL)
W

labL =
NW i =1, I ∈W i

Figure 2 The Example of Image Label and 1 NW

Segmentation labA =
NW
∑ I i (labA) (23)
Some experiment data in Figure 2 are as follows: i =1, I i ∈W
NW
the window parameters for clustering algorithm are 1
r=0.3, K=5, S/D=0.3928, and the processing time is
labB =
NW i =1, I ∈W
I i (labB) ∑
1.62 seconds, and the regions are 5, using Matlab7.0.
i

Where, the signs are the same with that in

Description for regions plays an important role in
Eq.(19).
image segmentation, and this paper adopts the
(4)Shape Descriptors for Region
following method, in which let W be a region.
This paper adopts ellipse Fourier method[6] to
(1)Texture Feature of Region
describe the region features. Firstly, get the region
Average contrast( con ) and average anisotropy boundary pixels and boundary contour and boundary
( ani ) are computed as Eq.(19) for region W. function by smoothing. Then, we use several ellipses to
denote region boundary contour, and treat Fourier

369
coefficients as region shape feature with Fourier 12
transformation. Each ellipse is denoted by 4 Fourier 10 EM-MDL

coefficients. This paper describes the region shape with 8

MSC-SD

Fourier average coefficients as F1 and F2.

t(s)
6

4
5.Experiment Results and Analysis 2
image
The paper proposes the region-based image 0
0 1 2 3 4
segmentation method(MSC-SD), under windows xp
with Matlab7.0. We validate the MSC-SD efficiency in Figure 4 The time curve for two method
two sides, that is, the experiment effect of MSC-SD sof image segmentation
method and comparesion with that in literature [3],
where, the method in literature [3] is a fine CBIR
6. Conclusions
system, and its image segmentation adopts EM and
MDL theory, called EM-MDL in this paper.
This paper introduces a method of region-based
Figure 3 shows the segmentation effects using the
image segmentation with mean-shift clustering
MSC-SD in this paper and EM-MDL in literature [3].
algorithm and realizes the image segmentation method
From the figure, we can conclude that the effect of
using computing program.The experiment results and
MSC-SD is better.
analysis and the comparison with other methods
validate that this method is more valuable with fast
speed and good effect.

Acknowledgements
δ

This work was Supported by the National Natural

Science Foundation of China (60673191) and the
Natural Science Foundation of Guangdong Province of
China(7300450)

Reference
[1] LIN Kai-yan,WU Jun-hui,XU Li-hong. A Survey on
Figure 3 The Effect and Comparison for Color Image Segmentation Techniques. Journal of Image
Image Segmentation and Graphics,2005,10(1):1-10.
[2] M.Ameer Ali, Laurence S Dooley ,Gour C Karmakar.
Table.1 is the features for tiger regions in Figure Fuzzy Image Segmentation using Location and Intensity.
3(d) with the method proposed in this paper. In:Third IASTED International Conference on
Visualization, Imaging and Processing 2003: 399-404.
Table.1 The Features of Tiger Region in [3] Chad Carson, Serge Belongie, Hayit Greenspan et
Figure 3(d) al.Blobworld: Image Segmentation Using Expectation-
Feature Feature Feature
Name Value Name
Feature Value Maximization and Its Application to Image Querying. IEEE
Trans. on Pattern Analysis and Machine Intelligence,
con 0.2471 ani 0.1362
2002,24(8): 1026-1038.
texbin 9 X 0.5220 [4]Georgescu,B. Shimshoni etc. Mean shift based clustering
locbin 561 0.4174 in high dimensions: a texture classification example.
Y Proceedings of the ninth IEEE International Conference on
labL 0.6560 labA 0.5030 Computer Vision,2003,(1):456-463.
labB 0.7921 [5] Ai-min Yang,Yong-mei Zhou etc.A Region-Based Image
F1 65.4838 F2 38.7338 Segmentation Method with Kernel FCM.Fuzzy Informa -tion
Figure 4 shows the time curves for segmentation and Engineering(ICFIE'07),2007,902-910.
using the MSC-SD and EM-MDL in literature [3] in [6] Lawrence H. Staib, James S.Duncan.Boundary Finding
with Parametrically Deformable Models.IEEE Trans.Pattern
Figure 3. From the figure, we can conclude the Anal. Mach. Intell, 1992,14(11): 1061-1075
processing time of MSC-SD is less than that of EM-
MDL, so we think of the MSC-SD better and more
valuable.

370