Pattern Recognition Letters xxx (2010) xxx–xxx

Contents lists available at ScienceDirect

Pattern Recognition Letters

journal homepage: www.elsevier.com/locate/patrec

A de-texturing and spatially constrained K-means approach for image segmentation

Max Mignotte
DIRO, Univ. de Montréal, C.P. 6128, Succ. Centre-ville, Montréal, Canada H3C 3J7

a r t i c l e i n f o a b s t r a c t

Article history: This paper presents a new and simple segmentation method based on the K-means clustering procedure
Received 17 December 2009 and a two-step process. The first step relies on an original de-texturing procedure which aims at convert-
Available online xxxx ing the input natural textured color image into a color image, without texture, that will be easier to
Communicated by M. Couprie
segment. Once, this de-textured (color) image is estimated, a final segmentation is achieved by a spa-
tially-constrained K-means segmentation. These spatial constraints help the iterative K-means labeling
Keywords: process to succeed in finding an accurate segmentation by taking into account the inherent spatial
Unsupervised segmentation
relationships and the presence of pre-estimated homogeneous textural regions in the input image. This
De-texturing
Spatially constrained K-means
procedure has been successfully applied on the Berkeley image database. The experiments reported in
Berkeley image database this paper demonstrate that the proposed method is efficient (in terms of visual evaluation and quanti-
tative performance measures) and performs competitively compared to the best existing state-of-the-art
segmentation methods recently proposed in the literature.
Ó 2010 Elsevier B.V. All rights reserved.

1. Introduction problem of the unsupervised segmentation of textured color

image;
Image segmentation is an important pre-processing step which
consists in dividing the image scene into spatially coherent regions (1) First, this clustering utilizes an iterative procedure that con-
sharing similar attributes. This preliminary low-level vision task verges monotonically to a local minima. This convergence to
has been widely studied in the last decades since it is of critical a bad local solution may be due either (or both) to a bad ini-
importance in many image understanding algorithms, computer tialization (i.e., a bad choice of the initial cluster centers) or
vision and graphics applications. Presently, the problem of finding to the complexity or to the high-dimensional of the dataset
a fast, simple and automatic method, able to efficiently segment a (such as the high dimensional feature descriptors required
color textured image remains open. to characterize different color textures).
Due to their simplicity and efficiency, clustering approaches (2) Such clustering method only involves a partitioning in the
were one of the first techniques used for the segmentation of (tex- feature space without (at all) considering spatial constraints.
tured) natural images (Banks, 1990). After the selection and the This considerably reduces the efficiency of this algorithm for
extraction of the image features (usually based on color and/or tex- the image segmentation issue. Since spatial relationships are
ture and computed on (possibly) overlapping small windows cen- essential attributes of any image, spatial constraints should
tered around the pixel to be classified), the feature samples, be necessarily taken into account to achieve an efficient
handled as vectors, are grouped together in compact but well- noise robust image segmentation.
separated clusters corresponding to each class of the image. The (3) Finally, the K-means assumes, often wrongly, the presence of
set of connected pixels belonging to each estimated class thus spherical clusters with equal volumes (or equivalently, the
defined the different regions of the scene. The method known as shapes of each feature distribution associated to each class
K-means (or Lloyd’s algorithm) (Lloyd, 1982) (and its fuzzy version are assumed to be Gaussian with identical variance).
called fuzzy C-means) are some of the most commonly used tech-
niques in the clustering-based segmentation field, and more gener- In this paper, we propose an original and simple segmentation
ally, ‘‘by far, the most popular clustering algorithm used in strategy based on the K-means procedure that remedies these
industrial applications and machine learning” (Berkhin, 2002). above-enumerated problems. First, in order to apply the K-means
Nevertheless, despite its popularity for general clustering, clustering algorithm in a reduced dimension space, in which the
K-means suffers from three major shortcomings for the difficult search of well-separated clusters can converge faster to a better
local minima (related to a more accurate segmentation map), we
E-mail address: [email protected] propose, in a first step, to simplify the input color textured image
URL: http://www.iro.umontreal.ca/~mignotte/ into a color image without texture. Thanks to this pre-processing

0167-8655/$ - see front matter Ó 2010 Elsevier B.V. All rights reserved.
doi:10.1016/j.patrec.2010.09.016

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
2 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx

step (that we will call in the following a de-texturing procedure) the

difficult textured image segmentation problem (which necessarily
should use a set of high dimensional texture feature descriptors) is
reduced to solving a (noisy) color image segmentation which is
drastically less complex than a color textured image segmentation
problem.
In our application, this de-texturing procedure is simply based
on a simple combination of several K-means segmentations
applied on the input image expressed by different color spaces
and using, as input multidimensional feature descriptor, the set
of values of the re-quantized color histogram estimated around
the pixel to be classified. Once, a de-textured (color) image is
estimated, the final segmentation is simply achieved by a spa-
tially-constrained K-means segmentation using, as simple cues, a
feature vector with the set of color values contained around the
pixel to be classified. The spatial constraint allows to take into ac-
count the inherent spatial relationships of any image and helps the
iterative K-means labeling process to succeed in finding an optimal
solution, i.e., an accurate segmentation map.
The remainder of this paper is organized as follows: Section 2
describes the proposed model i.e., the so-called de-texturing
procedure and the spatially constrained K-means segmentation
model. Finally, Section 3 presents a set of experimental results
and comparisons with existing segmentation techniques.

Fig. 2. Algorithm 2: De-texturing procedure.

2. Proposed model

2.1. De-texturing procedure regions of this image (see Fig. 3(b)–(d)), the second step of
this de-texturing procedure consists in [1] subdividing all
This de-texturing approach (described in Algorithm 2) is a the regions with more than W0 (=5000 in our application)
three-step process as follows: pixel size into disjoint pieces of size lower than W0 pixels
in order to limit regions with a large number of pixels which
(1) The first step is simply based on a K-means segmentation of could be due to a bad segmentation.1 To this end, we have
the input image using the Manhattan distance (L1 norm) and used the simple subdivision algorithm presented in pseudo-
as simple cues (i.e., as input multidimensional feature code in (Mignotte, 2007), and [2] simply replacing each pixel
descriptor) the set of values of the re-quantized color histo- value belonging to each small regions Rj of R by its mean
gram, with equidistant binning, estimated around each pixel color value. This defines a simulation Jc of the input image into
to be classified. In our application, this local histogram is a region partition constrained to be spatially piecewise
equally re-quantized (for each of the three color channels) uniform (in the color sense).
in a Nb = q3 bin descriptor, computed on an overlapping (3) The one and two above-mentioned steps are repeated on the
squared fixed-size (N0) neighborhood centered around the input image expressed in different color spaces and the
pixel to be classified (see Fig. 1 and Mignotte, 2008 for more averaging of these different partitions into regions with an
details). uniform (i.e., constant) color level value allows to give our
(2) The preceding step defines an segmentation of the image de-textured image (see Fig. 2 and Fig. 3(e)).
into small homogeneous texture regions (each one is defined
by the set of connected pixels belonging to a same texture Let us note that this de-texturing approach can be viewed as the
class or equivalently to a same cluster and to this end, we edge-preserving denoising algorithm proposed in (Mignotte, 2007)
use a simple flood fill algorithm). Given this partition R into for which, respectively:

(1) Each texture applied on each one of these distinct regions is

considered as a noise (a de-texturing is thus herein consid-
ered as a denoising approach).
(2) Each input (de-textured) image can be modeled by an union
of a number of nonoverlapping and distinct regions of uni-
form (i.e., constant) color level value.
(3) The set of Markov Chain Monte-Carlo (MCMC) simulations
of region partition maps used in (Mignotte, 2007) is herein
replaced by a set of K-means segmentations into regions,
on the input image expressed in different color spaces.

1
After this splitting procedure, for example, a region with a size of 12,000 pixels
will be divided into two regions of W0 = 5000 pixels and one region with a size of
2000 pixels. W0 = 5000 is thus the maximum size allowed of each region in the
Fig. 1. Algorithm 1: Bin descriptor estimation for each pixel. segmentation map.

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx 3

Fig. 3. From left to right; (a) a natural image from the Berkeley database (numbers 239,007 and 134,052), (b)–(d) its partition into regions obtained by the K-means
segmentation (K0 classes) using as feature vector h[] (see Algorithm 1) for the input image expressed in respectively, RGB, HSV, YIQ color spaces, (e) the averaging of these
different partition into regions (for which we have replaced each pixel value belonging to each region by its mean color value) finally defined the final de-textured image.

2.2. Spatially constrained K-means clustering vector with the set of color values contained around the pixel to
be classified.
As noted above, once a de-textured (color) image is estimated, Besides, in order to further help this K-means clustering process
the final segmentation is simply achieved by a spatially-con- to succeed in finding an accurate partition, a hard constraint
strained K-means segmentation using, as simple cues, a feature enforcing the spatial contiguity of each (likely) textural region is

Fig. 4. From top to bottom and left to right; (a) a natural image from the Berkeley database (number 134,052), (b) its soft edge map and (c) and (d) the sets of connected pixels
(i.e., regions) whose edge potential is respectively below n = 0.25 and n = 0.35. These regions are represented by colored regions. The white region corresponds to the sets of
pixels whose edge potential is above the threshold (and thus corresponding to inhomogeneous regions in the textural sense). (e) and (f) results of the final segmentation using
their respective map of homogeneous textured regions.

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
4 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx

imposed during the iterative K-means labeling process. To this end, (6) Fuse each small region (i.e., set of connected pixels belong-
we first compute an edge map in the textural sense (see Fig. 4 and ing to the same cluster whose size is below 300 pixels) with
the estimation of this edge map will be explained in the following the neighboring region sharing its longest boundary.
subSection 2.3). The most likely regions in this edge map are easily
estimated by identifying the sets of connected pixels whose edge 3. Experimental results
potential is below a given (and low) threshold n. In this way, we
easily identify regions in which only low textural gradient magni- 3.1. Set up
tudes have been detected. This procedure allows us to define a map
of (likely) homogeneous textural regions. The hard constraint In all the experiments, in order to promote diversity in the
enforcing the spatial contiguity of each of the K-means cluster is K-means segmentation results or in the different estimated edge
then simply done by assigning the majority class label in each maps (and also to somewhat increase the accuracy around
(pre-estimated) homogeneous textural region, for each iteration the boundaries between distinct texture regions, after the
of the K-means algorithm. This spatially constrained K-means is averaging process), we have considered ten (Ns = 10) different color
ensured after a few iterations (e.g. typically 10 iterations in our spaces (each color channel has been normalized between 0 and
application) of the classical K-means clustering. Implicitly, this 255), namely RGB, HSV, YIQ, XYZ, LAB, LUV, I1I2I3, H1H2H3, YCbCr,
procedure allows to consider non-spherical clusters in the K-means TSL, (Martinkauppi et al., 2001; Banks, 1990; Braquelaire and Brun,
clustering scheme since the distribution of each textural features 1997). For all K-means-based segmentations (first and second
are no more spherical after the spatial constraint. step), the L1 norm is used as clustering distance measure (experi-
ments have shown that the L1 norm gives the same performance
2.3. Edge map measure as the euclidean norm while consuming less computa-
tional resources in our application).
In order to estimate an edge map (in the textural sense), which
will then be used to find the most likely textured regions and then  For the K-means based-de-texturing approach and the estima-
to constrain the K-means procedure, we compute, at each pixel, the tion of the edge map, the number of bins for each local re-quan-
following distance tized color histogram, the number of classes, and the size of the


overlapping squared window are respectively set to q = 5, K0 = 5
D hiðN0 =2Þ;j ; hiþðN0 =2Þ;j þ D hi;jðN0 =2Þ ; hi;jþðN0 =2Þ
and N0 = 7.
where h is the Nb-bin (descriptor) vector, i.e., the re-quantized local  For the spatially-constrained K-means clustering used in the
color histogram, located at row i and column j (see Section 2.1 and second step, on the de-textured color image, we have used as
Algorithm 1), and Dðhid;j ; hiþd;j Þ is the L1 norm between vectors (or input multidimensional feature descriptor, the set of color val-
bin descriptors) hid,j and hi+d,j computed on a squared N0-size win- ues estimated on an overlapping squared fixed-size (N1 = 5)
dow centered respectively at location (i  d, j) and (i + d, j). In order neighborhood centered around each pixel to be classified and
to propose a reliable edge map, we have averaged the different edge finally K1 = 9 classes.
maps obtained for the same input image but expressed in different
color spaces (and normalized this resulting and averaged potential In order to obtain a more reliable color segmentation, we have
map between the interval [0  1]). Fig. 4 shows an example of a soft noticed that better segmentation results are achieved if we con-
edge map and two sets of regions obtained by two different thresh- sider, in the feature descriptor vector, all the color values ex-
old values n. The spatial constraints of the K-means procedure con- pressed by each color space. The feature extraction step thus
sists in assigning, at each iteration of the K-means segmentation yields to a [N1  N1  3  10]-dimensional feature vector (i.e.,
map, the majority class label existing in each colored region. Let number of pixels in the squared overlapping sliding window  3
us point out that the white regions shown in Fig. 4(c)–(d) are not color channels  10 color spaces). A final merging step is added
exploited in these spatial constraints since they represent inhomo- to each segmentation map that simply consists of fusing each small
geneous regions in the textural sense, thus corresponding to the region (i.e., regions whose size is below 300 pixels) with the region
sets of pixels whose edge potential is above the threshold value n. sharing its longest boundary.
The spatially-constrained K-means segmentation algorithm is out- In these experiments, we tested our segmentation algorithm
lined below: on the Berkeley segmentation database (Martin et al., 2001) con-
sisting of 300 color images of size 481  321 (beforehand divided,
½1 ½1
(1) Randomly choose K1 initial cluster centers c1 ; . . . ; cK 1 . by their creators, into a disjoint training and test sets of respec-
(2) At the kth step, assign sample xm to the cluster with the tively 200 and 100 images). For each color image, a set of bench-
½k
nearest center, i.e., to the cluster i if: xm  ci < mark segmentation results, provided by human observers is
½k
xm  cj ; 8j – i. available and will be used to quantify the reliability of the pro-
(3) posed segmentation algorithm. In order to ensure the integrity
(a) For each region Ri (defined by the set of connected pix- of the evaluation, the internal parameters of the algorithm are
els whose edge potential is below n) of the edge map: tuned on the train image set. The algorithm is then bench-marked
(b) Find the majority cluster of samples xm 2 Ri , i.e., by using the optimal training parameters on the independent test
½k
the cluster l if: cl ¼ arg maxc½k xm 2Ri 16j6K 1 j–l I set.
P P
n o l
½k ½k
xm  c l < xm  c j where I is the indicator
function. 3.2. Training
(c) Assign the majority cluster l to each sample xm belonging
to Ri . Experiments have shown that our overall segmentation proce-
½k
(4) Let C i denote the i-th cluster with ni samples after step 3c. dure is relatively sensitive to two parameters, namely, in order of
Determine new cluster centers by the mean of the samples importance, the threshold value n used to spatially constrain the
½kþ1
in the cluster ci
P
¼ ð1=ni Þ xm 2C ½k xm K-means procedure used on the de-textured color image and, to
i ½kþ1
(5) Repeat until convergence is achieved (i.e., until ci ¼ a very lesser degree, the number of classes K1 of the final cluster-
½k
ci ; 8i). ing. These two parameters are thus tuned on the train image set

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx 5

Table 1 PRI distribution

Average performance, in term of PRI measure, of our algorithm for the optimal 70
training set of its two internal parameters on the independent test set of the Berkeley

Nb. of Segmentations
image database (Martin et al., 2001) and comparison with other algorithms: 60
(Unnikrishnan et al., 2005; Yang et al., 2008; Ilea and Whelan, 2008; Krinidis and
50
Pitas, 2009; Mignotte, 2008; Felzenszwalb and Huttenlocher, 2004; Hedjam and
Mignotte, 2009; Deng and Manjunath, 2001; Ma et al., 2007; Sfikas et al., 2008; 40
Comaniciu and Meer, 2002; Shi and Malik, 2000). The first value is the performance
metric on the entire database and values between square brackets correspond to 30
performances on the train and test image sets.
20
ALGORITHMS PRI (Unnikrishnan et al.,
2005) 10
HUMANS in (Yang et al., 2008) 0.8754 0
SCKM[K1=9jn=0.37] 0.796 [train . 0.803 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
test / 0.782] PRI
CTex (Ilea and Whelan, 2008) 0.800
MIS[k=50] (Krinidis and Pitas, 2009) 0.798 Fig. 5. Distribution of the PRI performance measure over the 300 images of the
FCR[K1=13jK2=6jj=0.l35] (Mignotte, 2008) 0.788 Berkeley database (for SCKM½K 1 ¼9jn¼0:37 ).
FH (Felzenszwalb and Huttenlocher, 2004) in 0.784
(Yang et al., 2008)
HMC (Hedjam and Mignotte, 2009) 0.783
JSEG (Deng and Manjunath, 2001) in (Ilea and 0.770
n = 0, and K1 = 9), we obtain about PRI = 0.754. We can thus notice
Whelan, 2008) than a good value of n allows to significantly improve the segmen-
CTMg=0.15 (Yang et al., 2008; Ma et al., 2007) 0.762 tation result. Consequently, it is really necessary to help the opti-
St-SVGMM[K=15] (Sfikas et al., 2008) 0.759 mization process to succeed in finding an optimal solution. To
Mean-Shift (Comaniciu and Meer, 2002) in 0.755
this end, the hard constraint enforcing the spatial contiguity is use-
(Yang et al., 2008)
NCuts (Shi and Malik, 2000) in (Yang et al., 2008) 0.722 ful in this very difficult segmentation method. Fig. 5 shows the dis-
tribution of the PRI measure over the 300 images of the Berkeley
image database (for the algorithm SCKM½K 1 ¼9jn¼0:37 ).
Figs. 6 and 7 display some examples of segmentations obtained
by doing a local discrete grid search routine,2 with a fixed step-size, by our algorithm. The results for the entire database are avail-
on the parameter space and in the feasible ranges of parameter val- able on-line at http address www.iro.umontreal.ca/mignotte/
ues (namely n 2 [0.0  1.0] [step-size: 0.01], K1 2 [4  10] [step-size: ResearchMaterial/sckm.html. While being simple, this segmenta-
1]). The best couple of parameter is obtained for [K1 = 9jn = 0.37]. tion procedure gives excellent segmentation results with a very
competitive PRI score (which indicates that, on average, 79.6% of
pairs of pixels are correctly classified on the entire Berkeley data-
3.3. Comparison with state-of-the-art methods base and 78.2% on the test base).
The segmentation procedure takes, on average, less than 20 s
We have compared our segmentation algorithm (called SCKM½K 1 jn per image (for an AMD Athlon 64 Processor 3500+, 2.2 GHz,
for Spatially Constrained K-Means, K1 and n being its two internal 4435.67 bogomips and non-optimized code running on Linux) for
parameters) against several unsupervised algorithms. For each of the entire segmentation procedure. Table 2 compares the average
these algorithms, the internal parameters are set to optimal values computational time for an image segmentation and for different
and/or correspond to the internal values suggested by the authors. segmentation algorithms whose PRI is greater than 0.76.
All color images are normalized to have the longest side equals to
320 pixels. The segmentation results are then supersampled in or- 3.4. Sensitivity to parameters
der to obtain segmentation images with the original resolution
(481  321) before the estimation of the performance metric. We have tested the influence of the final merging procedure and
The comparison is based on the PRI3 performance measure its threshold on the final PRI score. Without the final merging pro-
(Unnikrishnan et al., 2005) which seems to be also highly correlated cedure (i.e., with a threshold equals to zero), we obtain a PRI equals
with human hand-segmentations (Yang et al., 2008) (a score equal to to 0.789 (on the entire Berkeley database) and by considering a
PRI = 0.796, for example, simply means that, on average, 79.6% of threshold twice greater, i.e., by merging each region whose size
pairs of pixel labels are correctly classified in the segmentation is below 600 pixels, with the region sharing its longest border,
results). we obtain a PRI equals to 0.795. These experiments thus show that
Table 1 shows the obtained PRI results for the different algo- our segmentation model tends to slightly over-segment and that
rithms. In terms of PRI measure, we observe that the discussed fu- our model is not too sensitive to this parameter to the extent that
sion strategy gives competitive results over the set of images of the it is around 300. We have also tested and quantified the influence
Berkeley image database. Without the spatial constraint (i.e., with of the variation of the parameters N0, N1, q, K0, K1 and n as respec-
tively a function of the number of pixels, bins or classes. Fig. 8
shows the evolution of the PRI score for several discrete values of
2
The grid search routine is a standard optimization algorithm which consists, for these parameters (holding all other parameters constant at their
several initial guesses of the parameter vector to be estimated, in employing a moving optimal values). Experiments show that our model is not too sen-
n-dimensional (n is the number of parameters and n = 2 in our application) grid
sitive to parameters N0, N1, K0 and q but sensitive to K1 and n and
containing three values per parameter low, medium, and high with spacing
determined by step-size. The algorithm tries to center the grid around the best this justifies the learning phase on these two parameters. Let us
performance score for each dimension (parameter), moving in an appropriate also note that for N1 = 1, we obtain a PRI score equals to 0.792. This
direction during each iteration. The optimization is successful when the grid becomes shows that this parameter is not essential (i.e., it is not essential to
centered on the best performance score across all dimensions.
3
also take the color value of the neighbors of the pixel to be classi-
We have used the Matlab source code, proposed by Allen Y. Yang in order to
estimate the PRI performance measure presented in the following Section. This code
fied), although N1 = 5 allows to ‘‘regularize” a bit the K-means pro-
is kindly available on-line at address http://www.eecs.berkeley.edu/yang/software/ cedure on the de-textured color image and to get a slightly better
lossy_segmentation/. PRI score.

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
6 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx

Fig. 6. Example of segmentations obtained by our algorithm SCKM½K 1 ¼9jn¼0:37 on several images of the Berkeley image database (see also Table 1 and Fig. 5 for quantitative
performance measures and http://www.iro.umontreal.ca/mignotte/ResearchMaterial/sckm.html for the segmentation results on the entire database).

We have also tested the influence and efficiency of our de-tex- to 0.773. This shows that our de-texturing helps the K-means
turing procedure in our segmentation method. To this end, we clustering process to succeed in finding a better solution of seg-
have used the proposed spatially constrained K-means on the ori- mentation, by simplifying the textural data to be segmented and/
ginal input image, instead of the de-textured image (with K1, n, q or by helping the K-means to converge toward a better local
and N0 keeping their optimal values). We obtain a PRI score equals minima.

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx 7

Fig. 7. Example of segmentations obtained by our algorithm SCKM½K 1 ¼9jn¼0:37 on several images of the Berkeley image database (see also Table 1 and Fig. 5 for quantitative
performance measures and http://www.iro.umontreal.ca/mignotte/ResearchMaterial/sckm.html for the segmentation results on the entire database).

Table 2
Average CPU time for different segmentation algorithms.

ALGORITHMS PRI CPU time (s) On [image size] Given in reference

CTex 0.800 85 [184  184] Ilea and Whelan (2008)
MIS[k=50] 0.798 2.9 [320  200] Krinidis and Pitas (2009)
SCKM½K 1 ¼9jn¼0:37 0.796 20 [320  200]
FCR½K 1 ¼13jK 2 ¼6jj¼0:135 0.788 60 [320  200] Mignotte (2008)
FH 0.784 1 [320  200] Felzenszwalb and Huttenlocher (2004)
HMC 0.783 80 [320  200] Hedjam and Mignotte (2009)
JSEG 0.770 6 [184  184] Ilea and Whelan (2008)
CTMg=0.15 0.762 180 [320  200] Yang et al. (2007)

We have shown, in Fig. 9, the four worst segmentations (in the a part of an animal (with its texture camouflage) with its natural
PRI sense), obtained by our segmentation method. Experiments environment (or more generally a textured region with the same
show that our method tends to over-segment some images con- average color of its background). In this latter case, this merging
taining regions with large texture elements or sometimes to merge may be due to the fact that two different textured regions, but with

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
8 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx

Xi
N0, N1 (Nb. Pixels) 0.1 0.2 0.3 0.4 0.5 0.6 0.7
N0
0.81 K1
q Xi
0.795 K0
N1 0.8

0.79 0.79

0.78

PRI
PRI

0.785 0.77

0.76
0.78
0.75

0.775 0.74
2 4 6 8 10 12 14 0 2 4 6 8 10 12 14
K0 (Nb. Classes) K1 (Nb. Classes)

Fig. 8. Left: evolution of the PRI for respectively: [1] N0 and N1 as a function of the number of pixels, [2] q as a function of the number of bins, [3] K0 as a function of the
number of classes. Right: evolution of the PRI for respectively: K1 and n.

Fig. 9. The four Berkeley images associated with the four worst PRI scores obtained by our segmentation method SCKM½K 1 ¼9jn¼0:37 , from left to right, image number 26,031
(PRI = 0.393), 54,005 (PRI = 0.459), 175,043 (PRI = 0.370), 69,040 (PRI = 0.438).

the same average color, will be assigned the same color value after model the texture elements (i.e., the so-called texton) but should
the de-texturing step. To some extent, the spatial constraints of the not be too large in order not to wrongly detect the boundaries, be-
final K-means procedure on the de-textured image, which is com- tween distinct textured regions, as an existing region (and thus to
puted with the original textural cues (based on local histogram of affect the accuracy of the boundary location). A good compromise,
each region) of the input image, is supposed to counterbalance a between good classification of the segmentation and contour accu-
bit this. Nevertheless, two textured regions, with almost identical racy, seems to be the value that we gave, i.e., N0 = 7. In order to
local histograms and two identical average colors will be merged quantify this contour accuracy, we have computed the Boundary
by our segmentation method. It seems that this is the main draw- Displacement Error (BDE) measure (lower distance is better) pro-
back of our segmentation method which could be improved on this posed by Freixenet et al. (2002) on the entire Berkeley database.
point. The BDE measures the average displacement error of boundary
We also present, in Fig. 10, some segmentation results for pixels between two segmented images. Particularly, it defines the
increasing values of N0. Although we have previously shown that error of one boundary pixel as the distance between the pixel
our segmentation model is less sensitive to N0 (comparatively to and the closest pixel in the other boundary image. For our algo-
n and K1), this parameter, however, seems to be an important inter- rithm, we obtain for N1 = 1 and N1 = 5, respectively, BDE = 9.8 and
nal parameter of our segmentation algorithm because it is related BDE = 10.0 compared to 9.9 for the algorithm called CTM and
to the accuracy of the detected boundary location. Indeed, experi- 10.0 for the algorithm called FH (results available in (Yang et al.,
ments show that this parameter must be large enough to fully 2008)), which, in terms of contour accuracy is comparable.

Fig. 10. Segmentation results for increasing values of N0. From left to right, N0 is equal to 3, 7, 15 and 3, 7 (holding all other parameters constant at their optimal values).

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016
 M. Mignotte / Pattern Recognition Letters xxx (2010) xxx–xxx 9

Let us finally add that, ideally, in order to get a more efficient Berkhin, P., 2002. Survey of Clustering Data Mining Techniques. Technical Report,
Accrue Software, San Jose, CA.
segmentation algorithm, N0, n and K1 should be automatically esti-
Braquelaire, J.-P., Brun, L., 1997. Comparison and optimization of methods of color
mated for each image. image quantization. IEEE Trans. Image Process. 6, 1048–1952.
Comaniciu, D., Meer, P., 2002. Mean shift: A robust approach toward feature space
analysis. IEEE Trans. Pattern Anal. Machine Intell. 24 (5), 603–619.
3.5. Algorithm Deng, Y., Manjunath, B.S., 2001. Unsupervised segmentation of color-texture
regions in images and video. IEEE Trans. Pattern Anal. Machine Intell. 23 (8),
The source code (in C++ language under Linux) of our algo- 800–810.
Felzenszwalb, P., Huttenlocher, D., 2004. Efficient graph-based image segmentation.
rithm (with the set of segmented images) are publicly available Internat. J. Comput. Vision 59, 167–181.
at the following http address www.iro.umontreal.ca/mignotte/ Freixenet, J., Munoz, X., Raba, D., Marti, J., Cufi, X., 2002. Yet another survey on
ResearchMaterial/sckm.html in order to make possible eventual image segmentation: Region and boundary information integration. In: ECCV02,
vol. III, p. 408.
comparisons with future segmentation algorithms or different per- Ilea, D.E., Whelan, P.F., 2008. Ctex – An adaptive unsupervised segmentation
formance measures. algorithm on color-texture coherence. IEEE Trans. Image Process. 17 (10), 1926–
1939.
Krinidis, M., Pitas, I., 2009. Color texture segmentation based on the modal energy of
4. Conclusion deformable surfaces. IEEE Trans. Image Process. 7 (18), 1613–1622.
Lloyd, S.P., 1982. Least squares quantization in PCM. IEEE Trans. Inf. Theory 28 (2),
129–136.
In this paper, we have presented an original, simple and effi- Ma, Y., Derksen, H., Hong, W., Wright, J., 2007. Segmentation of multivariate mixed
cient segmentation procedure based on the K-means procedure. data via lossy coding and compression. IEEE Trans. Pattern Anal. Machine Intell.
The proposed method overcomes the inherent problems of this 29 (9), 1546–1562.
Martin, D., Fowlkes, C., Tal, D., Malik, J., 2001. A database of human segmented
simple automatic clustering procedure using spherical distribu- natural images and its application to evaluating segmentation algorithms and
tions on the difficult problem of the textured color image segmen- measuring ecological statistics. In: Proc. Eighth Internat. Conf. on Computer
tation. First, the clustering is done on a simplified (de-textured Vision, vol. 2, pp. 416–423.
Martinkauppi, J.B., Soriano, M.N., Laaksonen, M.H., 2001. Behavior of skin color
image), on which the search of well-separated clusters is easier. under varying illumination seen by different cameras at different color spaces.
Besides, some spatial constraints help the iterative K-means label- In: Proc. SPIE, Mach. Vision Appl. Ind. Inspection IX, San Jose, California, pp.
ing process to succeed in finding an accurate segmentation by tak- 102–113.
Mignotte, M., 2007. Image denoising by averaging of piecewise constant
ing into account the inherent spatial relationships and the simulations of image partitions. IEEE Trans. Image Process. 16 (2), 523–533.
presence of pre-estimated homogeneous textural regions in the Mignotte, M., 2008. Segmentation by fusion of histogram-based k-means clusters in
input image. Implicitly, this procedure allows to consider non- different color spaces. IEEE Trans. Image Process. 17 (5), 780–787.
Hedjam, R., Mignotte, M., 2009. A hierarchical graph-based Markovian clustering
spherical clusters in the K-means clustering scheme since the dis-
approach for the unsupervised segmentation of textured color images. In: 16th
tribution of each textural feature vector is no more spherical after IEEE Internat. Conf. on Image Process, ICIP’09, Cairo, Egypt, pp. 1365–1368.
the spatial constraint. While being simple to implement or to Sfikas, G., Nikou, C., Galatsanos, N., 2008. Edge preserving spatially varying mixtures
for image segmentation. In: Proc. IEEE Internat. Conf. on Computer Vision and
understand and relatively fast compared to the best existing algo-
Pattern Recognition, Anchorage, AK, USA, vol. 1, pp. 1–7.
rithms, the proposed segmentation procedure performs competi- Shi, J., Malik, J., 2000. Normalized cuts and image segmentation. IEEE Trans. Pattern
tively among the recently reported state-of-the-art segmentation Anal. Machine Intell. 22, 888–905.
methods. Unnikrishnan, R., Pantofaru, C., Hebert, M., 2005. A measure for objective evaluation
of image segmentation algorithms. In: Proc. 2005 IEEE Conf. on Computer
Vision and Pattern Recognition (CVPR’05), Workshop on Empirical Evaluation
References Methods in Computer Vision, vol. 3, pp. 34–41.
Yang, A.Y., Wright, J., Sastry, S., Ma, Y., 2008. Unsupervised segmentation of natural
images via lossy data compression. Comput. Vision and Image Understanding
Banks, S., 1990. Signal Processing, Image Processing and Pattern Recognition.
110 (2), 212–225.
Prentice Hall.

Please cite this article in press as: Mignotte, M. A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recognition
Lett. (2010), doi:10.1016/j.patrec.2010.09.016