Pattern Recognition 40 (2007) 596608

www.elsevier.com/locate/patcog
Thresholding based on variance and intensity contrast
Yu Qiao
a,
, Qingmao Hu
a
, Guoyu Qian
a
, Suhuai Luo
b
, Wieslaw L. Nowinski
a
a
Biomedical Imaging Lab, Agency for Science, Technology and Research, 30 Biopolis Street, #07-01 Matrix, Singapore 138671, Singapore
b
School of Design, Communication and Information Technology, The University of Newcastle, University Drive, Callaghan NSW 2308, Australia
Received 11 October 2005; received in revised form 28 March 2006; accepted 21 April 2006
Abstract
A new thresholding criterion is formulated for segmenting small objects by exploring the knowledge about intensity contrast. It is
the weighted sum of within-class variance and intensity contrast between the object and background. Theoretical bounds of the weight
are given for the uniformly distributed background and object, followed by the procedure to estimate the weight from prior knowledge.
Tests against two real and two synthetic images show that small objects can be extracted successfully irrespective of the complexity of
background and difference in class sizes.
2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Thresholding; Histogram; Intensity contrast; Small object segmentation; Prior knowledge
1. Introduction
Thresholding is one of the most important and effective
methods for image segmentation, especially, when the image
consists of an object well separated from the background in
terms of intensity. By choosing a suitable threshold between
the dominant intensities of the background and object, the
original image can be partitioned into regions of the object
and background.
Many thresholding approaches have been proposed and
studied. They can be classied into approaches purely based
on intensity histogram and those based on knowledge. The
former can further be classied as methods based on valley
detection [1,2] and those based on criteria optimization. Var-
ious criteria have been proposed. The minimum error thresh-
olding method (MinError) assumes the normal distribution
for both object and background [3,4]. The optimum thresh-
old is achieved by optimizing a criterion function related to
the Bayes risk. Lee and Yang [5] estimated the parameters

Corresponding author. Tel.: +65 64788418; fax: +65 64789049.

E-mail addresses: [email protected] (Y. Qiao),
[email protected] (Q. Hu), [email protected] (G. Qian),
[email protected] (S. Luo),
[email protected] (W.L. Nowinski).
0031-3203/$30.00 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
doi:10.1016/j.patcog.2006.04.027
of normal distribution corresponding to the object and back-
ground from a truncated normal distribution. The thresh-
old is then determined by the Bayes decision rule. Pun [6]
is the rst to propose thresholding based on the maximum
entropy principle. Cross entropy is employed in threshold
selection by Li and Lee [7]. Cheng et al. [8] applied the fuzzy
c-partition to gray-level images and selected the threshold by
maximizing fuzzy entropy. Saha and Udupa [9] introduced
an energy criterion formulated by intensity-based class un-
certainty and region homogeneity. The threshold is selected
by minimizing the energy.
Otsu [10] applied the idea of maximizing the between-
class variance into histogram-based thresholding. The
method shows satisfactory results in various applications.
However, it tends to split the larger part when the sizes of
object and background are unequal [11].
Lee and Yang [5] pointed out that the separation of object
and background cannot generally be determined uniquely
by the image histogram. Prior knowledge about the relation
and properties of object and background can be helpful to
improve the performance of thresholding methods. Hu et
al. [12] took advantage of the knowledge about the range
of background proportion to the region of interest (ROI) to
conne the range of threshold selection and achieved reli-
able results in segmenting magnetic resonance (MR) and
Y. Qiao et al. / Pattern Recognition 40 (2007) 596608 597
Fig. 1. The histogram of uniformly distributed object and background.
computed tomography (CT) images of the human brain.
Other prior knowledge may also be employed for proper se-
lection of threshold.
The intensity contrast is dened as the difference in mean
intensities of the background and object, which character-
izes the intensity difference between the object and back-
ground. We explore the employment of intensity contrast
into thresholding in this paper. The threshold is obtained by
optimizing a new criterion, which is the weighted sum of
the within-class variance and the intensity contrast. Theo-
retical bounds of the weight are derived for the uniformly
distributed background and object (Fig. 1), followed by the
estimation of the weight from prior knowledge about the
range of object proportion and the range of the intensity
contrast. Analysis and comparison are made against the
Otsu method and MinError. Experimental results show the
superiority of the proposed method for segmenting small
objects.
2. Thresholding algorithm
We rst analyze the limitation of thresholding based on
within-class variance for images whose background and
object have very different sizes. A new thresholding cri-
terion and the corresponding algorithm are then proposed
to overcome this limitation. Without losing generality,
we assume that the background appears darker than the
object.
2.1. Within-class variance
Within-class variance is one of the most frequently used
criteria for thresholding [10]. The following two examples
exemplify that this criterion tends to classify an image into
the object and background of similar sizes irrespective of
the large intensity contrast between the small object and the
background.
The original image of example 1 (Fig. 2(a)) is a slice of
three-dimensional (3D) MR angiography (MRA) along the
axial direction. In the predened ROI marked by a white
Fig. 2. Example 1: (a) original MRA slice; (b) ROI selected using prior
knowledge; (c) segmentation obtained by the Otsu method with a threshold
of T =19; and (d) segmentation using the proposed method ( =0.054)
with the optimum threshold of T =69.
rectangle (Fig. 2(b)), there are three cerebral vessels (the
object) whose intensities are signicantly brighter than those
of the background. The distribution of background inten-
sities is bimodal while the histogram of object is a small
peak around intensity 100 (Fig. 3(a)). Although the intensity
contrast between the vessels and the background is visu-
ally distinct, the Otsu method provides a threshold (T =19)
that is located within the valley between the two back-
ground modes in the histogram. Almost half of the back-
ground is classied as object (Fig. 2(c)). As a result, the
intensity contrast is around 20, which is much smaller than
the real intensity contrast (73) between the vessels and the
background.
Example 2 (Fig. 4(a)) is a synthesized image containing
a small bright object at the image center. The intensities of
the object and background vary smoothly. The histogram
of the image consists of a small and a big rectangle-shaped
curves representing the object and background, respec-
tively (Fig. 5(a)). There is a wide intensity valley (in the
range from 90 to 200) between the object and background
(Fig. 5(a)). The Otsu method misses the valley and sets its
optimum threshold (T = 52) near the middle of the back-
ground distribution (Fig. 4(b)). Consequently, the intensity
contrast (44) obtained is much smaller than the real intensity
contrast (160).
598 Y. Qiao et al. / Pattern Recognition 40 (2007) 596608
Fig. 3. Example 1: (a) the histogram of the ROI in Fig. 2(b); (b) the value
of criterion with various ; and (c) the optimum threshold of different .
2.2. A new criterion
Examples 1 and 2 reveal one serious weakness of the
criterion of within-class variance, i.e., when the size of back-
ground is much larger than that of object and the distri-
bution of background intensities is complex (bimodal or
Fig. 4. Example 2: (a) original image; (b) segmentation obtained by the
Otsu method with a threshold of T =52; and (c) segmentation using the
proposed method ( =0.205) with the optimum threshold of T =90.
rectangular-shaped distributed), the optimum threshold will
tend to dichotomize the image into object and background
of similar sizes. The intensity contrast between the object
and background, which is a key factor of histogram-based
thresholding, however, has much less inuence on the image
segmentation.
To overcome the drawback, we propose a new criterion
that combines the within-class variance and the intensity
contrast,
J(, T ) =(1 )
W
(T ) |m
o
(T ) m
b
(T )|, (1)
where m
o
(T ) and m
b
(T ) are mean intensities of the object
and background, respectively.
W
(T ) is the square root of
within-class variance,

2
W
(T ) =P
b
(T )
2
b
(T ) +P
o
(T )
2
o
(T ), (2)
where P
b
(T ) and P
o
(T ) are probabilities of the background
and object,
2
b
(T ) and
2
o
(T ) are corresponding variances of
the background and object, respectively (see Ref. [10] for
their calculation). In this criterion, the within-class variance
measures the intensity homogeneity within the object and
background while intensity contrast captures the intensity
difference between them. The parameter is a weight that
balances their contributions. When =0, the new criterion
degenerates to the within-class variance. If = 1, thresh-
olding is determined only by the intensity contrast, which
may yield the threshold at the largest intensity of the image.
Therefore, the weight should be in the range of [0, 1).
The optimum threshold T

is selected by minimizing the

new criterion,
J(, T

) =min
T
J(, T ). (3)
Eq. (3) actually tries to decrease the within-class variance
and increase the intensity contrast simultaneously. In this
way, the intensity contrast becomes an explicit factor for
determining the optimum threshold.
2.3. The algorithm
An appropriate selection of the weight is desirable such
that a proper balance between the variance and contrast is
achieved to yield the correct threshold.
Theoretical bounds of are generally difcult to obtain
as both the variance and contrast are hard to be expressed
Y. Qiao et al. / Pattern Recognition 40 (2007) 596608 599
Fig. 5. Example 2: (a) the histogram of the image Fig. 4(a); (b) the value
of criterion with various ; and (c) the optimum threshold of different .
analytically. To simplify, we assume that both the object
and background are uniformly distributed (Fig. 1). More
specically, suppose the background is uniformly distributed
in [0, 2L], while the object is evenly located in [C qL,
C +qL]. The probability of the background and object are
k/(k + 1) and 1/(k + 1), respectively. When k approaches
innity (for small object), we have the following bounds
for
1/2

3
C/L (2 1/2

3)

3(C/L 1)
2
q
2
2

3(qk C/L +1) +3(C/L 1)

2
q
2
. (4)
Details of the derivation can be found in Appendix A.
This theoretic analysis provides guideline regarding s
existence and dependance on image features. In practice, the
distribution of the object and/or background is complicated.
can be approximated through training in the following two
ways.
One way to estimate is by supervised learning of a
number of images with similar histogram distributions or
similar object proportions. During learning, the desired value
of can be manually determined by visual judgment on a
single image through trying different . After learning, the
selected value of can then be used in the segmentation of
other similar images. For example, the proportion of cerebral
vessels in most two-dimensional (2D) slices of a 3D MRA
data varies within a small range. The histograms of vessels
and background are also quite similar. A neuroradiologist
can manually adjust the value of to extract the desired
vessels from background on a selected slice. This value of
can then be applied to the segmentation of other slices.
When supervised learning is not available, we can ex-
plore prior knowledge to conne the choice of . Here we
explore estimation using prior knowledge about the propor-
tion of the object/background and the intensity contrast. The
knowledge about the proportion of object in medical images
can be derived from anatomical knowledge or other prior
knowledge while the information of the intensity contrast
may be evaluated directly from the image to be analyzed or
indirectly from statistical analysis of similar images. Prior
knowledge about the proportion of the object/background
is proved to be very useful for improving the quality of
thresholding [12]. However, knowledge may not always be
accurately available or may be very variable for different ap-
plications. In these situations, we have to either set a loose
range or just take the high limit as 50% for small object if
the proportion of object is extremely variable. On the other
hand, we may explore knowledge about the intensity con-
trast of the desired object and background. When the inten-
sity contrast is very variable, the low and high limits of the
contrast range can be set as zero and the difference between
the maximum and minimum intensity of the image, respec-
tively. Eventually we can assume the availability of prior
knowledge of both object proportion and intensity contrast
and combine them to estimate .
Let P
o
be the proportion of object. The prior knowledge
about the proportion of object is
P
o
l
P
o
P
o
h
, (5)
600 Y. Qiao et al. / Pattern Recognition 40 (2007) 596608
where P
o
l
and P
o
h
are the lower and upper bounds of the
object proportion. The prior constraint about the intensity
contrast can be expressed as
C
l
|m
o
m
b
| C
h
, (6)
where C
l
and C
h
are the low and high limits of the intensity
contrast, respectively.
Let P
o
() represent the object proportion obtained with
a specied and the corresponding optimum threshold. Let
|m
o
()m
b
()| be the corresponding intensity contrast. The
weight can be estimated as follows:
1. For each [0, 1), calculate the object proportion P
o
()
and intensity contrast |m
o
() m
b
()|.
2. Find the minimum and maximum and denote them as

p
min
and
p
max
, respectively, such that P
o
l
P
o
()P
o
h
if
p
min

p
max
.
3. Search the minimum and maximum and denote them
as
c
min
and
c
max
, respectively, such that C
l
|m
o
()
m
b
()| C
h
if
c
min

c
max
.
4. Dene
min
= max(
p
min
,
c
min
) and
max
= min
(
p
max
,
c
max
).
5. Estimate

=(
min
+
max
)/2.
Since obtained with this procedure satises both condi-
tions (Eqs. (5) and (6)), it is a valid choice. Furthermore,
the interval [
min
,
max
] can be a constraint for the online
adjustment of .
The thresholding algorithm consists of two steps as fol-
lows:
(1) Estimate an appropriate weight

either through super-

vised learning or through the above mentioned proce-
dure satisfying constraints (Eqs. (5) and (6)).
(2) Find the threshold T

such that J(

, T

) dened by
Eq. (1) is minimized.
3. Experimental results
We have applied the proposed thresholding method to
two synthetic and two real images. For all experiments,
the weight

is approximated by the ve-step estimation

procedure through specifying the ranges of proportion of
object and the intensity contrast (Table 1). Tables 25 sum-
marize the optimum threshold, intensity contrast and object
proportion of MinError, the Otsu thresholding method and
the proposed method with various values of , respectively.
When the optimum threshold can be in a range which cor-
respond to the valley of J(, T ), we choose the minimum
of this range as the threshold for simplicity. However, any
threshold value in this valley is valid and yields the same
segmentation.
Fig. 2(d) is the segmentation of example 1 (Fig. 2(b)) with

=0.054. Unlike the Otsu method, the optimum threshold

(T = 69) is dragged out of the valley between background
Table 1
The prior knowledge about the intensity contrast and the proportion of
object of the four examples used for estimation (C
l
and C
h
are the
low and high limits of the intensity contrast, P
o
l
and P
o
h
are the lower
and upper bounds of the object proportion)
Example C
l
C
h
P
o
l
(%) P
o
h
(%)
1 50 75 1 50
2 50 170 1 40
3 80 170 1 50
4 60 100 7 10
Table 2
Example 1: optimum thresholds, the corresponding intensity contrasts
and proportions of object of the Otsu method and the proposed method
(
c
min
and
c
max
are minimum and maximum weights for the constraint
of intensity contrast,
p
min
and
p
max
are minimum and maximum weights
for the constraint of object proportion and

is the estimate of weight)

Optimum
threshold T

Intensity con-
trast |m
o
m
b
|
Proportion of
object P
o
(%)
Otsu 19 20 60.9

c
min
=0.013 58 70 1.8

c
max
=0.094 76 74.99 1.44

p
min
=0.013 58 70 1.8

p
max
=0.165 83 77 1.3

=0.054 69 73 1.6
Table 3
Example 2: optimum thresholds, the corresponding intensity contrasts and
proportions of object of the Otsu method and the proposed method
Optimum
threshold T

Intensity con-
trast |m
o
m
b
|
Proportion of
object P
o
(%)
Otsu 52 44 47.6

c
min
=0.043 90 160 1.05

c
max
=0.999 219 168 0.026

p
min
=0.043 90 160 1.05

p
max
=0.366 90 160 1.05

=0.205 90 160 1.05

Table 4
Example 3: optimum thresholds, the corresponding intensity contrasts and
proportions of object of MinError, the Otsu method and the proposed
method
Optimum
threshold T

Intensity con-
trast |m
o
m
b
|
Proportion of
object P
o
(%)
MinError 32 72 90
Otsu 97 58 40

c
min
=0.084 137 110 5

c
max
=0.999 214 120 0.0005

p
min
=0.0 97 58 40

p
max
=0.49 186 110 5

=0.287 137 110 5

Y. Qiao et al. / Pattern Recognition 40 (2007) 596608 601
Table 5
Example 4: optimum thresholds, the corresponding intensity contrasts and
proportions of object of MinError, the Otsu method and the proposed
method
Optimum
threshold T

Intensity con-
trast |m
o
m
b
|
Proportion of
object P
o
(%)
MinError 103 32.56 60
Otsu 137 58.29 23

c
min
=0.051 140 60.22 21

c
max
=0.999 209 88.3 0.08

p
min
=0.306 168 76.5 9.9

p
max
=0.422 173 77.3 9.3

=0.364 171 77.1 9.4

Fig. 6. Example 3: (a) the original synthesized image; (b) segmentation
with = 0.287; (c) segmentation with = 0.05; and (d) segmentation
with =0.99.
modes by the intensity contrast factor and located within
the valley between the small object and complex back-
ground (Fig. 3(a)). The intensity contrast is 73, which is
the one between the true object and background. As a re-
sult, the three cerebral vessels in the ROI are successfully
extracted.
In example 2, the optimum threshold (T =90) is obtained
with

=0.205, and the corresponding intensity contrast is

160 (Table 3). The intensity contrast term drives the thresh-
old to the valley ([90, 200]) between the real object and
background (Fig. 5(a)). The small object is correctly recog-
nized even though its size is much smaller than that of the
background (Fig. 4(c)).
Example 3 is a synthesized image (Fig. 6(a)). The his-
togram (Fig. 8(a)) shows that the background is composed
of three regions whose histograms are all Gaussian centered
at 30, 80 and 120, respectively. The object is the small area
whose intensities are normally distributed around 200.
For the Otsu method, one of the background regions (the
brightest part) is misclassied as object (Fig. 7(a)). The opti-
mumthreshold T =97 is between the two background modes
Fig. 7. Example 3: (a) segmentation obtained by the Otsu method; (b)
segmentation obtained by 2-mode MinError; (c) segmentation obtained by
3-mode MinError; and (d) segmentation obtained by 4-mode MinError.
80 and 120 (Fig. 8(a)). For MinError, when the number of
classes is assumed to be 2, two of the background regions are
classied as the object (Fig. 7(b)) and the optimum thresh-
old T = 32 is between the two background modes 30 and
80 (Fig. 8(a)). The intensity contrast obtained by both the
Otsu method and 2-mode MinError are less than 80 (Table
4). When three classes are to be segmented using MinError
(Fig. 7(c)), the two thresholds are T
1
=34 and T
2
=89. One
of the background regions is classied as the object. All the
four regions are well segmented by the multi-thresholds of
MinError (Fig. 7(d)) when the histogram of the image is
assumed to be a mixture of four normal densities.
The employment of the estimation procedure yielded
the desired

= 0.287. The object is extracted from the

complex background (Fig. 6(b)) with the optimum thresh-
old T = 137 that is within the valley between the real ob-
ject and background (Fig. 8(a)). Fig. 6(c) shows that the
segmentation with small (0.05) is the same as that of the
Otsu method. Fig. 6(d) is the segmentation with too large
(0.99) to classify only few brightest points (0.0005%) as the
object.
Example 4 is an axial slice of MR image (Fig. 9(a)).
Fig. 9(b) denes the ROI that contains the lateral ventricles
(the left and right deep dark areas), the interhemispheric
ssure (the deep dark region in the middle), the gray matter
(light dark part surrounding the interhemispheric ssure) and
the white matter (bright background). We need to extract
the lateral ventricles and the interhemispheric ssure. As the
object is assumed to be bright, the image Fig. 9(b) is inverted
to Fig. 9(c) before segmentation.
Fig. 10(a) presents the result of the Otsu method
(T =137), in which gray matter is regarded as object. The re-
sult of 2-mode MinError (T =103) is the worst (Fig. 10(b)).
The intensity contrast with respect to the above two thresh-
olds are less than 60 (Table 5). All gray matter and even
part of white matter are misclassied as object. The results
602 Y. Qiao et al. / Pattern Recognition 40 (2007) 596608
Fig. 8. Example 3: (a) the histogram of the original image Fig. 6(a); (b)
the value of criterion with various ; and (c) the optimum threshold of
different .
of 3-mode (Fig. 10(c)) and 4-mode (Fig. 10(d)) MinError
are also not desirable (the gray matter is still classied as
object).
The medical knowledge about the cerebral ventricles
helps to give the estimate

=0.364 (Table 5). Both lateral

Fig. 9. Example 4: (a) the original MR image; (b) the ROI in the original
image; (c) the inverted ROI image; (d) segmentation with =0.364; (e)
segmentation with =0.051; and (f) segmentation with =0.46.
ventricles and interhemispheric ssure in the ROI are well
segmented (Fig. 9(d)) with this

. Fig. 9(e) shows that part

of gray matter is treated as object when = 0.051. But all
interhemispheric ssures are misclassied as background
(Fig. 9(f)) if = 0.46 that leads to a too high threshold of
T =190.
4. Discussion
4.1. Comparison with Otsu method and MinError
thresholding method
Many algorithms have been developed for global
histogram-based thresholding. MinError and the Otsu
method are two of the most commonly used methods.
Y. Qiao et al. / Pattern Recognition 40 (2007) 596608 603
Fig. 10. Example 4: (a) segmentation obtained by the Otsu method; (b)
segmentation obtained by 2-mode MinError; (c) segmentation obtained by
3-mode MinError; and (d) segmentation obtained by 4-mode MinError.
MinError seems to perform best among 11 thresholding
methods when the histogram of the image is a mixture of
Gaussian distributions [13]. The Otsu method works quite
effectively in segmenting a well separated object and back-
ground in intensity [10]. Therefore, we only compare the
proposed method with MinError and the Otsu method.
The Otsu method [10] uses within-class variance for im-
age segmentation and works well when the object and back-
ground are distinct in intensity and of similar sizes. The dif-
ference in sizes between the object and background plays a
more important role than the intensity contrast in the Otsu
method. Consequently, the image is more likely to be seg-
mented into two classes of similar sizes (Figs. 2(c), 4(b),
7(a) and 10(a)) rather than those with big intensity contrast
(Figs. 2(d), 4(c), 6(b) and 9(d)).
MinError [3] uses a criterion related to average pixel clas-
sication error rate for threshold selection,
J(T
1
, . . . , T
k1
) =1 +2
k

i=1
[P
i
(ln
i
ln P
i
)], (7)
where P
i
is the probability of the ith class and
i
the cor-
responding variance. It is based on the assumption that the
histogram of image is a mixture of multiple normal densi-
ties.
MinError works best when the assumption is satised
and prior knowledge about the number of classes is correct
(Fig. 7(d)). The real object may not be extracted if the as-
sumption about the number of classes is wrong (Figs. 7(b)
and (c)). The segmentation may become worse when the
histogram of the image cannot be well approximated by
a mixture of n-mode normal distribution. When n is too
large, this method is too computationally expensive to be
applicable. In example 4, the intensity histogram can not
be well approximated by a mixture of n-mode normal dis-
tributions (n = 2, 3, 4). Thus, the real object cannot be ex-
actly recognized with the multiple thresholding of MinError
(Figs. 10(b)(d)).
In addition, the absence of an explicit term of intensity
contrast in the criterion (Eq. (7)) may cause MinError to
share a similar tendency to the Otsu method, i.e., the se-
lection of threshold ignores the information about intensity
contrast (Figs. 7 and 10).
Hu et al. [12] explored prior knowledge about the pro-
portion of object for thresholding. Their method may not be
well applicable to segmenting small objects as the tight range
of object proportion may not be available. For example, for
the segmentation of cerebral vessels (example 1), we only
know the loose range of object proportion ([1%, 50%)]. The
threshold based on this range using the Otsu method [12] is
25 (P
o
=44%), which is too small. The proposed criterion,
however, is still applicable (Table 2).
The desired segmentation of all four examples by the pro-
posed method clearly illustrates its advantage in extracting
small objects from complex background (Figs. 2(d), 4(c),
6(b) and 9(d)). The new criterion can segment the small
object because it explicitly tries to balance the within-class
variance and the intensity contrast.
4.2. Weight selection
The parameter uniquely determines the optimum thresh-
old and thus the quality of segmentation results. The op-
timum threshold will be the same as that of Otsu method
when is too small (0.012 for example 1 (Fig. 3(c)),
0.042 for example 2 (Fig. 5(c)), 0.083 for example 3
(Fig. 8(c)), and 0.016 for example 4 (Fig. 11(c))). On the
other hand, a too large value of will result in over-weight
of intensity contrast and thus yields a too high threshold
(Figs. 6(d) and 9(f)).
Figs. 3(b), 5(b), 8(b) and 11(b) show the relationship
between the value of the new criterion (Eq. (1)) and the
threshold for different weights of the four examples. When
is small, the new criterion takes the minimum at the
lower end where within-class variance dominates. So the
yielded threshold is close to that of the Otsu method. As
increases, within-class variance and the intensity con-
trast are balanced to yield the desirable threshold ( in
the ranges of [0.013, 0.094], [0.043, 0.366], [0.084, 0.49]
and [0.306, 0.422] for examples 14, respectively). When
is too large, the intensity contrast overwhelms to
yield a too high threshold. In this situation the selected
604 Y. Qiao et al. / Pattern Recognition 40 (2007) 596608
Fig. 11. Example 4: (a) the histogram of ROI for the inverted image
Fig. 9(c); (b) the value of criterion with various ; and (c) the optimum
threshold of different .
threshold is usually the maximum intensity of the image or
very close to it. For example, Figs. 3(b) and (c) in example
1 indicate that when 0.166 the threshold T = 112 that
maximizes the criterion (Eq. (1)) is the maximum image in-
tensity (Fig. 3(a)). So J(, T ) is only dened with T 112.
Similarly, the optimum threshold is around the maximum
image intensity for example 2 (0.368 (Figs. 5(b) and
(c)), T =219 (Fig. 5(a))), example 3 (0.491 (Figs. 8(b)
and (c)), T = 214 (Fig. 8(a))) and example 4 (0.573
(Figs.11(b) and (c)), T =209 (Fig. 11(a))).
An appropriate threshold can only be obtained with a suit-
able value of that well balances the inuences of within-
class variance and intensity contrast in the criterion (Eq. (1)).
Though theoretically there exists a range of to give the de-
sirable segmentation results (Figs. 2(d), 4(c), 6(b) and 9(d)),
the range is application dependent. We observe that if the ob-
ject is well separated from the background by some valleys
in the histogram there may be jumps of optimum threshold
around some . Fig. 3(c) indicates that the optimum thresh-
old has two big jumps in example 1, one from T

=19 to 58
around =0.013 and the other from T

=83 to 112 around

=0.166. In example 2 (Fig. 5(c)), the optimum threshold
jumps from T

=52 to 90 at =0.043 and from T

=90 to
200 when =0.367. The jumps in example 3 (Fig. 8(c)) are
also substantial, the rst from T

=97 to 137 at =0.084

and the second from T

= 186 to 214 around = 0.491.

Fig. 11(c) shows a jump at = 0.423 from T

= 173 to
187. These jumps mark the sharp transitions of the new cri-
terion from the state of dominated within-class variance to
the state of balanced within-class variance and intensity con-
trast or from the latter to the state of over-weighted intensity
contrast. Hence,these jumps may be useful for searching an
appropriate .
However, jumps that mark the state transition of the new
criterion may not always be obvious. In Fig. 11(c) the jump
at = 0.573 happens at the state of over-weighted inten-
sity contrast. No jumps indicate the state transition from
dominated within-class variance to balanced within-class
variance in this example. In this situation, estimation of
based on prior knowledge about intensity contrast and
the proportion of object should be employed. The appro-
priate value of can be estimated with the knowledge
from the original images (examples 2 and 3) or from the
experience in the special eld (medical knowledge in ex-
amples 1 and 4). Tables 25 list the range of obtained
with the knowledge and the suitable estimate for the four
examples.
4.3. Choice of ROI
An ROI is adopted to simplify analysis for easier approx-
imation of object proportion and intensity contrast and more
efcient exploration of prior knowledge. The basic guide-
line is that the ROI should include the object in the presence
of possible variations in both the size and the position of the
object, and the ROI should be as small as possible. When
the size of ROI varies, both the object proportion and the in-
tensity contrast can change. When ROI increases, the object
Y. Qiao et al. / Pattern Recognition 40 (2007) 596608 605
proportion decreases, while the intensity contrast may
increase or decrease. For example 1, the position and
variation of the three vessels are roughly known from
medical literature, and the drawn ROI is based on the
study of literature. Expanding the ROI will decrease the
object proportion but have little impact on the intensity
contrast. With the modied object proportion, new valid
can be found to yield similar segmentation. For exam-
ple, if the size of ROI increase 10% along both horizon-
tal and vertical directions, the lower bound of object P
o
l
decreases from 1% to 0.83%, which generates the corre-
sponding estimated weight (

= 0.053). The new thresh-

old (T = 71) is very close to the one with the original
ROI (T =69).
In real applications, an appropriate ROI is strongly re-
commended by studying possible variation of object/
background. In the worse case, ROI can be determined
manually through training. Eventually, an appropriate ROI
will facilitate determination of object and intensity contrast
to yield a reasonable and hence optimum threshold.
5. Conclusions
Thresholding based on within-class variance tends to clas-
sify an image as the object and background of similar sizes.
In order to overcome this drawback for segmenting small
objects, we have proposed a new criterion, which is the
weighted sumof within-class variance and intensity contrast.
In this way, both the intensity homogeneity of object and
background and the intensity difference between them are
well combined and balanced to yield a desirable threshold.
Theoretically there exists a bound for the weight when
the intensity contrast and variances meet some conditions
similar to that of Eq. (4) for uniformly distributed back-
ground and object. In practice, due to the complicated distri-
bution of the object and/or background, the weight can be
estimated by supervised learning or based on prior knowl-
edge about the intensity contrast and the proportion of ob-
ject. Tests against two synthetic and two real images have
conrmed that small objects can be extracted successfully
with a suitable irrespective of the complexity of back-
ground and difference in class sizes.
Appendix A. The weight range for uniform distribution
Suppose we have uniform distributions for both the back-
ground and object. Without losing generality, we can assume
that the background is distributed within [0, 2L] as shift-
ing will not impact thresholding. Suppose the object is dis-
tributed within [C qL, C + qL], the ratio of probability
between the background and object is P
b
/P
o
= k. As this
is for small object segmentation, it is assumed that k?1.
Fig. 1 shows the distribution of histogram.
Let
1
stand for rst order Taylor series and stand for
the limit with 1/k approaching 0. The density function is
f (x) =

k
(k +1)
1
2L
, 0x 2L,
1
(k +1)
1
2qL
, C qLx C +qL,
0 otherwise.
(A.1)
1. When T belongs to (0, 2L), the object function is de-
noted as J
1
(, T ).
P
b
(T ) =

T
0
f (x) dx
=

T
0
k
(k +1)
1
2L
dx =
T
2L(1 +1/k)

T
2L
,
P
o
(T ) =1 P
b
(T )
2L T
2L
,
m
b
(T ) =
1
P
b
(T )

T
0
xf (x) dx
=
1
P
b
(T )

T
0
x
k
(k +1)2L
dx =
T
2
,
m
o
(T ) =
1
P
o
(T )

C+qL
T
xf (x) dx
=
1
P
o
(T )

2L
T
x
k
(k +1)2L
dx
+

C+qL
CqL
x
1
(k +1)2qL
dx

=
1
1 P
b
(T )

k
k +1
4L
2
T
2
4L
+
C
k +1

L +
T
2
,

2
b
(T ) =
1
P
b
(T )

T
0
[x m
b
(T )]
2
f (x) dx =
T
2
12
,

2
o
(T ) =
1
P
o
(T )

C+qL
T
[x m
o
(T )]
2
f (x) dx
=
1
P
o
(T )

2L
T
[x m
o
(T )]
2
k
(k +1)2L
dx
+

C+qL
CqL
[x m
o
(T )]
2
1
(k +1)2qL
dx

(2L T )
2
12
,

2
w
(T ) =P
b
(T )
2
b
(T ) +P
o
(T )
2
o
(T )

(L T )
2
4
+
L
2
12
,
J
1
(, T ) =(1 )
w
(T ) [m
o
(T ) m
b
(T )]
(1 )

(L T )
2
4
+
L
2
12
L
(1 )
L

12
L. (A.2)
606 Y. Qiao et al. / Pattern Recognition 40 (2007) 596608
2. When T belongs to (2L, C qL), the object function
is denoted as J
2
(, T ).
m
b
(T ) =L, P
b
(T ) =
k
(k +1)
1
1
1
k
1,
m
o
(T ) =C, P
o
(T ) =
1
(k +1)
1

1
k
0,

2
b
(T ) =
1
P
b
(T )

2L
0
[x m
b
(T )]
2
f (x) dx =
L
2
3
,

2
o
(T ) =
1
P
o
(T )

C+qL
CqL
[x m
o
(T )]
2
f (x) dx =
q
2
L
2
3
,

2
w
(T ) =P
b
(T )
2
b
(T ) +P
o
(T )
2
o
(T )
1

L
2
3
+
L
2
3k
(q
2
1)
L
2
3
,
J
2
(, T ) =(1 )
w
(T ) [m
o
(T ) m
b
(T )]
1
(1 )
L

3
(C L) +
L(q
2
1)(1 )
2

3k
(1 )
L

3
(C L). (A.3)
3. When T belongs to (C qL, C + qL], the object
function is denoted as J
3
(, T ).
P
b
(T ) =

T
0
f (x) dx =
k
k +1
+
T C +qL
2qL(k +1)
1
1

1
k
(C +qL T )
2qL
1,
P
o
(T ) =1 P
b
(T )
1

1
k
(C +qL T )
2qL
0,
m
b
(T ) =
1
P
b
(T )

T
0
xf (x) dx
=
1
P
b
(T )

2L
0
x
k
2L(k +1)
dx
+

T
CqL
x
1
2qL(k +1)
dx

1
L +
1
k

C +qL T
2q
L +
T
2
(C qL)
2
4qL

L,
m
o
(T ) =
1
P
o
(T )

C+qL
T
xf (x) dx =
C +qL +T
2
,
m
o
(T ) m
b
(T )
1

C +T +(q 2)L
2

1
k

C +qL T
2q
L +
T
2
(C qL)
2
4qL

2
b
(T ) =
1
P
b
(T )

T
0
[x m
b
(T )]
2
f (x) dx
=
1
P
b
(T )

2L
0
[x m
b
(T )]
2
k
2L(k +1)
dx
+

T
CqL
[x m
b
(T )]
2
1
2qL(k +1)
dx

1 +
C +qL T
2qLk

L
2
(1 1/k)
3
+
1
k
(T L)
3
(C qL L)
3
6qL
1

L
2
3
+
1
k

L
2
3
+
L(C +qL T )
6q
+
(T L)
3
(C qL L)
3
6qL

2
o
(T ) =
1
P
o
(T )

C+qL
T
[x m
o
(T )]
2
f (x) dx
=
(C +qL T )
2
12
,

2
w
(T ) =P
b
(T )
2
b
(T ) +P
o
(T )
2
o
(T )
1

1
C +qL T
2kqL

L
2
3
+
1
k

L
2
3
+
L(C +qL T )
6q
+
(T L)
3
(C qL L)
3
6qL

+
(C +qL T )
3
24kqL
1

L
2
3

1 +
1
k

1 +
(T L)
3
(C qL L)
3
2qL
3
+
(C +qL T )
3
8qL
3

,
J
3
(, T ) =(1 )
w
(T ) [m
o
(T ) m
b
(T )]
1
(1 )
L

1 +
1
2k

1
+
(T L)
3
(C qL L)
3
2qL
3
+
(C +qL T )
3
8qL
3

C +qL +T 2L
2
+

k

C +qL T
2q
L +
T
2
(C qL)
2
4qL

.
Y. Qiao et al. / Pattern Recognition 40 (2007) 596608 607
As J
3
(, T ) takes minimum at T =(C +qL), so
J
3
(, T )
(1 )
L

3
(C +qL L) +
1
k

(1 )

L
2

(C +qL L)
3
(C qL L)
3
2qL
3
1

+

k
(C L).
As q is small, we further simplify the above expression as
J
3
(, T )(1 )
L

3
(C +qL L)
+
L(1 )
2

3k

C
L
1

2
1

+

k
(C L). (A.4)
In order to nd the constraints on such that T will be in
the range [2L, C qL), we need the following condition:
J
1
J
2
and J
3
J
2
. (A.5)
When J
1
J
2
, we have (using limits of J
1
and J
2
)
(1 )
L

12
L(1 )
L

3
(C L)

1/(2

3)
C/L 2 +1/(2

3)
. (A.6)
When J
2
J
3
, we have (using rst order Taylor series of J
2
and J
3
)
(1 )
L

3
(C L) +
L(q
2
1)(1 )
2

3k
(1 )
L

3
(C +qL L)
+
L(1 )
2

3k

C
L
1

2
1

+

k
(C L)
qL +
L(q
2
1)(1 )
2

3k

L(1 )
2

3k

C
L
1

2
1

+

k
(C L)

3(C/L 1)
2
q
2
2

3(qk C/L +1) +3(C/L 1)

2
q
2
.
(A.7)
Combining (A.6) and (A.7), we have the constraints on
1/(2

3)
C/L 2 +1/(2

3)

3(C/L1)
2
q
2
2

3(qk C/L+1) +3(C/L1)

2
q
2
. (A.8)
In (A.8), the lower bound should be smaller than the upper
bound, which sets constraint on C, L, q and k as follows:
1/(2

3)
C/L 2 +1/(2

3)
<
3(C/L 1)
2
q
2
2

3(qk C/L +1) +3(C/L 1)

2
q
2
. (A.9)
Take note of small q to simplify (A.9), we have
1/(2

3)
C/L 2 +1/(2

3)
<
3(C/L 1)
2
2

3(qk C/L +1) +3(C/L 1)

2
qk <

C
L
1

C
L
1

C
L
2 +
1
2

+1

3
2

C
L
1

. (A.10)
To have a valid lower bound for in [0, 1), C should be
larger than 2L; the larger C/L, the smaller the low bound
of . To have a valid upper bound, qk should be larger than
(C/L1); the smaller (qk C/L+1), the larger the upper
bound of .
For a given C/L, the lower bound of is uniquely deter-
mined by
1/(2

3)
C/L2+1/(2

3)
, qk should be small enough such
that the upper bound be larger than the lower bound, which
means q should be small. When q is not small enough, the
upper bound does not exist, which implies that the found
threshold will be larger than C qL so some object pixels
will be classied as background.
Let us give a specic example to illustrate the above re-
lationships. Suppose k =40, L=40, C/L=2.5. To have a
valid upper bound, we have q <0.125. When q =0.05, the
theoretical bounds of derived from (A.8) is [0.366, 0.796],
while the calculated valid bound is [0.35, 0.76]. The calcu-
lated and real bounds are very close. When q =0.10, the cal-
culated and theoretical bounds of are [0.366, 0.438] and
[0.35, 0.39], respectively. As qk becomes larger, the discrep-
ancy between the real and theoretical bounds will get larger
as we have used rst order Taylor series and limits to derive
the theoretical bounds assuming k approaches innity.
References
[1] J.M.S. Prewitt, M.L. Mendelsohn, The analysis of cell images, Ann.
N. Y. Acad. Sci. 128 (3) (1966) 10351053.
[2] J.S. Weszka, A. Rosenfeld, Histogram modication for threshold
selection, IEEE Trans. Syst. Man Cybern. 9 (1) (1979) 3852.
[3] J. Kittler, J. Illingworth, Minimum error thresholding, Pattern
Recognition 19 (1) (1986) 4147.
608 Y. Qiao et al. / Pattern Recognition 40 (2007) 596608
[4] S. Cho, R. Haralick, S. Yi, Improvement of Kittler and Illingworths
minimum error thresholding, Pattern Recognition 22 (5) (1989)
609617.
[5] J.S. Lee, M.C.K. Yang, Threshold selection using estimates from
truncated normal distribution, IEEE Trans. Syst. Man Cybern. 19
(2) (1989) 422429.
[6] T. Pun, Entropic thresholding: a new approach, CVGIP: Graphical
Models Image Process. 16 (1981) 210239.
[7] C.H. Li, C.K. Lee, Minimum cross entropy thresholding, Pattern
Recognition 26 (4) (1993) 617625.
[8] H.D. Cheng, J.R. Chen, J.G. Li, Threshold selection based on fuzzy
c-partition entropy approach, Pattern Recognition 31 (7) (1998)
857870.
[9] P.K. Saha, J.K. Udupa, Optimum image thresholding via class
uncertainty and region homogeneity, IEEE Trans. Pattern Anal.
Mach. Intell. 23 (7) (2001) 689706.
[10] N. Otsu, A thresholding selection method from gray-level histograms,
IEEE Trans. Syst. Man Cybern. 9 (1) (1979) 6266.
[11] J. Kittler, J. Illingworth, On threshold selection using clustering
criteria, IEEE Trans. Syst. Man Cybern. 15 (5) (1985) 652655.
[12] Q. Hu, Z. Hou, W.L. Nowinski, Supervised range-constrained
thresholding, IEEE Trans. Image Process. 15 (1) (2006) 228240.
[13] C.A. Glasbey, An analysis of histogram-based thresholding
algorithms, CVGIP: Graphical Models Image Process. 55 (6) (1993)
532537.
About the AuthorYU QIAO received his B. Eng. and M. Eng. degrees from Shanghai Jiao Tong University in 1991 and 1997, respectively, and Ph.D.
degree from National University of Singapore in 2004. He is currently a post-doc research fellow with the Biomedical Imaging Laboratory, the Agency
for Science, Technology and Research, Singapore. His research interests include medical image processing, computer vision and pattern recognition.
About the AuthorQINGMAO HU received his Ph.D. from Huazhong University of Science and Technology in 1990. From 1990 to 1996, he was a
lecturer and associate professor with the First Military Medical University China. From 1996 to 2000 he was a post-doc research fellow with University
of Bern Switzerland. From 2000, he is a research scientist and team leader with the Biomedical Imaging Laboratory, the Agency for Science, Technology
and Research, Singapore. His research interests include medical image processing, computer vision and pattern recognition.
About the AuthorGUOYU QIAN received his B.Eng. degree from Shanghai Jiao Tong University in 1986 and M. Eng. Degree from Nanyang
Technological University in 1995. He is currently a senior research associate with the Biomedical Imaging Laboratory, the Agency for Science, Technology
and Research, Singapore.
About the AuthorSUHUAI LUO received his B.E. and M.E. degrees from Nanjing University of Posts and Telecommunications China in 1982 and
1987, respectively, and Ph.D. degree from the University of Sydney, Australia in 1995. From 1995 to 2004, he was a senior research scientist with
the Commonwealth Scientic and Industrial Research Organization Australia and the Bioinformatics Institute Singapore. He is now a lecturer with the
University of Newcastle, Australia. His research interests include biomedical signal processing, image and video processing, and information visualization.
About the AuthorWIESLAW L. NOWINSKI D.Sc., Ph.D. is principal scientist and director of the Biomedical Imaging Lab, Agency for Science,
Technology and Research, Singapore. His research interests include neuroinformatics, brain atlases, computer-aided surgery, virtual reality, modeling,
segmentation, registration, and future directions in computer-aided surgery. He has developed 14 brain atlas products used world-wide in neurosurgery,
brain mapping, neuroradiology, and neuroeducation. He has led 33 patent applications and contributed above 330 articles. He received Magna cum
Laude awards from RSNA, ECR, and ASNR radiological societies.