IET Image Processing

Research Article

Automatic thresholding using a modified ISSN 1751-9659

Received on 10th February 2019
Revised 23rd August 2019
valley emphasis Accepted on 14th October 2019
E-First on 27th January 2020
doi: 10.1049/iet-ipr.2019.0176
www.ietdl.org

Jiangwa Xing1, Pei Yang2 , Letu Qingge3

1Research Center of Basic Medical Sciences, Medical College, Qinghai University, Xining 810016, People's Republic of China
2Department of Computer Technology and Application, Qinghai University, Xining 810016, People's Republic of China
3College of Computing and Informatics, University of North Carolina Charlotte, NC 28223, USA

E-mail: [email protected]

Abstract: Otsu's method is one of the most well-known methods for automatic thresholding, which serves as an important
algorithm category for image segmentation. However, it fails if the histogram is close to unimodal or has large intra-class
variances. To alleviate this limitation, improved Otsu's methods such as the valley emphasis method and weighted object
variances method have been proposed, which still yield non-optimal segmentation performance in some cases. In this study, a
modified valley metric using second-order derivative is proposed to improve the Otsu's algorithm. Experiments are firstly
conducted on five typical test images whose histograms are unimodal, multimodal or have large intra-class variances, and then
expanded to a larger data set consisting of 22 cell images. The proposed algorithm is compared with original Otsu's method and
existing improved algorithms. Four evaluation metrics including misclassification error, foreground recall, Dice similarity
coefficient and Jaccard index are adopted to quantitatively measure the segmentation performance. Results show that the
proposed algorithm achieves best segmentation results on both data sets quantitatively and qualitatively. The proposed
algorithm adapts the Otsu's method to more image subtypes, indicating a wider application in automatic thresholding and image
segmentation field.

1 Introduction In 1979, Nobuyuki Otsu published the well-known global

thresholding method named Otsu's method [9]. It is one of the most
Image segmentation is an important technique widely used in many commonly used techniques for thresholding real-world images
application fields [1–5]. A lot of segmentation algorithms including such as ice-covered cables [22], lung CT images [23], and images
traditional technologies [6–13] and deep learning-based methods in various other fields [24]. Although Otsu's method has been
[14, 15] have been presented so far for various kinds of proposed for a long time, there are still a lot of research works
segmentation tasks. Among all the segmentation algorithms, focusing on improving the algorithm [10–13, 18] and analysing its
automatic thresholding is still widely used due to its simplicity and performance [25]. The main idea of Otsu's method is to find the
effectiveness. As a typical kind of segmentation method and a optimal threshold value by maximizing the between-class variances
preprocessing tool for many other computer vision tasks [16, 17], of the histogram. It works well on images with bimodal or
automatic thresholding has been widely studied over the past multimodal distributed histograms while fails on images with
decades. The main idea of automatic thresholding algorithms is to unimodal histograms or histograms of large intra-class variances.
select proper threshold values to separate pixels of a grey-level In order to overcome this disadvantage, many researchers have
image into several subsets. For a single thresholding segmentation, been trying to improve the original Otsu's method. Ng [18]
the two subsets correspond to the background and the foreground, modified the between-class variance function and introduced a
respectively. Many automatic thresholding algorithms have been valley emphasis (VE) technique. In Ng's work, grey-level
introduced recently, which can be further categorized into global probability was adopted as a valley metric, and a penalty factor
thresholding and local thresholding [18, 19]. Local thresholding constructed based on valley metric was introduced to the objective
selects multiple threshold values based on localized grey-level function to make the threshold close to the valley of the image
information, resulting in better performance for non-uniform histogram. Although the modified thresholding algorithm produces
illumination images; while global thresholding chooses only one a better result on defect detection problem, it fails in some other
threshold value for the entire image, which is much simpler and cases, especially when the variance of the object is very different
more efficient. from that of the background [10]. Later, Fan and Lei [10] pointed
Representative global thresholding includes iterative out that Ng's method only weighted the objective function with the
thresholding [20], entropic thresholding [21] and Otsu-based valley-point value of the histogram, and proposed to add in the
methods [9]. Iterative thresholding separates pixels of a grey-level neighbourhood information of the valley point (neighbourhood
image into two classes using a predefined threshold value, valley emphasis (NVE)). Xu et al. [11] analysed the characteristics
calculates their respective average grey-level value g1 and g2, and of Otsu's threshold and got the conclusion that Otsu's threshold
then iteratively uses (g1 + g2)/2 as the new threshold value to biased toward the class with larger variance if the within-class
reclassify the image pixels until the value of (g1 + g2)/2 stays variances of background and foreground were different. Partly
constant. The iterative thresholding method is easy to implement, inspired by Xu's work, Yuan et al. [12] proposed an improved
but its algorithm complexity is O(n2) (n is the pixel number of the Otsu's method named weighted object variances (WOVs). WOV
image). Entropic thresholding divides the image pixels into two ensured the threshold located at the valley of two peaks or at the
classes based on their grey levels, calculates the inter-class entropy, left bottom of a single peak histogram by adding a weight on the
and sets the threshold value which achieves the largest entropy as object variance. In 2019, Cao et al. [26] published a new Otsu-
the optimal threshold. Entropic thresholding also suffers from based parameter-free thresholding method for image segmentation
relatively high time complexity, which is O(n!). by adding a new term to maximize the distance between the mean
values of each class, which claimed to be more accurate and robust

IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544 536

© The Institution of Engineering and Technology 2019
 2.1 Otsu's method
Otsu's method tries to find the optimal threshold value by
maximizing the between-class variances of the histogram. Let I be
a grey-level image of size M*N, and I(x, y) whose value is ranging
from 0 to L − 1 represent the pixel grey level with coordinates (x,
y). If the number of pixels with grey level i is denoted as f(i), the
probability of occurrence of grey-level i can be defined as

f (i)
pi = , i ∈ [0, L) (1)
M*N

For a single threshold situation, given a threshold t, pixels of an

image will be divided into two classes C0 = {I(x, y) I(x, y) ≤ t}
and C1 = {I(x, y) t < I(x, y) < L}. Let P0(t) and P1(t) be the
probabilities of the two classes respectively, and μ0(t), μ1(t) and μ(t)
be the average grey levels of C0, C1 and the entire image, then they
can be defined as follows:
t
P0(t) = ∑ pi (2)
i=0

L−1
P1(t) = ∑ pi = 1 − P0(t) (3)
i=t+1
Fig. 1  Thresholds of various Otsu methods on image coins
(a) Original image, (b) Ground truth, (c) The desired threshold and threshold values
t t
acquired by Otsu, VE, NVE, WOV, Cao's method and the proposed algorithm ∑i = 0 i ⋅ pi ∑i = 0 i ⋅ pi
μ0(t) = = (4)
t
∑i = 0 pi P0(t)
compared with standard Otsu's method. Besides these improved
one-dimensional Otsu's methods mentioned above, many two- L−1 L−1
dimensional Otsu's methods have also been published [27–30]. ∑i = t + 1 i ⋅ pi ∑i = t + 1 i ⋅ pi
μ1(t) = = (5)
Despite better performance, the two-dimensional Otsu's algorithms L−1
∑i = t + 1 pi P1(t)
are generally more time consuming compared to one-dimensional
Otsu's methods. In this paper, we are focusing on improving the L−1 L−1
∑i = 0 i ⋅ pi
performance of one-dimensional Otsu's algorithm. μ(t) = L−1
= ∑ i ⋅ pi
Fig. 1 shows the threshold values acquired by several one- ∑i = 0 pi i=0 (6)
dimensional Otsu's algorithms on a typical image named coins.
Despite the effectiveness declared in the published literature, these = P0(t) ⋅ μ0(t) + P1(t) ⋅ μ1(t)
algorithms could not find the desired threshold value, resulting in
non-optimal segmentation results. To further improve Otsu's For a threshold t, the between-class variance in Otsu's method is
method to fit more cases, in this paper, we introduce a modified defined as
valley metric and propose an improved Otsu's method based on
that. The main contributions of this paper are as follows: σ(t) = P0(t) ⋅ (μ0(t) − μ(t))2 + P1(t) ⋅ (μ1(t) − μ(t))2 (7)

(i) We have introduced a modified valley metric using second- Then the best threshold t* can be acquired by solving the
order derivative instead of grey-level probability used in original optimization issue below
VE, and proposed an improved Otsu's algorithm based on the
modified valley metric; t* = arg max σ(t)
t
(ii) Four quantitative evaluation metrics including misclassification
error (ME), foreground recall (FRecall), Dice similarity coefficient = arg max P0(t) ⋅ (μ0(t) − μ(t))2 (8)
t
(DSC) and Jaccard index (Jac) are employed for algorithm
validation. Experiments on five typical test images as well as on a +P1(t) ⋅ (μ1(t) − μ(t))2
larger data set consisting of 22 cell images demonstrate that the
proposed method has the overall best segmentation performance 2.2 Improved Otsu's methods
compared with existing Otsu's method, VE, NVE, WOV and Cao's
method. With the observation that the best threshold value should locate at
(iii) The proposed algorithm could achieve much better the valley of two peaks, or at the bottom rim of a single peak, Ng
segmentation results on images having unimodal histograms or proposed an improved VE scheme. The main idea of VE is to
large intra-class variances, without compromising its segmentation select a threshold value with a small probability of occurrance and
performance on images with bimodal or multimodal histograms. It to maximize the between-class variance in Otsu's method [13]. The
adapts the Otsu's method to more image subtypes and makes it best threshold of VE can be determined as follows:
suitable for wider applications in real-world image segmentation
scenarios. t* = arg max σ(t) ⋅ (1 − pt)
t

= arg max (P0(t) ⋅ (μ0(t) − μ(t))2 (9)

2 Otsu's algorithms t
In this section, original Otsu's method will be introduced briefly, +P1(t) ⋅ (μ1(t) − μ(t))2) ⋅ (1 − pt)
followed by improved algorithms including VE, NVE, WOV and
Cao's method. The VE was proved to be useful experimentally on defect detection
in Ng's work, but there are still some cases where it does not fit.
Fan introduced a modified strategy using neighbourhood
information for valley metric. Let F( ⋅ , n) denote average filtering

IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544 537

© The Institution of Engineering and Technology 2019
where • is a one-dimensional input signal and n represents the filter where min (s′′( ⋅ )) and max (s′′( ⋅ )) correspond to the minimum
size. The valley metric and optimization issue in NVE Otsu are and maximum value of the second-order derivative respectively,
defined as follows: and their definition domains are [0, L − 1]. The range of wv(x) is
[0, 1].
p̄ = F(p, n), p̄ = [p̄0, …, p̄L − 1], p = [p0, …, pL − 1] (10) Since it is usually difficult to get a differentiable envelope
function in practice, we use difference instead of derivative. Let
t* = arg max σ(t) ⋅ (1 − p̄t) f = [ f (0), …, f (L − 1)] be the histogram of an image, f (i) be the
t
number of pixels with grey-level i as defined in Section 2, the
= arg max P0(t) ⋅ (μ0(t) − μ(t))2 ⋅ (1 − p̄t) (11) difference representation for s′(x) and s′′(x) can be defined as:
t

+P1(t) ⋅ (μ1(t) − μ(t))2 ⋅ (1 − p̄t) Δ f (i) = f (i + 1) − f (i)

(16)
Δ2 f (i) = Δ f (i + 1) − Δ f (i)
It is obvious that if n equals 1 in (10), NVE is exactly the same as
VE. In other words, VE is a special case of NVE. Using the difference in (16), we define the valley metric using the
Yuan had experimentally studied the performance of VE and formula below
NVE, and proposed a modified Otsu's method named WOVs which
adds a weight factor on the first term in (7). The between-class Δ2 f (x) − min (Δ2 f ( ⋅ ))
variance in WOV is then defined as wv(x) = (17)
max (Δ2 f ( ⋅ )) − min (Δ2 f ( ⋅ ))
σ(t) = ω ⋅ P0(t) ⋅ (μ0(t) − μ(t))2 + P1(t) ⋅ (μ1(t) − μ(t))2 (12) Combining (1) and (17), wv(x) can be finally defined as
where ω is a weight factor ranging from 0 to 1 and set to P0(t) for
Δ2 px − min (Δ2 p( ⋅ ))
adaptive weight selection. Consequently, the threshold can be wv(x) = (18)
determined by solving the following optimization problem: max (Δ2 p( ⋅ )) − min (Δ2 p( ⋅ ))

t* = arg max σ(t) where px is the probability of occurrence of grey level x, and p( ⋅ )
t represents the probability of all grey levels.
= arg max P0(t) ⋅ P0(t) ⋅ (μ0(t) − μ(t))2 (13) In our implementation, the histogram of the image is first
t smoothed using an average filter in order to alleviate the second-
+P1(t) ⋅ (μ1(t) − μ(t))2 order difference anomaly caused by the abnormal fluctuation of the
histogram. The average smooth process can be described as
Cao et al. presented an improved Otsu's method whose objection
function not only makes the between-class variance maximum but ~ ∑ j = − k : 1: k f (i + j)
f (i) = (19)
also maximizes the distance between the mean values of each class. 2k + 1
The new object function in Cao's work is defined as
where 2k + 1 is the filter size, and k is set to five as suggested in
σ(t) = P0(t) ⋅ P1(t) ⋅ ((μ0(t) − μ1(t)) 2 the NVE method [18].
(14)
+(μ0(t) − μ(t))2 + (μ1(t) − μ(t))2) 3.2 Automatic thresholding using modified valley metric
Although the VE, NVE and WOV algorithms as well as Cao's Taking the valley metric in (18) into consideration, the final object
method perform better than standard Otsu's method in most cases, function of our proposed algorithm is defined as follows:
there are still certain types of images they cannot properly process.
As shown in Fig. 1, the thresholds acquired by VE, NVE, WOV δ(t) = σ(t) ⋅ wv(t) (20)
and Cao's method are not located at the real valley and
significantly different from the desired value. In the following In (20), the between term σ(t) and the valley metric wv(t) are
section, a new valley metric is introduced, and automatic multiplied together. There are two advantages using (20) as the
thresholding using the modified VE is proposed. objective function. On one hand, the two terms in (20) are as large
as possible when δ(t) reaches its largest value. On the other hand,
3 Proposed automatic thresholding the objective function is parameter-free which means we do not
have to discuss the value of tradeoff parameters between σ(t) and
3.1 Valley metric using second-order derivative wv(t).
As is known from the discussion above, VE is not enough in some
thresholding cases. In order to further improve the performance of
Otsu's method, we are trying to evaluate the valley using the
second-order derivative. Let s(x) be the envelope function of a
histogram and second-order differentiable, and s′(x), s′′(x) be the
first-order and second-order derivative, respectively, it is obvious
that s′(x) equals 0 at both peaks and valleys, whereas s′′(x) is
positive at valleys and negative at peaks. Fig. 2 shows the envelope
curve of the histogram of image coins and its corresponding to
second-order difference curve. We can clearly find that points
around the valley of the histogram envelope curve correspond to
positive second-order difference values. Especially, each valley
point in histogram envelope curve corresponds to a peak point with
a positive value in the second-order difference curve using original
histogram.
Motivated by the mathematical property mentioned above, we
define the valley metric as
Fig. 2  Histogram of coins and its corresponding second-order difference
s′′(x) − min (s′′( ⋅ )) curves
wv(x) = (15)
max (s′′( ⋅ )) − min (s′′( ⋅ ))

538 IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544

© The Institution of Engineering and Technology 2019
 As Otsu's method, the best threshold t* of our proposed Fo ∩ FT + Bo ∩ BT
ME = 1 − (22)
algorithm is the solution of the optimization issue below: BT + FT

t* = arg max σ(t) ⋅ wv(t) where Fo and Bo are pixel sets of foreground and background
t
2 2
segmented by automatic thresholding method, and FT, BT are
= arg max (P0(t) ⋅ (μ0(t) − μ(t)) + P1(t) ⋅ (μ1(t) − μ(t)) ) manually labelled foreground and background pixel sets which
t (21)
2 2
serve as ground truth. Consequently, ME represents the ratio of
Δ pt − min (Δ p( ⋅ )) misclassified pixels in an image by automatic thresholding method.
⋅
max (Δ2 p( ⋅ )) − min (Δ2 p( ⋅ )) Obviously, better automatic thresholding corresponds to a smaller
ME value. On the other hand, FRecall is used to assess the ability
Consequently, the proposed improved Otsu's method is as follows: of foreground extraction of a method, and it can be expressed as

Step 1: For a given grey level image, calculating its histogram Fo ∩ FT

FRecall = (23)
f = [ f (0), …, f (L − 1)]. FT
~ ~
Step 2: f ← f , where f is the smoothed histogram obtained by
(19). where Fo and FT are defined as the same as those in (22). Large
Step 3: Calculating the between σ(t) defined by (7) and valley FRecall value represents a higher foreground extraction ability.
metric wv(t) defined by (18) for each t ranging from 0 to 255 using The third adopted metric is DSC, which is also named as F1
the smoothed histogram f. score. It is usually used to measure the similarity of two samples.
Step 4: Calculating the optimal threshold value by solving the The larger the DSC value is, the more similar the two samples are.
optimization problem in (21). Equation (24) is the definition of DSC

2TP
4 Experimental results and analysis DSC = (24)
2TP + FP + FN
Experiments are conducted using Matlab R2012b on a PC with
Intel Core 2.30 GHz CPU and 4.0 GB memory. First, five typical In this paper, TP represents the number of correctly detected
test images named rice, coins, handwriting, printing character and foreground pixels, and FP, FN are the number of incorrect
license plate are selected to test algorithm performance, whose foreground pixels and number of miss detected foreground pixels,
histograms include unimodal, bimodal and multimodal respectively.
distributions (Fig. 3). Among them, image coins has significant The last evaluation metric is Jac and it is defined as
large intra-class variances in its histogram. Each of the images is
manually labelled with the binary segmentation ground truth, partly Fo ∩ FT
Jac = (25)
based on which the algorithm evaluation will be conducted. Fo ∪ FT
Subsequently, a larger data set containing 22 cell images is
constructed to further test the proposed algorithm. All the 22 where Fo and FT are the same as those introduced in ME.
images are collected from internet, and we manually label the In this paper, the performance of the proposed method is
segmentation ground truth for each image. More details about the evaluated on both data sets mentioned above and compared with
data set and the validation experiments will be introduced later. original Ostu's method, VE, NVE, WOV and Cao's method. In the
In order to verify the effectiveness of the proposed algorithm, following parts, algorithm evaluation using a data set of five
results are compared with those of Otsu's method [15], VE [13], typical test images is first conducted in four aspects, including the
NVE [19], WOV [23] and Cao's method [26], and four evaluation object function comparison, rationality analysis of the threshold
metrics including ME [13, 19, 23], FRecall, DSC [31, 32] and Jac value, quantitative evaluation and qualitative evaluation. Next a
[33, 34] are adopted to quantitatively measure the algorithm's larger data set of 22 cell images is introduced, and further
performance. The metric ME for effectiveness evaluation is experiments are conducted on the proposed larger data set.
defined as follows:

Fig. 3  Test images and their manually labelled binary segmentation ground truth
(a) Original image, (b) Ground truth, (c) Histograms of each image

IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544 539

© The Institution of Engineering and Technology 2019
Fig. 4  Object function curve comparison of the proposed method and Otsu's algorithm on
(a) Image rice, (b) Image coins. σ(t): Otsu's algorithm, δ(t): the proposed method

Table 1 Threshold values of five algorithms on test images

Methods Rice Coins Handwriting Printing character License plate
Otsu's method [9] 131 126 113 98 155
VE [18] 131 125 113 98 155
NVE [10] 131 125 113 98 155
WOV [12] 136 134 127 105 172
Cao et al. [26] 138 132 94 109 155
proposed method 121 82 98 96 137

Fig. 5  Positions of threshold values of each method on test images

(a) Rice, (b) Handwriting, (c) Printing character, (d) License plate

4.1 Object function evaluation 4.2 Rationality analysis of the threshold value
The main idea of the proposed method is taking the second-order To validate the rationality of the threshold of the proposed
difference-based valley metric into consideration to construct a algorithm, all five typical test images are used for testing. Through
more effective object function. In this section, the objective the whole experiments, the filter size used in NVE is set to 11 as
function of the proposed method will be evaluated on image rice suggested in the literature work [19]. Table 1 demonstrates the
and coins and compared with Otsu's method. threshold values of each method, and it is obvious that the results
Fig. 4 demonstrates the difference between our proposed object of Otsu's method, VE and NVE are extremely close on all images.
function and that of Otsu's method. The primary impression of the Fig. 5 shows the threshold locations of each method on the
figure is that our modified object functions are not as smooth as histograms of all test images except for image coins, which is
that of the Otsu's, which may be due to the fluctuation of the shown in Fig. 1c. The histogram distributions of image handwriting
histogram. However, the trends are distinct that the objective and printing character are close to unimodal, the case of which
function values are enhanced at valley points and weakened at Otsu's method is known to perform badly on. In particular, the grey
peaks. This effect is more obvious on image coins shown in level of foreground character for image handwriting is lower and
Fig. 4b. closer to background grey level, making the problem more
difficult. From Fig. 5 we can find that threshold values acquired by

540 IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544

© The Institution of Engineering and Technology 2019
Table 2 ME values of each method on test images
Methods ME
Rice Coins Handwriting Printing character License plate Average
Otsu's method [9] 0.0530 0.0248 0.2289 0.0383 0.0111 0.0712
VE [18] 0.0530 0.0241 0.2289 0.0383 0.0111 0.0711
NVE [10] 0.0530 0.0241 0.2289 0.0383 0.0111 0.0711
WOV [12] 0.0638 0.0306 0.4640 0.0456 0.0416 0.1291
Cao et al. [26] 0.0686 0.0292 0.0452 0.0499 0.0111 0.0408
proposed method 0.0399 0.0015 0.0556 0.0369 0.0201 0.0308
Bold values indicate the best segmentation method for images tested.

Table 3 FRecall values of each method on test images

Methods FRecall
Rice Coins Handwriting Printing character License plate Average
Otsu's method [9] 0.8324 0.9241 0.9177 0.9249 0.9829 0.9164
VE [18] 0.8324 0.9264 0.9177 0.9249 0.9829 0.9169
NVE [10] 0.8324 0.9264 0.9177 0.9249 0.9829 0.9169
WOV [12] 0.7951 0.9063 0.9800 0.8765 0.9223 0.8960
Cao et al. [26] 0.7791 0.9107 0.7486 0.8501 0.9853 0.8548
proposed method 0.9011 0.9956 0.7914 0.9375 0.9951 0.9241
Bold values indicate the best segmentation method for images tested.

Table 4 DSC values of each method on test images

Methods DSC
Rice Coins Handwriting Printing character License plate Average
Otsu's method [9] 0.9063 0.9606 0.4466 0.9179 0.9893 0.8441
VE [18] 0.9063 0.9618 0.4466 0.9179 0.9893 0.8444
NVE [10] 0.9063 0.9618 0.4466 0.9179 0.9893 0.8444
WOV [12] 0.8848 0.9508 0.2983 0.8991 0.9585 0.7983
Cao et al. [26] 0.8750 0.9533 0.7609 0.8875 0.9853 0.8941
proposed method 0.9330 0.9978 0.7413 0.9217 0.9810 0.9150
Bold values indicate the best segmentation method for images tested.

the proposed method for these two images both locate at the Table 3 shows the FRecall values of all test methods. The
bottom rim of the peak and only the proposed algorithm and Cao's proposed method obtains the best average result and has the largest
method generate thresholds close to the left bottom rim for image FRecall values for almost all test images except for image
handwriting. For the remaining images with bimodal or multimodal handwriting. Although a larger FRecall value generally
histogram distribution, the proposed algorithm chooses thresholds corresponds to a better foreground detection, it may not be the case
closest to the real valleys compared with all other compared if the foreground pixels are overly labelled compared with ground
methods as well, indicating its general effectiveness in all test truth. According to (23), if we label all pixels in an image as
cases. In particular, as is shown in Fig. 1c, the proposed method foreground pixels, the FRecall can reach 100%, which is not the
could identify a threshold value most similar to the desired value result we expect. Here the result for image handwriting encounters
for image coins, whose histogram has large intra-class variances similar situation and the good segmentation result of our proposed
and challenging for existing Otsu's method and other improved method will be demonstrated in the qualitative evaluation section.
methods. The DSC values of each algorithm on the five test images are
shown in Table 4. As introduced before, larger DSC values indicate
4.3 Quantitative evaluation better algorithm performance. From Table 4 we can come to the
conclusion that the proposed algorithm, whose average value is
In order to evaluate the performance of all test methods, we have 0.92, generally perform better than the compared methods on test
manually labelled the binary segmentation ground truth for the five images. Table 5 shows the Jac values of each algorithm on five test
test images (Fig. 3). The ground truth images are labelled in pixel images. The average Jac value of the proposed method is larger
level and saved as png format files. ME, FRecall, DSC and Jac than those of all compared algorithms, once again confirming the
metric values are then calculated based on the ground truth images outperformance of the proposed algorithm.
to quantitatively evaluate the segmentation performance of each
method. 4.4 Qualitative evaluation of segmentation results
Table 2 lists the ME values of each algorithm on all of the five
images. We can see that the proposed algorithm gains the smallest Fig. 6 shows the thresholding segmentation results of each method
ME values on three of the five test images except for image on the five test images. Images of each column from left to right
handwriting and image license plate, which are both one percent are original images, results of Otsu's method, VE, NVE, WOV,
larger than the smallest ME. In addition, the average ME of the Cao's method and the proposed method. There is no much
proposed method on five test images is far less than most of the difference among all methods for image printing character and
other Otsu-based algorithms including Otsu's method, VE, NVW image license plate. For image rice, although all methods miss
and WOV, and one percent smaller than that of the Cao's method. some foreground object pixels at the bottom area due to uneven
As smaller ME values indicate more effective segmentation ability, illumination, our algorithm misses the least. As for image coins
the ME results have verified the effectiveness of our proposed whose histogram has large intra-class variances, the proposed
algorithm. method is the only one which could correctly segment out all coins

IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544 541

© The Institution of Engineering and Technology 2019
Table 5 Jac values of each method on test images
Methods Jac
Rice Coins Handwriting Printing character License plate Average
Otsu's method [9] 0.8287 0.9241 0.2875 0.8482 0.9787 0.7734
VE [18] 0.8287 0.9264 0.2875 0.8482 0.9787 0.7739
NVE [10] 0.8287 0.9264 0.2875 0.8482 0.9787 0.7739
WOV [12] 0.7934 0.9063 0.1753 0.8166 0.9203 0.7224
Cao et al. [26] 0.7777 0.9107 0.6251 0.7977 0.9809 0.8184
proposed method 0.8744 0.9955 0.5889 0.8548 0.9627 0.8553
Bold values indicate the best segmentation method for images tested.

Fig. 6  Segmentation results of each automatic thresholding method. From left to right: original images, Otsu's results, VE's results, NVE's results, WOV's
results, Cao's results, and the proposed results

Fig. 7  Segmentation results of each automatic thresholding method on a subset of the larger data set. From left to right: original images, Otsu's results, VE's
results, NVE's results, WOV's results, Cao's results, and proposed results

without any local cavities. When applying the proposed method on performance, indicating its overall effectiveness for various image
image handwriting, despite a much lower FRecall value acquired subtypes.
compared with most of the other Otsu-based methods shown in
Table 3, the lower ME and higher DSC and Jac values assure that 4.5 Further evaluation on a larger data set
the proposed method could almost segment all words from the
image, generating the best result. When compared with Cao's In order to further evaluate the proposed algorithm, we conduct
method for image handwriting, three of the four evaluation metric experiments on a larger data set consisting of 22 cell images. All
values including ME, DSC and Jac are better for Cao's method, and the original cell images are collected from internet, and the ground
only the value of FRecall shows a better result for our proposed truth of each image is manually labelled before algorithm
method. However, from the segmentation results in Fig. 6 we could validation.
see that despite some over labelled foreground at the left lower The proposed algorithm and the five compared methods are
corner, our proposed method could segment much clearer evaluated both quantitatively and qualitatively on the 22 cell
characters compared with Cao's method. Therefore, based on the images. Fig. 7 shows part of the segmentation results. It is obvious
thresholding segmentation results on all five test images described that most segmentation results of the proposed method are better
above, the proposed method shows the best segmentation compared to other algorithms. On one hand, from the segmentation

542 IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544

© The Institution of Engineering and Technology 2019
Table 6 Average metric values of each algorithm on all cell images
Metrics Otsu's method [9] VE [18] NVE [10] WOV [12] Cao's method [26] Proposed method
ME 0.1640 0.1611 0.1626 0.2165 0.1639 0.1457
FRecall 0.7001 0.7002 0.6990 0.8826 0.6677 0.8604
DSC 0.7293 0.7312 0.7298 0.7202 0.7148 0.7996
Jac 0.6337 0.6359 0.6341 0.6115 0.6148 0.6986
Bold values indicate the best segmentation method for images tested.

Table 7 Average running time of each algorithm on different size of images (ms)
Image size Method
Otsu's method [9] VE [18] NVE [10] WOV [12] Cao's method [26] Proposed method
256 × 256 0.3588 0.3588 0.4524 0.3588 0.3588 0.7644
512 × 512 0.4524 0.4680 0.5928 0.4526 0.4524 0.8736
1024 × 1024 1.1544 1.1700 1.3104 1.1388 1.1388 1.5912
2048 × 2048 4.1496 4.3680 4.2588 4.1184 4.1340 4.6488
4096 × 4096 15.6313 15.8653 16.2709 15.6937 15.6937 16.1461

images in rows 2, 3 and 5, we can see that foreground objects in methods have the same time complexity of O(n). When image size
the segmentation images generated by the proposed method reaches 4096 × 4096, all methods could still achieve real-time
contain less black holes which correspond to miss detection of image processing since the average running time is <50 ms.
foreground pixels. On the other hand, the threshold values obtained
using the proposed method and WOV are more reasonable than 5 Conclusion
those obtained using Otsu's method, VE, NVE and Cao's method
on the image in the 4th row in Fig. 7. While the results of Otsu's, In this paper, we have introduced a modified valley metric and
VE, NVE and Cao's method in the 4th row obviously miss most of presented an improved Otsu's method for automatic image
the foreground objects, the results of the proposed method and thresholding. The proposed valley metric is constructed using
WOV are more consistent with the manually labelled ground truth. second-order derivative and introduced into the objective function
Moreover, compared to the results of WOV, foreground objects of Otsu's method to make the threshold more likely to locate at the
segmented using the proposed method are more complete. Despite valley of peaks of the image histogram. Experiment verification is
the advantages of the proposed method, we should also notice its conducted on five typical images as well as on a larger data set of
drawbacks in some of the segmentation results. Taking the results 22 cell images with manually labelled ground truth. The proposed
in rows 1 and 5 as an example, while the proposed method detects method is compared with existing Otsu-based methods including
more complete foreground objects, more background pixels are standard Otsu's method, VE, NVE and WOV, as well as recently
incorrectly classified. Taking all the segmentation results into published Cao's method. The proposed method could significantly
consideration, the proposed method performs better than all the improve Otsu's method in segmenting images with unimodal
compared algorithms. histograms and images having large intra-class variances. It has
Table 6 shows the average metric values of each method on the similar time complexity as compared methods and is shown to be
22 cell images. The proposed method obtains best values on three the most effective algorithm among all test methods by both
of the four metrics, which are ME, DSC and Jac. For FRecall, the quantitative and qualitative evaluations, exhibiting best flexibility
result of the proposed method (86.04%) is slightly smaller than and performance for segmenting images of various histogram
WOV method (88.26%). However, WOV method has the largest distributions.
average ME value (21.65%) among all test algorithms, which is
7% larger than the proposed method (14.57%). The quantitative 6 Acknowledgments
evaluation results on the larger data set verify the effectiveness of
the proposed algorithm. This work is partly supported by the National Natural Science
Foundation of China (Grant Nos. 61866031, 61862053, 61762074
and 31860030), and the Science Technology Foundation for
4.6 Algorithm complexity Middle-aged and Young Scientist of Qinghai University (Grant
We compare the algorithm complexity for all test methods shown Nos. 2016-QGY-5, 2017-QGY-4 and 2018-QGY-6).
in this paper. Let n represent the pixel number of the image, L be
the grey level, the time complexity of Otsu's method would be 7 References
O(n). The extra computation burden of the proposed algorithm
[1] Cao, J.F., Chen, L.C., Wang, M., et al.: ‘Implementing a parallel image edge
includes (i) the average filtering of the histogram, which needs (2k  detection algorithm based on the Otsu-Canny operator on the Hadoop
+ 1)L times of addition and multiplication respectively; (ii) the platform’, Comput. Intell. Neurosci., 2018, 2018, pp. 1–12
calculation of the second order difference, which needs 2L times of [2] Chen, C.T., Tsao, C.K., Lin, W.C.: ‘Medical image segmentation by a
subtraction; (iii) the calculation of valley metric, which includes 2L constraint satisfaction neural network’, IEEE Trans. Nucl. Sci., 1991, 38, (2),
pp. 678–686
times of subtraction and L times of division; (iv) the calculation of [3] Lin, L., Yang, W., Li, C., et al.: ‘Inference with collaborative model for
new object function, which needs L times of multiplication. The interactive tumor segmentation in medical image sequences’, IEEE Trans.
total computation increase is (4k + 8)L, where k and L are constant Cybern., 2017, 46, (12), pp. 2796–2809
whose values are 5 and 256, respectively. Therefore, our proposed [4] Han, B., Wu, Y.: ‘A novel active contour model based on modified symmetric
cross entropy for remote sensing river image segmentation’, Pattern Recogn.,
method has same time complexity as Otsu's method, which is O(n). 2017, 67, pp. 396–409
Similarly, the time complexity of all other compared methods in [5] Zhang, L., Kong, H., Chin, C.T., et al.: ‘Segmentation of cytoplasm and
this paper is also O(n). Table 7 lists the average running time of nuclei of abnormal cells in cervical cytology using global and local graph
100 times implementation for each algorithm on different size of cuts’, Comput. Med. Imaging Graph., 2014, 38, (5), pp. 369–380
[6] Wang, D.W.: ‘Hybrid fitting energy-based fast level set model for image
images. When the image size is small (no more than 1024 × 1024), segmentation solving by algebraic multigrid and sparse field method’, IET
the extra time consumption for our proposed method compared Image Process., 2017, 12, (4), pp. 539–545
with all other methods is about 0.3–0.4 ms. However, as image size [7] Zheng, J., Zhang, D.H., Huang, K.D., et al.: ‘Adaptive image segmentation
increases, the average running time will get much similar for all method based on the fuzzy c-means with spatial information’, IET Image
Process., 2018, 12, (5), pp. 785–792
methods, which is consistent with the previous analysis that all

IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544 543

© The Institution of Engineering and Technology 2019
[8] Cao, J.F., Chen, S.G.: ‘Active contour model based on variable exponent p- [23] Wei, Y., Chang, C., Jia, T., et al.: ‘Segmentation of regions of interest in lung
Laplace equation for image segmentation’, J. Mod. Opt., 2019, 66, (7), pp. CT images based on 2-D Otsu optimized by genetic algorithm’. Chinese
726–738 Control and Decision Conf., Guilin, China, 2009, pp. 5185–5189
[9] Otsu, N.: ‘A threshold selection method from gray-level histograms’, IEEE [24] Rinn, J.L., Kertesz, M., Wang, J.K., et al.: ‘Functional demarcation of active
Trans. Syst. Man Cybern., 1979, 9, (1), pp. 62–66 and silent chromatin domains in human HOX loci by noncoding RNAs’, Cell,
[10] Fan, J.L., Lei, B.: ‘A modified valley-emphasis method for automatic 2007, 129, (7), pp. 1311–1323
thresholding’, Pattern Recognit. Lett., 2012, 33, (6), pp. 703–708 [25] Goh, T.Y., Basah, S.N., Yazid, H., et al.: ‘Performance analysis of image
[11] Xu, X., Xu, S., Jin, L., et al.: ‘Characteristic analysis of Otsu threshold and its thresholding: Otsu technique’, Measurement, 2018, 114, pp. 298–307
applications’, Pattern Recognit. Lett., 2011, 32, (7), pp. 956–961 [26] Cao, X., Li, T., Li, H., et al.: ‘A robust parameter-free thresholding method
[12] Yuan, X.C., Wu, L.S., Peng, Q.: ‘An improved Otsu method using the for image segmentation’, IEEE Access, 2019, 7, pp. 3448–3458
weighted object variance for defect detection’, Appl. Surf. Sci., 2015, 349, pp. [27] Fan, J., Zhao, F.: ‘Two-dimensional Otsu's curve thresholding segmentation
472–484 method for gray-level images’, Acta Electron. Sin., 2007, 35, (4), pp. 751–755
[13] Merzban, M.H., Elbayoumi, M.: ‘Efficient solution of Otsu multilevel image [28] Chen, Q., Zhao, L., Lu, J., et al.: ‘Modified two-dimensional Otsu image
thresholding: a comparative study’, Expert Syst. Appl., 2019, 116, pp. 299– segmentation algorithm and fast realisation’, IET Image Process., 2012, 6,
309 (4), pp. 426–433
[14] Chen, L.C., Papandreou, G., Kokkinos, I., et al.: ‘DeepLab: semantic image [29] Sha, C.S., Hou, J., Cui, H.X.: ‘A robust 2D Otsu's thresholding method in
segmentation with deep convolutional nets, atrous convolution, and fully image segmentation’, J. Vis. Commun. Image Represent., 2016, 41, pp. 339–
connected CRFs’, IEEE Trans. Pattern Anal., 2018, 40, (4), pp. 834–848 351
[15] Chen, L., Bentley, P., Mori, K., et al.: ‘DRINet for medical image [30] Zhu, X.Y., Xiao, Y., Tan, G.H., et al.: ‘GPU-accelerated 2D OTSU and 2D
segmentation’, IEEE Trans. Med. Imaging, 2018, 37, (11), pp. 2453–2462 entropy-based thresholding’, J. Real-Time Image Process., 2019, pp. 1–13
[16] Yilmaz, A., Javed, O., Shah, M.: ‘Object tracking: a survey’, ACM Comput. [31] Li, H, Song, D, Liu, Y, et al.: ‘Automatic pavement crack detection by multi-
Surv., 2006, 38, (4) scale image fusion’, IEEE Trans. Intell. Transp., 2019, 20, (6), pp. 2025–2036
[17] Lai, Y.K., Rosin, P.L.: ‘Efficient circular thresholding’, IEEE Trans. Image [32] Huang, D, Zhang, J, Huang, K.: ‘Automatic microblog-oriented unknown
Process., 2014, 23, (3), pp. 992–1001 word recognition with unsupervised method’, Chin. J. Electron., 2018, 27,
[18] Ng, H.F.: ‘Automatic thresholding for defect detection’, Pattern Recognit. (1), pp. 1–8
Lett., 2006, 27, (14), pp. 1644–1649 [33] Karasawa, K., Oda, M., Kitasaka, T., et al.: ‘Multi-atlas pancreas
[19] Yazid, H., Arof, H., Yazid, H.: ‘Automated thresholding in radiographic segmentation: atlas selection based on vessel structure’, Med. Image Anal.,
image for welded joints’, Nondestr. Test. Eval., 2012, 27, (1), pp. 69–80 2017, 39, pp. 18–28
[20] Ridler, T. W., Calvard, S.: ‘Picture thresholding using an iterative selection [34] Cunningham, R.J., Harding, P.J., Loram, I.D.: ‘Real-time ultrasound
method’, IEEE Trans. Syst. Man Cybern., 1978, 8, (8), pp. 630–632 segmentation, analysis and visualisation of deep cervical muscle structure’,
[21] Pun, T.: ‘Entropic thresholding, a new approach’, Comput. Graph. Image IEEE Trans. Med. Imaging, 2017, 36, (2), pp. 653–665
Process., 1981, 16, (3), pp. 210–239
[22] Sun, F., Wang, H., Fan, J.: ‘2D Otsu segmentation algorithm based on
simulated annealing genetic algorithm for iced-cable images’. Int. Forum on
Information Technology & Applications, Chengdu, China, 2009, pp. 600–602

544 IET Image Process., 2020, Vol. 14 Iss. 3, pp. 536-544

© The Institution of Engineering and Technology 2019