Noname manuscript No.

(will be inserted by the editor)

Vegetation segmentation based on variational level set using

multi-channel local wavelet texture and color
Tiejun Yang · Yaowen Chen · Zhun Fan

Received: date / Accepted: date

Abstract The existing spectrum index-based methods tomation applications. DVC methods are generally based
for detecting vegetation coverage suffer from an overde- on visible spectral indexes (VSIs), including the Excess
pendence on spectrum. To address these issues, this pa- Green index (ExG) [1], Excess Red index (ExR) [2],
per proposes a graph cut based variational level set seg- Color Index of Vegetation Extraction (CIVE) [3], Ex-
mentation algorithm that combines multi-channel local cess Green minus excess Red index (ExGR) [4] and the
wavelet texture (MCLWT) and color. First, the prior Vegetative index (VEG) [5]. The advantages of VSI-
color is generated by automatic estimation based on based methods are that they can accurately segment
the mathematical morphology with a color histogram. vegetation, soil and sky and that visible spectral sen-
Then, local wavelet texture features are extracted using sors are not very expensive. Other DVC methods based
a multi-scale-and-orientation Gabor wavelet transfor- on multi-spectral images have advantages of wider cov-
mation followed by local median and entropy filtering. erage and better accuracy than VSIs, although only at
Next, in addition to the energy of color, that of M- considerably higher cost. However, both the VSI-based
CLWT is integrated into the variational level set model and multi-spectral-based methods share a common de-
based on kernel density estimation. Consequently, all ficiency: they have an over-dependence on spectrum or
energies are integrated into the graph cut based vari- color while neglecting wider spatial information such as
ational level set model. Finally, the proposed energy texture.
functional is made convex to obtain a global optimal Therefore, combinations of spectral indexes and oth-
solution, and a primal-dual algorithm with global rela- er methods have been proposed to improve the accura-
beling is adopted to accelerate the evolution of the lev- cy of DVC. Ponti [6] presented a combination of spec-
el sets. A comparison of the segmentation results from tral indexes and mean shift (MS) to segment balloon-
our proposed algorithm and other state-of-the-art al- captured remote sensing images, including MS+CIVE,
gorithms showed that our algorithm effectively reduces MS+ExG and MS+VVI. To improve the robustness of
the over-dependence on color and yields more accurate DVC in the presence of illumination variations or plan-
results in detecting vegetation coverage. t canopy shadows, Bai et al. [7] used particle swarm
Keywords Variational level set · detecting vegetation optimization (PSO) clustering and morphology mod-
coverage · local wavelet texture · graph cut elling to segment vegetation from soil in color images
acquired by an off-the-shelf digital camera affixed to an
image acquisition device. Unsupervised methods such
1 Introduction as MS or PSO clustering help in determining an ap-
propriate threshold for spectral indexes from images,
Detection of vegetation coverage (DVC) is a highly im- although errors always occur when the spectra of the
portant aspect of machine vision-based agricultural au- foreground and background overlap.
Zhun Fan
In addition to spectrum information or color, other
Department of Electrical Engineering, Shantou University, cues such as texture and shape have been considered in
Guangdong, P. R. China DVC. Feng et al. [8] used colors in the RGB color space
E-mail: [email protected] and textures based on the gray-level co-occurrence ma-
2 Tiejun Yang et al.

trix (GLCM) as the input for a random forest to map 2 Background

urban vegetation from UAV-captured remote sensing
images. Aksoy et al. [9] exploited both spectral- and Ga- The graph cut divides all pixels in an image into three
bor filter-based texture and shape properties to detect categories: source, sink and boundary pixels. Only when
hedgerows using decision-making methods. Texture fea- a pixel is classified into the category to which it truly
tures based on GLCM have a problem determining the belongs is the cut set minimized. The graph cut prin-
scale and orientation of objects, while multi-resolution ciple can be used to model the energy functional of a
analysis based methods such as Gabor filters can ex- level set [14][17].
tract the textural features of vegetation at different s- Based on GCTV, Bresson et al. [17] also presented
cales and orientations. Machine learning methods are a VLS model and defined a boundary weight function
also often used [10], but these methods rely on the avail- for the total variation. In these methods, the source
ability of numerous good training samples, and they are pixels indicate the object, whose cost is denoted by cf .
prone to overfitting and other issues. The sink pixels indicate the background; their cost is
denoted by cb . The boundary pixels separate the object
from the background; their cost is denoted by ce . Using
Image segmentation methods based on the varia-
the graph cut principle, an image segmentation problem
tional level set (VLS) model [11] have attracted a sub-
can be viewed as one of classifying all the pixels into the
stantial amount of attention [12, 13]. Unger M. et al.
above three classes. A minimal cut set is obtained only
[14] proposed a variational level set segmentation mod-
when all pixels have been correctly classified. Thus, the
el based on graph cut and total variation (GCTV) that
segmentation problem can be viewed as minimizing the
further exploited the primal-dual and global relabeling
following energy functional:
(PDGRL) method to minimize the primal-dual ener-
Z Z Z
gy interval. However, GCTV uses only prior color as
Egctv = ce |∇u| + cb udx + cf (1 − u) dx
an external energy and, consequently, suffers from an Ω Ω Ω (1)
over-reliance on it. The VLS segmentation model has s.t.u (x) ∈ {0, 1} ,
a different problem: a non-convex functional leads to
a local optimal solution. To eliminate the non-convex where Ω ⊂ R2 is an open set representing the image
factors, Chan et al. [15] proposed convexity transform domain and u is a characteristic function.
algorithms for certain non-convex minimization prob- To obtain a minimum of the functional in Eq. (1),
lems. These transform algorithms are called Algorithms ce is set to be inversely proportional to the boundary
for Finding Global Minimizers (AFGM). gradient and cf and cb are estimations of the probabil-
ity distribution of the object color and the background
Hence, we propose an improved method based on color, respectively. These estimations can be obtained
GCTV to perform DVC that improves the segmenta- automatically using the method described in Section
tion accuracy by integrating more features, such as tex- 3.1.
ture and automatically generated prior color, and by One problem in GCTV is that the functional in Eq.
convexity transforms of the non-convex energy func- (1) contains a binary function u. In other words, it is
tionals. In our method, multi-channel textures are ex- a non-convex problem that has only a local optimal
tracted by the Gabor wavelet transform combined with solution. Therefore, the problem must be transformed
local mean and entropy filters in a process called multi- into a convex one. We use a convexity transform method
channel local wavelet texture (MCLWT). MCLWT was proposed by Chan et al. [15] to solve this problem. We
shown to be effective in vegetation segmentation in our refer readers to [15] for details.
previous work [16]. Prior color is obtained automatical-
ly by mathematical morphology and color histogram.
3 Method
Then, these features are integrated into the energy func-
tional of GCTV to evolve the level sets.
Our method includes three major processes: feature
extraction, model adaptation and post-processing, as
The rest of this paper is organized as follows: Sec- shown in Fig. 1. To improve segmentation accuracy
tion 2 discusses the VLS model based on GCTV and by integrating more features, three kinds of features—
the convexity transform method. Then, the proposed automatically estimated prior color, the a* channel col-
algorithm is described in detail in Section 3. Section or from the L*a*b* color space and multi-channel local
4 presents the experimental results and a discussion. wavelet texture (MCLWT)—are extracted. To utilize
Finally, Section 5 provides conclusions. the extracted features as supplementary energies in the
Vegetation segmentation based on variational level set using multi-channel local wavelet texture and color 3

Fig. 1: Flow chart of the proposed method, including pre-processing, segmentation and post-processing.

level set model, the features are integrated as external b) Next, a mathematical morphology thicken oper-
energies into the graph cut based level set segmentation ator is applied, making fobj shk one pixel thicker.
model. To achieve global optimization more convenient- c) Finally, a mathematical morphology open oper-
ly, the proposed energy functional is transformed into a ator is imposed to reserve the most likely areas
convexity functional using Chan’s convexity transform of vegetation using a structuring element (SE),
method [15] as described above. We also adopt PDGRL which is a line 10 pixels long at a 2-degree angle.
to speed up the level set evolution in our method. Fi- The result is denoted by fobj .
nally, in the post-processing stage, mathematical mor- 3) To obtain the AEPC for non-vegetation, we use the
phology operations are used to erase tiny islands and following steps:
fill in small holes in the segmentation results. a) A shrink operator is applied to fbw .
b) Then, a mathematical morphology dilate opera-
tor is used to find the likely areas of non-vegetation
3.1 Automatic estimation of prior color using an SE line 8 pixels long at a 2-degree angle.
The result is denoted by fbkg .
To obtain prior color, manual labelling is widely used in 4) Using fobj and fbkg as masks [14], color histograms
[18,14]. However, manual labelling requires human in- of the vegetation hf and non-vegetation hb are cal-
teraction and its efficiency is low—especially for large culated.
data sets. To address this problem, automatic estima- 5) The histograms are smoothed by a Gaussian filter
tion of prior color (AEPC) is required. Here, we pro- and then linearly interpolated at each pixel’s inten-
pose an approach based on CIVE and mathematical sity value from the original image, yielding the final
morphology that can automatically generate approxi- prior color histograms, Hf and Hb , respectively.
mate probability distributions of vegetation and non-
vegetation. CIVE is used here to generate estimated Fig. 2 shows an example of AEPC. The visualized image
segmentation results. Then, mathematical morphology of AEPC fAEP C in Fig. 2h is composed as follows:
is applied to obtain the primary components of the ob-
jects and background. These processes are described fAEP C = [R, G, B] ,
(3)
below: R = fbkg , G = 0, B = fobj .
1) A given image f (x, y) is first segmented using CIVE
to obtain binary segmentation results. CIVE is com-
puted as follows:
3.2 Multi-channel local wavelet texture
fCIV E = 0.441R − 0.811G + 0.385B + 18.787, (2)
where R, G and B are the values of the color chan- Because vegetation usually has different textural fea-
nels of f (x, y) in the RGB color space. Then Otsu tures than those of surrounding soil, water, etc., we
thresholding is used to transform fCIV E into a bi- introduce a texture feature called multi-channel local
nary value, fbw . wavelet texture (MCLWT) into the proposed model for
2) To obtain the AEPC for vegetation, we use the fol- vegetation segmentation. A Gabor wavelet transform
lowing steps: is insensitive to illumination variation and geometric
a) A mathematical morphology shrink operator is transformation. Therefore, we use it here to transfor-
applied to the inverse of fbw , shrinking the areas m the original image into a multi-resolution frequency
of vegetation to connected lines or points. The domain. The two-dimensional Gabor wavelet transform
results are denoted by fobj shk . w (·, ·) for an image f (x, y) with a frequency of f0 and
4 Tiejun Yang et al.

(a)

(a) (b) (c) (d)

(b)

Fig. 3: Gabor filter bank with 2 frequencies and 4

(e) (f) (g) (h) orientations (θ = 0,π/4,π/2,3π/4): (a) f0 = 0.2, (b)
f0 = 0.5.
Fig. 2: An AEPC example: (a) the original image from
DB1, (b) the segmentation results by CIVE, (c) the
shrink operator results on the inverse of (b), (d) the 3.3 Integration of color and texture energies
thicken operator results on (c), (e) the prior color es-
timation results of vegetation by the open operator on First, the energy of prior color Ep can be produced by
(d), (f) the shrink operator results on (b), (g) the prior AEPC and integrated directly into the proposed energy
color estimation results of non-vegetation by the dilate functional using Eq. (1) as follows:
R
operator on (f), and (h) a visualized image of AEPC. Ep = Ω (cb − cf ) udxdy,
(6)
s.t. u (x, y) ∈ {0, 1} ,
R
an orientation of θ can be written as follows: where cb = Hb and cf = Hf . The item Ω cf dx at the
right of Eq. (1) is omitted because it is independent of
ZZ
wf0 ,θ (u, v) = f (x, y) h∗G (u − x, v − y) dxdy, u.
f02
  2 2
f0 xr f02 yr2
 Second, the energy of the a* color channel in the
hG (x, y) = exp − + 2 CIE L*a*b* color space, denoted by Ec , is added as a
πσx σy σx2 σy (4) fidelity term based on the idea of the C-V model and
· exp [j2πf0 xr ] , [19], which is given by
xr = x cos (θ) + y sin (θ) , 2
H (φ) (c0 1 − fa∗ (x, y))
Z !
yr = −x sin (θ) + y cos (θ) , Ec = 2
dxdy,
Ω + (1 − H (φ)) (c0 2 − fa∗ (x, y))
where * denotes the complex conjugate, hG (·) is the Z 
Gabor wavelet function, and σx and σy are the scal- c0 1 = fa∗ (x, y)dxdy |Σ|, (7)
ing parameters along the wave and perpendicular to Σ
,
the wave, respectively. Here, σx = σy = 1, and we
Z
use a bank of filters defined in a specific frequency c0 2 = fa∗ (x, y)dxdy |Ω\Σ|,
Ω\Σ
range and orientation. An example of this filter bank
with 2 frequencies (f0 = 0.2, 0.5) and 4 orientations where fa∗ is the a* color channel. Because H (φ) ∈
(θ = 0,π/4,π/2,3π/4) is shown in Fig. 3, which yields {0, 1}, it can be replaced by u. Then, Ec is transformed
8 channels after the transform. Then, the local wavelet as follows:
2!
texture feature fMCLWT is extracted based on the Ga- (1 − u)(c0 1 − fa∗ (x, y))
Z
bor wavelet transform defined in Eq. (4): Ec = 2
dxdy
Ω +u(c0 2 − fa∗ (x, y))
fMCLWT = (fmed , fetp ) , 2 !
(8)
(c0 2 − fa∗ (x, y))
Z
= udxdy.
!
2
fmed = median wf0 ,θ (p, q) , Ω −(c0 1 − fa∗ (x, y))
(p,q)∈W (5) R  
2
! The item Ω (c0 1 − fa∗ (x, y)) dxdy at the right of
fetp = entropy wf0 ,θ (p, q) , Eq. (8) is omitted because it is independent of u.
(p,q)∈W
Third, the texture energy based on MCLWT is in-
where median (·) and entropy (·) are functions of the tegrated into our model. There are several approaches
median and entropy filters of a filtered window W with for integrating texture into a variational level set mod-
a size of n in wf0 ,θ , respectively. Using this feature ex- el, including mean value-based approaches such as the
traction approach, fMCLWT has 16 channels of MCLWT- C-V model [20], the Gaussian mixed model (GMM)-
based textures in total. based [21], kernel density estimation (KDE)-based [22],
Vegetation segmentation based on variational level set using multi-channel local wavelet texture and color 5

and so on. However, because the mean value-based ap- 4 Experiments

proaches are not reliable enough for textured objects,
and GMM based approaches are well-known for overfit- Our experiments were performed on a PC with a 2.5
ting, we adopt KDE to build the texture energy based GHz Intel Core 2 Q8300 processor and 4 GB of 800
on MCLWT using the following steps: MHz DDR2 RAM under MATLAB 2013b. We used t-
i wo image databases of vegetation, DB1 and DB2 from
a) For each channel of fMCLWT , i = 1, 2, ..., C, KDE is
[6] and [7] respectively, to perform segmentation ac-
used to estimate each channel’s probability distri-
i

curacy comparisons. DB1 contains 12 images of bean
bution pj fMCLWT , j = 1, 2, where C is the total
fields, while DB2 contains 200 images of rice fields. The
number of channels of fMCLWT , and p1 and p2 de-
ground truths for all the images were manually labeled
note the probability distributions of the object and
by specialists. The default parameter settings were as
background, respectively.
follows: n = 11, λ1 = λ2 = 1, and λ3 = 100.
b) Calculate the total texture probability distributions
of the object and background by
X X
i i



tf = p1 fMCLWT , tb = p2 fMCLWT , (9) 4.1 Accuracy evaluation
i i
where tf is the MCLWT texture energy of the ob- We used the metric from [6], denoted as Acc1 , to com-
ject, and tb is the texture energy of the background. pare the proposed segmentation algorithms on the DB1
Subsequently, the MCLWT texture energy is integrated images. The Acc1 metric is defined as follows:
Xr .
into the GCTV model. The proposed texture energy Acc1 = 1 − Erri 2r
functional Et is i=1
Z Z Erri = ei,1 + ei,2 , (12)
Et = tb udxdy + tf (1 − u) dxdy F Pi F Ni
Z Ω Ω
(10) ei,1 = , ei,2 = , i = 1, ..., r,
N − Ni Ni
= (tb − tf ) udxdy.
Ω where FP is the number of false positives, and FN is
the number of false negatives. F Pi is the number of
R
The item Ω tf dxdy at the right of Eq. (10) is omit-
ted because it is independent of u. Therefore, the final pixels of region j 6= i misclassified into region i, F Ni
functional of our model EMCLWT is given by represents the pixels of region i misclassified into other
regions, N is the number of pixels in a given image,
Z and Ni denotes the pixels that belong to region i. For
EMCLWT = ce |∇u| dxdy + λ1 Ep + λ2 Ec + λ3 Et ,(11) binary segmentation, here r = 2.
Ω
We use the metric called the Jaccard index (J ) from
where λ1 , λ2 and λ3 are positive constants. The first
[7] to evaluate the segmentation performance on the
term to the right of Eq. (11) is the total variation; the
DB2 images. The Jaccard index is calculated as follows:
others are the fidelity terms of prior color, a* color and
MCLWT texture energies, respectively. Using Eq. (11), J = T P /(T P + F P + F N ). (13)
the color and texture energies are integrated into the
GCTV framework in a unified manner. where TP is the number of true positive pixels.
In addition to the above measures, the metrics of
sensitivity (Sens), specificity (Spec), accuracy (Acc2 ),
3.4 Global optimization based on convexity transform precision (Prec) and F-measure (F ) were calculated in
our experiments for further quantitative comparison.
To achieve global optimization, we adopt AFGM to e- The formulas of these metrics are given in Eqs. (14)–
liminate the non-convex factors in the proposed func- (18), respectively:
tional defined by Eq. (11). First, u ∈ {0, 1} is relaxed by
Sens = T P /T P + F N , (14)
u : R2 → [0, 1]. Then, a globally optimal solution, uλ ,
can be achieved. Finally, a binarized solution of the o- Spec = T N /F P + T N , (15)
riginal problem is obtained using the threshold method.
The details can be found in [15]. Acc2 = (T P + T N )/(T P + F P + T N + F N ), (16)
For fast convergence, we use PDGRL [14] to min-
imize the energy functional in Eq. (11). The PDGRL P rec = T P /T P + F P , (17)
algorithm reduces the energy interval between the orig- F = 2 ∗ P rec ∗ Sens/(P rec + Sens). (18)
inal and its dual problem using iterative and threshold
methods, thereby increasing the algorithm’s efficiency. where TN is the number of true negative pixels.
6 Tiejun Yang et al.

4.2 Numerical tests Table 1: Comparison data of the state-of-the-art algo-

rithms using DB1
First, the proposed algorithm was compared to 8 state-
Algorithm µ (%) σ (%)
of-the-art vegetation segmentation algorithms on the CIVE [3] 66.1 11.9
DB1 images. These algorithms cover the major type- ExG [24] 78.5 8.6
s of effective segmentation techniques. Among these MS [23] 76.5 10.7
algorithms, the one proposed by Cheng et al. [23] is VVI [6] 70.4 10.5
MS+VVI [6] 72.6 13.3
clustering-based; the ones presented by Kataoka et al. MS+ExG [6] 85 8.4
[3], Gée et al. [24] and VVI [6] are VSI-based; and the MS+CIVE [6] 86.4 7.2
Mean Shift combined with VVI (MS+VVI), Mean Shift GCTV [14] 88.9 6.1
combined with ExG (MS+ExG) and Mean Shift com- Proposed 89.1 5.7
bined with CIVE (MS+CIVE) algorithms proposed in
[6] are hybrid methods that combine the preceding t-
wo types of methods. The GCTV algorithm [14] is the
most similar to ours. Table 1 lists the accuracy scores
of the comparison results measured by Acc1 as defined
in Eq. (12) for the DB1 images, where µ and σ denote (a) (b) (c) (d)
the mean and standard deviation of Acc1 , respective-
ly. As shown in Table 1, CIVE has the lowest accu-
racy (66.1%). This result occurs because CIVE simply
uses each pixel’s color to perform segmentation. The
algorithms that combine clustering and VSI such as M- (e) (f) (g)
S+ExG and MS+CIVE use the mean values of region-
al features to merge or split areas based on the seg- Fig. 4: Comparison of our algorithm with 4 other state-
mentation results of the VSI algorithms. These hybrid of-the-art algorithms reported in [6]. The black pix-
algorithms achieve higher accuracies than do the VSI- els denote vegetation and the white pixels represen-
based ones, 85% and 86.4%, respectively. Using AEPC, t the background. (a) An original image from DB1,
GCTV achieves a high accuracy of 88.9%. Our method (b) ground truth, (c)–(g) the segmentation results by
integrates both color and textures in the proposed level ExG, CIVE, MS+ExG, MS+CIVE and our algorithm,
set segmentation model; consequently, it achieves the respectively.
highest accuracy (89.1%) and the lowest standard de-
viation (5.7%).
A sample image from DB1 and the segmentation are more precise compared to those obtained the other
results of ExG, CIVE, MS+ExG, MS+CIVE and our two methods. Note that the ground truth obtained by
method are shown in Fig. 4. These compared algorithm- manual labelling tends to lose some details of the vege-
s were claimed to be superior to other VSI-based algo- tation and background, while our method catches these
rithms in [6]. Fig. 4a and Fig. 4b show the original im- subtle differences surprisingly well.
age and the ground truth, respectively. From a visual We also compared the segmentation qualities of the
comparison, the index methods (ExG and CIVE) ob- proposed algorithm with GCTV [14] and those of six
tain the worst results because they contain numerous other methods reported in [7] on the DB2 images. A-
FPs or FNs, as shown in Fig. 4c and Fig. 4d. The hy- mong these algorithms, ExG&Otsu and ExGExR are
brid methods (MS+ExG and MS+CIVE) perform bet- VSI-based methods; EASA, GMM and ColourHist are
ter than the indexed methods, as depicted in Fig. 4e statistical methods; and ClusterMorph is a hybrid method
and Fig. 4f, respectively, because they consider region- that uses clustering and mathematical morphology. The
al color features during segmentation. Our results are performances of these algorithms are compared using
quite similar to those of the hybrid methods, although the means µ and standard deviations σ of the metric
some details are different, as demonstrated in Fig. 5. defined by Eq. (13) in Table 2.
Fig. 5 shows two areas cropped from the original According to the experiments, the VSI-based meth-
image marked by the two labeled white boxes in Fig. ods (ExG&Otsu and ExGExR) performed the worst,
4a. The 1st row of Fig. 5 refers to box No. 1 and the their means on the J metric were only 76.2% and 62.3%,
2nd row to box No. 2. As shown, the hybrid methods respectively. EASA, GMM and ColourHist use the prob-
segment more background pixels as vegetation. Appar- ability density function of color instead of the thresh-
ently, the background areas segmented by our method olding used by the VSI-based methods, and they yielded
Vegetation segmentation based on variational level set using multi-channel local wavelet texture and color 7

(a) (b) (c) (d) (e)

Fig. 5: Comparison of our algorithm and the hybrid al-

gorithms on an image of DB1 [6]: (a) Two areas cropped
from the original image are marked by the two labeled
white boxes in Fig. 4a. The first row refers to box No.
1 and the second row refers to box No. 2. (b) Ground
(a) (b) (c) (d)
truth. (c)–(e) the segmentation results of the areas by
MS+ExG, MS+CIVE and our method, respectively.
Fig. 6: Examples of segmentation results by our al-
gorithm and ClusterMorph from [7]. The black pix-
Table 2: Segmentation results of state-of-the-art algo- els represent rice and the white pixels represent the
rithms on the DB2 images background: (a) The original images from DB2; (b) the
ground truth; (c)–(d) the results by ClusterMorph and
Algorithm µ (%) σ (%)
ExG&Otsu [25] 76.2 7.7 our algorithm, respectively.
ExGExR [25] 62.3 18.1
EASA [26] 80.2 7.8 Table 3: Other performance metrics of our algorithm
GMM [27] 86.9 6.9
ColourHist [28] 82.1 6.4 on the DB1 images (µ and σ denote the mean values
ClusterMorph [7] 88.1 4.7 and standard deviations, respectively)
GCTV [14] 82.9 9.7
Proposed 92.7 4.3 Sens Spec Acc2 Prec J F
µ (%) 95.04 83.17 93.90 98.07 93.31 96.47
σ (%) 4.98 10.55 4.67 1.72 5.27 2.91

better results: µ > 80%. ClusterMorph obtained a high

µ of 88.1 through its use of particle swarm optimiza- Table 4: Other performance metrics of our algorithm
tion clustering and morphology modelling in the CIE on the DB2 images (µ and σ denote the mean values
L*a*b* color space, although it also required an extra and standard deviations, respectively)
offline learning stage. Our algorithm performs the best
Sens Spec Acc2 Prec F Acc1
(µ = 92.7% and σ = 4.3%), which is much better than µ (%) 99.08 82.95 94.65 93.49 96.14 91.15
GCTV. σ (%) 2.52 9.24 3.08 4.04 2.50 3.84
Fig. 6 depicts some segmentation examples from D-
B2 obtained by ClusterMorph and our algorithm. Clus-
terMorph was chosen for comparison here because it- 5 Conclusion
s performance is superior to the other methods based
on the results in Table 2. The 3 rows demonstrate 3 The proposed method, which integrates both multi-
examples under different illumination: the black pixels channel local wavelet texture and color, obtains a higher
represent rice areas and the white pixels represent the vegetation segmentation accuracy compared to several
background. As shown, ClusterMorph produced more other state-of-the-art methods. By combining the prior
FPs than did our algorithm under these different illu- color of the estimated object and the background with
mination conditions—especially in shaded areas. local wavelet texture features, the proposed method us-
In addition to the preceding evaluation, Table 3 and es both the colors of pixels and regional texture fea-
Table 4 list the means and standard deviations of the tures. In addition, it avoids the strong dependence on
other performance metrics defined by Eqs. (14)–(18) prior color exhibited by GCTV-based methods and sub-
obtained by our algorithm on the DB1 and DB2 im- stantially improves the accuracy of segmentation result-
ages, respectively. Among these metrics, the mean value s. Due to the regional texture fidelity term, the segmen-
of Spec is approximately 83%, while those of the other tation results for areas with similar colors but different
metrics are all above 90%. textures are more accurate.
8 Tiejun Yang et al.

Acknowledgements This research was partly supported by 14. M. Unger, T. Pock, and H. Bischof. Global relabeling
the Guangdong Provincial Key Laboratory of Digital Signal for continuous optimization in binary image segmenta-
and Image Processing Techniques (2013GDDSIPL-03) and by tion. Energy Minimization Methods in Computer Vi-
the Guangxi Basic Ability Promotion Project for Young and sion and Pattern Recognition, Lecture Notes in Comput-
Middle-aged Teachers (2017KY0247). er Science, 6819:104–117, 2011.
15. Nikolova M Chan T. F., Esedoglu S. Algorithms for
finding global minimizers of image segmentation and de-
noising models. SIAM Journal on Applied Mathematics,
References 66(5):1632–1648, 2006.
16. Tiejun Yang, Lin Huang, and Ping Hu. Vegetation seg-
1. Von Bargen K. Woebbecke D.M., Meyer G.E. and mentation based on level set using local wavelet texture.
Mortensen D.A. Color indices for weed identification un- In 2016 IEEE 13th International Conference on Signal
der various soil, residue, and lighting conditions. Trans- Processing, pages 222–227.
actions of the ASAE (USA), 38(1):259–269, 1995. 17. Xavier Bresson, Selim Esedoglu, Pierre Vandergheynst,
2. George E. Meyer, Timothy W. Hindman, and Koppolu Jean-Philippe Thiran, and Stanley Osher. Fast global
Laksmi. Machine vision detection parameters for plant minimization of the active contour/snake model. Jour-
species identification. Photonics East, 1999. nal of Mathematical Imaging and Vision, 28(2):151–167,
3. T. Kataoka, T. Kaneko, H. Okamoto, and S. Hata. Crop 2007.
growth estimation system using machine vision. In Ad- 18. Carsten Rother, Vladimir Kolmogorov, and Andrew
vanced Intelligent Mechatronics, 2003. AIM 2003. Pro- Blake. Grabcut: interactive foreground extraction us-
ceedings. 2003 IEEE/ASME International Conference ing iterated graph cuts. Acm Transactions on Graphics,
on, pages b1079–b1083. 23(3):309–314, 2004.
4. George E. Meyer and Joao Camargo Neto. Verification 19. Tiejun Yang, Lin Huang, and Chuanxian Jiang. Wavelet
of color vegetation indices for automated crop imaging and total variation based level set segmentation method
applications. Computers and Electronics in Agriculture, for portraits. International Journal of Digital Content
63(2):282–293, 2008. Technology and its Applications, 7(4):973–982, 2013.
5. T. Hague, N. Tillett, and H. Wheeler. Automated crop 20. Chan T. F. and Vese L. A. Active contours without edges.
and weed monitoring in widely spaced cereals. Precision IEEE Transactions on Image Processing, 10(2):266–277,
Agriculture, 7(1):21–32(12), 2006. 2001.
6. M.P.Ponti. Segmentation of low-cost remote sensing im- 21. Zhenping Xie, Shitong Wang, and Dewen Hu. New in-
ages combining vegetation indices and mean shift. IEEE sight at level set & gaussian mixture model for natu-
Geoscience and Remote Sensing Letters, 10(1):67–70, ral image segmentation. Signal Image Video Processing,
2013. 7(3):521–536, 2013.
7. Xiaodong Bai, Zhiguo Cao, Yu Wang, Zhenghong Yu, 22. J. Chang and J.W. Fisher. Efficient mcmc sampling
Zhu Hu, Xuefen Zhang, and Cuina Li. Vegetation seg- with implicit shape representations. In IEEE Conference
mentation robust to illumination variations based on on Computer Vision and Pattern Recognition (CVPR),
clustering and morphology modelling. Biosystems En- pages 2081–2088, 2011.
gineering, 125(3):80–97, 2014. 23. Yizong Cheng. Mean shift, mode seeking clustering.
8. Quanlong Feng, Jiantao Liu, and Jianhua Gong. Uav re- IEEE Transactions on Pattern Analysis and Machine
mote sensing for urban vegetation mapping using random Intelligence, 17(8):790–799, 1995.
forest and texture analysis. Remote Sensing, 7(1):1074– 24. C. Gée, J. Bossu, G. Jones, and F. Truchetet. Crop/weed
1094, 2015. discrimination in perspective agronomic images. Com-
9. S. Aksoy, H. G. Akcay, and T. Wassenaar. Automat- puters and Electronics in Agriculture, 60(1):49–59, 2008.
ic mapping of linear woody vegetation features in agri- 25. J. C. Neto. A Combined Statistical-Soft Computing Ap-
cultural landscapes using very high resolution imagery. proach for Classification and MappingWeed Species in
IEEE Transactions on Geoscience and Remote Sensing, Minimum-Tillage Systems. Thesis, 2006.
48(1):511–522, 2010. 26. Lei F. Tian and David C. Slaughter. Environmentally
adaptive segmentation algorithm for outdoor image seg-
10. M. S. Nia, Zhe Wang, P. Gader, S. A. Bohlman, S. J.
mentation. Computers and Electronics in Agriculture,
Graves, and M. Petrovic. Impact of atmospheric correc-
21(3):153–168, 1998.
tion and image filtering on hyperspectral classification of
27. L. M. Bergasa, M. Mazo, A. Gardel, M. A. Sotelo, and
tree species using support vector machine. Journal of
L. Boquete. Unsupervised and adaptive gaussian skin-
Applied Remote Sensing, 9(1):095990, 2015.
color model. Image and Vision Computing, 18(12):987–
11. Osher S. and Sethian J. Fronts propagating with cur-
1003, 2000.
vature dependent speed: Algorithms based on hamilton-
28. M. J. Jones and J. M. Rehg. Statistical color models
jacobi formulations. Journal of Computational Physics,
with application to skin detection. International Journal
79(1):12–49, 1988.
of Computer Vision, 46(1):81–96, 2002.
12. Kangwei Liu, Jianhua Wan, and Zhongzhi Han. Ab-
normal event detection and localization using level set
based on hybrid features. Signal Image Video Process-
ing, 11(6):1–7, 2017.
13. Zahra Shahvaran, Kamran Kazemi, and Mo-
hamad Sadegh Helfroush. Simultaneous vector-valued
image segmentation and intensity nonuniformity correc-
tion using variational level set combined with markov
random field modeling. Signal Image Video Processing,
10(5):887–893, 2016.