Weed detection and classification for autonomous farming

S. Kodagodaa, Z. Zhanga, D. Ruizb, G. Dissanayakea

a
ARC Centre of Excellence for Autonomous Systems (CAS), University of Technology, Sydney, AUSTRALIA
b
Australian Centre for Precision Agriculture (ACPA), University of Sydney, Sydney, AUSTRALIA

Abstract

Autonomous weed control concepts have recently being extensively researched due to the advantages that they
possess. One of the critical modules of such systems is the sensing and classification of weeds within crops. In this
paper, we systematically chose the sensing setup and cues to be used for classification of two common weed species
(Bidens pilosa L. and Lolium rigidum L.) in a wheat crop. An automatic cue selection followed by classification
procedure is proposed. Some classification results are presented while discussing problems leading to future direction
of research.

Keywords: Weed detection, spectral analysis, classification

1. Introduction sensing and data logging. It can be programmed for

autonomous navigation. It has accurate GPS
There are a number of field operations that can be localisation system and weighs 89kg with approximate
executed by autonomous vehicles, giving more benefits size of 800L x 550W x 400H (mm).
than conventional man-guided machines [1]. These
vehicles would be able to work unattended over long It is planned to implement a
periods of time, carrying out useful tasks such as fully insulated and isolated
cultivation and seeding, weed control, soil scouting, electrocution cradle extending
application of fertilizers, irrigation, and harvesting. out at the back of the robot to
Automatic guidance may avoid over-application of be used to destroy the weeds. It
will have five independent
chemicals and fertilizers, reducing environmental
electrodes at 20kV covering
impact [2]. As weed populations have been found to be Fig. 1 CASPA weeder 250 mm width and a spiked
distributed heterogeneously in time and space within
castor wheel as the ground
agricultural fields, weed control systems based on electrode. The authors’ main contribution to this
vision have been developed to spray specifically the project is to synthesise a low cost, real time crop-weed
weed infested areas in real-time, reducing treatment classification algorithm.
costs as well as herbicide loading to the environment In the literature, crop-weed classification falls into
[3-4]. two basic categories: spectral data based classification
The authors are collaborating with [5] to realize an and computer vision based classification. Spectral data
autonomous weed control system based on CASPA based technology relies on the difference of spectral
weeder shown in Fig. 1. The CASPA weeder has the response of each plant species. Jurado et al [6] use a
capabilities of remote control, joystick based control, NIRS monochromator to measure plants’ spectral
reflectance in a lab environment. The spectral fields in southern Australia yield reduction can be up to
reflectance data was then analyzed statistically. They 50% at weed densities of 200-500 plants/m2 [12].
have shown that the spectral difference in the band of Bidens pilosa L (commonly known as cobbler's peg) is
750nm to 950nm is suitable for discriminating an annual broad leaf weed widely distributed in
sunflower and wheat stubble. However, the tropical and subtropical regions of the world and is
experimental results also contained overlapping reported to be a weed of 31 crops [13]. Therefore, in
spectral responses from dissimilar plants. They this work our particular attention is paid to
proposed to use an airborne hyper spectral camera for classification of wheat, Bidens and Lolium species.
better classification results albeit higher costs of the The rest of the paper is arranged as follows. Section 2
equipment. Borregaard et al [7] adopted a line scan discusses the spectral analysis of the concerned plant
spectral device to implement on site weed detection. In species. Section 3 discribes the experimental system.
their work, a line scan spectrometer was mounted Crop weed classification algorithm is given in Section
pointing down observing plants. The line scanning 4. In Section 5, experimental results are given. Section
spectrometers provide additional spatial and textural 6 concludes the paper.
information to that of normal spectrometers with a spot
foot print. Classification methods such as LDA, QDA, 2. Spectral analysis
PCA, and PLS were used to classify the spectral data.
However, the accuracy of Borregaard’s algorithm was It is a well-known fact that various plant species
between 60% and 90%. Eddy et al [8] used a hyper have different optical and light absorption properties.
spectral camera with a resolution of 640x480. Each That is mainly due to pigments in the UV and visible
pixel from this camera can have a spectral resolution of wavelengths, chemical composition in the NIR range,
10nm within the spectral band of 400nm to 1000nm. A and numerous leaf tissue structures. Leaf pigments,
feed forward neural network was trained to classify the which contain Chlorophyll absorb large amount of light
spectral data, and a detection rate of 88%-95% was in the UV band, blue (450nm) and red (680nm) part of
achieved. The hyper spectral camera can capture more the spectrum, whilst slightly lower absorption in the
information than that of line scan spectrometers. green (550nm) band (see Fig. 2). The leaf pigments do
However, the high cost of the hyper spectral camera not have a good NIR light absorption property, hence
prevent its feasibility of using in low cost weed the leaves reflect or diffuse large amount of radiation.
detection systems. The transmitted radiation is further affected by the leaf
Computer vision based techniques have also been internal structure, which can be used for classification
extensively exploited for crop-weed discrimination. A of diverse plant species.
color image is rich in information providing cues such Fig.2 shows the spectral response of three types of
as color, texture, shape,…. etc. Perez et al [9] have plant species under laboratory conditions. The plants
chosen color and shape as appropriate visual cues. The were grown in trays and spectral responses were
cues were then analyzed by a heuristical approach, measured with artificial lighting. Oceanoptic
Bayes classification and k-nearest Neighbor spectrometer was used and measurements were taken
classification. They managed to achieve a detection on leaves of the species. Fig. 2 shows a large variation
rate of 71% to 89%. Aitkenhead et al [10] proposed to in the responses of three species within the spectral
use shape features in a neural network to achieve a band, 750nm to 950nm (NIR). It also shows up another
detection rate of 50%-90% for crop/weed band around 550nm (green). The steep slope 700nm to
classification. Hemming et al [11] could only achieve a 750nm is called the “red edge”.
detection rate of 50% to 85% using shape features. After analyzing several number of data sets taken
from the same type of species, it was noted that the
Large variation of the detection performance indicates
spectral responses within a particular plant species can
that the accuracy of the shape parameter calculation is
significantly vary causing a straightforward
not robust. This can be due to occlusions introduced by
segmentation based on spectral analysis is erroneous.
proximity leaves. Fig.3 shows the spectral variations of each type of
Lolium rigidum L (commonly known as ryegrass) is species and the obvious overlapping regions. For
a widespread grass weed in cereal fields, causing example, considering the spectral band 750nm to
important yield reductions and having evolved 900nm, it can be seen that the spectral variation of
resistance to five major groups of herbicides. In wheat
wheat is large enough to contain some Bidens and rye 550nm and 750nm – 950nm.
grass responses.
3. Experimental setup
60
max bidens
min bidens
max lolium
Experiments were carried out in a laboratory consisting
50
min lolium
max wheat
of a camera set up, data logging software and trays of
40 min wheat three plant species namely wheat, Bidens and Lolium.
As in Fig. 4, the experimental setup is consisted of a
30
color camera, a NIR camera, a LED lighting source, a
20 laptop computer and data logging software. NIR
10
camera is a monochrome camera fitted with a visual
light block filter. Lenses of both cameras were chosen
0
500 550 600 650 700 750 800 850 900 to be identical. The lighting source is specially
wavelength-nm

Fig. 2 Variance in spectral reflectance of different plant

designed consisting of 5x4 LED arrays with spectral
species band of 780nm. The colour images were captured with
fluorescent lighting and the NIR images were captured
This variation in spectral response can be due to age of with the LED lighting. The image resolution is
leaves, their orientation, spatial position of the leaves 640x480.
and plant health. The spectral responses of a wheat
leave obtained at different orientations from the light
source are shown in Fig. 3. It could be noted that the
spectral responses become weak with larger angles to
the light source. In some cases, relatively larger
footprint of the spectrometer caused the measurements
to be affected by the background, especially for thin
leaves such as rye grass.

Fig. 4 Experimental setup

4. Crop/Weed classification algorithm

Vision cues have been extensively used in the

literature for crop/weed classification. In this section,
an unsupervised crop/weed classification algorithm
based on vision information alone is presented. Fig. 5
Fig. 3 Variance in spectral reflectance of different measuring shows a block diagram of the proposed crop/weed
angles. Angles are measured from the wheat surface. The classification algorithm. It mainly consisted of two
light source is positioned at 90 degrees to wheat surface. parts, namely model generation and classification.
The spectrometers may not be a viable solution for 4.1. Automatic model generation
online weed/crop classification due to either their
higher costs or due to practical constraints. One such In order to classify plant species, we need to choose
practical limitation is the requirement of a reference appropriate vision cues and models defining the
spectral response at the measurement point for
species. We adopt a strategy, where such a selection is
calculating the ratio. Further, it needs a sweeping
carried out automatically requiring minimal input from
mechanism for obtaining the special information due to
the operator. This leads the algorithm to be used in
its limited circular footprint. The spectrometer probe
also needs to be closely located to the surface before diverse classification applications enhancing the
taking measurements. Therefore, it is proposed to use a generality. Further, it will help an unskilled worker to
color camera and a NIR (Near Infra Red) camera to carry out the operations without having to consult
capture the discriminative bands, which are around vision experts in selecting appropriate parameters.
 Generated Foreground
model (plants)

(a) input image (b) Segmented plant species

Pixel categorization

Morphological operation

0 50 100 150 200 250

Classification results
(c) Hue (d) Saturation

(a) Model generation (b) classification

Fig. 5 Classification algorithm

The automatic model generation algorithm is given in

Fig. 5(a). Once a raw image is received, a simple color
based classification is used for detection of plant/soil (e) Texture (f) Hue & Saturation
(see Fig. 6 (b)). Foreground consisting of pixels
belonging to plant species are further processed for
calculating various visual cues such as, color, texture,
NIR, etc. The cues and their combinations are grouped
into several number of clusters based on k-means
algorithm. The estimated clusters are further analyzed
by Mahalanobis distance (MD), which is a good
measure for discrimination of the clusters. The cues or
(g) Probabilistic model of
their combinations corresponding to larger MDs are (h) Probabilistic model of
Bidens wheat.
chosen as the appropriate cues or their combinations
for the classification problem. The MD is a better
choice than Euclidian distance (ED) between cluster
centers as latter does not consider the distribution of
the clusters. For example, high ED between cluster
centers does not necessarily leading to good
classification results due to possible larger overlap
between the cluster distributions. Given the problem,
(i) Probability map based on (j) Classification results
once the cues or their combinations are chosen as most MD
appropriate, data surrounding the cluster centers are
used to determine the model of a particular plant Fig. 6 Automatic model generation and classification: wheat/
species. Bidens

4.2.Classification algorithm

Once the model is established, it is possible to

synthesize a classification algorithm based on [14],
which is shown in Fig. 5(b). By going through each
foreground pixel, the MD is calculated with the
established model. Pixels that have MD values less wheat is ambiguous as shown in Fig. 6 (h), when
than 1 are given a value of 1 and the other pixels are comparing with that of Bidens (Fig. 6 (g)). Therefore,
given a value of 1/MD. This is resulted in an “image” the classification results (Fig. 8) shows degraded
describing how confidence a pixel to be belonging to performances than that of using Bidens model ( as in
the established model. The confidence image is then Table 1, it has a detection rate of 77% and false alarm
thresholded and morphological operations are carried rate of 30%).
out to improve the connectivity of highly probable
pixels while rejecting outliers. The final image shows
classified pixels based on the established model.

5. Experimental results

In this section, some classification results are

presented with an analytical discussion. The plant
species considered here are wheat, Bidens and Lolium.
Cues considered are, Hue, Saturation, Texture and their
combinations. For all scenarios, we assumed the
number of clusters to be 2 for the k-means algorithm.

5.1.Classification of Wheat and Bidens

Fig.6 (a) shows a color image of wheat and Bidens

plants captured in a laboratory condition with artificial
lighting. We used a classification algorithm given in (a) raw (b) Segmented Bidens
Section 4. Fig.6 (b) shows the classified plant species Fig. 7 Classification based on Bidens model
from soil based on color. The distribution of hue,
saturation, texture, and hue and saturation of the
image Fig. 6 (b) are shown in Fig. 6(c,d,e,f)
respectively. Texture reveals the spatial distribution of
an image, typically repeated patterns. The common
methods of determining texture are Gabor filter, run-
length statistics and co-occurrence matrices. In this
work, we have used the output of Gabor filter as a
measure of texture. It could be noted that the hue is
tightly distributed and the saturation has a broad
distribution. Although hue and saturation as individual
components do not show separable clusters, they form
discriminative clusters once combined as in Fig. 6 (f).
Therefore, the model generation algorithm has chosen
hue and saturation combination to be the most
appropriate for Bidens/wheat classification. The
resulting models of Bidens and wheat are shown in
Fig.6 (g) and (h) respectively. Fig. 6 (i) shows the
probability map image generated using the model of
(a) raw (b) Segmented wheat
the Bidens (Fig. 6 (g)) followed by classification
Fig. 8 Classification based on wheat model
results in Fig. 6 (j).
Further, classification results based on Bidens
Table 1: Classification results of wheat and Bidens
model is shown in Fig.7. The results are appealing with detection rate % false alarm rate %
97% detection rate and just 2% of false alarms (see wheat 77 30
Table 1). Fig. 8 shows classification results based on Bidens 97 2
the derived wheat model. The probabilistic model of
5.2. Classification of wheat and Lolium References

We used the same algorithm for wheat/Lolium [1] Pedersen S. M., Fountas, S., Have H. and Blackmore
classification without much success (Fig. 9). As given B. S., Agricultural robots-systems analysis and
in Table 2, while using the wheat model, it has a economic feasibility. Precision Agriculture (2006),
7:295-308.
reasonably good detection rate of 85% accompanied by
[2] Blackmore S., Fountas S.,Tang L. and Have H.,
bad false alarm rate of 82% bringing overall Design specifications for a small autonomous tractor
classification results to be unacceptable. While using with behavioural control, J. Int. Comm. Agric. Eng.
the Lolium model, the detection rate is too low (26%) (CIGR). Manuscript PM 04 001. vol 6. July 2004.
and the false alarm rate is reasonable high (27%) [3] Gerhards R. and Oebel H., Practical experiences with a
causing the overall classification to be unacceptable. system for site-specific weed control in arable crops
That is mainly due to the similarities of the two species using real-time image analysis and GPS-controlled
in the distributions of hue, saturation, and texture cues. patch system. Weed Research (2006), 46:185-193.
Therefore, we are currently working on incorporating [4] Gee C. H., Bossu J., Jones G. and Truchetet F.,
NIR images and their cues for improved classification Crop/weed discrimination in perspective agronomic
images. Com. and Electr. in Agri. (2008), 60:49-59.
accuracies in such challenging situations.
[5] Eaton R., Katupitiya J., Cole A. and Meyer C.,
Architecture of an automated agricultural tractor:
Hardware, software and control systems, Proceedings
of the 2005 IFAC World Congress. Prague: Elsevier,
July 2005.
[6] Jurado-Exposito M., Lopez-Granados F., Stenciano
(a) Raw image (b) Segmented (c) Segmented wheat S., Garcia-Torres L., and Gonzalez-Sndujar J. L.,
Lolium Discrimination of Weed Seedlings, Wheat (Tritium
Fig. 9 Classification results: wheat/Lolium aestivum).
[7] Borregarrd T., Nielsen H., Norgarrd L. and Have H.,
Table 2: Classification results of wheat and Lolium Crop-weed Discrimination by Line Imaging
detection rate % false alarm rate % Spectroscopy, J. agric. Eng. Res. vol. 75, pp 389-400,
wheat 85 82 2000.
Lolium 26 27 [8] Eddy, P.; Smith, A.; Hill, B.; Peddle, D.; Coburn, C.;
Blackshaw, R.; Proceedings of Geoscience and
6. Conclusions and future work Remote Sensing Symposium, 2006. IEEE
International Conference; p116 – 119; 2006.
Automatic crop - weed classification is a crucial [9] Pérez A. J., López F., Benlloch J. V. and Christensen
S., Colour and shape analysis techniques for weed
module in autonomous weed control. In this paper, we detection in cereal fields, Computers and Electronics
have investigated a method of classifying wheat from in Agriculture, 25(3):197-212, 2000.
Bidens. We have also shown that visual cues such as [10] Aitkenhead M. J., Dalgetty I. A., Mullins C. E.,
color and texture alone do not provide enough McDonald A. J. S. and Strachan J. J. C., Weed and
information to obtain high classification accuracies crop discrimination using image analysis and
especially for wheat and Lolium. This made us to artificial intelligence methods, Computers and
Electronics in Agriculture 39:157-171, 2003.
choose NIR image cues along with color camera cues.
[11] Hemming J. and Rath T., Computer-Vision-based
It leads to other challenges, related to the behavior of Weed Identification under Field Conditions using
the lighting source and multimodal correspondence Controlled Lighting, J. agric. Engng Res., 78(3): 233-
problem, which we are currently working on. 243, 2001.
[12] Lemerle D., Verbeek B. and Orchard B., Ranking the
ACKNOWLEDGEMENTS ability of wheat varieties to compete with Lolium
rigidum. Weed Research 41 (3): 197–209, 2001.
This work is supported by the ARC Centre of [13] Holm G. L., Plucknett D. L., Pancho J. V. and
Excellence program, funded by the Australian Herberger J. P. , The World's Worst Weeds.
Research Council (ARC) and the New South Wales Distribution and Biology, FL, USA, 1991.
State Government. The authors like to thank Dr Jay [14] Bradski G., Kaehler A. and Pisarevsky V., Learning-
Katupitiya and Mr. Kim Son Dang for providing a Based computer vision with Intel’s open source
computer vision library, Intel Technology Journal,
picture of CASPA weeder and it’s description. 9(2):119-130,2005.