Detection and Classification of Plant Leaf Diseases

by using Deep Learning Algorithm

Abstract—Plant leaf diseases and destructive insects are a major

challenge in the agriculture sector. Faster and an accurate prediction of
Rajasthan, India
leaf diseases in crops could help to develop an early treatment technique
while considerably reducing economic losses. Modern advanced
developments in Deep Learning have allowed researchers to extremely
improve the performance and accuracy of object detection and
recognition systems. In this paper, we proposed a deep-learning-based
approach to detect leaf diseases in many different plants using images of
plant leaves. Our goal is to find and develop the more suitable deep-
learning methodologies for our task. Therefore, we consider three main
families of detectors: Faster Region-based Convolutional Neural Network
(Faster R-CNN), Region-based Fully Convolutional Network (R-FCN),
and Single Shot Multibox Detector (SSD), which was used for the purpose
of this work. The proposed system can effectively identified different
types of diseases with the ability to deal with complex scenarios from a
plant’s area.

Keywords— Plant Leaf Diseaases, Deep Learning, faster R-CNN,

R- FCN, SSD

I. INTRODUCTION
Agriculture is the mainstay of the Indian economy.
Immense commercialisation of an agriculture has creates a
very negative effect on our environment. The use of chemical
pesticides has led to enormous levels of chemical buildup in
our environment, in soil, water, air, in animals and even in our
own bodies. Artificial fertilisers gives on a short-term effect on
productivity but a longer-term negative effect on the
environment, where they remain for years after leaching and Fig.1 Diseases affected leaf images
running off, contaminating ground water. Another negative Deep learning constitutes a recent, modern technique for
effect of this trend has been on the fortunes of the farming image processing and data analysis, with accurate results and
communities worldwide. Despite this so-called increased large potential. As deep learning has been successfully applied
productivity, farmers in practically every country around the in various domains, it has recently entered also the domain of
world have seen a downturn in their fortunes. This is where agriculture. So we will apply deep learning to create an
organic farming comes in. Organic farming has the capability algorithm for automated detection and classification of plant
to take care of each of these problems. The central activity of leaf diseases. Nowadays, Convolutional Neural Networks are
organic farming relies on fertilization, pest and disease control. considered as the leading method for object detection. In this
Plant disease detection through naked eye observation of paper, we considered detectors namely Faster Region-Based
the symptoms on plant leaves, incorporate rapidly increasing Convolutional Neural Network (Faster R-CNN), Region-based
of complexity. Due to this complexity and to the large number Fully Convolutional Networks (R-FCN) and Single Shot
of cultivated Crops and their existing phytopathological Multibox Detector (SSD). Each of the architecture should be
problems, even experienced agricultural experts and plant able to be merged with any feature extractor depending on the
pathologists may often fail to successfully diagnose specific application or need. We consider some of the commercial/cash
diseases, and are consequently led to mistaken conclusions and crops, cereal crops, and vegetable crops and fruit plants such
concern solutions. An automated system designed to help as sugarcane, cotton, potato, carrot, chilly, brinjal, rice, wheat,
identify plant diseases by the plant’s appearance and visual banana and guava, these leaves images are selected for our
symptoms could be of great help to amateurs in the purpose. Fig. 1 shows images of the diseased affected leaves
agricultural process. This will be prove as useful technique for on various crops. The early detection of plant leaf diseases
farmers and will alert them at the right time before spreading could be a valuable source of information for executing proper
of the disease over large area. diseases detection, plant growth management strategies and
disease control measures to prevent the development and the
spread of diseases.

1
 II. RELATED WORK In paper [5] authors presented disease detection in Malus
Here, we take some of the papers related to Plant leaf domestica through an effective method like K-mean clustering,
diseases detection using various advanced techniques and texture and color analysis. To classify and recognize different
some of them shown below, agriculture, it uses the texture and color features those
In paper[1], author described as an in-field automatic generally appear in normal and affected areas.
wheat disease diagnosis system based on a weekly supervised In paper [6] authors compared the performance of
deep learning framework, i.e. deep multiple instance learning, conventional multiple regression, artificial neural network
which achieves an integration of identification for wheat (back propagation neural network, generalized regression
diseases and localization for disease areas with only image- neural network) and support vector machine (SVM). It was
level annotation for training images in wild conditions. concluded that SVM based regression approach has led to a
Furthermore, a new in- field image dataset for wheat disease, better description of the relationship between the
Wheat Disease Database 2017 (WDD2017), is collected to environmental conditions and disease level which could be
verify the effectiveness of our system. Under two different useful for disease management
architectures, i.e. VGG-FCN- VD16 and VGG-FCN-S, our
system achieves the mean recognition accuracies of 97.95% III. PROPOSED METHODOLOGY
and 95.12% respectively over 5-fold cross validation on Plants are susceptible to several disorders and attacks
WDD2017, exceeding the results of 93.27% and 73.00% by caused by diseases. There are several reasons that can be
two conventional CNN frameworks, characterizable to the effects on the plants, disorders due to the
i.e. VGG-CNN-VD16 and VGG-CNN-S. Experimental results environmental conditions, such as temperature, humidity,
demonstrate that the proposed system outperforms nutritional excess or losses, light and the most common
conventional CNN architectures on recognition accuracy under diseases that include bacterial, virus, and fungal diseases.
the same amount of parameters, meanwhile maintaining Those diseases along with the plants may shows different
accurate localization for corresponding disease areas. physical characteristics on the leaves, such as a changes in
Moreover, the proposed system has been packed into a real- shapes, colors etc. Due to similar patterns, those above
time mobile app to provide support for agricultural disease changes are difficult to be distinguished, which makes their
diagnosis. recognition a challenge, and an earlier detection and treatment
can avoid several losses in the whole plant. In this paper, we
In paper [2], author discussed and to perform a survey of are discussed to use recent detectors such as Faster Region-
40 research efforts that employ deep learning techniques, Based Convolutional Neural Network (Faster R-CNN),
applied to various agricultural and food production challenges. Region-based Fully Convolutional Networks (R-FCN) and
Examine the particular agricultural problems under study, the Single Shot Multibox Detector (SSD) to detection and
specific models and frameworks employed the sources, nature classification of plant leaf diseases that affect in various plants.
and pre-processing of data used, and the overall performance The challenging part of our approach is not only deal with
achieved according to the metrics used at each work under disease detection, and also known the infection status of the
study. Moreover, study comparisons of deep learning with disease in leaves and tries to give solution (i.e., name of the
other existing popular techniques, in respect to differences in suitable organic fertilizers) for those concern diseases.
classification or regression performance. Findings indicate that
deep learning provides high accuracy, outperforming existing A. Faster Region-Based Convolutional Neural
commonly used image processing techniques. Network (Faster R-Cnn)
In paper [3], author discussed about convolutional neural Faster R-CNN is one of the Object detection systems,
network models were developed to perform plant disease which is composed of two modules. The first module is a deep
detection and diagnosis using simple leaves images of healthy fully convolutional network that proposes regions. For training
and diseased plants, through deep learning methodologies. the RPNs, the system considers anchors containing an object
Training of the models was performed with the use of an open or not, based on the Intersection-over-Union (IoU) between
database of 87,848 images, containing 25 different plants in a the object proposals and the ground-truth. Then the second
set of 58 distinct classes of [plant, disease] combinations, module is the Fast R-CNN detector [13], [14] that uses the
including healthy plants. Several model architectures were proposed regions. Box proposals are used to crop features
trained, with the best performance reaching a 99.53% success from the same intermediate feature map which are
rate in identifying the corresponding [plant, disease] subsequently fed to the remainder of the feature extractor in
combination (or healthy plant). The significantly high success order to predict a class and class-specific box refinement for
rate makes the model a very useful advisory or early warning each proposal. Fig. 2 shows the basic architecture of Faster R-
tool, and an approach that could be further expanded to CNN. The entire process happens on a single unified network,
support an integrated plant disease identification system to which allows the system to share full-image convolutional
operate in real cultivation conditions.
features with the detection network, thus enabling nearly cost-
In paper [4] author describes a methodology for early and free region proposals.
accurately plant diseases detection, using artificial neural
network (ANN) and diverse image processing techniques. As
the proposed approach is based on ANN classifier for
classification and Gabor filter for feature extraction, it gives
better results with a recognition rate of up to 91%. An ANN
based classifier classifies different plant diseases and uses the
combination of textures, color and features to recognize those
diseases.

2
 OVERVIEW OF THE SYSTEM

Fig. 2. Basic architecture of Faster R-CNN

Fig. 3. System design
B. REGION-BASED FULLY CONVOLUTIONAL
NETWORK (R-FCN)
D. DATA COLLECTION
We develop a framework called Region-based Fully Dataset contains images with several diseases in many
Convolutional Network (R-FCN) for object detection. While different plants. In this System we consider some of the
Faster R-CNN is an order of magnitude faster than Fast R- commercial/cash crops, cereal crops, and vegetable crops and
CNN, the fact that the region-specific component must be fruit plants such as sugarcane, cotton, potato, carrot, chilly,
applied several hundred times per image led [12], [13], [16], brinjal, rice, wheat, banana and guava. Diseased leaves,
[19] to propose the R-FCN (Region-based Fully Convolutional healthy leaves all of them were collected for those above crops
Networks) method which is like Faster R-CNN, but instead of from different sources like images download from Internet, or
cropping features from the same layer where region proposals simply taking pictures using any camera devices or any else.
are predicted, crops are taken from the last layer of features
prior to prediction. R-FCN object detection strategy consists E. IMAGE PRE-PROCESSING
of: Image annotation and augmentation
(i) region proposal, and (ii) region classification. This Image annotation, the task of automatically generating
approach of pushing cropping to the last layer minimizes the description words for a picture, is a key component in various
amount of per-region computation that must be done. The image search and retrieval applications. But in this system, we
object detection task needs localization representations that manually annotate the areas of every image containing the
respect translation variance and thus propose a position- disease with a bounding box and class. Some diseases might
sensitive cropping mechanism that is used instead of the more look similar depending on its infection status.
standard ROI pooling operations used in object detection [13],
[20]. They show that the R FCN model could achieve
comparable accuracy to Faster R-CNN often at faster running
times.
C. Single Shot Detector (Ssd)
The SSD approach is based on a feed-forward
convolutional network that produces a fixed-size collection of
bounding boxes and scores for the presence of object class
instances in those boxes, followed by a non-maximum
suppression step to produce the final detections. This network
is able to deal with objects of various sizes by combining Fig. 4. Annotated Image
predictions from multiple feature maps with different Annotation process might able to label the class and
resolutions [13], [15]. Furthermore, SSD encapsulates the location of the infected areas in the leaf image. The outputs of
process into a single network, avoiding proposal generation this step are the coordinates of the bounding boxes of different
and thus saving computational time. sizes with their corresponding class of disease, which
IV. EXPERIMENTAL RESULT consequently will be evaluated as the Intersection over-Union
(IoU) with the predicted results during testing. Fig. 4 shows
In our system processing starts with Data collection, the annotated image.
through some the pre-processing, feature extractor steps to be
allowed and then finally detect the diseases from image. Fig. 3
shows the overview of our proposed system.

3
 Images are collected from various sources were in various SSD
formats along with different resolutions and quality. In order SSD generates anchors that select the top most convolutional
to get better feature extraction, images are intended to be used feature maps and a higher resolution feature map at a lower
as dataset for deep neural network were pre-processed in order resolution. Then, a sequence of the convolutional layer
to gain consistency. Images used for the dataset were image containing each of the detection per class is added with spatial
resized to 256×256 to reduce the time of training, which was resolution used for prediction [13], [15]. Thus, SSD is able to
automatically computed by written script in Python, using the deal with objects of various sizes contained in the images. A
OpenCV framework [7], [8]. Non-Maximum Suppression method is used to compare the
In machine learning, as well as in statistics, overfitting estimated results with the ground-truth.
appears when a statistical model describes random noise or G. Feature Extraction
error rather than underlying relationship [9]. The image There are some conditions that should be taken into
augmentation contained one of several transformation consideration when choosing a Feature Extractor, such as the
techniques including affine transformation, perspective type of layers, as a higher number of parameters increases the
transformation, image rotations [10] and intensity complexity of the system and directly influences the speed,
transformations (contrast and brightness enhancement, color, and results of the system. Although each network has been
noise). Fig 5 and Fig 6 shows an example for affine designed with specific characteristics, all share the same goal,
transformations and simple rotations. which is to increase accuracy while reducing computational
complexity. In this system each object detector to be merged
with some of the feature extractor. [13]The system
performance is evaluated first of all in terms of the
Intersection-over-Union (IoU), and the Average Precision
(AP) that is introduced in the Pascal VOC Challenge [17]
A∩B
( )
IoU A, B = |A𝖴B| (1)
where A represents the ground-truth box collected in the
annotation, and B represents the predicted result of the
network. If the estimated IoU outperforms a threshold value,
Fig. 5. Affine transformations the predicted result is considered as a true positive, TP, or if
not as a false positive, FP. TP is the number of true positives
generated by the network, and FP corresponds to the number
of false positives. Ideally, the number of FP should be small
and determines how accurate the network to deal with each
case is. The IoU is a widely used method for evaluating the
accuracy of an object detector. [13][17]The Average Precision
is the area under the Precision-Recall curve for the detection
task. As in the Pascal VOC Challenge, the AP is computed by
averaging the precision over a set of spaced recall levels [0,
Fig. 6. Rotations 0.1, . . . , 1], and the mAP is the AP computed over all classes
in our task.
F. IMAGE ANALYSIS 1
𝐴𝑃 = ∑ 𝑝 (𝑟) (2)
Our system main goal is to detect and recognize the 𝑟𝗀{0,0.1,…,1} 𝑖𝑛𝑡𝑒𝑟 𝑝
11
class disease in the image. We need to accurately detect the
object, as well as identify the class to which it belongs. We pinterp (r) = max p(r̃) (3)
r̃:r̃ ≥r
extend the idea of object detection framework to adapt it with
different feature extractors that detect diseases in the image. where p(r̃) is the measure precision at recall r̃ .
Faster R-CNN Faster R-CNN for each object proposal, [14]we extract the
features with a RoI Pooling layer and perform object
Faster R-CNN [13], [14] for object recognition and its
classification and bounding-box regression to obtain the
Region Proposal Network (RPN) to estimate the class and
estimated targets. We used batch normalization for each
location of object that may contain a target candidate. The
feature extractor, and train end-to-end using an Image Net Pre
RPN is used to generate the object a proposal, including their
trained Network.
class and box coordinates.
To perform the experiments, our dataset has been divided
into training set, validation set and testing set. Evaluation is
R-FCN
performed on the Validation set after that training is process is
Similar to Faster R-CNN,[13], [16], [20] R-FCN uses a
performed on the training set and then final evaluation done in
Region Proposal Network to generate object proposals, but
testing phase. As in the Pascal Visual Object Classes (VOC)
instead of cropping features using the RoI pooling layer it
Challenge [17], the validation set is a technique used for
crops them from the last layer prior to prediction.
minimizing over fitting and is a typical way to stop the
network from learning. We use the training and validation sets
to perform the training process and parameter selection,
respectively, and the testing set for evaluating the results on
4
unknown data.

5
 V. CONCLUSION [7] Srdjan Sladojevic, Marko Arsenovic, Andras Anderla, Dubravko
Crop protection in organic agriculture is not a simple Culibrk, and Darko Stefanovic, Deep Neural Networks Based
matter. It depends on a thorough knowledge of the crops Recognition of Plant Diseases by Leaf Image Classification, Hindawi
grown and their likely pests, pathogens and weeds. In our Publishing Corporation, Computational Intelligence and Neuroscience
Volume 2016, Article ID 3289801, 11 pages
system specialized deep learning models were developed, http://dx.doi.org/10.1155/2016/3289801.
based on specific convolutional neural networks architectures, [8] J.Howse,OpenCV ComputerVision with Python, PacktPublishing,
for the detection of plant diseases through leaves images of Birmingham, UK, 2013.
healthy or diseased plants. Our detector applied images [9] D. M. Hawkins, “The problem of over-fitting,” Journal of Chemical
captured in-place by various camera devices and also collected information and Computer Sciences, vol. 44, no. 1, pp. 1–12, 2004.
[10] C. C. Stearns and K. Kannappan, “Method for 2-D affine transformation
from various resources. Our experimental results and
of images,” US Patent No. 5,475,803, 1995.
comparisons between various deep-architectures with feature [11] Sankaran, S. Mishra, A.; Ehsani, R. A review of advanced techniques for
extractors demonstrated how our deep-learning-based detector detecting plant diseases. Comput. Electron. Agric. 2010, 72, 1–13.
is able to successfully recognize different categories of [12] Huang, J., Rathod, V., Sun, C.; Zhu, M., Korattikara, A., Fathi, A.,
diseases in various plants and also give solution for concern Fischer, I., Wojna, Z.,Song, Y., Guadarrama, S., et al. Speed/accuracy
diseases. Pests/diseases are generally not a significant problem trade-offs for modern convolutional object detectors. In Proceedings of
the IEEE Computer Society Conference on Computer Vision and Pattern
in organic systems, since healthy plants living in good soil Recognition, Honolulu, HI, USA, 22–25 July 2017.
with balanced nutrition are better able to resist pest/disease [13] Alvaro Fuentes , Sook Yoon , Sang Cheol Kim and Dong Sun Park A
attack. We hope our proposed system will make a suggestive Robust Deep Learning-Based Detector for Real-Time Tomato Plant
contribution to the agriculture research. Diseases and Pests Recognition, Sensors 2017, 17, 2022;
doi:10.3390/s17092022.
REFERENCES [14] Ren, S., He, K., Girshick, R., Sun, J. Faster R-CNN: Towards Real-Time
[1] Jiang Lu, Jie Hu, Guannan Zhao, Fenghua Mei, Changshui Zhang, An Object Detection with Region Proposal Networks. IEEE Trans. Pattern
in-field automatic wheat disease diagnosis system, Computers and Anal. Mach. Intell. 2016, 39, 1137–1149.
[15] Liu, W., Anguelov, D, Erhan, D., Szegedy, C., Reed, S.; Fu, C.; Berg,
Electronics in Agriculture 142 (2017) 369–379.
[2] Andreas Kamilaris, Francesc X. Prenafeta-Boldu Deep learning in A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the
agriculture: A survey, Computers and Electronics in Agriculture 147 European Conference on Computer Vision ECCV, Amsterdam, The
(2018) 70–90. Netherlands, 8–16 October 2016; pp. 21–37.
[3] Konstantinos P. Ferentinos, Deep learning models for plant disease [16] Dai, J., Li, Y., He, K., Sun, J. R-FCN: Object Detection via Region-
detection and diagnosis Computers and Electronics in Agriculture 145 based Fully Convolutional Networks. arXiv 2016, arXiv:1605.06409v2.
(2018) 311–318. [17] Everingham, M., Van Gool, L., Williams, C., Winn, J., Zisserman, A.
[4] Kulkarni Anand H, Ashwin Patil RK. Applying image processing The Pascal Visual Object Classes (VOC) Challenge. Int. Comput. Vis.
technique to detect plant diseases. Int J Mod Eng Res 2012;2(5):3661–4. 2010, 88, 303–338.
[18] R. Girshick. Fast r-cnn. In Proceedings of the IEEE International
[5] Bashir Sabah, Sharma Navdeep. Remote area plant disease detection
Conference on Computer Vision, pages 1440–1448, 2015.
using image processing. IOSR J Electron Commun Eng 2012;2(6):31–4.
[19] J. Dai, K. He, and J. Sun. Instance-aware semantic segmentation via
ISSN: 2278-2834.
multi-task network cascades. arXiv preprint arXiv:1512.04412, 2015.
[6] Rakesh Kaundal, Amar S Kapoor and Gajendra PS Raghava “Machine
[20] R. Girshick. Fast r-cnn. In Proceedings of the IEEE International
learning technique in disease forecasting: a case study on rice blast
Conference on Computer Vision, pages 1440–1448, 201
prediction,” BMC Bioinformatics, 2006.