Received March 10, 2021, accepted March 24, 2021, date of publication April 8, 2021, date of current version

April 19, 2021.

Digital Object Identifier 10.1109/ACCESS.2021.3069646

Plant Disease Detection and Classification

by Deep Learning—A Review
LILI LI1 , SHUJUAN ZHANG 2, AND BIN WANG 2
1 College of Information Science and Engineering, Shanxi Agricultural University, Jinzhong 030800, China
2 College of Agricultural Engineering, Shanxi Agricultural University, Jinzhong 030800, China

Corresponding author: Shujuan Zhang ([email protected])

This work was supported in part by the Innovation Project of Shanxi for Postgraduate Education under Grant J202082047.

ABSTRACT Deep learning is a branch of artificial intelligence. In recent years, with the advantages of
automatic learning and feature extraction, it has been widely concerned by academic and industrial circles.
It has been widely used in image and video processing, voice processing, and natural language processing.
At the same time, it has also become a research hotspot in the field of agricultural plant protection, such as
plant disease recognition and pest range assessment, etc. The application of deep learning in plant disease
recognition can avoid the disadvantages caused by artificial selection of disease spot features, make plant
disease feature extraction more objective, and improve the research efficiency and technology transformation
speed. This review provides the research progress of deep learning technology in the field of crop leaf disease
identification in recent years. In this paper, we present the current trends and challenges for the detection
of plant leaf disease using deep learning and advanced imaging techniques. We hope that this work will be
a valuable resource for researchers who study the detection of plant diseases and insect pests. At the same
time, we also discussed some of the current challenges and problems that need to be resolved.

INDEX TERMS Deep learning, plant leaf disease detection, visualization, small sample, hyperspectral
imaging.

I. INTRODUCTION Farmers with less experience may misjudgment and use drugs
The occurrence of plant diseases has a negative impact on blindly during the identification process. Quality and output
agricultural production. If plant diseases are not discovered in will also bring environmental pollution, which will cause
time, food insecurity will increase [1]. Early detection is the unnecessary economic losses. To counter these challenges,
basis for effective prevention and control of plant diseases, research into the use of image processing techniques for plant
and they play a vital role in the management and decision- disease recognition has become a hot research topic.
making of agricultural production. In recent years, plant dis-
ease identification has been a crucial issue.
Disease-infected plants usually show obvious marks or
lesions on leaves, stems, flowers, or fruits. Generally, each
disease or pest condition presents a unique visible pattern
that can be used to uniquely diagnose abnormalities. Usually,
FIGURE 1. Traditional image recognition processing.
the leaves of plants are the primary source for identifying
plant diseases, and most of the symptoms of diseases may
begin to appear on the leaves [2]. The general process of using traditional image recognition
In most cases, agricultural and forestry experts are used processing technology to identify plant diseases is shown
to identify on-site or farmers identify fruit tree diseases and in Fig. 1. Dubey and Jalal [3] used the K-means clustering
pests based on experience. This method is not only sub- method to segment the lesions regions, and combined the
jective, but also time-consuming, laborious, and inefficient. global color histogram (GCH) color coherence vector (CCV)
local binary pattern (LBP), and completed local binary pat-
The associate editor coordinating the review of this manuscript and tern (CLBP) was used to extract the color and texture features
approving it for publication was Yongming Li . of apple spots, and three kinds of apple diseases were detected

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 9, 2021 56683
 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

and identified based on improved support vector machine Recently, the convolutional neural networks (CNN), a spe-
(SVM), and the classification accuracy reached 93%. cial of deep learning techniques, are quickly becoming the
Chai et al. [4] studied four tomato leaf diseases, including preferred methods [7]. CNN is the most popular classifier
early blight and late blight leaf mildew and leaf spot, and for image recognition, and it has shown outstanding ability
extracted 18 characteristic parameters such as color, texture, in image processing and classification [8]. Deep learning
and shape information of tomato leaf spot images, using step- approaches were first introduced in plant image recognition
wise discriminant and Bayesian discriminant principal com- based on leaf vein patterns [9]. They used 3-6 layers CNN
ponent analysis (PCA), respectively. Principal component classified three leguminous plant species: white bean, red
analysis and fisher discriminant methods were used to extract bean, and soybean. Mohanty et al. [10] trained a deep learning
the characteristic parameters and construct the discriminant model to recognize 14 crop species and 26 crop diseases. The
model. The accuracy of the two methods reached 94.71% and trained model achieved an accuracy of 99.35% on the test set.
98.32%, respectively. Ma et al. [11] used a deep CNN to conduct symptom-wise
Li and He [5] selected 5 kinds of apple leaf diseases recognition of four cucumber diseases (i.e., downy mildew,
(speckled deciduous disease, yellow leaf disease, round spot anthracnose, powdery mildew, and target leaf spots). The
disease, mosaic disease, and rust disease) as the research recognition accuracy reached 93.4%. Kawasaki et al. [12]
objects. By extracting 8 features of the apple leaf spot image, introduced a system based on CNN to recognize cucumber
such as color, texture, and shape. The BP neural network leaf disease, which realized an accuracy of 94.9%.
model was used to classify and recognize the diseases, and Although very good results have been reported in the
the average recognition accuracy reached 92.6%. literature, however, the diversity of the used datasets is lim-
Guan et al. [6] extracted 63 parameters including morphol- ited. Large datasets (comprised of thousands of images) are
ogy, color, and texture features of rice leaf disease spots, and required for the training of CNNs. Unfortunately, for plant
applied step-based discriminant analysis and Bayesian dis- leaf disease recognition, such large and diverse datasets have
criminant method to classify and recognize three rice diseases not yet been collected for use by researchers. At present,
(blast, stripe blight, and bacterial leaf blight) with the highest transfer learning is the most effective way to train the robust-
recognition accuracy of 97.2%. ness of CNN classifiers for plant leaf disease recognition.
In short, it can be concluded that studies on plant disease Transfer learning enables the adaptation of pre-trained CNNs
recognition based on traditional image processing technology by retraining them with smaller datasets whose distribution is
have achieved certain results, with high accuracy of disease different from the larger datasets previously used to train the
recognition, but there are still deficiencies and limitations as network from scratch [13]. Indeed, it is effective that using
follow: CNN models pre-trained on the ImageNet dataset and then
1) The research links and processes are cumbersome, retraining them for leaf disease recognition. Therefore, the
highly subjective, time-consuming and labor-consuming; combination of deep learning and transfer learning provides
2) It is heavily dependent on spot segmentation; 3) It is a new way to solve the problem of limited datasets of plant
heavily dependent on artificial feature extraction; 4) It is diseases.
difficult to test the disease recognition performance of the There are some research papers previously presented to
model or algorithm in more complex environments. summarize the research about agriculture (including plant
Therefore, it is of great significance to realize intelligent, disease recognition) by DL [8], [14], but they lacked some of
rapid, and accurate plant leaf disease recognition. the recent developments in terms of visualization techniques
In recent years, deep learning technology in the study of implemented along with the DL and modified the famous DL
plant disease recognition made more progress. Deep learn- models, which were used for plant disease identification.
ing (DL) technology in the face of the user is transparent, The article [15] presented many imaging techniques for
the researchers of plant protection and statistics professional plant disease detection, and the focus was on imaging tech-
level is not high, can be automatically extracted image fea- niques. The major techniques presented for plant diseases and
tures and classification of plant disease spot, eliminating the classification are SVM, K-means, and KNN.
traditional image recognition technology of feature extraction The article [16] presented many developed/modified DL
and classifier design a lot of work, can express original architectures implemented to detect and classify plant dis-
image characteristics, has the characteristics of the end-to- eases. And provided a comprehensive explanation of DL
end. These characteristics make deep learning technology in models used to visualize various plant diseases. But there is
plant disease recognition-obtained-widespread attention, and no mention of the early detection of the diseases and how to
it has become a hot research topic. This is due to three factors: detect and classify plant diseases based on small samples.
the availability of larger datasets, the adaptability of multi- In the paper [17], the authors had presented a com-
core graphics processing units (GPUs), and the development prehensive review of recent research work done in plant
of training deep neural networks and supporting software disease recognition using IPTs, from the perspective of fea-
libraries, such as the computing unified device architecture ture extracted based on hand-crafted or using deep learning
(CUDA) from NVIDIA. techniques. And it is concluded that the deep learning

56684 VOLUME 9, 2021

L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

techniques have superseded shallow classifiers trained using DL models/architectures were used for image detection, seg-
hand-crafted features. But they lacked some of the recent mentation, and classification, and these architectures were
developments in terms of visualization techniques, and there successively applied to plant disease detection.
is no mention of the early detection of the diseases and how
to detect and classify plant diseases based on small samples. 2) METRICS
This paper aim at the shortcomings of the existing review In order to evaluate these algorithms/architectures, top-
papers on disease detection, we provide a review of recent 1%/top-5% error [6], [20]–[22], precision and recall
studies carried out in the area of plant leaf disease recognition [10], [23], [24], F1 score [24], [25], training/validation
using image processing, hyper-spectral imaging, and deep accuracy and loss [25], [26], classification accuracy (CA)
learning techniques. We hope that this work will be helpful [27], [28], and mean average precision (mAP) are usually
for researchers in the area of plant leaf disease recognition selected as the indicators of judgment.
using DL methods.
The rest of this paper is organized as follows. In Section 2, B. DISEASE DATASETS
review some basic knowledge including deep learning con- Common diseases datasets are: 1) P1antVillage, an open
cept, foundation, framework, development history, model dataset, has now collected 54309 plant leaves disease images,
evaluation criteria, the plant leaves disease datasets, and the covers 14 kinds of fruit and vegetable crops, such as
data enhancement methods, etc. In Section 3, we review apple, blueberry cherries, grapes, orange peach bell pep-
research work done so far towards the application of deep per potato raspberry soybean pumpkin strawberry, and
learning in crop leaves disease recognition from some tomatoes, corn contains 26 diseases (17 kinds of fungal
aspects. In Section 4, plant disease detection based on small disease, 4 kinds of bacteria disease, 2 kinds of myco-
sample data set is discussed. In Section 5, some applica- sis, 2 kinds of viral diseases and 1 kind of diseases
tions of hyper-spectral imaging in plant disease detection are caused by mite), also includes 12 healthy crop leaf images.
discussed. Section 6, summarizes and discusses gaps in the 2) ‘Plant Pathology Challenge’ for CVPR 2020-FGVC7
existing literature that need to be addressed. (https://www.kaggle.com/c/plantpathology - 2020 fgvc7),
it consists of 3,651 high-quality annotated RGB images
II. BASIC KNOWLEDGE OF DEEP LEARNING of 1,200 apple scab and 1,399 cedar apple rust symptoms
A. THE FRAMEWORK, DEVELOPMENT, AND EVALUATION and 187 complex disease patterns (the leaves with more than
OF MODELS FOR DEEP LEARNING one disease in the same leaf) and 865 healthy apple leaves.
1) THE HISTORY OF DEEP LEARNING 3) While others constitute datasets of real images collected
Deep Learning (DL) is a subclass of Machine Learning (ML), by the authors for their research needed(corn, tea, soybeans,
It was introduced in 1943 [18] and then went into three stages cucumbers, apples, grapes). 4) Growing the plants themself
of development. The first generation of neural network-MCP and inoculating them with the virus, the method of data acqui-
(1943∼1969): originated in 1943, is a linear model, can only sition is commonly seen in applications that use hyperspectral
deal with linear classification problems. images for disease detection.
The second generation of neural network-back propagation
(BP) (1986∼1998): Hinton invented the BP algorithm suit- C. DATA AUGMENTATION
able for multi-layer perceptron (MLP) in 1986 and adopted In leaf disease detection, collection and label a large num-
sigmoid function for nonlinear mapping, which effectively ber of disease images require lots of manpower material
solved the problem of nonlinear classification and learning. resources and financial resources. For some certain plant
This method caused the second upsurge of neural networks. diseases, their onset period is shorter, it is difficult to collect
However, in 1991, the BP algorithm was pointed out that there them. In the field of deep learning, the small sample size
was a gradient vanishing problem. and dataset imbalance are the key factors leading to the poor
The third generation neural network-DL(2006-present): recognition effect. Therefore, the deep learning model for leaf
In 2006, Hinton gradient disappeared in the deep web training disease detection, expand the amount of data is necessary.
are put forward in this problem solution, but because there is Data augmentation to meet the requirements for the practical
no special effective experimental verification and no atten- application, and not at liberty to expand (the color is one of
tion. It was not until 2011 that the ReLU activation func- the main manifestations of different diseases, for example,
tion (the activation function that can effectively restrain the when doing image enhancement can’t change the color of the
gradient disappeared problem) was put forward, then enter original image). There are two common ways to augment the
the outbreak period in 2012, in the famous ImageNet image datasets.
recognition contest, the Hinton team used a deep learning
model-AlexNet to win, and far more than the second method 1) TRADITIONAL AUGMENTATION
(SVM). Since then CNN has attracted the attention of many The typical methods are the physical expansion method (ten-
researchers. sile rotation adjustment resolution image translation distur-
After the introduction of AlexNet [19], the DL architecture bance, etc.), web crawler, variational auto-encoder (VAE),
began to evolve over time as shown in VI. Many advanced and autoregressive model, etc. The shortcomings of the

VOLUME 9, 2021 56685

 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

FIGURE 2. The history of DL architectures.

FIGURE 3. Diagram of GANS structure sketch.

produced samples by the traditional expansion method are In the paper [34], four different kinds of grape leaf disease
poor quality, inadequate diversity, and unevenness. images were expanded by a novel Leaf GAN model. The
experimental results showed that the Leaf GAN model could
2) GENERATE ADVERSARIAL NETWORKS (GANS) make the grape leaf disease images highlight the disease and
GANs is a kind of generating model proposed by Goodfel- generate enough grape leaf disease images. It was proved that
low et al. [29] in 2014. Subsequently, many variations of Leaf GAN was superior to those of the DCGAN and WGAN.
GAN have emerged successively, such as DCGAN, CGAN,
PGGAN, LAPGAN, InfoGAN, WGAN, F-GAN, SeqGAN,
LeakGAN, etc. The major goal is to generate synthetic sam- D. VISUALIZATION TECHNIQUE
ples with the same characteristics as the given training distri- In recent years, the successful application of deep learning
bution. The GANs models mainly consist of two parts, that is, technology in plant disease classification provides a new
generator and discriminator. The structure diagram is shown idea for the research of plant disease classification. However,
in Fig. 3. DL classifiers lack interpretability and transparency. The
Generative network approaches have been extensively used DL classifiers are often considered black boxes without any
to generate samples in recent years. Nazki et al. [30] is explanation or details about the classification mechanism.
the first work that uses GANs to synthetically augment the High accuracy is not only necessary for plant disease clas-
dataset to improve the plant disease recognition performance. sification but also needs to be informed how the detection
By optimizing the activation reconstruction loss (ARL) func- is achieved and which symptoms are present in the plant.
tion and put forward an improved AR-GAN, compared with Therefore, in recent years, many researchers have devoted
most prominent existing models, the proposed model is intro- themselves to the study of visualization techniques such as
duced into composite images, and nine kinds of tomato on the the introduction of visual heat maps and salient maps to better
test data set (2789), the results showed that the classification understand the identification of plant diseases. Among them,
accuracy is significantly increased (+ 5.2%), compared with the works of [35] and [36] are crucial to understanding how
the classic way. CNN recognizes disease from images.
Tian et al. [31] proposed an approach (CycleGAN) that For example, Brahimi et al. [35] introduced saliency
can generate more apple disease images. Generated images maps to visualize the symptoms of plant diseases.
augmented by conditional deep convolutional generative Mohanty et al. [10] used AlexNet and GoogLeNet architec-
adversarial networks (C-DCGAN) [32] use the segmented tures, through the precision (P), recall (R), F1 score, and the
tea disease spot image as the input of VGG16. The result overall accuracy to evaluate the performance of the models on
showed that the average accuracy is about 28% higher by the PlantVillage. Used the three scenarios (color gray and seg-
using C-DCGAN than rotation and translation. mentation) to assess the performance of the 2 CNN famous
The article [33] generated images by using deep con- architectures, and come to the conclusion that GoogLeNet
volutional generative adversarial networks (DC-GAN), and outperformed AlexNet, the first layer of the visual results
achieved a top-1 average identification accuracy of 94.33% clearly showed the disease spots also. In Cruz et al. [37],
on GoogLeNet. The T-distribution random neighborhood the improved LeNet model was used to detect olive plant
embedding (T-SNE) verified that the image distribution gen- diseases, that is, segmentation and edge maps were used to
erated by this method was closer to the sample distribution of identify plant diseases. Brahimi et al. [38] proposed a new
the real image. visualization method, that is, a new DL model teacher/student

56686 VOLUME 9, 2021

L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

network was introduced to identify the spots of plant diseases, the color information of the disease, and the other for identi-
compared with the existing plant disease treatment methods, fying the texture of the heat map. The preliminary hot-spot
the new method obtained a clearer visualization effect. detection and its ulterior description by color and textural
According to the author Dechant et al. [39], using dif- descriptors allow real-time performance as only the suspi-
ferent CNN combinations, the visual heat map of maize cious regions are trained and described by the higher level
disease images was used as the inputs, and the probabil- classifiers and descriptors.
ity associated with the occurrence of a particular type of Khan et al. [47] proposed a new visualization technology
disease was given. The ROC curve was used to evaluate using correlation coefficient and DL model (e.g., AlexNet and
the performance of the model. In addition, the characteris- VGG16 architecture). Kerkech et al. [48] variety vegetation
tic map of maize diseases was also drawn. Lu et al. [40] indices in color space combined with the LeNet model were
realized that wheat disease detection by using VGG-FCN used to detect the grape diseases. The article [49] for the
and VGG-CNN model and visualized the module features. reason of interpreting the deep learning model, compared
The results showed that the DMIL-WDDS based on VGG- with some of the most popular explanatory methods: sig-
FCN-VD16 achieved a progressive learning process for fine nificant figure, Smooth-Grad, boot back-propagation, depth
characteristics of the disease. The feature visualization was a Taylor decomposition, integration gradient layered associ-
good demonstration of what the DMIL-WDDS was learning. ated transmission, and gradient time input. And trained the
Moreover, the results indicated that Softmax aggregation was DenseNet121 network to identify eight different soybean
a superior choice for DMIL-WDDS to improve the recog- stresses(biological and non-biological). And concluded that
nition accuracy. Ha et al. [41] used the VGG-CNN model the interpretability methods identified the infected regions of
to test the blight of radish and used the k-means clustering the leaf as important features for some (but not all) of the
method to show the disease markers. And the method was correctly classified images.
able to detect the individual infected areas. That is, the regions Taken tea leaf diseases images(tea of 261 images of 5 kinds
of healthy radish and moderate Fusarium wilt of radish were of common disease) in the complex background as the
successfully detected by the method. The results showed that research object, Sun et al. [50] proposed a method combining
the method can also be applied to other crops and plants, simple linear iterative clustering (SLIC) and support vector
including tomato, tobacco, banana, and etc. machine (SVM), and gain a significant figure accurate tea
Barbedo [42] explored the use of individual lesions and leaf disease images, with 98.5% accuracy, precision is 96.8%,
spots for the task, rather than considering the entire leaf, the recall rate was 98.6%, the F1 score was 97.7%. The results
and by using the DL models to identify the plant diseases. showed that the method can effectively extract tea leaves from
The accuracy obtained using the approach was, on average, complex background significant figure.
12% higher than those achieved using the original images. Hu et al. [51] put forward a kind of new convolution neural
Ghosal et al. [43], developed a deep CNN framework to network model ARNet (Attention residual network) combin-
identify and classify 8 kinds of soybean stress. And also ing the attention mechanism with the residual idea, and the
present an explanation mechanism, used the top-K high- leaves of five tomato diseases in the early and late periods
resolution feature maps that isolate the visual symptoms to were studied. The results of the study concluded that, com-
make predictions. The unsupervised identification of visual pared with the existed models such as VGG16, the ARNet
symptoms provided a quantitative measure of stress severity, had a better classification performance. The different layers
allowing for identification (a type of foliar stress), classifica- of (Attention convolutional block, ACB), are visualized in
tion (low, medium, or high stress), without detailed symptom the form of heat maps, and the attention information of the
annotation by experts. different layers obtained by the module are shown in Fig. 4.
Lu et al. [44] used CNN to identify rice diseases, early Fig. 4 (a) and (b) represent the output heat maps of late
disease detection, and the characteristic maps of disease spots early blight disease and late leaf frost disease in the ACB
were also obtained. Picon et al. [45] proposed an adapted module at different levels of the ARNET model respectively.
algorithm based on a deep residual neural network to deal Among them, the heat output of each type of image showed
with the detection of multiple crop diseases in real condi- in line 1, and line 2 showed that heat superimposed on the
tions for early disease detection. And developed a mobile original image, from left to right in turn for layer 2, 3, 4,
application in which heat maps were used to identify plant and 5 layers of the last ACB output module. As you can see
diseases. Obtained results reveal an overall improvement of in Fig. 4, the ACB module can more accurately extract the
the balanced accuracy up to 0.87 under exhaustive testing, key feature of each type of disease, shallow ACB module to
and the accuracy greater than 0.96 on a pilot test performed extract the characteristics of relatively scattered, not as a cat-
in Germany. egory, but the deep ACB module to extract the characteristics
Johannes et al. [46] used an algorithm based on heat map of more concentrated which the color is more close to red,
technology to extract the diseased objects. In addition, each that is the corresponding place a greater contribution to the
heat map is described by two descriptors, one for evaluating final classification decision.

VOLUME 9, 2021 56687

 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

Ahmad et al. [54] used four different pretraining convolu-

tion neural networks VGG19, VGG16, ResNet, and Inception
V3, and the models were trained by fine-tuning parameters.
The experimental results showed that the Inception V3 had
the best performance on the two datasets(the laboratory
dataset and the field dataset). And the average performance
superior to 10% to 15% on the laboratory dataset compared
with on the field dataset. Bi et al. [55] showed that the
recognition accuracy rates of apple leaf spot and rust models
collected by agricultural experts were 77.65%, 75.59%, and
73.50%, by using ResNet152, Inception V3, and MobileNet,
respectively.
Jiang et al. [56] used the Mean Shift algorithm to segment
four kinds of rice disease spot (red blight stripe disease to
rice blast and sheath blight) at first, and then extract shape
feature by artificial calculation (put forward three new shape
characteristic lesions number N, S lesion area, number of
FIGURE 4. The output heat maps of the ACB module in reference [47].
lesions ratio R) and CNN extracts color feature, at last,
the SVM classifier was used to identify the diseases, and the
results showed that the CNN used segmentation algorithm
III. LEAF DISEASE DETECTION BY DEEP accuracy was 92.75%, the accuracy was 82.26% without the
LEARNING ARCHITECTURES segmentation algorithm, and the accuracy of the CNN in
This section presents the recent researches done by using combination with the SVM model was 96.8%.
famous DL architectures for the identification and classi- Liang et al. [57] established a dataset contains 2906 of the
fication of leaf diseases. Moreover, there are some related positive samples and 2902 of the negative samples to identify
works in which modified/improved versions of DL architec- rice blasts. And the experimental results showed that the
tures were introduced to achieve better results and software senior characteristics extracted from CNN than the traditional
development of disease identification systems. manual extraction of local binary pattern histogram (LBPH)
and wavelet transform (Haar-WT) had better identification
A. LEAF DISEASE DETECTION BY WELL-KNOWN DEEP and effectiveness.
LEARNING ARCHITECTURES Huang et al. [58] put forward a kind of plant leaf image
1) CLASSIC DEEP LEARNING ARCHITECTURES FOR disease recognition method based on the neural structure
LEAF-DISEASE DETECTION search algorithm, the method can learn the structure of the
Since each disease region has its own characteristics, neural network to the appropriate depth on the P1antVillage,
Barbedo [42] and Lee et al. [52], discussed the use of indi- automatically. According to the results of the studied methods
vidual lesions and spots rather than considering the whole on the dataset of imbalanced and balanced searched out a
leaf. The advantages of this method were that occurrence suitable network structure, and the recognition accuracy of
of multiple diseases on the same leaf could be detected and the model was 98.96% and 99.01% respectively. However,
the data can be augmented by cutting up the leaf image if the balance of the gray images was not improved, the
into multiple sub-images. The article [55] taken 79 diseases accuracy fell to 95.40%.
of 14 species of plants in the experimental environment and Long et al. [59] used AlexNet for 2 kinds of training, that is,
complex field environment as the research object and used training from scratch and transfer learning from the ImageNet
the GoogLeNet model to identify diseases. The overall accu- to detect the camellia leaf diseases (4 kinds of diseases and
racy of using a single lesion and spot was 94%, which was healthy). The results showed that transfer learning can sig-
higher than using the whole image (82%). Lee et al. [52] put nificantly improve the convergence speed and classification
forward a new view of leaf disease detection that focused on performance of the models, and the classification accuracy as
identifying diseases disease area method (i.e. by the common high as 96.53%.
name of disease rather than crops - diseases on the target Xu et al. [60] in order to realize image recognition of
category), and through the experiments showed that whatever corn leaf disease (healthy, leaf blight, rust) in complex field
crops, the model training with the common disease were more background with small samples, proposed a convolutional
universal, especially for the new data obtained in different neural network model(VGG16) based on transfer learning.
fields or that crops have not been seen. The weight parameters of the VGG16 model were trained
Qiu et al. [53] used the Mask-RCNN whose feature extrac- on ImageNet and transferred to the model, and the average
tion network was ResNet50 or ResNet101 to detect the wheat recognition accuracy was 95.33%.
disease areas, and the average accuracy on the test dataset The ResNet50 network pre-trained on ImageNet was used
was 92.01%. to study 4 types of apple leaf diseases in the Plant Pathology

56688 VOLUME 9, 2021

L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

TABLE 1. Summary of recent research works about the application of DL framework directly.

2020 Challenge dataset, and the overall test accuracy of the accuracy of the proposed AlexNet-precursor + Cascade-
model was 97%. But except for the complex disease pat- Inception network was 97.62%.
tern category (the combination of several disease symptoms), Picon et al. [45] in order to extract the detailed features
the recognition accuracy was only 51% [61]. of the wheat disease symptoms, the first 7 × 7 convolutional
Li et al. [62] used VGG16 and Inception V3 models to layer of the ResNet50 network was replaced with two 3 × 3
identify different degrees of Ginkgo biloba diseases, the accu- convolutions and the sigmoid activation function was used
racy of the VGG16 was 98.44% in the laboratory dataset and instead of the softmax layer for improvement. And used the
92.19% in the field dataset. The accuracy of the Inception improved ResNet50 network to detect the early three wheat
V3 model was 92.3% and 93.2%, respectively. diseases (septoria, tan spot, and rust), and achieved 96%
Table 1 offers a brief overview of recent research works accuracy on the balanced dataset.
about the application of the DL framework directly. For the existing deep network model existed problems such
as a large number of parameters, long training time, high
storage cost and computational cost, etc. Wang et al. [64]
2) NEW/MODIFIED DL ARCHITECTURES FOR based on the ResNet18, by adding a multi-scale feature
LEAF-DISEASE DETECTION extraction module to change the residual layer connec-
Dechant et al. [39] integrated multiple CNN classifiers to tion method, decomposes the large convolution kernel and
study high-resolution corn disease images. The experimental performs group convolution operations, and proposes an
results showed that when a single CNN classifier was used, improved multi-scale residual (Multi-scale ResNet) model,
the accuracy rate was 90.8%, when two first-level classifiers which significantly reduced the model parameters, storage
were used, the accuracy rate rise to 95.9%, and when three space and computing overhead. The accuracy rate of 95.95%
first-level classifiers were used, the accuracy rate was 97.8%. was achieved on the PlantVillage dataset, and 93.05% was
Liu et al. [63] proposed a new CNN structure to identify achieved in the self-collected dataset of 7 real environmental
the apple leaf disease. The network was formed by cascading diseases.
an AlexNet-precursor network and an Inception network. Aiming at the problem that the current plant leaf dis-
The Inception network replaced the fully connected layers ease recognition model is easily interfered with by shadows,
in the traditional AlexNet model, significantly reducing the occlusions, and light intensity, and the feature extraction is
number of trainable parameters, thereby reducing storage blind and uncertain, Ren et al. [65] and others had constructed
requirements. Use Nesterov’s accelerated gradient (NAG) a deconvolution-guided VGG network (Deconvolution-
optimization algorithm instead of the stochastic gradient Guided VGGNet, DGVGGNet) model, which can identify
descent (SDG) algorithm to update the weights to improve plant leaf disease and segment disease spot. This model
the convergence speed. The performance of this network was had a recognition accuracy of 99.19% for the 10 types
compared with SVM, BP, AlexNet, GoogLeNet, ResNet20, of tomato leaf disease images in the PlantVillage dataset.
and VGG16. The accuracy of these models were 68.73%, The pixel accuracy and average intersection ratio of disease
54.63%, 91.19%, 95.69%, 92.76% and 96.32%, while the spots segmentation were 94.66% and 75.36%, respectively.

VOLUME 9, 2021 56689

 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

And it had good robustness in occlusion, low light, and other modifying the activation function. And the average recogni-
environments. tion accuracy of corn plant leaf diseases reached 92%.
Guo et al. [66] designed a multi-receptive field recognition Zhang et al. [71] combined the expansion convolution
model based on AlexNet (Multi-Scale AlexNet) by remov- and global pooling for the problem of the AlexNet model
ing the local response normalization layer of the AlexNet with too many parameters and a single feature scale and
network, modifying its fully connected layers, and setting proposed a global pooling extended convolutional neural net-
a multi-scale convolution kernel to extract features. The work (GPDCNN) based on the AlexNet model. After the
PlantVillage dataset and self-collected 7 kinds of tomato expansion, an accuracy of 95.18% was obtained on the dataset
diseased leaves dataset are the research objects. The model of 6 common cucumber leaf diseases taken in the field.
reduced the memory requirements of the original AlexNet by Due to the problem of low recognition accuracy of grape
95.4%, and the average recognition accuracy of tomato leaf leaves with different degrees of disease, He et al. [72] pro-
diseases and each disease in the early, middle, and late stages posed a Multi-Scale ResNet based on ResNet18 by changing
was up to 92.7%. the conv1 layer to a combination of multiple convolution
Fan et al. [67] added a batch standardization layer to kernels and adding the SENet module to ResNet18 to identify
the convolutional layer of the Faster R-CNN model, intro- grape leaf disease. The model had an average recognition
duced a central cost function to construct a mixed cost accuracy of 90.83% for seven grape diseases including dif-
function, and used a stochastic gradient descent algorithm ferent severity.
to optimize the training model. They used 9 kinds of corn Agarwal et al. [73] developed a CNN model with 3 convo-
leaf diseases with complex backgrounds in the field as the lutional layers, 3 maximum pooling layers, and 2 fully con-
research object. Under the same experimental environment, nected layers, and each layer had a different number of filters
the improved method had an average accuracy increase to detect 9 types of tomato leaf diseases. The experimental
of 8.86%, and a single image detection time was reduced by results showed that the average accuracy of the proposed
0.139s; compared with the SSD algorithm, the average accu- model on the test set reached 91.2%, and its performance was
racy was 4.25% higher, and a single image detection time was much better than VGG16, MobileNet, and Inception.
reduced by 0.018 s. Table 2 briefly introduces the research progress of the
Wang et al. [68] in order to solve the problems of a long improvement of DL in plant disease detection in recent
time of training, poor segmentation effect, and susceptibility years.
to illumination and background during the image segmen-
tation of cucumber leaf lesions in traditional convolutional B. TARGET DETECTION OF PLANT DISEASES FOR
neural networks, they proposed a method based on the full LEAF DISEASE DETECTION
convolution neural network (in which the activation function Fuentes et al. [74] used Faster R-CNN, R-FCN, and SSD
of rectified linear units (RELU) was replaced by the exponen- architectures to locate lesion areas of 9 kinds of tomato leaf
tial linear unit (ELU), and the batch normalization function diseases and insect pests, and classified them according to
was used to stabilize the model training process, and the the bounding box. And explored the influence of different
softmax of the original CNN was replaced with support vector CNN architectures on the detector. The results showed that
machine (SVM)). The average pixel segmentation accuracy ResNet50 as the feature extractor achieved a mean average
was 80.46% and the average cross-combination ratio was accuracy (mAP) of 85.98%, and the detection time was about
70.43% on the 6 kinds of cucumber leaf disease dataset. 160 ms per image. Subsequent work [75] refined the Faster
Hu et al. [51] tried to solve the problem of insufficient iden- R-CNN by introducing a single-class CNN, and the results
tification methods for fine-grained tomato diseases. Taken showed that the mAP increased by 13%.
5 kinds of tomato diseased leaves in the early and late stages Jiang et al. [76] proposed a novel method that is the
as the research objects, and proposed a new convolutional SSD with inception module and rainbow concatenation
neural network model ARNet based on the combination of (INAR-SSD). And the VGG16 feature extractor used in
attention and residual ideas. Compared with existing mod- the INAR-SSD network was a modification by replacing
els such as VGG16, ARNet had better classification perfor- two convolution layers (Conv4_1 and Conv4_2) with incep-
mance, with an average recognition accuracy rate of 88.2%. tion modules, fully connected layers of VGG16 were also
Picon et al. [69] proposed three different CNN archi- replaced with 1 × 1 convolutions. On a dataset of 5 kinds
tectures (RESNET-MC-1, RESNET-MC-2, and RESNET- of apple leaf diseases, the proposed INAR-SSD network
MC-3) to integrate contextual non-image meta-data (such achieved the highest mAP of 78.8% compared with the Faster
as crop information) into image-based convolutional neural R-CNN (73.78%) and SSD (75.82%). Meanwhile, the detec-
Network. RESNET-MC-1 achieved 98% accuracy in a field tion speed of the model was 23.13 FPS.
environment containing 17 diseases and 5 crops (wheat, bar- Li et al. [77] took 5 kinds of bitter gourd leaf diseases
ley, corn, rice, and rapeseed). taken in the field as the research object, modified the Faster
Chen et al. [70] proposed an improved VGG model R-CNN by increasing the size of the regional suggestion
(INC-VGGN) based on the VGG model framework by intro- frame and integrating the feature pyramid network (FPN)
ducing two Inception modules, adding a pooling layer, and based on ResNet50. The research results showed that after

56690 VOLUME 9, 2021

L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

TABLE 2. Summary of recent research works about application about the improvement of DL in plant disease detection.

integrating the feature pyramid network, the average accuracy the local receptive field of the original convolution kernel
of the obtained model was 86.39%, higher than the original and segment the lesion area accurately. Segmentation exper-
model (7.54%), and the accuracy of gray spot detection was iments were carried out on the field-photographed datasets
improved by 16.56%. The detection time of each image is of corn leaf spot, corn round spot, wheat stripe rust, wheat
0.322s, which can guarantee real-time detection. anthracnose, cucumber target spot disease, and cucumber
Aiming at the problem of difficulty in real-time detection brown spot. The segmentation accuracy was 87.04% and the
of apple leaf disease images under actual conditions due to the recall rate was 78.31%. The comprehensive evaluation index
complex background and small lesions, Li et al. [78] modified value was 88.22% and the single image segmentation speed
the Faster R-CNN by using the feature pyramid network was 0.23 s.
(FPN) and adopting precise region of interest pooling (PROI Table 3 offers a brief overview of recent research works in
Pooling). The research results showed that the improved target detection of plant diseases.
model can effectively detect five apple leaf diseases under
natural conditions, with a mean average accuracy of 82.28%. C. THE SYSTEM OF LEAF-DISEASE DETECTION
Compared with Faster R-CNN, YOLOv3, and Mask R-CNN, In an era when smart agricultural technology is so advanced,
the mean average accuracy increased by 5.81%, 13.92%, and mobile phones have become a new type of ‘‘farming tool’’
4.86%, and the detection time of a single image was reduced for farmers, which can help farmers in identifying diseases
by 43ms, respectively. and insect pests. Currently, researchers develop small pro-
Li et al. [79] proposed a video detection architecture of grams or mobile apps to help farmers identify crop pests and
plant diseases and insect pests based on deep learning and a diseases. The farmer takes pictures and uploads the diseased
custom backbone, which can better reflect the quality of video parts of the crop, and the system will return the recognition
detection in experiments. Experiments showed that compared result within a few seconds. And provide users with the
with VGG16, ResNet50, ResNet101 backbone systems, and diagnosis results, similarity, disease characteristics, causes,
YOLOv3, the custom backbone system was more suitable for and prevention and control plans for users, so that farmers
detecting untrained rice videos. The custom DCNN backbone can treat diseases and insects in a scientific way and increase
had eminent detection sensitivity in withered leaves of rice crop yields.
sheath blight and rice stem borer symptoms. And the detec- Ozguven and Adem [81] modified the Faster R-CNN by
tion speed was 30 frames per second (FPS). increasing the size of the input layer from 32 × 32 pixels
Aiming at the problem of low segmentation accuracy of to 600 × 600 pixels and developed an automatic detection
traditional convolutional neural networks in crop disease leaf and recognition system for leaf spot disease in 3 levels of
images, Wang et al. [80] constructed a regional disease detec- sugar beet disease severity (mild, moderate, and severe). The
tion network (RD-net) based on the traditional VGG16 model developed Faster R-CNN achieved an accuracy of 95.48%
and replaced the fully connected layer with a global pool- compared to 92.89% achieved by Faster R-CNN.
ing layer. Based on the Encoder-Decoder model structure, Aiming at the problem that the classification accuracy of
a regional segmentation network (RS-net) was established, the classification model for the severity of crop diseases and
and the multi-scale convolution kernel was used to improve insect pests is not high enough, Yu et al. [82] proposed an

VOLUME 9, 2021 56691

 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

TABLE 3. Summary of recent research works about the application in target detection of plant diseases.

TABLE 4. A brief overview of recent research works in the development of plant leaf disease identification system.

improved ResNet50 model (CDCNNv2) combined with deep Liu et al. [86] deployed the MobileNet network on the
transfer learning and developed a classification system for the mobile phone, and the average recognition accuracy of the
severity of crop diseases and insect pests. In addition to real- 6 kinds of grape diseased leaves collected in the field was
time and fully automatic detection of crop pests and diseases, 87.5%, and the average calculation time for a single image
the system also implements a series of supporting functions was 134ms.
such as prevention and control recommendations and drug Based on the ResNet50 architecture, Esgario et al. [87]
recommendations. developed a system that can identify and estimate the severity
Li et al. [83] combined the attention mechanism with the of stress caused by biological agents on coffee leaves. The
residual structure to build the PARNet model and completed system had an accuracy of 95.24% for the classification of
the development of the WEB application. The average accu- biological stress on coffee leaves, and an accuracy of 86.51%
racy of the platform for 5 tomato leaf diseases can reached for estimation of the severity.
96.84%. It was 2.25%∼11.58% higher than other models Xiong et al. [88] proposed an automatic image segmenta-
(VGG16, ResNet50, and SENet). tion algorithm based on the GrabCut algorithm and selected
Jiang et al. [84] redesigned and optimized the convo- the MobileNet as DL classification model, and designed a
lutional neural network structure based on the traditional crop disease recognition system for mobile smart devices.
LeNet-5 network, and proposed a convolutional neural net- The system had a recognition accuracy of more than 80% for
work system for ginger disease recognition based on the a total of 27 diseases of 6 crops in the laboratory environment
four kinds of ginger disease collected in the natural environ- and the field.
ment. The recognition rate of four kinds of ginger diseases Table 4 offers a brief overview of recent research works in
reached 96%. the development of plant leaf disease identification systems.
Zhou [85] identified 5 kinds of apple leaf diseases based
on transfer learning and the Faster R-CNN and developed IV. LEAF-DISEASE DETECTION BASED
an apple leaf disease detection system based on the Android ON SMALL SAMPLES
platform. The detection system had an average recognition In practical applications, the incidence of some plant diseases
accuracy of 76.55% for apple leaf diseases. is low and the cost of acquiring disease images is high,

56692 VOLUME 9, 2021

L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

resulting in only a few or dozens of disease images collected. fine-grained transfer learning (73%). Zhong et al. [99] pro-
The transfer learning method can transfer the knowledge posed a generative model based on conditional adversarial
learned from the general large dataset to the professional auto-encoder (CAAE), which was used to perform general-
fields with relatively little data. But for the datasets with only ized one-shot and few-shot learning in the case of few or even
a few or dozens of images, the transfer learning method also zero training samples to solve the problem of citrus diseases
has the problem of low recognition accuracy [23]. This is identification.
because it is difficult for the deep network to learn different Ren et al. [100] proposed a plant disease identification
features, which leads to problems that are difficult to converge method based on one-shot learning for the small sample prob-
or over-fitting. Therefore, plant disease datasets with single lem of plant leaf diseases. Taking 8 kinds of plant disease with
or small samples can hardly support the training of DL archi- a small number of samples in the public dataset P1antVillage
tecture. On the other hand, for the recognition of new classes as the identification object, the focal loss function (FL) was
that do not appear in the training set, the deep learning model used to train the plant disease classifier based on the relation
needs to be retrained. network. The results showed that the recognition accuracy
Recent advances in DL have proven the effectiveness of of the method in 5-way and 1-shot tasks reached 89.90%,
several architectures to learn new classes using small datasets, which was 4.69% higher than the original relation network
a famous sub-field known as Few-Shot Learning (FSL) [89]. model. At the same time, compared with matching network
FSL can not only solve the recognition problem of new and transfer learning, the improved method had increased the
classes that did not appear in the training but also solve the recognition accuracy on the experimental dataset by 25.02%
problem of the neural network, which difficult to converge and 41.90%, respectively.
due to the small number of experimental samples, thereby Argüeso et al. [101] taken 38 kinds of plant disease images
improving the accuracy of small datasets recognition. in the public dataset P1antVillage as the identification object,
The FSL methods used for image classification include Siamese networks, and triplet loss was used and compared to
model initialization, metric learning and data generate meth- classical fine-tuning transfer learning. The median accuracy
ods. The initialization method focuses on the adjustable was 55.5 % learning for 1 image per class. Median accuracy
parameters in the network so that a new class of classifier can were 80.0 % and 90.0 % for 15 and 80 images per class. The
be learned from a limited set of examples [90]–[92]. The aim FSL method outperformed the classical fine-tuning transfer
of metric learning methods is learning to compare. It means learning which had an accuracy of 18.0 % and 72.0 % for
that once a network learns to compare classes, it will be able 1 and 80 images per class, respectively. The author Wu [32]
to learn new classes from few labeled samples [93]. Finally, took 3 kinds of tea leaf diseases as the research object, seg-
generate data methods, the methods learn a generator from mented the lesions and expanded the dataset of the segmented
the data in the base classes, and use that generator to generate lesions at first, and then used the combination of depth trans-
data for new classes. fer and Cayley-Klein metric to realize the identification of
FSL solutions for plant leaves classification have been tea diseases, and a result of 100% recognition accuracy was
introduced recently, for example, Hu et al. [94] present a achieved.
low shot learning method for tea leaf disease identification, Table 5 offers a brief overview of recent research works in
used the improved conditional deep convolutional genera- plant leaf disease detection based on small samples.
tive adversarial networks (C-DCGAN) for data augmenta-
tion. And the average identification accuracy of the proposed V. HYPER-SPECTRAL IMAGING(HSI) WITH DL MODELS
method was 90%. Wang and Wang [95] proposed a few- Plants may be affected by multiple pathogens at the same
shot learning method based on the Siamese network with time during the growth process, and some different pathogens
contrastive loss and kNN classifiers to solve plant leaf classi- may produce similar symptoms and signs [102], [103] and the
fication problem with a small sample. Das and Lee [96] pro- symptoms are not obvious at the early stage of plant diseases,
posed a two-stage multilayer neural network for the few-shot which makes it easy to use naked eyes or simple computer
recognition of new categories and a detailed mathematical vision has become very difficult to detect plant diseases.
theory derivation process. The electromagnetic spectrum range of hyperspectral sen-
Aiming at the problem of too few samples in the training sors is mainly concentrated in the visible and near-infrared
set, Li et al. [97] proposed a one-shot learning method for (400 ∼ 1000nm), and sometimes includes shortwave infrared
the first time and used Bayesian functions to build a network. (SWIR, 1000 ∼ 2500nm). This sensor can obtain spectral
Subsequently, many DL architectures for one-shot learning information from hundreds of narrow spectral bands [104].
tasks were proposed and achieved remarkable results. The These narrow bands are highly sensitive to subtle plant
author verified in his work [98] that using FSL can trans- leaf changes caused by diseases and can distinguish differ-
fer knowledge from a clear source domain (colon tissue) ent types of diseases so that early asymptomatic detection
to a more general domain (composed of colon, lung, and can be carried out. Therefore, HSI is the focus of recent
breast tissues) by using few training images. Experimental research, for the early detection of plant diseases. For exam-
results showed that the FSL can obtain an average accuracy ple, the review [105] provided an overview of advanced
of 90% with only 60 training images, which was better than hyperspectral technologies for plant disease detection.

VOLUME 9, 2021 56693

 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

TABLE 5. A brief overview of recent research works in plant leaf disease detection based on small samples.

FIGURE 5. Comparison of segmentation results of a typical healthy plant. (a)direct CNN model. (b)AC-GAN model. (c)the
proposed OR-AC-GAN model.

Xie et al. [106] investigated the feasibility of using a hyper- Through spectral sensitivity analysis, the disease sensitive
spectral imaging technique to identify two kinds of diseases bands were determined, and two new disease indexes were
on tomato leaves. Both imaging information and spectral established using these bands: tea tree anthracnose ratio index
information were investigated in the study. The ELM model (TARI) and tea tree anthracnose normalized index (TANI).
was established to identify the diseased samples and the A method combined unsupervised classification and adaptive
successive projection algorithm (SPA) was applied to select two-dimensional threshold detection is proposed, based on a
useful wavelengths. The classification accuracy was 97.1% set of optimized spectral features. The results showed that
of the SPA-ELM model on the testing set. The early hyper- the overall accuracy of identifying diseases was 98% at the
spectral images of cucumber downy mildew in greenhouses leaf level, and the overall accuracy of identifying diseases
collected infield, it is influenced by environmental illumi- was 94% at the pixel level. In [110], a detailed review of
nation and difficult to extract effective features from them. DL with the HSI technique was provided. In order to avoid
Qin et al. [107] proposed a novel method of extracting fea- the over-fitting and improve accuracy, a detailed comparison
ture bands based on disease difference information. Which was provided between several DL models like 1D/2D-CNN
improved combining adaptive weighting algorithm (CARS) (2D-CNN better result) LSTM/GRU (both faced over-fitting),
and successive projection algorithm (SPA), and an early 2D-CNN-LSTM/GRU (still over-fitting). Therefore, a novel
detection model of cucumber downy mildew was established. hybrid method (2D-CNN-BidLSTM/GRU) which from a
For the hyperspectral image of healthy cucumber leaves and convolutional and bidirectional gated recurrent network was
the daily hyperspectral images within 12 days of infection, proposed for the hyperspectral images. The model resolved
the detection rate of 100% can be obtained from the 2∼12 day the problem of over-fitting and achieved 75% F1-score and
of infection, and the detection rate of the test set for 1 day 73% accuracy for wheat disease detection [111]. In [112],
of infection reached 95.8%. Abdulridha et al. [108] used the author developed a supervised 3D-CNN model to learn
hyper-spectral imaging technology combined with the MLP the spectral and spatial information of hyperspectral images
classification method, had an accuracy of 99% for the four for the classification of healthy and charcoal rot infected
stages of tomato bacterial spot disease and bacterial target samples. A visualization method based on a saliency map was
spot disease (healthy asymptomatic stage, early stage, and used to identify the classification accuracy of hyper-spectral
late-stage). Yuan et al. [109] proposed a method for detect- wavelengths. The importance of wavelength can be inferred
ing tea tree anthracnose based on hyperspectral imaging. by analyzing the size of the gradient distribution of the

56694 VOLUME 9, 2021

L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

TABLE 6. A brief overview of recent research works in plant leaf disease detection using hyperspectral images.

saliency map of the image on the hyper-spectral wavelength. have been discussed. At the same time, there are also some
Based on the hyper-spectral imaging of the inoculated and inadequacies.
simulated inoculated stem images, the classification accuracy Most of the DL frameworks proposed in the literature have
of the model 3D-CNN was 95.73%, and the F1 score of good detection effects on their datasets, but the effects are not
the infection category was 87%. For the detection of potato good on other datasets, that is the model has poor robustness.
virus, DL was used to describe the hyperspectral images Therefore, better robustness DL models are needed to adapt
and achieved acceptable values of precision (78%) and recall the diverse disease datasets.
(88%) [113]. In [114], developed a DL model of multiple In most of the researches, the PlantVillage dataset was used
Inception-ResNet, which uses both spatial and spectral data to evaluate the performance of the DL models. Although this
on hyperspectral UAV images to detect the yellow rust in dataset has a lot of images of several plant species with their
wheat. The model achieved an accuracy of 85%, which was diseases, it was taken in the lab. Therefore, it is expected to
quite a lot higher than the RF-classifier (77%). Gui et al. [115] establish a large dataset of plant diseases in real conditions.
divided the early soybean mosaic virus disease (SMV) into Although some studies are using hyperspectral images of
0, 1, and 2 degrees according to its severity. In the case of diseased leaves, and some DL frameworks are used for early
a small number of experimental soybean samples, they pro- detection of plant leaves diseases, problems that affect the
posed a novel SMV early detection method which combined widespread use of HSI in the early detection of plant dis-
convolutional neural network and a support vector machine eases remain to be resolved. That is, for early plant disease
(CNN-SVM), and achieved an accuracy rate of 96.67% on detection, it is difficult to obtain the labeled datasets, and
the training set and 94.17% on the testing set. The litera- even experienced experts cannot mark where the invisible
ture [116] taken corn seedlings after cold stress as the research disease symptoms are, and define purely invisible disease
object, extract the spectral curve of the comprehensive eval- pixels, which is very important for HSI to detect plant disease.
uation index of cold damage based on hyper-spectral images,
and used DL to construct a corn seedling damage detection REFERENCES
model. According to [117], a novel hyper-spectral analysis [1] F. Fina, P. Birch, R. Young, J. Obu, B. Faithpraise, and C. Chatwin,
proximal sensing method based on generative adversarial ‘‘Automatic plant pest detection and recognition using k-means clustering
algorithm and correspondence filters,’’ Int. J. Adv. Biotechnol. Res., vol. 4,
nets (GANs), named as outlier removal auxiliary classifier no. 2, pp. 189–199, Jul. 2013.
generative adversarial nets (OR-AC-GAN) was proposed in [2] M. A. Ebrahimi, M. H. Khoshtaghaza, S. Minaei, and B. Jamshidi,
order to detect tomato plant disease before its clear symp- ‘‘Vision-based pest detection based on SVM classification method,’’
Comput. Electron. Agricult., vol. 137, pp. 52–58, May 2017.
toms appeared. The classification accuracy achieved 96.25% [3] S. R. Dubey and A. S. Jalal, ‘‘Adapted approach for fruit disease identi-
before visible symptoms show up at plant leaf level (as shown fication using images,’’ Int. J. Comput. Vis. Image Process., vol. 2, no. 3,
in Fig. 5). pp. 44–58, Jul. 2012.
[4] A.-L. Chai, B.-J. Li, Y.-X. Shi, Z.-X. Cen, H.-Y. Huang, and J. Liu,
Table 6 offers a brief overview of recent research works in ‘‘Recognition of tomato foliage disease based on computer vision tech-
plant leaf disease detection using hyperspectral images. nology,’’ Acta Horticulturae Sinica, vol. 37, no. 9, pp. 1423–1430,
Sep. 2010.
[5] Z. R. Li and D. J. He, ‘‘Research on identify technologies of apple’s
VI. CONCLUSION AND FUTURE DIRECTIONS
disease based on mobile photograph image analysis,’’ Comput. Eng. Des.,
In this paper, we have introduced the basic knowledge of vol. 31, no. 13, pp. 3051–3053 and 3095, Jul. 2010.
deep learning and presented a comprehensive review of recent [6] Z.-X. Guan, J. Tang, B.-J. Yang, Y.-F. Zhou, D.-Y. Fan, and Q. Yao,
‘‘Study on recognition method of rice disease based on image,’’ Chin.
research work done in plant leaf disease recognition using J. Rice Sci., vol. 24, no. 5, pp. 497–502, May 2010.
deep learning. Provided sufficient data is available for train- [7] J. G. A. Barbedo, ‘‘Factors influencing the use of deep learning for plant
ing, deep learning techniques are capable of recognizing plant disease recognition,’’ Biosyst. Eng., vol. 172, pp. 84–91, Aug. 2018.
leaf diseases with high accuracy. The importance of collect- [8] A. Kamilaris and F. X. Prenafeta-Boldú, ‘‘Deep learning in agriculture:
A survey,’’ Comput. Electron. Agricult., vol. 147, pp. 70–90, Apr. 2018.
ing large datasets with high variability, data augmentation, [9] G. L. Grinblat, L. C. Uzal, M. G. Larese, and P. M. Granitto, ‘‘Deep learn-
transfer learning, and visualization of CNN activation maps ing for plant identification using vein morphological patterns,’’ Comput.
in improving classification accuracy, and the importance of Electron. Agricult., vol. 127, pp. 418–424, Sep. 2016.
[10] S. P. Mohanty, D. P. Hughes, and M. Salathé, ‘‘Using deep learning for
small sample plant leaf disease detection and the importance image-based plant disease detection,’’ Frontiers Plant Sci., vol. 7, p. 1419,
of hyper-spectral imaging for early detection of plant disease Sep. 2016.

VOLUME 9, 2021 56695

 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

[11] J. Ma, K. Du, F. Zheng, L. Zhang, Z. Gong, and Z. Sun, ‘‘A recognition [33] Q. Wu, Y. Chen, and J. Meng, ‘‘DCGAN-based data augmenta-
method for cucumber diseases using leaf symptom images based on deep tion for tomato leaf disease identification,’’ IEEE Access, vol. 8,
convolutional neural network,’’ Comput. Electron. Agricult., vol. 154, pp. 98716–98728, 2020.
pp. 18–24, Nov. 2018. [34] B. Liu, C. Tan, S. Li, J. He, and H. Wang, ‘‘A data augmentation method
[12] Y. Kawasaki, H. Uga, S. Kagiwada, and H. Iyatomi, ‘‘Basic study of based on generative adversarial networks for grape leaf disease identifi-
automated diagnosis of viral plant diseases using convolutional neural cation,’’ IEEE Access, vol. 8, pp. 102188–102198, 2020.
networks,’’ in Proc. Int. Symp. Vis. Comput., Las Vegas, NV, USA, [35] M. Brahimi, K. Boukhalfa, and A. Moussaoui, ‘‘Deep learning for tomato
Dec. 2015, pp. 638–645. diseases: Classification and symptoms visualization,’’ Appl. Artif. Intell.,
[13] Y. Kessentini, M. D. Besbes, S. Ammar, and A. Chabbouh, ‘‘A two- vol. 31, no. 4, pp. 299–315, May 2017.
stage deep neural network for multi-norm license plate detection and [36] M. Brahimi, M. Arsenovic, S. Laraba, S. Sladojevic, K. Boukhalfa,
recognition,’’ Expert Syst. Appl., vol. 136, pp. 159–170, Dec. 2019. and A. Moussaoui, ‘‘Deep learning for plant diseases: Detection
[14] A. K. Singh, B. Ganapathysubramanian, S. Sarkar, and A. Singh, ‘‘Deep and saliency map visualisation,’’ in Human and Machine Learning.
learning for plant stress phenotyping: Trends and future perspectives,’’ Boston, MA, USA: Springer, Jun. 2018, pp. 93–117. [Online]. Available:
Trends Plant Sci., vol. 23, no. 10, pp. 883–898, Oct. 2018. https://link.springer.com/book/10.1007/978
[15] V. Singh, N. Sharma, and S. Singh, ‘‘A review of imaging techniques [37] A. C. Cruz, A. Luvisi, L. De Bellis, and Y. Ampatzidis, ‘‘Vision-based
for plant disease detection,’’ Artif. Intell. Agricult., vol. 4, pp. 229–242, plant disease detection system using transfer and deep learning,’’ in Proc.
Oct. 2020. Asabe Annu. Int. Meeting, Spokane, WA, USA, Jul. 2017, pp. 16–19.
[16] M. H. Saleem, J. Potgieter, and K. M. Arif, ‘‘Plant disease detection [38] M. Brahimi, S. Mahmoudi, K. Boukhalfa, and A. Moussaoui, ‘‘Deep
and classification by deep learning,’’ Plants, vol. 8, no. 11, pp. 468–489, interpretable architecture for plant diseases classification,’’ in Proc. Sig-
Oct. 2019. nal Process., Algorithms, Archit., Arrangements, Appl. (SPA), Poznan,
[17] L. C. Ngugi, M. Abelwahab, and M. Abo-Zahhad, ‘‘Recent advances Poland, Sep. 2019, pp. 111–116.
in image processing techniques for automated leaf pest and disease [39] C. DeChant, T. Wiesner-Hanks, S. Chen, E. L. Stewart, J. Yosinski,
recognition—A review,’’ Inf. Process. Agricult., vol. 180, pp. 26–50, M. A. Gore, R. J. Nelson, and H. Lipson, ‘‘Automated identifica-
Apr. 2020. tion of northern leaf blight-infected maize plants from field imagery
[18] W. S. McCulloch and W. Pitts, ‘‘A logical calculus of the ideas immanent using deep learning,’’ Phytopathology, vol. 107, no. 11, pp. 1426–1432,
in nervous activity,’’ Bull. Math. Biophys., vol. 5, no. 4, pp. 115–133, Nov. 2017.
Dec. 1943. [40] J. Lu, J. Hu, G. Zhao, F. Mei, and C. Zhang, ‘‘An in-field automatic
[19] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘ImageNet classification wheat disease diagnosis system,’’ Comput. Electron. Agricult., vol. 142,
with deep convolutional neural networks,’’ in Proc. Adv. Neural Inf. pp. 369–379, Nov. 2017.
Process. Syst., vol. 25, Sep. 2012, pp. 1097–1105.
[41] J. G. Ha, H. Moon, J. T. Kwak, S. I. Hassan, M. Dang, O. N. Lee, and
[20] K. He, X. Zhang, S. Ren, and J. Sun, ‘‘Deep residual learning for H. Y. Park, ‘‘Deep convolutional neural network for classifying Fusarium
image recognition,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. wilt of radish from unmanned aerial vehicles,’’ J. Appl. Remote Sens.,
(CVPR), Sacramento, CA, USA, Jun. 2016, pp. 770–778. vol. 11, no. 4, p. 1, Dec. 2017.
[21] K. Simonyan and A. Zisserman, ‘‘Very deep convolutional networks for
[42] J. G. A. Barbedo, ‘‘Plant disease identification from individual lesions
large-scale image recognition,’’ in Proc. Int. Conf. Learn. Repr., London,
and spots using deep learning,’’ Biosyst. Eng., vol. 180, pp. 96–107,
U.K., Apr. 2014, pp. 1–14.
Apr. 2019.
[22] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
[43] S. Ghosal, D. Blystone, A. K. Singh, B. Ganapathysubramanian,
V. Vanhoucke, and A. Rabinovich, ‘‘Going deeper with convolutions,’’ in
A. Singh, and S. Sarkar, ‘‘An explainable deep machine vision framework
Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Sacramento,
for plant stress phenotyping,’’ Proc. Nat. Acad. Sci. USA, vol. 115, no. 18,
CA, USA, Jun. 2015, pp. 1–9.
pp. 4613–4618, Apr. 2018.
[23] J. Amara, B. Bouaziz, and A. Algergawy, ‘‘A deep learning-based
approach for banana leaf diseases classification,’’ in Proc. Datenbanksys. [44] Y. Lu, S. Yi, N. Zeng, Y. Liu, and Y. Zhang, ‘‘Identification of rice diseases
Für Bus., Technol. Web (BTW), Workshopband, CA, USA, Jul. 2017, using deep convolutional neural networks,’’ Neurocomputing, vol. 267,
pp. 1–24. pp. 378–384, Dec. 2017.
[24] M. Rußwurm and M. Körner, ‘‘Multi-temporal land cover classification [45] A. Picon, A. Alvarez-Gila, M. Seitz, A. Ortiz-Barredo, J. Echazarra, and
with long short-term memory neural networks,’’ in Proc. Int. Arch. Pho- A. Johannes, ‘‘Deep convolutional neural networks for mobile capture
togramm. Remote Sens. Spat. Inf. Sci., Hannover, Germany, Jun. 2017, device-based crop disease classification in the wild,’’ Comput. Electron.
pp. 551–558. Agricult., vol. 161, pp. 280–290, Jun. 2019.
[25] M. Türkoglu and D. Hanbay, ‘‘Plant disease and pest detection using deep [46] A. Johannes, A. Picon, A. Alvarez-Gila, J. Echazarra,
learning-based features,’’ TURKISH J. Electr. Eng. Comput. Sci., vol. 27, S. Rodriguez-Vaamonde, A. D. Navajas, and A. Ortiz-Barredo,
no. 3, pp. 1636–1651, May 2019. ‘‘Automatic plant disease diagnosis using mobile capture devices,
[26] A. K. Mortensen, M. Dyrmann, H. Karstoft, R. N. Jørgensen, and applied on a wheat use case,’’ Comput. Electron. Agricult., vol. 138,
R. Gislum, ‘‘Semantic segmentation of mixed crops using deep convolu- pp. 200–209, Jun. 2017.
tional neural network,’’ in Proc. CIGR-AgEng Conf., Aarhus, Denmark, [47] M. A. Khan, T. Akram, M. Sharif, M. Awais, K. Javed, H. Ali, and
Jun. 2016, pp. 26–29. T. Saba, ‘‘CCDF: Automatic system for segmentation and recognition
[27] M. Dyrmann, H. Karstoft, and H. S. Midtiby, ‘‘Plant species classification of fruit crops diseases based on correlation coefficient and deep CNN
using deep convolutional neural network,’’ Biosystems Eng., vol. 151, features,’’ Comput. Electron. Agricult., vol. 155, pp. 220–236, Dec. 2018.
pp. 72–80, Nov. 2016. [48] M. Kerkech, A. Hafiane, and R. Canals, ‘‘Deep leaning approach with
[28] W. Xinshao and C. Cheng, ‘‘Weed seeds classification based on PCANet colorimetric spaces and vegetation indices for vine diseases detection
deep learning baseline,’’ in Proc. Asia–Pacific Signal Inf. Process. Assoc. in UAV images,’’ Comput. Electron. Agricult., vol. 155, pp. 237–243,
Annu. Summit Conf. (APSIPA), Hong Kong, Dec. 2015, pp. 408–415. Dec. 2018.
[29] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, [49] K. Nagasubramanian, A. K. Singh, A. Singh, S. Sarkar, and
S. Ozair, A. Courville, and Y. Bengio, ‘‘Generative adversarial nets,’’ in B. Ganapathysubramanian, ‘‘Usefulness of interpretability methods
Proc. Adv. Neural Inf. Process. Sys. (NIPS), Quebec City, QC, Canada, to explain deep learning based plant stress phenotyping,’’ Comput. Sci.,
Jun. 2014, pp. 2672–2680. vol. 4, pp. 18–32, Jul. 2020.
[30] H. Nazki, S. Yoon, A. Fuentes, and D. S. Park, ‘‘Unsupervised image [50] Y. Sun, Z. Jiang, L. Zhang, W. Dong, and Y. Rao, ‘‘SLIC_SVM based
translation using adversarial networks for improved plant disease recog- leaf diseases saliency map extraction of tea plant,’’ Comput. Electron.
nition,’’ Comput. Electron. Agricult., vol. 168, Jan. 2020, Art. no. 105117. Agricult., vol. 157, pp. 102–109, Feb. 2019.
[31] Y. Tian, G. Yang, Z. Wang, E. Li, and Z. Liang, ‘‘Detection of apple [51] Z.-W. Hu, H. Yang, J.-M. Huang, and Q.-Q. Xie, ‘‘Fine-grained tomato
lesions in orchards based on deep learning methods of CycleGAN and disease recognition based on attention residual mechanism,’’ J. South
YOLOV3-dense,’’ J. Sensors, vol. 2019, pp. 1–13, Apr. 2019. China Agricult. Univ., vol. 40, no. 6, pp. 124–132, Jul. 2019.
[32] H.-Y. Wu, ‘‘Identification of tea leaf’s diseases in natural scene images [52] S. H. Lee, H. Goëau, P. Bonnet, and A. Joly, ‘‘New perspectives on
based on low shot learning,’’ M.S. thesis, Dept. Inf. Eng., Anhui Univ., plant disease characterization based on deep learning,’’ Comput. Electron.
Hefei, China, 2020. Agricult., vol. 170, Mar. 2020, Art. no. 105220.

56696 VOLUME 9, 2021

L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

[53] R. Qiu, C. Yang, A. Moghimi, M. Zhang, B. J. Steffenson, and [75] A. F. Fuentes, S. Yoon, J. Lee, and D. S. Park, ‘‘High-performance deep
C. D. Hirsch, ‘‘Detection of fusarium head blight in wheat using a deep neural network-based tomato plant diseases and pests diagnosis system
neural network and color imaging,’’ Remote Sens., vol. 11, no. 22, with refinement filter bank,’’ Frontiers Plant Sci., vol. 9, Aug. 2018,
Nov. 2019, Art. no. 2658. Art. no. 1162.
[54] I. Ahmad, M. Hamid, S. Yousaf, S. T. Shah, and M. O. Ahmad, [76] P. Jiang, Y. Chen, B. Liu, D. He, and C. Liang, ‘‘Real-time detection
‘‘Optimizing pretrained convolutional neural networks for tomato leaf of apple leaf diseases using deep learning approach based on improved
disease detection,’’ Complexity, vol. 2020, pp. 1–6, Sep. 2020, doi: convolutional neural networks,’’ IEEE Access, vol. 7, pp. 59069–59080,
10.1155/2020/8812019. 2019.
[55] C. Bi, J. Wang, Y. Duan, B. Fu, J.-R. Kang, and Y. Shi, ‘‘MobileNet based [77] J.-H. Li, L.-J. Lin, and K. Tian, ‘‘Detection of leaf diseases of balsam pear
apple leaf diseases identification,’’ Mobile Netw. Appl., vol. 10, pp. 1–9, in the field based on improved faster R-CNN,’’ Trans. Chin. Soc. Agricult.
Aug. 2020, doi: 10.1007/s11036-020-01640-1. Eng., vol. 36, no. 12, pp. 179–185, Jun. 2020.
[56] F. Jiang, Y. Lu, Y. Chen, D. Cai, and G. Li, ‘‘Image recognition of four [78] X.-R. Li, S.-Q. Li, and B. Liu, ‘‘Apple leaf disease detection method based
rice leaf diseases based on deep learning and support vector machine,’’ on improved faster R-CNN,’’ Comput. Eng., vol. 46, no. 11, pp. 59–64,
Comput. Electron. Agricult., vol. 179, Dec. 2020, Art. no. 105824. Nov. 2020.
[57] W.-J. Liang, H. Zhang, G.-F. Zhang, and H.-X. Cao, ‘‘Rice blast disease [79] D. Li, R. Wang, C. Xie, L. Liu, J. Zhang, R. Li, F. Wang, M. Zhou, and
recognition using a deep convolutional neural network,’’ Sci. Rep., vol. 9, W. Liu, ‘‘A recognition method for rice plant diseases and pests video
no. 1, pp. 1–10, Feb. 2019. detection based on deep convolutional neural network,’’ Sensors, vol. 20,
[58] J.-P. Huang, J.-X. Chen, K.-X. Li, J.-Y. Li, and H. Liu, ‘‘Identification no. 3, Jan. 2020, Art. no. 578.
of multiple plant leaf diseases using neural architecture search,’’ Trans. [80] W. Zhen, Z. Shanwen, and Z. Baoping, ‘‘Crop diseases leaf segmentation
Chin. Soc. Agricult. Eng., vol. 36, no. 16, pp. 166–173, Aug. 2020. method based on cascade convolutional neural network,’’ Comput. Eng.
[59] M.-S. Long, C.-J. OuYang, H. Liu, and Q. Fu, ‘‘Image recognition Appl., vol. 56, no. 15, pp. 242–250, Aug. 2020.
of camellia oleifera diseases based on convolutional neural network & [81] M. M. Ozguven and K. Adem, ‘‘Automatic detection and classification of
transfer learning,’’ Trans. Chin. Soc. Agricult. Eng., vol. 34, no. 18, leaf spot disease in sugar beet using deep learning algorithms,’’ Phys. A,
pp. 194–201, Sep. 2018. Stat. Mech. Appl., vol. 535, Dec. 2019, Art. no. 122537.
[60] J.-H. Xu, M.-Y. Shao, Y.-C. Wang, and W.-T. Han, ‘‘Recognition of corn [82] X.-D. Yu, M.-J. Yang, H.-Q. Zhang, D. Li, Y.-Q. Tang, and X. Yu,
leaf spot and rust based on transfer learning with convolutional neural ‘‘Research and application of crop diseases detection method based on
network,’’ Trans. Chin. Soc. Agricult. Mach., vol. 51, no. 2, pp. 230–236, transfer learning,’’ Trans. Chin. Soc. Agricult. Eng., vol. 51, no. 10,
Feb. 2020. pp. 252–258, May 2020.
[61] R. Thapa, N. Snavely, S. Belongie, and A. Khan, ‘‘The plant pathology [83] X.-Z. Li, Y. Xu, Z.-H. Wu, Z. Gao, and L. Liu, ‘‘Recognition system
2020 challenge dataset to classify foliar disease of apples,’’ Comput. Sci., of tomato leaf disease based on attentional neural network,’’ Jiangsu J.
vol. 3, Feb. 2020, Art. no. 11958. Agricult. Sci., vol. 36, no. 3, pp. 561–568, Mar. 2020.
[62] K.-Z. Li, J.-H. Lin, J.-R. Liu, and Y.-D. Zhao, ‘‘Using deep learning [84] F.-Q. Jiang, C. Li, D.-W. Yu, M. Sun, and E.-B. Zhang, ‘‘Design
for image-based different degrees of ginkgo leaf disease classification,’’ and experiment of tobacco leaf grade recognition system based
Information, vol. 11, no. 2, pp. 95–104, Feb. 2020. on caffe,’’ J. Chin. Agricult. Mech., vol. 40, no. 1, pp. 126–131,
[63] B. Liu, Y. Zhang, D.-J. He, and Y.-X. Li, ‘‘Identification of apple leaf dis- Jan. 2019.
eases based on deep convolutional neural networks,’’ Symmetry, vol. 10,
[85] M.-M. Zhou, ‘‘Apple foliage diseases recognition in Android system with
no. 1, pp. 11–19, Dec. 2018.
transfer learning-based,’’ M.S. thesis, Dept. Inf. Eng., Northwest A&F
[64] C.-S. Wang, J. Zhou, H.-R. Wu, G.-F. Teng, C.-J. Zhao, and J.-X. Li,
Univ., Yangling, China, 2019.
‘‘Identification of vegetable leaf diseases based on improved multi-scale
[86] Y. Liu, Q. Feng, and S.-Z. Wang, ‘‘Plant disease identification method
ResNet,’’ Trans. Chin. Soc. Agricult. Eng., vol. 36, no. 20, pp. 209–217,
based on lightweight CNN and mobile application,’’ Trans. Chin. Soc.
Doc. 2020.
Agricult. Eng., vol. 35, no. 17, pp. 194–204, Jun. 2019.
[65] S.-G. Ren, F.-W. Jia, X.-J. Gu, P.-S. Yuan, W. Xue, and H.-L. Xu,
[87] J. G. M. Esgario, R. A. Krohling, and J. A. Ventura, ‘‘Deep learning
‘‘Recognition and segmentation model of tomato leaf diseases based on
for classification and severity estimation of coffee leaf biotic stress,’’
deconvolution-guiding,’’ Trans. Chin. Soc. Agricult. Eng., vol. 36, no. 2,
Comput. Electron. Agricult., vol. 169, Feb. 2020, Art. no. 105162.
pp. 186–195, Jan. 2020.
[66] X. Q. Guo, T. J. Fan, and X. Shu, ‘‘Tomato leaf diseases recognition [88] Y. Xiong, L. Liang, L. Wang, J. She, and M. Wu, ‘‘Identification of cash
based on improved multi-scale AlexNet,’’ Trans. Chin. Soc. Agricult. crop diseases using automatic image segmentation algorithm and deep
Eng., vol. 35, no. 13, pp. 162–169, Jul. 2019. learning with expanded dataset,’’ Comput. Electron. Agricult., vol. 177,
Oct. 2020, Art. no. 105712.
[67] X.-P. Fan, J.-P. Zhou, and Y. Xu, ‘‘Recognition of field maize leaf diseases
based on improved regional convolutional neural network,’’ J. South [89] H. Larochelle, ‘‘Few-shot learning,’’ in Computer Vision: A Reference
China Agricult. Univ., vol. 41, no. 6, pp. 82–91, Jun. 2020. Guide, 2nd ed., K. Ikeuchi, Ed. Tokyo, Tokyo: Univ. Tokyo Press, 2014.
[68] Z. Wang, S.-W. Zhang, and X.-F. Wang, ‘‘Method for segmentation of [Online]. Available: http://www.doc88.com/
cucumber leaf lesions based on improved full convolution neural net- [90] C. Finn, K. Xu, and S. Levine, ‘‘Probabilistic model-agnostic meta-
work,’’ Jiangsu J. Agricult. Sci., vol. 35, no. 5, pp. 1054–1060, May 2019. learning,’’ Comput. Sci., vol. 2, pp. 9516–9527, Oct. 2018.
[69] A. Picon, M. Seitz, A. Alvarez-Gila, P. Mohnke, A. Ortiz-Barredo, [91] A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, R. Pascanu, S. Osindero,
and J. Echazarra, ‘‘Crop conditional convolutional neural networks for and R. Hadsell, ‘‘Meta-learning with latent embedding optimization,’’
massive multi-crop plant disease classification over cell phone acquired Comput. Sci., vol. 3, pp. 18–27, Mar. 2018.
images taken on real field conditions,’’ Comput. Electron. Agricult., [92] A. Nichol and J. Schulman, ‘‘Reptile: A scalable meta learning algo-
vol. 167, Dec. 2019, Art. no. 105093. rithm,’’ Comput. Sci., vol. 2, no. 2, pp. 1–8, Mar. 2018.
[70] J. Chen, J. Chen, D. Zhang, Y. Sun, and Y. A. Nanehkaran, ‘‘Using deep [93] G. Koch, ‘‘Siamese neural networks for one-shot image recognition,’’
transfer learning for image-based plant disease identification,’’ Comput. M.S. thesis, Dept. Comput. Sci., Toronto Univ., Toronto, ON, Canada,
Electron. Agricult., vol. 173, Jun. 2020, Art. no. 105393. 2015.
[71] S. Zhang, S. Zhang, C. Zhang, X. Wang, and Y. Shi, ‘‘Cucumber leaf [94] G. Hu, H. Wu, Y. Zhang, and M. Wan, ‘‘A low shot learning method for
disease identification with global pooling dilated convolutional neural tea leaf’s disease identification,’’ Comput. Electron. Agricult., vol. 163,
network,’’ Comput. Electron. Agricult., vol. 162, pp. 422–430, Jul. 2019. Aug. 2019, Art. no. 104852.
[72] X. He, S.-Q. Li, and B. Liu, ‘‘Grape leaf disease identification based [95] B. Wang and D. Wang, ‘‘Plant leaves classification: A few-shot
on multi-scale residual network,’’ Comput. Eng., vol. 46, no. 4, pp. 1–8, learning method based on Siamese network,’’ IEEE Access, vol. 7,
Apr. 2020. pp. 151754–151763, 2019.
[73] M. Agarwal, A. Singh, S. Arjaria, A. Sinha, and S. Gupta, ‘‘ToLeD: [96] D. Das and C. S. G. Lee, ‘‘A two-stage approach to few-shot learning
Tomato leaf disease detection using convolution neural network,’’ Pro- for image recognition,’’ IEEE Trans. Image Process., vol. 29, no. 5,
cedia Comput. Sci., vol. 167, pp. 293–301, Jan. 2020. pp. 3336–3350, Dec. 2020.
[74] A. Fuentes, S. Yoon, S. Kim, and D. Park, ‘‘A robust deep-learning- [97] L. Fei-Fei, R. Fergus, and P. Perona, ‘‘One-shot learning of object
based detector for real-time tomato plant diseases and pests recognition,’’ categories,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 4,
Sensors, vol. 17, no. 9, p. 2022, Sep. 2017. pp. 594–611, Apr. 2006.

VOLUME 9, 2021 56697

 L. Li et al.: Plant Disease Detection and Classification by Deep Learning—A Review

[98] A. Medela, A. Picon, C. L. Saratxaga, O. Belar, V. Cabezón, R. Cicchi, [114] X. Zhang, L. Han, Y. Dong, Y. Shi, W. Huang, L. Han,
R. Bilbao, and B. Glover, ‘‘Few shot learning in histopathological images: P. González-Moreno, H. Ma, H. Ye, and T. Sobeih, ‘‘A deep learning-
Reducing the need of labeled data on biological datasets,’’ in Proc. based approach for automated yellow rust disease detection from
IEEE 16th Int. Symp. Biomed. Imag. (ISBI), Venice, Italy, Apr. 2019, high-resolution hyperspectral UAV images,’’ Remote Sens., vol. 11,
pp. 1860–1864. no. 13, Jun. 2019, Art. no. 1554.
[99] F. Zhong, Z. Chen, Y. Zhang, and F. Xia, ‘‘Zero- and few-shot learning [115] J. Gui, J. Fei, Z. Wu, X. Fu, and A. Diakite, ‘‘Grading method of soybean
for diseases recognition of citrus aurantium L. Using conditional adver- mosaic disease based on hyperspectral imaging technology,’’ Inf. Process.
sarial autoencoders,’’ Comput. Electron. Agricult., vol. 179, Dec. 2020, Agricult., vol. 160, no. 7, pp. 1–6, Nov. 2020.
Art. no. 105828. [116] W. Yang, C. Yang, Z. Hao, C. Xie, and M. Li, ‘‘Diagnosis of plant
[100] S.-N. Ren, Y. Sun, H.-Y. Zhang, and L.-X. Guo, ‘‘Plant disease identifi- cold damage based on hyperspectral imaging and convolutional neural
cation for small sample based on one-shot learning,’’ Jiangsu J. Agricult. network,’’ IEEE Access, vol. 7, pp. 118239–118248, 2019.
Sci., vol. 35, no. 5, pp. 1061–1067, May 2019. [117] D. Wang, R. Vinson, M. Holmes, G. Seibel, A. Bechar, S. Nof, and Y. Tao,
[101] D. Argüeso, A. Picon, U. Irusta, A. Medela, M. G. San-Emeterio, ‘‘Early detection of tomato spotted wilt virus by hyperspectral imaging
A. Bereciartua, and A. Alvarez-Gila, ‘‘Few-shot learning approach for and outlier removal auxiliary classifier generative adversarial nets (OR-
plant disease classification using images taken in the field,’’ Comput. AC-GAN),’’ Sci. Rep., vol. 9, no. 1, pp. 1–14, Mar. 2019.
Electron. Agricult., vol. 175, Aug. 2020, Art. no. 105542.
[102] P. Baranowski, M. Jedryczka, W. Mazurek, D. Babula-Skowronska,
A. Siedliska, and J. Kaczmarek, ‘‘Hyperspectral and thermal imaging of
oilseed rape (Brassica napus) response to fungal species of the genus
alternaria,’’ PLoS ONE, vol. 10, no. 3, Mar. 2015, Art. no. e0122913.
[103] S. Graeff, J. Link, and W. Claupein, ‘‘Identification of powdery mildew
LILI LI received the B.S. degree from Datong
(Erysiphe graminis sp. Tritici) and take-all disease (Gaeumannomyces University, China, in 2012, and the M.S. degree
graminis sp. Tritici) in wheat (Triticum aestivum L.) by means of leaf from Chongqing University, China, in 2015. She
reflectance measurements,’’ Open Life Sci., vol. 1, no. 2, pp. 275–288, is currently pursuing the Ph.D. degree with the
Jun. 2006. College of Agricultural Engineering, Shanxi Agri-
[104] P. Ghamisi, J. Plaza, Y. Chen, J. Li, and A. J. Plaza, ‘‘Advanced spectral cultural University. She is currently working
classifiers for hyperspectral images: A review,’’ IEEE Geosci. Remote with Shanxi Agricultural University. Her research
Sens. Mag., vol. 5, no. 1, pp. 8–32, Mar. 2017. interest includes applications of computer vision
[105] N. Zhang, G. Yang, Y. Pan, X. Yang, L. Chen, and C. Zhao, ‘‘A review in agriculture, using deep learning methods to
of advanced technologies and development for hyperspectral-based plant study plant leaf disease detection in complex
disease detection in the past three decades,’’ Remote Sens., vol. 12, no. 19, background.
Sep. 2020, Art. no. 3188.
[106] C. Xie, Y. Shao, X. Li, and Y. He, ‘‘Detection of early blight and late
blight diseases on tomato leaves using hyperspectral imaging,’’ Sci. Rep.,
vol. 5, no. 1, pp. 1–11, Nov. 2015.
[107] L.-F. Qin, X. Zhang, and X.-Q. Zhang, ‘‘Early detection of cucumber
downy mildew in greenhouse by hyperspectral disease differential fea-
SHUJUAN ZHANG received the Ph.D. degree in
ture extraction,’’ Trans. Chin. Soc. Agricult. Mach., vol. 51, no. 11, agricultural mechanization engineering from Zhe-
pp. 212–220, Nov. 2020. jiang University. She is currently a Professor and
[108] J. Abdulridha, Y. Ampatzidis, S. C. Kakarla, and P. Roberts, ‘‘Detection the Doctoral Director with the College of Agri-
of target spot and bacterial spot diseases in tomato using UAV-based cultural Engineering, Shanxi Agricultural Univer-
and benchtop-based hyperspectral imaging techniques,’’ Precis. Agricult., sity. She has published over 60 articles in her
vol. 21, no. 5, pp. 955–978, Oct. 2020. research-related fields. Her major research inter-
[109] L. Yuan, P. Yan, W. Han, Y. Huang, B. Wang, J. Zhang, H. Zhang, ests include digital agriculture information collec-
and Z. Bao, ‘‘Detection of anthracnose in tea plants based on hyper- tion technology and equipment, technology and
spectral imaging,’’ Comput. Electron. Agricult., vol. 167, Dec. 2019, equipment for harvesting, primary processing, and
Art. no. 105039. non-destructive testing of agricultural products.
[110] A. Signoroni, M. Savardi, A. Baronio, and S. Benini, ‘‘Deep learn-
ing meets hyperspectral image analysis: A multidisciplinary review,’’
J. Imag., vol. 5, no. 5, p. 52, May 2019.
[111] X. Jin, L. Jie, S. Wang, H.-J. Qi, and S.-W. Li, ‘‘Classifying wheat
hyperspectral pixels of healthy heads and Fusarium head blight disease
using a deep neural network in the wild field,’’ Remote Sens., vol. 10,
BIN WANG received the B.S. and M.S. degrees
no. 3, Mar. 2018, Art. no. 395. from Shanxi Agricultural University, China, in,
[112] K. Nagasubramanian, S. Jones, A. K. Singh, S. Sarkar, A. Singh, and 2011and 2015, respectively, where he is currently
B. Ganapathysubramanian, ‘‘Plant disease identification using explain- pursuing the Ph.D. degree with the College of
able 3D deep learning on hyperspectral images,’’ Plant Methods, vol. 15, Agricultural Engineering. His research interests
no. 1, pp. 1–10, Aug. 2019. include hyperspectral technology and nondestruc-
[113] G. Polder, P. M. Blok, H. A. C. de Villiers, J. M. van der Wolf, and tive testing technology.
J. Kamp, ‘‘Potato virus Y detection in seed potatoes using deep learn-
ing on hyperspectral images,’’ Frontiers Plant Sci., vol. 10, Mar. 2019,
Art. no. 209.

56698 VOLUME 9, 2021