Plant Disease Classification Using Convolutional Networks and Generative

Adverserial Networks

Emanuel Cortes
Stanford University
[email protected]

Abstract ages of diseased and healthy plant leaves. I will be using

a deep convolutional network, a generative adversarial net-
Crop diseases serve as a major threat to food supply. As work, and a semi supervised learning approach that utilizes
a result of the growing of smartphone technology through- a ladder network. These different approaches will be used
out the world, it has now become technical feasible to lever- to output a predicted disease type or a type of healthy plant
age image processing techniques to identity type of plant species.
disease from a simple photo. Identifying disease can lead
to quicker interventions that can be implemented to reduce 2. Related Work
the effects of crop diseases on food supply.
2.1. Plant Disease Classificaiton
Using a public dataset of 86,147 images of diseased and
healthy plants, a deep convolutional network and semi su- Before the problem of crop disease detection can be
pervised methods are trained to classify crop species and solved, the problem of identifying different species of plants
disease status of 57 different classes. need to be addressed. Fortunately, there has been much
work already completed in this problem domain. In the
research article, A Model of Plant Identification System
1. Introduction Using GLCM, Lacunarity, and Shen Features, researchers
dived into many preprocessing steps that can be done to
Despite having seen many improvements in the mass extract important features for binary-class classification.
production and accessibility of food, food security remains Images are transformed using Polar Fourier Transform to
threatened by a variety of factors such as the decline of pol- achieve translational and rotational invariance. Color fea-
linators and plant diseases. In the developing world, more tures, such as the mean, standard deviation, skewness,
than 80 percent of the agricultural production is generated and kurtosis are made on the pixel values of the leaves.
by smallholder farmers, and reports of yield loss of more Lastly, the research paper incorporated features that come
than 50% due to pests and diseases are common. Further- from gray-level co-occurrence matrix (GLCM). The GLCM
more, the majority of individuals suffering from hunger live functions characterize the texture of an image by calculat-
in smallholder farming households. Fortunately, diseases ing how often pairs of pixel with specific values and in a
can be managed by identifying the diseases as soon as it ap- specified spatial relationship occur in an image, creating a
pears on the plant. In addition, with the rise of the internet GLCM, and then extracting statistical measures from this
and mobile technology worldwide, it easy to access diagno- matrix. Although crop disease detection is moving away
sis information on a particular type of disease. As a result, from hand-engineered features, such as SIFT, HoG, and
the prevalence of smartphones with powerful cameras can SURF, there are still a variety of techniques to help deep
help to scale up any type of solution that involves crop de- learning networks extract important features, such as back-
tection feasible and practical. ground segmentation. The method that was used in the ex-
Smartphones in particular offer very novel approaches periments will be explained in the method section.
to help identify diseases because of their computing power, In the research paper, Plant Leaf and Disease Detection
high-resolution displays, and extensive built-in sets of ac- by Using HSV Features and SVM, the researchers proposed
cessories, such as advanced HD cameras. In fact it is es- using a neural network to classify whether a leaf was in-
timated that around 6 billion phones would be available fected or not. If a leaf was infected, the images were further
around 2050. processed by a neural network, where a genetic algorithm
The input to the algorithm in this paper will 2D im- was implemented to optimize the SVM loss to determine

4321
identify the type of disease. This method is quite interesting 5. Spatial Pooling: Spatial Pooling (also called subsam-
in that it breaks down the process disease identification in pling or downsampling) reduces the dimensionality of
two steps. It will be interesting to compare and contrast with each feature map but retains the most important infor-
more recent papers, where healthy leaves are treated as just mation. Spatial Pooling can be of different types: Max,
another class label. As a result, classification is just done Average, Sum etc. These networks have grown in the
in a single step. In addition, the research paper introduces a number of layers leading to architectures such as Rest-
method for optimizing the loss function using a genetic al- Net and AlexNet that have been trained on images such
gorithm, which can be compared to natural selection where as Cifar-10 and then fined tune to other problems, such
only the strong hyper parameters will survive. I need to do as plant classification.
more research in how genetic algorithms compare in other
forms of computing loss, such as Adam, RmsProp, and etc. 3. Method
Lastly, similar to the paper, A Model of Plant Identifica-
3.1. Background Segmentation
tion System Using GLCM, Lacunarity, and Shen Features,
there was a focus on identifying important features of the In the research paper, ”Using Deep Learning for Image-
image to help in the classification process. In both papers, Based Plant Disease Detection,” Mohanty and his col-
they used gray-level co-occurrence matrix (GLCM) to ex- leagues worked with three different versions of the leaf im-
tract information about the distribution of pixel values. ages from PlantVillage. One of these versions included leaf
images that were segmented to to exclude the background.
2.2. Plant Disease Classification with Convolutions According to Mohanty and his colleagues, these segmented
Other work moved towards using convolutional net- images of the whole dataset was prepared to investigate the
works to get better performances. Convolutional Networks role of the background on the learning algorithm.
are a category of Neural Networks that have proven very ef- In order to segment the background, I would be using
fective in areas such as image recognition and classification. a model that is inspired by the Boris Conforty tutorial on
ConNets have been successful in identifying faces, objects, segmenting images from PlantVillage. If you take a pic-
and traffic signs apart from powering vision in robots and ture of a uniform background under non-uniform lightning
self driving cars. conditions, you’ll end up with a gradient in all R, G and
ConvNets derive their name from the ”convolution” op- B channels (in the RGB color space), whereas only the L
erator. The primary purpose of Convolution in case of a channel will be affected (in the Lab color space). Here is an
ConvNet is to extract features from the input image. Con- example image expressed in the channels of the lab colore
volution preserves the spatial relationship between pixels by space:
learning image features using small filters.
There are many important parts of the Convolution Net-
work. These includes the following properties

1. Depth: Depth corresponds to the number of filters we

use for the conolution operation.
Because the negative values in the a channel corresponds
2. Stride: Stride is the number of pixels by which we slide to green pixels, I created a mask that is able to zero out the
our filter matrix over the input matrix. pixel locations of the original picture that is located outside
the region where the values in the lab space are negative.
3. Zero-padding: Sometimes, it is convenient to pad the In addition, I blurred the image with a guassian to ensure
input matrix with zeros around the border, so that we that the negative values that were chosen truly belong to the
can apply the filter to bordering elements of our input area that represented the leaf of the original leaf. Here is the
image matrix. following example of the image:

4. Non-Linearity: ReLU is an element wise operation

(applied per pixel) and replaces all negative pixel val-
ues in the feature map by zero. The purpose of ReLU is
3.2. Discriminator as a Classifier
to introduce non-linearity in our ConvNet, since most
of the real-world data we would want our ConvNet to The discriminator of a generative adversarial network is
learn would be non-linear (Convolution is a linear op- used to discriminate the fake images from real images. We
eration element wise matrix multiplication and addi- can further generalize the discriminator to be a classifier that
tion, so we account for non-linearity by introducing a with two classes: FAKE and REAL. If this is the case, then
non-linear function like ReLU). it may not be too hard to write a discriminator that operatons

4322
on more than two classes. As part of my attempt to design loss = None
a semi-supervised approach, I implemented a loss function N, C = logits_real.size()
that could create a discriminator that could act as a classifier fake_labels =
for different plant diseases. Variable((torch.zeros(N)+23).type(dtype).long())
Before explaining my multi-label categorical generative return softmax_loss(logits_real, true_labels) +
network, previous work has been done that influenced how softmax_loss(logits_fake, fake_labels)
I constuctured my model. In ”Semi-Supervised Learning
with Generative Adverserial Networks,” Augustus Odena
proposed that the descirminator network D can be imple- The discriminator loss function will contain the sum of
mented with a softmax output layer with one unit for each of two softmax functions. One of softmax will try to reduce
the classes [REAL, FAKE]. Once this modification is made, the negative log likilhood with respect to the given labels
its simple to see that D could have N+1 output units corre- of the real data. The other softmax function will reduce
sponding to CLASS-1, CLASS-2, . . . CLASS-N, FAKE]. the negative log likilhood with respect the fake labels of the
Therefore the discriminator can act as a classifier. In both fake data. A loss function that reduces the sum of these soft-
my research paper and in Odena’s work, this type of dis- max functions would create a discriminator that discrimi-
crimnatory is referred to as D/C. nate between real and fake data and if given an real image,
Odena further describes that training this type of GAN the discriminator can produce a label or identify a particular
is quite similar to training a normal GAN. D/C is trained type of disease for that image.
to minimize the negative log likelihood with respect to the
given labels and G is trained to maximize it. Therefore, the 4. Dataset and Features
following algorithm is proposed to train my algorithm
There is a total of 86,147 images of diseased and healthy
Algorithm 1 SGAN Training Algorithm plants. These images span 25 different species of plants.
1: Input:I : number of total iterations
Each set of images, such as the training, validation, and
2: for i > 1 to i == 10 do
testing, span 25 different species of plants. The training set
3: Draw m noise samples includes 55,135 images of disease and healthy plant leaves.
4: Draw m examples from the real data that also span 25 different species of plants. The valida-
5: Perform gradient descent to minimize the D/C loss tion set includes 13,783 images of disease and healthy plant
6: function with respect to real and fake lables leaves. The testing set includes 17, 229 images of disease
7: Draw m noise samples and healthy plant leaves.
8: Perform gradient descent on the parameters of G All images were segmented to remove the background.
w.r.t. In addition, all images were resized to have a width of 64
9: the NLL of D/Cs outputs on the minibatch of size pixels and a height of 64 pixels. Then this was repeated for
m. images that were segmented.

In addition, here is the following code for my loss func- 5. Experiments/Results

tion for the descriminator: The experiment that performed well on the unsupervised
def discriminator_loss_for_mcgan data was rsnet. It was able to score above 80% in the
(logits_real, logits_fake, training phrase under 5 epochs with the learning rate
true_labels, of 1e-5. This was our baseline and was expected to do
softmax_loss): well. Most of the activiations were completey dark after
finetuning the the data. The following images are part of an
Inputs: activation layer that shows that resnet is learning to identify
- logits_real: diseases in few of the activations.
PyTorch Variable of shape (N, C)
giving scores for the real data.
- logits_fake:
PyTorch Variable of shape (N, C)
giving scores for the fake data.
- true_labels:
PyTorch Variable of Shape (N,) There were two experiments that were done for imple-
giving labels for real data. menting DCGAN. In one case, I used the following archi-
tecture for the descriminator:

4323
Unflatten(BATCH_SIZE, NUM_CHANNELS, I
MAGE_HEIGHT, IMAGE_WIDTH),
nn.Conv2d(NUM_CHANNELS, 32,
kernel_size=5, stride=1),
nn.LeakyReLU(0.01, inplace=True),
nn.MaxPool2d(kernel_size=2, stride=2),
nn.Conv2d(32, 64, kernel_size=5, stride=1),
nn.LeakyReLU(0.01, inplace=True),
nn.MaxPool2d(kernel_size=2, stride=2),
Flatten(),
nn.Linear(10816, 4*4*64),
nn.LeakyReLU(0.01, inplace=True),
nn.Linear(4*4*64, NUM_CLASSES)

, where batch size is 64, num channels is 3, image height

is 64, image width is 64, and num of classes is 37. This
model lachieved an accuracy of 78% for testing data. How- In order to improve the quality of images produced by
ever, it performed poorly on the unstructured images, which generative model and increase the accuracy of the unstruc-
only performed an acurracy of 6%. tured data, I increased the complexity of both the descrim-
I used the following architecture for the generator: inator and generator. Here is the model for the descrimina-
tor:
nn.Linear(noise_dim, 1024),
nn.BatchNorm1d(1024), Unflatten(BATCH_SIZE, NUM_CHANNELS,
nn.Linear(1024, 16*16*128), IMAGE_HEIGHT, IMAGE_WIDTH),
nn.BatchNorm1d(16*16*128), nn.Conv2d(NUM_CHANNELS,
Unflatten(BATCH_SIZE, 128, 16, 16), 128, kernel_size=5, stride=1),
torch.nn.ConvTranspose2d(128, 64, nn.LeakyReLU(0.01, inplace=True),
kernel_size=4, stride=2, padding=1), nn.MaxPool2d(kernel_size=2, stride=2),
nn.ReLU(inplace=True), nn.Conv2d(128, 64, kernel_size=5, stride=1),
nn.BatchNorm2d(64), nn.LeakyReLU(0.01, inplace=True),
torch.nn.ConvTranspose2d(64, 3, nn.MaxPool2d(kernel_size=2, stride=2),
kernel_size=4, stride=2, padding=1), nn.Conv2d(64, 32, kernel_size=5, stride=1),
nn.Tanh(), nn.LeakyReLU(0.01, inplace=True),
Flatten() nn.MaxPool2d(kernel_size=2, stride=2),
Flatten(),
In addition here is the first image produced the early it- nn.Linear(32*4*4, 4*4*64),
erations during training: nn.LeakyReLU(0.01, inplace=True),
nn.Linear(4*4*64, NUM_CLASSES)
Here is the model for the descriminator:
nn.Linear(noise_dim, 1024),
nn.BatchNorm1d(1024),
nn.Linear(1024, 32*32*128),
nn.BatchNorm1d(32*32*128),
Unflatten(BATCH_SIZE, 128, 32, 32),
torch.nn.ConvTranspose2d(128, 64,
kernel_size=4, stride=2, padding=1),
nn.ReLU(inplace=True),
torch.nn.ConvTranspose2d(64, 32,
kernel_size=4, stride=2, padding=1),
nn.ReLU(inplace=True),
torch.nn.ConvTranspose2d(32, 16,
And this the last image produced during training. kernel_size=1, stride=1, padding=0),

4324
nn.Conv2d(16, 3, kernel_size=2, stride=1),
nn.LeakyReLU(0.01, inplace=True),
nn.MaxPool2d(kernel_size=1, stride=2),
nn.Tanh(),
Flatten()

Here is the improved image at the beginning of training:

6. Conclusion
Using Gans may hold promise to classify diseases in
classifying plant disease. Unfortunately segmenting based
on background did not improve accuracy. One possible idea
for future work is to implement segmentation with Mask
RCNN.
1.) Strange RN, Scott PR (2005) Plant disease: a threat
to global food security. Phytopathology 43
2.) Dirzo, Rodolfo; Hillary S. Young; Mauro Galetti;
Gerardo Ceballos; Nick J. B. Isaac; Ben Collen (2014).”De-
faunation in the Anthropocene”(PDF).Science.345(6195):
401406. 3.) UNEP (2013) Smallholders, food security, and
the environment.
Here is the final improved image at the beginning of
4.) Harvey CA et al. (2014) Extreme vulnerability of
training:
smallholder farmers to agricultural risks andclimate change
in madagascar. Philosophical Transactions of the Royal So-
ciety of London B:Biological Sciences 369(1639). 5.) Mo-
hanty, S. P., Hughes, D. P., Salath, M. (2016). Using Deep
Learning for Image-Based Plant Disease Detection. Fron-
tiers in Plant Science, 7. doi:10.3389/fpls.2016.01419 6.)
Hanson, A.J., Joy, A., Francis, J. Francis, J. (2017). Plant
Leaf Disease Detection using Deep Learning and Convo-
lutional Neural Network. IJESC, Volume 7 Issue No.3 .
7.) Sladojevic, S, Arsenovic, M., Anderla, A., Culibrk,
Dubravko, Sefanovic, D. (2016). Deep Neural Networks
Based Recognition of Plant Disease by Leaf Image Classi-
fication. 8.) Naik, M.R., Sivappagari, C. (2016). Plant Leaf
and Disease Detection by Using HSV Features and SVM.
IJESC, Volume 6 Issue No.12. 9.) Kadir, A. (2014). A
Model of Plant Identification System Using GLCM, Lacu-
narity and Shen Features. Research Journal of Pharmaceu-
tical, Biological, and Chemical Sciences Vol.5(2)

In addition, the classifier performed worst in classifying

the test data (69%) However, the performance in the un-
structured data was improved to 12%

Here is the confusion matrix for the test data:

4325