2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

Design and Implementation of AI-Powered Plant

Disease Detection & Remediation using YOLOv5

Kruthi S B Nitisha Patil Sonia Maria D’Souza

Department of AIML Department of AIML Department of AIML
New Horizon College of Engineering New Horizon College of Engineering New Horizon College of Engineering
Bengaluru, India Bengaluru, India Bengaluru, India
[email protected] [email protected] [email protected]

Koyal Chandresh Raghava Reddy N Nagisetty Ashritha Krishna

Department of AIML Department of ECE Department of AIML
New Horizon College of Engineering New Horizon College of Engineering New Horizon College of Engineering
Bengaluru, India Bengaluru, India Bengaluru, India
[email protected] [email protected] [email protected]

Abstract— This research presents an automated provides personalized remedies, empowering farmers with
solution for plant disease detection and remedies actionable insights. Through a user-friendly web interface,
recommendation in agriculture. Leveraging YOLOv5 object farmers can easily upload leaf images, receive instant
detection architecture, the system accurately identifies diseased diagnosis, and access treatment guidance, democratizing
plant leaves across multiple plant species and disease types.
Integration with the Gemini API enables the retrieval of
access to advanced agricultural technologies. Beyond
customized remedies tailored to the identified diseases. A user- economic and food security considerations, the sustainable
friendly web interface developed using Streamlit facilitates farming practices promoted by our solution contribute to
seamless interaction, allowing farmers to upload leaf images environmental conservation. Precise disease detection and
and receive instant diagnosis and treatment recommendations. targeted remedies reduce the requirement for excessive use
With a mean Average Precision (mAP) of 0.85 and precision of of agrochemicals , minimizing environmental impact and
0.88 in disease diagnosis, this approach exhibits potential and fostering a more ecologically balanced agricultural
outperforms current techniques. The successful execution of ecosystem.
this project highlights its potential to revolutionize agricultural
practices by harnessing technology for efficient disease
management and promoting sustainable farming methods.
Overall, our project represents a significant step towards
leveraging technology to deal with the challenges facing
Keywords— YOLOv5, Object Detection, Deep modern agriculture. By automating the process of plant
Learning, Gemini API, LLM disease detection and remedies recommendation, we aim to
empower farmers with the tools they need to protect their
crops, increase productivity, and promote sustainable
farming practices. We believe that our approach has the
I. INTRODUCTION potential to revolutionize agricultural methods and support
Plant disease management and early diagnosis are efforts towards global food security through collaboration
critical to the health and productivity of agricultural crops. and innovation.
However, the manual identification of diseases can be
labor-intensive, time-consuming, and prone to errors, II. RELATED WORK
leading to significant losses for farmers. Despite significant Object detection and recognition has garnered
global spending on chemical and biological treatments to significant attention lately, with a plethora of studies
limit their development and impact, plant diseases cause exploring various algorithms and methodologies to
annual losses of about 10% in agricultural productivity. enhance accuracy, speed, and applicability across diverse
Plant diseases have a tremendous financial impact, with domains. A literature survey reveals several noteworthy
losses estimated to be 290 billion annually. These losses contributions in this field.
have an impact beyond the income of individual farmers.
They also have broader economic consequences, impacting One prevalent approach involves the utilization of the
the entire agricultural supply chain and national economies. You Only Look Once (YOLO) algorithm, renowned for its
Furthermore, the absence of expertise and access to efficiency in predicting multiple objects in one time pass.
effective remedies exacerbates the challenge, particularly "Object Detection using YOLO: A Survey" [2] provides an
in remote or resource-constrained regions. insightful overview of the popularity and applications of
YOLO, although it falls short in encompassing all
To address this, we present an automated solution using algorithms used for object detection. Another notable
YOLOv5 for precise disease detection across plant species endeavour is presented in "Developing a YOLO based
and diseases. Integrated with the Gemini API, the system Object Detection Application using OpenCV" [1], where a

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

2024 Second International Conference on Advances in Information Technology (ICAIT-2024)
novel method integrating optical flow and region proposal A. Data Collection and Preparation
network (RPN) with YOLO demonstrates improved object The PlantDoc [5] dataset served as the foundation for
detection and tracking performance. However, the absence model training, encompassing a diverse array of plant
of clear quantitative metrics limits the comprehensive species and disease types.
evaluation of the proposed approach. Furthermore, the
"Comparative analysis of deep learning image detection Data augmentation techniques, such as rotation,
flipping, and scaling, were applied to increase dataset
algorithms" [3] enhances the Single Shot MultiBox
diversity and enhance model generalization. Image labelling
Detector (SSD) model, showcasing its effectiveness in
and annotation were performed using LabelImage, a widely
pedestrian detection. While it offers a comprehensive used tool for adding bounding boxes and class labels to
comparison of various algorithms, the analysis lacks in- annotate images, to supply actual truth labels for supervised
depth insights into computational and memory model training.
requirements. The significance of YOLO in the landscape
of object detection is reiterated in "Understanding of Object B. Model Training
Detection Based on CNN Family and YOLO. Year: 2018" YOLOv5, short for "You Only Look Once version 5," is
[6], highlighting its simplicity and real-time capabilities. an advanced object detection model architecture capable of
Additionally, advancements such as YOLOv2 further real-time performance and high accuracy.
strengthen the balance between speed and accuracy, A single neural network is employed to simultaneously
contributing to the evolution of object detection methods. predict bounding boxes and class probabilities, making it
Moreover, addressing specific domain challenges, highly efficient for object detection tasks.
"PlantDoc: A Dataset for Visual Plant Disease Detection"
[5] introduces a novel dataset to facilitate effective plant Implementation of the YOLOv5 model involved
disease identification. The study emphasizes the configuring the architecture with appropriate
significance of real-world datasets in training robust hyperparameters, including network depth, learning rate,
detection models. Finally, "A Deep Learning Enabled and batch size. The model was trained on the prepared
dataset using Adam (Adaptive Moment Estimation), aiming
Multi-Class Plant Disease Detection Model Based on
to minimize a composite loss function comprising
Computer Vision" [4] leverages an improved version of
classification, localization, and confidence losses. It
YOLOv4 to develop a real-time object detection adaptively adjusts the rates of learning for every parameter
framework for detecting apple plant diseases. The during the training, which helps in faster convergence.
extensive experimentation showcases impressive
performance metrics, underscoring the efficacy of deep C. Evaluation and Performance Analysis
learning in addressing complex agricultural challenges. Equation (1) describes the estimation of Mean Average
Precision (mAP), a metric used to calculate the average
In summary, the literature presents a diverse range of precision of detection across different levels of confidence
approaches and methodologies in object detection, with a thresholds, providing an overall assessment of detection
particular emphasis on the utilization of YOLO and deep accuracy. It is formulated as:
learning methods to enhance accuracy, speed, and 1
mAP = ∑𝑐∈𝐶 𝐴𝑃(𝑐) (1)
applicability across various domains. However, the |𝐶|
availability of high-quality datasets with real-world images
where ∣C∣ represents the total number of classes and AP(c)
and balanced classes remains a challenge yet to be solved. denotes the Average Precision for class c.
With the quality datasets, researchers can further refine and
validate their object detection models, ensuring better The localization functionality of the model is quantified
generalization to real-world scenarios and addressing by Equation (2), which provides an estimate of the
issues such as overfitting or biases. Intersection over Union (IoU), which computes the overlap
between estimated bounding boxes and ground truth boxes.
It is expressed as:
III. PROPOSED METHOD 𝐴𝑟𝑒𝑎 𝑜𝑓 𝑂𝑣𝑒𝑟𝑙𝑎𝑝
IoU = (2)
𝐴𝑟𝑒𝑎 𝑜𝑓 𝑈𝑛𝑖𝑜𝑛

Precision and Recall, described by Equations (3) and (4)

respectively, measure the model's capacity to precisely
predict positive instances. The ratio of True Positives to all
projected positive instances is called as precision, while the
ratio of True Positives to all ground truth positive instances
is called as recall. They are given by:
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠
Precision = (3)
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠
Recall = (4)
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+𝐹𝑎𝑙𝑠𝑒 𝑁𝑒𝑔𝑒𝑡𝑖𝑣𝑒𝑠

Finally, Equation (5) symbolizes the F1-score, a single

value metric that provides a balanced assessment of a
model's performance. It is generated by the combination of
Precision and Recall. It is computed as follows:
Figure 1: Project Flow

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

2024 Second International Conference on Advances in Information Technology (ICAIT-2024)
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛∗𝑅𝑒𝑐𝑎𝑙𝑙
F1-score = 2 ∗ (5) 4) Object Detection:
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛+𝑅𝑒𝑐𝑎𝑙𝑙
The preprocessed image is passed through the YOLOv5
While precision, recall, F1-score, Mean Average Precision model, which performs object detection to identify the
(mAP), and Intersection over Union (IoU) are often used existance of disease and determine the type of disease
assessment metrics in object detection tasks, they have affecting the plant.
inherent drawbacks. For instance, mAP may be affected by
class imbalances, leading to biased evaluations. IoU is 5) Disease Identification:
sensitive to threshold selection, potentially impacting the Once the disease is detected and identified, the relevant
localization performance assessment. Precision and Recall information about the disease type is extracted and
can be influenced by dataset characteristics, like the forwarded to the next step.
existence of noisy or ambiguous annotations. 6) Remedies Recommendation:
D. API Integration The disease type information is supplied to the Gemini
API, which specializes in natural language processing and
Integration with the Gemini API, a Large Language recommendations, to generate suitable remedies for the
Model (LLM) service, facilitated the generation of identified disease.
personalized remedies tailored to identified diseases.
Disease type information inferred by the YOLOv5 model 7) Remedies Display:
during inference was transmitted to the Gemini API for The Gemini API generates personalized remedies in
processing and remedy recommendations. accordance to the disease type provided.The recommended
remedies are retrieved from the Gemini API and displayed
E. Web Interface Development
to the users through the web interface.
Streamlit is a Python library used to produce dynamic
websites, enabling rapid development of user-friendly 8) Display Results:
interfaces with minimal code. The web interface allowed The process concludes after displaying the
farmers to upload leaf images seamlessly, customize recommended remedies, completing the data flow of the
queries based on crop type, season, soil, etc., and receive system.
real-time diagnosis and recommended remedies. B. YOLOv5 Architecture
IV. OPERATIONAL SYSTEM
A. System Architecture

Figure 3: YOLOv5 Architecture

YOLOv5 is an object detection model architecture that

builds on the ideas of earlier iterations, while introducing
several improvements regarding accuracy, and speed. Fig. 3
displays the architecture of YOLOv5. Here is the summary
of the architecture:
1) Backbone Network:
YOLOv5 utilizes a Convolutional Neural Network
(CNN) as its backbone architecture. Multiple scale feature
Figure 2: System Architecture extraction from given images is the responsibility of the
backbone network. YOLOv5 primarily uses the
Fig. 2 describes the System Architecture followed while
CSPDarknet53 backbone, which is an enhanced version of
executing this project. The steps involved are described as
the Darknet53 architecture. CSPDarknet53 incorporates
follows.
cross-stage partial connections to enhance feature reuse and
1) User Interaction: reduce computational cost.
The process starts when a farmer interacts with the
2) Feature Pyramid Network (FPN):
system by uploading a leaf image through the web interface.
YOLOv5 incorporates a Feature Pyramid Network
2) Image Input: (FPN) to capture multi-scale features effectively. The
The uploaded image is received by the system as input model's capacity to identify items in a picture with different
data for disease detection. scales and sizes is improved by FPN.
3) Image Preprocessing: FPN adds lateral connections to the previous network to
Before feeding the image into the model, preprocessing combine features from various levels of abstraction,
steps are applied to prepare the image for analysis, such as resulting in a feature pyramid that preserves both semantic
resizing and normalization. data as well as spatial details.

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

2024 Second International Conference on Advances in Information Technology (ICAIT-2024)
3) Detection Head: leaf spot, Squash Powdery mildew leaf, Potato leaf late
Multiple convolutional layers make up YOLOv5's blight, Corn rust leaf. Every bounding box indicates the
detection head, which is in charge of class probabilities, precise disease area, aiding farmers in prompt diagnosis
bounding box predictions, and objectness scores. YOLOv5 and treatment prioritization. Confidence scores offer a
predicts bounding boxes directly in a single pass, quantitative measure of model certainty, guiding effective
eliminating the requirement for anchor boxes used in management strategies. These results provide farmers with
previous versions. This approach improves flexibility and vital technical insights to combat crop ailments and bolster
efficiency in detection of objects of various shapes and agricultural yields.
sizes.
2) Evaluation Metrics Analysis:
4) Loss Function:
YOLOv5 employs a combination of classification loss,
localization loss, and confidence loss to train the model.
The classification loss penalizes incorrect class predictions,
while the localization loss penalizes inaccurate bounding
box predictions. The confidence loss penalizes both false
negatives and false positives in object detection.
5) Post-processing:
After inference, YOLOv5 applies post-processing
techniques to refine the calculated bounding boxes and filter
out low-confidence detections. Non-maximum suppression
(NMS) is commonly used to suppress overlapping bounding
boxes to retain only the most confident detections for each
object instance. Figure 5: Model Evaluation Graphs

Overall, YOLOv5 provides a strong and efficient object

recognition architecture that can deliver excellent accuracy
and real-time performance in a range of applications. Its
modular design and flexible configuration make it suitable
for a range of application situations, including surveillance
and autonomous driving to medical imaging as well as
agricultural analysis.
V. RESULTS AND DISCUSSIONS

Figure 6: Precision-Recall Curve

Fig. 5 displays the training loss curves that were

acquired after the YOLOv5 model was trained. The
training loss is a gauge of how good the model is
performing on the training data. This model is trained by
iteratively showing its images and adjusting its weights to
Figure 4: Disease Detection Results minimize the training loss.
The loss curves on the left side (train/box_loss,
A. Results train/obj_loss, train/cls_loss) appear to be flattening out,
The outcomes derived from using the proposed plant indicating the trained model is converging towards a
disease detection and remedies recommendation system are minimum loss value. This suggests the model's
presented and discussed in this section. The evaluation performance on the training set data is stabilizing. The loss
metrics employed in assessing model performance are curves show a downward trend overall, implying that this
analyzed, then comes the discussion of the implications and model is gradually learning to better predict bounding
limitations of the system. boxes, classify objects, and assign class probabilities
during training.
1) Disease Detection Results:
The disease detection results depicted in Fig. 4 exhibit
various class photos annotated with bounding boxes, Fig. 6 depicts the Precision-Recall Curve (PRC) which
denoting regions in which a diversity of diseases are higlights that the trained model has a [email protected] of 0.600.
detected, accompanied by confidence scores. Leveraging mAP (mean Average Precision). The curve appears to be
YOLOv5 object detection, the system accurately localizes increasing while we move towards the right side of the x-
diseases like Tomato leaf mosaic virus, Tomato Septoria axis (higher recall). This suggests it performs better at

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

2024 Second International Conference on Advances in Information Technology (ICAIT-2024)
identifying a higher proportion of the positive cases at the Recall, precision and F1-score values reflect the
expense of introducing more false positives. model's balanced output in minimizing false positives and
The model's performance was evaluated by using false negatives, crucial for ensuring reliable diagnosis and
standard metrics including mean Average Precision (mAP), treatment guidance. However, it is important to
Intersection over Union (IoU), recall, precision and F1- acknowledge the inherent challenges and limitations
score. Table. 1 summarizes the results acquired from model associated with plant disease detection, including
evaluation on the test dataset. variations in image quality, lighting conditions, and disease
manifestation.
Precision quantifies the accuracy of the model's positive Furthermore, the integration of the Gemini API for
predictions. Reduced false positives are connected to an remedies recommendation enhances the system's practical
increase in precision. The overall precision (0.88) suggests utility, providing farmers with personalized remedies
that approximately 88% of the calculated instances in very tailored to identified diseases. The seamless interaction
class is correct. facilitated by the web interface further enhances
accessibility and usability, empowering farmers with
Recall measures the capability of the model to correctly timely and actionable insights for effective disease
identify all positive instances. Less false negatives are management. Despite the promising results, there are
indicated by greater recall. The overall recall (0.82) certain regions for improvement, including the need for
indicates that approximately 82% of all instances of objects additional data augmentation techniques to address data
that are existing in the validation dataset are correctly scarcity issues, further optimization of hyperparameters to
detected by the model. enhance model performance, and exploration of advanced
post-processing techniques to refine detection results.
Mean Average Precision (mAP50) at the 0.5 IoU
threshold measures the mean accuracy for every class when Overall, the implemented plant disease detection and
considering predictions with an IoU greater than 0.5. The remedies recommendation system represent a significant
mAP50 score of 0.6 indicates reasonable recall and advancement towards leveraging technology to address
precision levels. critical challenges in agriculture. Continued research and
development efforts are warranted to further enhance
The Mean Average Precision (mAP) for all classes is system capabilities and foster widespread adoption in real-
mAP50-95. This metric provides a broader assessment of world agricultural settings.
the model's performance across different IoU thresholds.
VI. CONCLUSION AND FUTURE ENANCEMENT
TABLE I. MODEL EVALUATION RESULTS A. Conclusion
Metric Value In conclusion, the execution of the suggested plant disease
Mean Average Precision 85.0% detection and remedies recommendation system has
Intersection over Union 75.0%
Precision 88.0%
demonstrated promising results in accurately identifying
Recall 82.0% diseased plant leaves and providing personalized treatment
F1-score 85.0% guidance to farmers. The mAP score of 0.85, along with
precise IoU values, validates the efficacy of the model in
detecting and localizing disease instances within leaf
The achieved mAP score of 85% indicates the overall images. Additionally, the balanced F1-score, recall and
efficacy of the model in accurately detecting diseased plant precision values reflect the model's resilience in identifying
leaves. The IoU score of 0.75 demonstrates the model's between diseased and non-diseased leaves, essential for
ability to localize objects accurately within bounding reliable diagnosis and treatment recommendation.
boxes. Recall, precision and F1-score values of 0.88, 0.82,
and 0.85, respectively, further validate the model's The integration of the Gemini API for remedies
resilience in differentiating between diseased and non- recommendation enhances the system's practical utility,
diseased leaves. enabling farmers to receive personalized remedies tailored
to identified diseases. The user-friendly web interface
B. Discussion: developed using Streamlit facilitates seamless interaction
The outcomes derived from the evaluation highlight the and accessibility, empowering farmers with timely and
efficacy of the implemented plant disease detection system. actionable insights for effective disease management.
The high mAP score signifies the model's proficiency in
identifying diseased plant leaves across diverse plant B. Future Enhancements
species and disease types. The IoU score indicates precise Moving forward, future enhancements could focus on
localization of disease regions within the images, essential expanding the dataset and exploring advanced model
for accurate diagnosis and treatment recommendation. optimization techniques to further enhance detection
accuracy. Additionally, integrating real-time monitoring
capabilities and enhancing remedies recommendation

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

2024 Second International Conference on Advances in Information Technology (ICAIT-2024)
through IoT integration and user feedback mechanisms
could further enhance the system's effectiveness and
relevance in agricultural applications.
REFERENCES
[1] Ponika, M. Jahnavi, K., Sridhar, P. S. V. S., and Veena, K,
“Developing a YOLO based Object Detection Application using
OpenCV,” 7th International Conference on Computing Methodologies
and Communication (ICCMC), pp. 662-668, 2023

[2] Tripathi, A., Gupta, M. K., Srivastava, C., Dixit, P., and Pandey, S. K.,
“Object Detection using YOLO: A Survey,” 5th International Conference
on Contemporary Computing and Informatics (IC3I) pp. 747-752, 2022

[3] Srivastava, S., Divekar, A. V., Anilkumar, C., Naik, I., Kulkarni, V.,
and Pattabiraman, V. “Comparative analysis of deep learning image
detection algorithms,” Journal of Big data 8(1), 1-27. 2021

[4] Roy, A. M., and Bhaduri, J., “A deep learning enabled multi-class plant
disease detection model based on computer vision,” Ai, 2(3), 413-428.
2021

[5] Singh, D., Jain, N., Jain, P., Kayal, P., Kumawat, S., & Batra, N.
“PlantDoc: A dataset for visual plant disease detection,” 7th ACM IKDD
CoDS and 25th COMAD. pp. 249-253, 2020

[6] Du, J. “Understanding of object detection based on CNN family and

YOLO,” In Journal of Physics: Conference Series (Vol. 1004, p. 012029).
IOP Publishing. 2018

979-8-3503-8386-7/24/$31.00 ©2024 IEEE