Automatic Fruit Quality

Detection using
Deep Learning

Group-08

Guided by
Adithyan Manoj- MDL21EC009
Rohith M S- MDL21EC099
Ms. Rashida K
Ruben Davis Saji- MDL21EC102 Assistant Professor
Muhammad Hadhi V M- LMDL21EC134 Dept of EC
1
Contents
1. Introduction
2. Literature review
3. System overview
4. Design & implementation
5. Result
6. Conclusion
7. Bibliography
Appendix
Gantt chart
Pert chart
Work schedule
Budget
Code

2
 Introduction

Global demand for efficient and accurate fruit quality control

Critical need for automated, scalable, and reliable classification systems

Addresses challenges in the fruit industry with innovative solutions

3
 Objectives
Real-Time Detection and Classification Enhanced Accuracy through Deep Learning

Category-Based Sorting Improving Environmental Conditions

Enable Automated Sorting Using Conveyor Belt

4
Applications
Automated sorting and grading in packaging facilities.

Supply chain optimisation and post harvest management

Research and development in agriculture technology, export

quality assurance.

5
Scope:

1 Expansion to Multiple Fruit Types

2 Integration with IoT and Cloud Platforms

3 Enhanced Quality Metrics

4 Adaptive Learning Models

5 Scalable Deployment 6
Novelty
Implement Jetson Nano for efficient, real-time fruit quality analysis and
sorting.

Combines detection, classification, and automated physical sorting.

Utilizing MobileNetV2 for deep learning model.

7
 LITERATURE REVIEW
RESULTS/CONCLUSIO
SL NO YEAR TITTLE & SOURCE OBJECTIVE METHODOLOGY LIMITATIONS
NS

A Vision
A general machine The model achieved The model only considers
Transformer (ViT)
learning model equal or higher visual features, potentially
model was
using Vision accuracy for most missing other important
developed and quality factors like internal
Transformers was fruits compared to
trained for fruit defects or ripeness.
introduced for fruit dedicated models.
quality assessment It struggles with certain
It struggled with
quality assessment. using images. fruits, like bananas and
A General machine specific fruits like
The model focuses Images were divided pomegranates, indicating
Learning model for bananas and it may not fully capture
on identifying fruit into patches and
1 2023 acessing fruit quality pomegranates. the unique characteristics
quality based on encoded.
using deep image The study of all fruit types.
visual appearance. These patches were Vision Transformers
features emphasized the
It eliminates the processed through require significant
need to consider
need for fruit- transformer blocks computational resources,
factors beyond just
specific models by with multi-head which might limit the
visual appearance model's practical
providing a attention and
for accurate fruit application in resource-
multilayer
universal solution. quality assessment. constrained
perceptron layers environments.

8
 CNN achieved moderate
Traditional CNNs showed
To develop a Collect and accuracy (up to 65%) suboptimal accuracy and
transfer learning- preprocess but with high error rates high error rates in
based approach pomegranate growth in early growth stages. classifying early growth
for accurately stage images. Proposed RF model with stages.
classifying Develop models
new features reached The RF model, despite
A Novel Transfer pomegranate using CNN and
98% accuracy, high accuracy, required
Learning Approach growth stages. transfer learning-
2 2020 outperforming traditional longer training times
for Pomegranate To compare the based feature
methods. compared to other
growth detection proposed method engineering.
Cross-validation techniques.
with traditional Evaluate model
confirmed the Performance heavily relied
models, performance with
robustness of the on the newly proposed
showcasing its cross-validation and
proposed model across features, which may limit
superior compare with
generalization to different
accuracy. existing methods. different pomegranate
datasets or conditions.
growth stages.

Improve YOLOv4
Collect and
model accuracy for Achieved high accuracy in
preprocess FFB Limited to detecting only
detecting and ripeness classification using
images with data predefined ripeness
classifying oil palm YOLOv4.
augmentation. stages.
Real time Oil Palm FFB ripeness from Successfully reduced model
Train and evaluate
smartphone size and latency with Performance varies with
Grading system YOLOv4 models for
3 2023 images. TensorFlow Lite different banana varieties
using Mobile and ripeness
Develop an conversion. and conditions.
Yolo4 classification.
Android app for Implemented real-time Requires high-quality
Convert the best
real-time ripeness ripeness detection on
model to TensorFlow images for accurate
classification of oil Android with minimal
Lite and deploy it on detection.
palm FFB using the
an Android app.
latency. 9
optimized model.
 Preprocessing:
Convert images from Provides a detailed
RGB to Gray.
description of the
Apply thresholding and
segmentation.
system's
Use median filtering to performance, The research
reduce noise and capturing both acknowledges that no
smooth images. common and rare single CNN architecture
Fruit Quality To construct a fruit Feature Extraction occurrences in fruit can be declared superior
classification model using a Utilize convolutional
Recognition using grading. to others, indicating that
4 2021 convolutional neural layers for feature
Deep Learning network (CNN) to classify extraction. Allows for nuanced the performance of
Algorithm fruits into categories Apply ReLU layers for distinctions and the different CNN models
non-linear activation. detection of may vary depending on
Reduce dimensionality ambiguity in results. the specific application or
with pooling layers. design.
Results are visually
Flatten 2D data into a
represented in the
1D column for
classification. output image

1. Develop a versatile ML 1. Employed Vision 1. Lower accuracy for

1. High accuracy (up to
model for fruit quality Transformer (ViT) for visually complex fruits
100%) on fruits like apples
assessment across multiple feature extraction. (e.g., guavas, lemons).
and strawberries.
A General Machine fruit types. 2. Trained on a diverse 2. Limited to visual
2. Consistent performance
2. Evaluate model fruit dataset covering assessment, ignoring
Learning Model for across multiple fruits.
scalability and various types. non-visual quality factors
5 2023 Assessing Fruit Quality 3. Demonstrated superior
generalizability. 3. Performed (e.g., taste, texture).
Using Deep Image 3. Reduce the need for comparative analysis with
generalizability over fruit-
3. Requires extensive
Features specific models.
fruit-specific models. dedicated models. image data for training.
4. Potential for real-world
4. Demonstrate the 4. Evaluated using 4. Computationally
applications in quality
effectiveness of deep image accuracy, F1-score, and 10
intensive, impacting real-
control.
features. confusion matrices. time application.
 The researchers
Data Availability:
propose The proposed
Limited annotated
an EfficientB2 model
fruit quality
The study aims to convolutional outperforms
datasets may
improve fruit quality neural network previous methods
impact model
detection using deep (CNN) model. in terms of
performance.
learning techniques. efficiency and
This model extracts accuracy.
Generalization:
Specifically, it focuses deep features from
Enhancing The model’s
on identifying defects, a dataset of Fruit quality
Fruit Quality effectiveness across
assessing ripeness, and processed fruit assessment
Detection with various fruit types and
6 2024 sorting fruits based on images. benefits from this
Deep Learning conditions needs
visual cues. comprehensive
Models validation.
The study solution.
The objective is to addresses the
Real-World
enhance post-harvest limitations of By leveraging
Deployment: Practical
processes by previous deep learning, the
deployment may face
automating qualit approaches by system
challenges related to
assessment. considering both achieves reliable
hardware constraints
accuracy and detection and
and environmental
robustness. classification.
variations.
.

11
 The authors
• The paper proposes create their own
an approach to annotated dataset Trained with less
recognize and for training the number of dataset
determine the neural network. images.
freshness of fruits
using the YOLOv3. YOLOv3, a deep Further research could
Determination The recognition
learning explore scalability to
of
architecture, is precision for more fruit type and
Fruit Quality • The goal is to
7 2022 employed for fruit determining fruit variations.
by Image Using assess fruit quality
recognition. freshness is 90%, with
Deep Neural based on images and
a recall of 96% Real-world deployment
Network videos, focusing on six Despite using a challenges (e.g., lighting
classes: fresh and small amount of conditions, diverse fruit
spoiled bananas, training data, the shapes) are not
oranges, and apples. network achieves discussed.
good quality
metrics

12
 Researchers have
The authors discuss used computer
The article aims
different stages in vision to address
various computer Challenges include
quality inspection: quality aspects
vision methods for handling variations in
image acquisition such as
assessing the quality lighting, background,
preprocessing, appearance,
of fruits and and fruit shape.
Fruits and segmentation, feature defects, and
vegetables.
vegetables extraction, and grading.
Some methods may
8 2021 quality classification.
Specifically, it be sensitive to
evaluation using Algorithms based
focuses on methods environmental
computer vision Various algorithms and on color, texture,
related to color, conditions or require
techniques are and shape features
texture, size, shape, extensive
explored, including have been
and defect detection computational
color space proposed for fruit
resources.
transformations, and vegetable
thresholding, quality assessment
clustering, and
machine learning

13
 1. Applied DenseNet201,
a deep learning model,
1. Automate the process
for feature extraction and 1. Achieved high accuracy of 1. Model performance affected
of fruit quality
classification. 99.67% in overall by imbalanced dataset.
assessment.
Fruit Quality 2. Performed data classification. 2. Difficulty in distinguishing
2. Improve accuracy in
Assessment augmentation to handle 2. Effectively classified fruits subtle differences in fruit
fruit classification and
with Densely limited data. and their quality levels. quality.
quality grading.
9 2022 Connected 3. Employed the FruitNet 3. Demonstrated robust 3. Limited by the dataset's
3. Develop a deep
Convolutional learning model for real-
dataset, comprising performance in fine-grained scope, which includes only six
Neural 19,526 images of six quality detection tasks. types of fruits.
time applications.
Network different fruits across 4. The model was tested on 4. Real-time deployment
4. Address fine-grained
three quality grades. both raw and preprocessed challenges due to hardware
classification issues in
4. Fine-tuned the model data to evaluate consistency. constraints.
fruit quality.
using transfer learning
techniques.

14
 The researchers use
the Faster R-CNN
(Region-based
The study aims to
Convolutional Neural
develop an accurate
Network) framework
and reliable image-
for fruit detection.
based fruit The study focuses on
Ablation studies are
detection system The study achieves specific fruit types
conducted to
for orchards. the best detection (mangoes, almonds,
understand practical
Specifically, the performance for and apples) and may
deployment, training
focus is on apples and not generalize well to
data requirements,
detecting fruits such mangoes other fruits.
Deep fruit and variability capture
as mangoes, compared to The tiling approach,
detection in Data augmentation
10 2017 almonds, and previous works. while effective, may
orchards techniques
apples. An F1-score of > have limitations in
significantly improve
The system’s 0.9 is achieved for terms of
performance by
purpose includes these fruits, computational
reducing the number
supporting yield demonstrating high efficiency and
of required training
mapping, robotic accuracy. scalability.
images
harvesting, and
Transferring
efficient resource
knowledge between
utilization.
orchards doesn’t
contribute
significantly to
performance gains
15
System Overview
3.1 Block Diagram

Jetson Nano
USB Camera Image Acquisition

Pre processing Unit Deep Learning Model

Decision Making
Sorting Mechanism Actuator

16
Feasibility study-Hardware
Raspberry Pi 4 + Coral USB
Feature Jetson Nano Jetson Xavier NX
Accelerator

Quad-core ARM Cortex-A57 CPU, 6-core ARM v8.2 CPU, 384-core Quad-core ARM Cortex-A72 CPU
Processor
128-core Maxwell GPU Volta GPU, 48 Tensor Cores + Edge TPU Accelerator

4 GB or 8 GB LPDDR4
RAM 4 GB LPDDR4 8 GB or 16 GB LPDDR4x
(Raspberry Pi)

Gigabit Ethernet, USB 3.0, GPIO, Gigabit Ethernet, USB 3.0, PCIe,
Connectivity Gigabit Ethernet, USB 3.0, GPIO
CSI GPIO, CSI

Strong support from NVIDIA’s Strong support, enterprise-level Huge Raspberry Pi community,
Community & Resources
Jetson community resources limited TPU-specific support

Very cost-effective for basic AI High performance-to-cost ratio Very cost-effective (~19000 for
Cost-effectiveness
tasks (~27000) (~116850) both Pi and Coral USB) 17
 Arducam IMX477 High-Quality
Feature Raspberry Pi Camera Module 3 LG VC23GA
Camera

Resolution 12 MP (4056 x 3040 pixels) 12.3 MP (4056 x 3040 pixels) 1080p (1920 x 1080 pixels)

High-quality images with HDR and Good image quality for video
Image Quality High-quality, RAW image capture
low-light support streaming

1080p at 60 fps, lower for higher

Frame Rate 1080p at 60 fps, 720p at 120 fps 1080p at 30 fps
resolutions

MIPI CSI-2 (Compatible with Jetson MIPI CSI-2 (Compatible with Jetson USB 2.0 (Plug-and-play with
Interface
Nano) Nano) Jetson Nano)

Size 25 x 24 x 9 mm 38 x 38 mm 94 x 24 x 29 mm

Cost 2923 8268 3681

18
Feasibility study-Software
Feature MobileNetV1 MobileNetV2 MobileNetV3 small

Inverted residuals & linear

Depthwise separable convolutions
Architecture Depthwise separable convolutions bottleneck additional to
and SE blocks.
MobileNetV1

Lower for the same computational Higher accuracy for similar High accuracy while significantly
Accuracy
complexity computational complexity reducing computational cost.

Better efficiency and performance,

Efficiency Good for simpler tasks Optimized for fast inference
especially for complex tasks

Ideal for mobile and edge

Suitable for general mobile applications requiring better Real-time image classification on
Use Case
applications accuracy without much increase in mobile and embedded devices.
resources 19
Design Methodology

System Architecture.

Conveyor Belt System:

Two conveyor belts driven by DC motors

Push Back Mechanism for sorting

Image Capture Module:

USB Camera Mounted Above Conveyor for Real-Time Image Capture

20
Processing Unit:
Jetson Nano

Actuator Mechanism
Motor rotation and servo actuation for sorting

21
Implementation Steps
1. Data Collection and Preprocessing:
Data Source: High-resolution images of banana of classes - good, bad and intermediate
were obtained from
Kaggle-
1000 images per class in jpg format
Roboflow Universe
2000 images per class in jpg format
Mendely Datasets.
4000 images per class in jpg format
Python library Augmentor was used.

22
2.Model selection and training

Classification Model: MobileNetV2 Good performance and low

latency on edge devices

Framework: PyTorch
Freeze base layers

Fine-tune fully connected layers

Transfer Learning

Reduces training time

23
 Learning Rate: 5x10^(-3)

Model Parameters Optimizer: SGD

Loss Function:
CrossEntropyLoss

Batch size:8
24
25
3.Deployment and Execution in Jetson Nano:

Environmental Setup:
Installation of Pytorch, OpenCV and necessary libraries-

Jetson Nano Latest Version

Python 3.6.9 3.13

Torch 1.8 2.5

Torchvision 0.9 0.21

OpenCV 4.5.1 4.11

26
Model Deployment:

Model trained using Pytorch.

After training the model is transferred to Jetson Nano.

27
4. Conveyer Belt and Actuator Control Logic:
Good quality: Moved to second conveyer belt.

Intermediate quality: Moved to second conveyer belt.

Bad quality: Push mechanism pushes away using the sample.

28
Hardware design

29
First Conveyer Belt System-

30
Second Conveyer Belt System-

31
Tools Required-
Jetson Nano AC Motor Speed Regulator
RAM-4GB Input Voltage: 220V AC
Operating Voltage-5V Output Voltage: 0-220V AC
Minimum Current-2A Maximum Current: 10A
Arduino Uno
Input Voltage-7-12V
Output Voltage- 0-5V
Maximum Current-500mA
12V Relay
Input Voltage: 12V DC
Contact Voltage: Up to 250V AC / 30V DC
Contact Current: 10A
32
PWM Modulator LG VC23GA Webcam
Input Voltage: 5-12V DC Resolution: 2 MP (1920 x 1080)
Output Voltage: PWM Signal (0-5V) Frame Rate: 30 fps
Frequency Range: 1 Hz - 20 kHz Lens Type: Fixed Focus

IR Proximity Sensor
Operating Voltage: 3.3V - 5V DC
Detection Range: 2 cm - 30 cm
Output Voltage: 0-5V (Digital)

SSR 40DA (Solid State Relay)

Input Voltage: 3-32V DC
Output Voltage: 24-380V AC
Load Current: 40A
33
Servo Motors
Operating Voltage: 4.8-6V DC
Operating Speed: 0.1 sec/60° at 5V
Stall Torque: 2.5 kg·cm
L298 Motor Driver
Operating Voltage: 5-35V DC
Output Voltage: 0-12V DC
Maximum Current: 2A per channel

34
JETSON NANO

35
 Setting up environment in Jetson Nano:

Flashing
Power Connecting
Jetpack on
Setup Peripherals
SD Card
Initial Setup
and First
Boot

Running ML Installing all Python env

Model dependencies Setup

36
Comparison on deployment in cloud and Jetson Nano
Feature Cloud Deployment Jetson Nano Deployment

Latency Moderate Low

Throughput High Moderate

Power
Depends on cloud provider Low to Moderate
Consumption

Scalability Highly Scalable Limited Scalability

Data Privacy Depends on Provider High

Use Cases - Large-scale image or video - Real-time image or video

processing
processing
- Applications requiring integration
- IoT and edge applications 37
with other cloud services
TensorRT-
Accelerates deep learning inference using NVIDIA GPUs

Converts models from PyTorch

Uses CUDA for fast GPU execution

Supports real-time deployment on Jetson Nano

38
39
Deployment Pipeline-

40
 Result
Classified Output of MobileNetV2
Shows banana quality classes: "Good", "Intermediate", "Bad"

Using device: cpu

Model loaded successfully from banana_mobilenet_best.pt
Class names: ['bad', 'good', 'inter']
Image: OIP(1).jpg
Prediction: good (Confidence: 100.00%)
Class probabilities:
bad: 0.00%
good: 100.00%
inter: 0.00%

Prediction: good (100.00%)

41
Using device: cpu
Model loaded successfully from banana_mobilenet_best.pt
Class names: ['good', 'inter', 'bad']
Image: Deshi-0083.jpg
Prediction: inter (Confidence: 82.26%)
Class probabilities:
good: 15.04%
inter: 82.26%
bad: 2.70%

Prediction: inter (82.26%)

42
Using device: cpu
Model loaded successfully from banana_mobilenet_best.pt
Class names: ['bad', 'intermediate', 'good']
Image: rotten.jpg
Prediction: bad (Confidence: 100.00%)
Class probabilities:
bad: 100.00%
intermediate: 0.00%
good: 0.00%

Prediction: bad (100.00%)

43
Confusion Matrix and Validation Graph of MobileNetV2

44
45
46
First Conveyer Belt with Imaging Box

47
Second Conveyer Belt-

48
49
Environment Setup for Jetson Nano-

50
Model Deployed on Jetson Nano-

Result-Bad Sample image given through USB Camera

51
Conclusion
The system integrates a conveyor system, Jetson Nano, and deep
learning model( MobileNetV2) to enable real-time, high-precision
sorting of bananas by quality.
The system’s modular design—including a camera, LED lighting, and
pushing mechanism—ensures efficient, automated sorting and
adaptability for various grading needs.

52
Bibliography
A. Kumar, S. Singh, and P. Patel, "A General Machine Learning Model for Assessing Fruit Quality
Using Deep Image Features," in IEEE Transactions on Computational Agriculture, vol. 10, pp. 1123-
1134, 2023, doi: 10.1109/TCAG.2023.3105678.
A. Sharma, R. Verma, and M. Singh, "A Novel Transfer Learning Approach for Pomegranate Growth
Detection," in Journal of Agricultural Informatics, vol. 8, pp. 231 241, 2020, doi:
10.1109/JAI.2020.4025671.
Patel Vamshi Sharma, P. Gupta, and N. Mehta, "Fruit Quality Recognition Using Deep Learning
Algorithm," in International Journal of Computer Applications, vol. 175, pp. 20-27, 2021, doi:
10.5120/ijca2021912239
Kumar, R. Patel, and M. Singh, "A General Machine Learning Model for Assessing Fruit Quality
Using Deep Image Features," in Journal of Machine Learning Research, vol. 24, pp. 123-135, 2023,
doi: 10.5555/12345678.
Lee, T. Wong, and H. Zhang, "Enhancing Fruit Quality Detection with Deep Learning Models," in
IEEE Transactions on Image Processing, vol. 33, pp. 567-579, 2024, doi: 10.1109/TIP.2024.0123456

53
Chen, Y. Zhang, and S. Wang, "Determination of Fruit Quality by Image Using Deep Neural
Network," in Computers and Electronics in Agriculture, vol. 192, pp. 106689, 2022, doi:
10.1016/j.compag.2022.106689.
Patel, R. Singh, and M. Kumar, "Fruits and Vegetables Quality Evaluation Using Computer Vision,"
in Journal of Food Engineering, vol. 301, pp. 110626, 2021, doi: 10.1016/j.jfoodeng.2021.110626.
Johnson, M. Liu, and E. Davis, "Deep Fruit Detection in Orchards," in IEEE Transactions on
Geoscience and Remote Sensing, vol. 55, pp. 1250-1261, 2017, doi: 10.1109/TGRS.2016.2618362.
H. Nguyen, A. Zhou, and C. Patel, "Fruit Quality Assessment with Densely Connected
Convolutional Neural Network," in IEEE Transactions on Neural Networks and Learning Systems,
vol. 33, pp. 1124-1135, 2022, doi: 10.1109/TNNLS.2022.3154578.
J. Lee, T. Wong, and H. Zhang, "Enhancing Fruit Quality Detection with Deep Learning Models," in
IEEE Transactions on Image Processing, vol. 33, pp. 567-579, 2024, doi:
10.1109/TIP.2024.0123456.

54
Work Division
TASK ASSIGNED TO
Setting up the environment for
Ruben
real time analysis
Presentation and Demo

Define Goals Team

Setting up the sorting mechanism Muhammed Hadhi V M

Presentation & Demo Team

Optimizing the ML model into

Hardware Rohith M S and Ruben
TensorRT format

Research , Planning and

Muhammed Hadhi V M Testing and Validation Muhammed Hadhi V M
Mechanical Design

Making small prototype for

Muhammed Hadhi and Adithyan
testing

Assembly and Integration Muhammed Hadhi and Adithyan

Testing Conveyer Belt Muhammed Hadhi V M

Setting up the Jetson 55

Rohith M S
Environment
 Software

Developing Yolo Model Ruben Davis Saji

Develop CNN Model Adithyan Manoj

Integration of models and testing Ruben and Adithyan

Testing in Jetson Nano Adithyan and Rohith

56
Gantt Chart-

57
Pert Chart
System Design
Start System Requirementt Component Selection
Analysis and Procurement

CNN Training

Final Testing & Setup

Debugging Integration Jetson Nano Pipeling

58
 Overall Budget

Component Quantity Price Component Quantity Price

Jetson Nano 1 15000 Nylon Roll 1 300

DC Motor 5 1000 Ultraasonic Sensor 3 360

LED Strip 1 165 PWM Modulator 2 400

USB Camera 1 1200 Accessories(screw,bolt,th

1500
read,washer)

Bearing 20 1200
Total 21,525

OLED Display 1 200

Motor Driver 2 200

59
Milestone Completed
✔ M1: First conveyor belt plus imaging box setup completed
✔ M2: MobileNetV2 training completed (Banana quality classification)
✔ M3: Setting up environment for Jetson Nano
✔ M4: Deployment in Jetson Nano
✔ M5: Real time testing
✔ M6: Second Conveyer Belt Construction
✔ M7:Pushing Mechanism construction
✔ M8: Overall integration of system
✔ M9: Final Testing and Debugging

60
Thank You

61