SXC JOURNAL

Volume 1, 2024

Data-driven Smart Farming to Grade and Classify Tomatoes using

CNN and FFNN for Agricultural Innovation
Binisha Joshi, Bikal Konda, and Rajan Karmacharya*
Department of Computer Science, St. Xavier’s College, Kathmandu, Nepal
Email: [email protected]*

ABSTRACT
Identifying images poses a challenge in computer vision, but the use of deep learning methods has greatly
enhanced the performance of image classification systems. In this research, Convolutional Neural Networks
(CNN) and Feed Forward Neural Networks (FFNN) have been utilized for image classification. CNN is extremely
effective in picture classification, which extracts relevant information from images using convolutional and
pooling layers to minimize the dimensionality of the derived features, while FFNN algorithm is a classic neural
network with fully linked layers. It can be used to further process the features extracted by CNN. The study
makes use of CNN and FFNN models to train a huge dataset of tomato images to categorize them based on
their type, ripeness, and damage status. CNN is found to be more effective in the case of tomato classification
as compared to FFNN algorithm in all the use cases. The accuracy for classification of an image (tomato
or not) using CNN is 95.83%, type classification using CNN is 81.52%, whereas using FFNN is 66.30%;
ripeness grading for CNN is 92.86%, whereas for FFNN it is 57.14%; and damage status grading is 92.86%
using CNN and 67.86% using FFNN. Therefore, it can be concluded that quality processing of tomatoes can
be improved using CNN.
Keywords: Image classification, tomatoes, CNN, FFNN

Introduction agriculture, n.d.).

The agricultural sector is a significant contributor The nightshade family includes fruits such as the
to Nepal’s economy – as much as 36% to the GDP tomato (Solanum lycopersicum), which is native to
– and provides employment to approximately South America. Lycopene, an antioxidant linked to
two-thirds of the population. Agriculture spans a variety of health benefits, including a lower risk of
about 30% of the country’s total land area. The cancer and heart disease, is mostly found in tomatoes.
government is actively engaged in modernizing, Tomatoes can be of a variety of hues, including yellow,
diversifying, commercializing, and promoting orange, green, and purple, in addition to its mature
the agricultural industry. The adoption of “smart color of red. Furthermore, there are numerous tomato
farming” concepts is being explored as a means to subspecies with varying shapes and flavors (Tomatoes:
enhance productivity and bring about benefits in Nutrition facts and health benefits, 2019).
the agricultural sector (Agriculture old, n.d.).
In the agricultural production system, information
and communication technology is integrated into
machinery, tools, and sensors. The integration
of technologies like the Internet of Things and
cloud computing is further advancing this trend
by incorporating additional robotics and artificial
intelligence into farming practices. The goal of
smart farming is to enhance the quality and quantity
of agricultural products while reducing the need
for human labor, ultimately striving for optimal
results (The complete guide to smart farming & Figure 1. Smart farming concept Note. (Amos, n.d.)

Published by St. Xavier’s College,

80 Kathmandu, Nepal
SXC JOURNAL
Volume 1

There are more than 45+ types and varieties of of the minor distinctions between ripe and unripe
tomatoes. But this research substantially classifies tomatoes, even the Human Visual System has
them into three introductory types of tomatoes (by difficulty distinguishing between them. In order to
shape and size) – Classic Tomatoes (Regular- Sized effectively classify tomato ripeness, digital images
Tomatoes), Cherry Tomatoes (Mini Tomatoes), and must overcome nonlinear obstacles. The research
Beefsteak Tomatoes (Large Tomatoes). cited here proposes using a Support Vector Machine
(SVM) classifier to differentiate ripe and unripe
fruits, and a Multiclass Support Vector Machine
(MSVM) classifier to detect faults. Because of its
simple testing setup and dependability, the proposed
method is ideal for incorporation into the tomato
supply chain. Implementing this approach may
result in enhanced early differentiation of tomatoes
in the value chain, which will benefit the producers
(Kumar et al., 2020).

Figure 2. Classification of tomatoes Tomato quality is critical for consistent marketing,

and customer impression is heavily influenced by
This study focuses mostly on grading tomato maturity. Recognizing ripeness phases is critical
maturity (ripe/unripe and damaged/undamaged) and for making high-quality items. Automation of
tomato type (cherry, classic, and beefsteak). To grade ripeness evaluation can improve quality product
and classify the tomatoes, no physical intervention manufacturing, benefiting the important worldwide
or human labor is required when employing the tomato sector. An automated method to classify
smart farming idea. The aim of this study is firstly different stages of tomato ripeness using color
image classification (whether the image is of tomato characteristics was proposed by the authors of
or not) and then further classification of tomato type this article. This suggested method extracts and
and grading its ripeness level and damage status. classifies features using Support Vector Machines
(SVMs), Linear Discriminant Analysis (LDA), and
Literature review
Principal Components Analysis (PCA) (El-Bendary
Tomatoes are becoming more popular, and et al., 2015).
their quality is becoming more important to the
A research carried out in 2020 had the objectives
consumers. Tomato ripeness has a big impact on
to enhance the packing and market value of
their quality. Traditional tomato categorization
cherry fruits by developing an efficient grading
is based on farmers’ experience, however it is
system that utilized an upgraded CNN algorithm.
frequently erroneous and time-consuming. Image
Image analysis was used to recognize normal and
processing, computer vision, and machine learning
irregularly shaped cherries in the study. To improve
methods, especially leveraging pre-trained CNN
the generalization potential of the CNN, a novel
models like VGG16, VGG19, and ResNet101, have
hybrid pooling strategy combining max pooling and
enabled the advancement of top-notch agriculture
average pooling was used. K-Nearest Neighbors
practices. In determining cherry tomato maturity,
(KNN), Artificial Neural Networks (ANN), Fuzzy
the VGG19 model has the highest precision
logic, and Ensemble Decision Trees (EDT) were
(94.14%). However, only three types of tomatoes
compared to older methods such Histogram of
were investigated, making high accuracy easier to
Oriented Gradients (HOG) and Local Binary
achieve. More tomato kinds will be added in the
Patterns (LBP). The enhanced CNN approach
future, and the CNN models will be retrained for
surpassed prior methods in simulation, obtaining
more precise categorization (Huynh et al., 2021).
an accuracy of 99.4%. This implies that CNN and
Because of their complicated physical features image processing techniques can efficiently replace
and substantial nonlinearity, categorizing tomatoes traditional cherry grading methods, resulting in
and diagnosing the disorders are difficult. Because improved market control and cherry fruit export

Published by St. Xavier’s College,

Maitighar, Kathmandu, Nepal 81
SXC JOURNAL
Volume 1

(Momeny et al., 2020). were utilized as inputs in the BPNN to ascertain the
maturity levels of the tomato samples (Wan et al.,
A research on Sun Bright tomatoes in 2015
2018).
investigated how ripeness affects the quality of
tomatoes for both processing and consumption. The After studying many relevant research papers and
purpose was to learn how the optical properties of articles, it was found that many algorithms, such
tomatoes, specifically their absorption and scattering as CNN, SVM, LDA, PCA, etc., were used in the
properties, altered as they ripened. 281 “Sun Bright” field of smart farming and image classification for
tomatoes at various stages of ripeness were studied classification of fruits such as banana, pear, tomato,
using hyperspectral imaging. The study’s goal was cherry, etc. Pre-trained CNN algorithms, such as
to create classification models for tomato maturity VGG16, VGG19, ResNet101, were also used in
based on optical absorption (μa) and scattering various researches. Of all the algorithms, CNN was
(μs’) spectra. The study attempted to categorize found to be a better choice for the research purpose
tomatoes into six or three maturity groups by using
Partial Least Squares-Discriminant Analysis (PLS-
Research methodology
DA) models utilizing these optical characteristics, The datasets used for the training are collected via
including solo and a combination (μa &μs’, eff) data GitHub along with real time images; for testing
(Zhu et al., 2015). purpose, real time images are used. The models
were trained on around 10,000 tomato images of
Another research in the early 2000s discussed the
different types, ripeness, and damage status.
use of color image processing to assess tomato
quality maturity. RGB and Lab* color schemes were
employed for the picture analysis. The findings were
as follows: The radical regression curve of G(36)
was judged to be 70% average correct, the pixels
count of G(36) showed the highest correlation
coefficient from tomato maturity, the level of a* also
rises in accordance with maturation while the b*
value did not change significantly, and the average
value of a* for the upper surface can be used for the Figure 3. Training images
maturity index (Gejima et al., 2004).
Artificial neural network (ANN)
Recent advances in computer vision have enabled
An artificial neuron network (also known as a neural
new agricultural applications, most notably accurate
network) is a computer model of how nerve cells
yield estimation for improved harvesting, marketing,
in the human brain work. Artificial neural networks
and logistics planning. A method for categorizing
(ANNs) use learning procedures to update their
fresh market tomatoes (Roma and Pear varieties)
answers on their own or to learn when fresh data is
based on their maturity levels (green, orange, and
presented to them. An artificial neural network has
red) was studied. Color features were combined
three or more linked layers. The top layer is made
with a backpropagation neural network (BPNN)
up of neurons that are used as input. These neurons
classification algorithm in this approach. To capture
send information to deeper layers, which send the
tomato images, a computer vision-based device
final output information to the final output layer
was developed, and image processing techniques
(Rouse, 2023). The numerical values that connect
were employed to isolate tomato targets. The area
the neurons are referred to as weight. The weights
for color feature extraction was determined as the
between neurons determine the neural network’s
largest inscribed circle on the tomato’s surface,
learning capacity. As artificial neural networks learn,
which was divided into five concentric circles for
the weights of the neurons change. Weights are
this purpose. The tomato’s maturity level was
assigned at random first. The “activation function” is
represented by the average hue values from each
used to standardize the output of neurons (Artificial
sub-region. Subsequently, these color characteristics

Published by St. Xavier’s College,

82 Kathmandu, Nepal
SXC JOURNAL
Volume 1

neural network - Applications, algorithms and whether or not the feature is there (Ratan, 2020).
examples, n.d.).
Pooling layer
Convolutional neural network
In convolutional neural networks, the feature map
A CNN is a deep learning neural network designed formed by a preceding convolutional layer and a
for processing structured arrays of data, such as non-linear activation function is often utilized as the
photos. The convolutional layer is a special type of basis for pooling. The essential phases of the pooling
layer that provides convolutional neural networks process are quite similar to those of the convolution
their strength. The design of a convolutional neural procedure. You select a filter and place it over the
network is a multi-layered feed-forward neural output feature map of the previous convolutional
network formed by progressively stacking many layer. Based on the type of pooling operation you
hidden layers on top of one another, allowing it to select, the pooling filter determines the output on the
acquire hierarchical features due to its sequential receptive field (the region of the feature map beneath
development (LeCun & Benaissa, n.d.). the filter). The most commonly used strategies are
max-pooling and average pooling (What is pooling
CNN has three main layers, namely, Convolutional
in a convolutional neural network (CNN): Pooling
layer, Pooling layer, and Fully-connected (FC) layer.
layers explained, 2021).
Convolutional layer
Fully connected layer
The convolutional layer is the central component of a
Neural networks are made up of a collection of
CNN, and it is also where majority of the processing
interdependent non-linear functions. Each function
occurs. The only components required are input data,
is carried out by a single neuron (or perceptron).
a filter, and a feature map. Assume that the input is
In fully connected layers, the neuron changes
a color image made up of a 3D pixel matrix. As a
the input vector linearly using a weights matrix.
result, the input will have three dimensions: height,
The result is then transformed nonlinearly using a
width, and depth, which correspond to the RGB
nonlinear activation function. Each input into the
values in a picture. In addition, we have a feature
input vector influences every output into the output
detector, also known as a kernel or filter, which
vector. However, not all weights have an effect on
will traverse the image’s receptive fields and assess
all outputs (Unzueta, 2022).

Figure 4. Convolutional neural network

Note. (Ratan, 2020)

Published by St. Xavier’s College,

Maitighar, Kathmandu, Nepal 83
SXC JOURNAL
Volume 1

Feed forward neural network an N-class multiclass classification issue. The

result of the softmax activation is an output vector
Feed-forward networks are artificial neural
with N elements, the entry at index i representing
networks that do not have looping nodes. Because
the likelihood that a certain input belongs to class
all input is simply transmitted forward, this type
i. Mathematically, it is expressed as (Softmax
of neural network is also known as a multi-layer
activation function: Everything you need to know,
neural network. The data received at the input nodes
n.d.).
transfer across covert layers, and the output at the
nodes comprises data flow. There are no network Result Analysis
links that can be altered to send data back from the
A total of 70 tomato and 26 non-tomato images
output node.
were used for testing purpose; a total of 92 images
The following is how a feed forward neural network for type classification, which included 27 cherry,
approximate functions: 55 classic, and 2 beefsteak tomatoes; 56 images
for damage status grading (48 undamaged and 8
● Classifiers are determined by an algorithm
damaged tomatoes); and 38 ripe and 18 unripe
employing the equation y = f*(x).
tomatoes, an accumulation of 56 images for ripeness
● Therefore, category y is given to input x. grading testing. The testing results and comparisons
are displayed and discussed below.
● The feed forward model states that y = f (x;
θ). The function’s closest approximation is
determined by this value (Understanding
feed forward neural networks in deep
learning, n.d.).
Activation function:
By including an activation function, an artificial
neural network may learn complex patterns in
data. The output of a neural network is determined
by mathematical equations known as activation
functions (Activation functions — All you need to
know! | By Sukanya Bag | Analytics vidhya, n.d.).
ReLU activation function:
If the input is positive, the rectified linear activation
function, or ReLU, which is a non-linear or piecewise
linear function, will output the input directly; if
the input is negative, it will output zero. It is the
activation function that is most frequently employed
in neural networks, particularly in convolutional
neural networks (CNNs) and multilayer perceptrons.
It is written as f(x) = max(0, x) in mathematics
(Praharsha, n.d.).
Softmax activation function
The neural network’s unprocessed outputs are
converted into a vector of probabilities—basically,
a probability distribution across the input classes—
by the softmax activation function. Think about

Published by St. Xavier’s College,

84 Kathmandu, Nepal
SXC JOURNAL
Volume 1

Figure 5. Workflow of the algorithm

Published by St. Xavier’s College,

Maitighar, Kathmandu, Nepal 85
SXC JOURNAL
Volume 1

Figure 6. Image not a tomato

Figure 7. Cherry, ripe and undamaged tomato

Published by St. Xavier’s College,

86 Kathmandu, Nepal
SXC JOURNAL
Volume 1

Figure 8. Classic, ripe and damaged tomato

Table 1. Accuracy results

Accuracy CNN FFNN

Classification of image (tomato or not) 95.83% -

Type classification 81.52% 66.30%

Grading ripeness 92.86% 57.14%

Grading damage status 92.86% 67.86%

Figure 9. Confusion matrix for image classification Figure 10. Confusion matrix for type classification
(tomato or not) using CNN using CNN

Published by St. Xavier’s College,

Maitighar, Kathmandu, Nepal 87
SXC JOURNAL
Volume 1

Figure 11. Confusion matrix for type classification Figure 12. Confusion matrix for grading damage
using FFNN status using CNN

Figure 13. Confusion matrix for grading damage

Figure 14. Confusion matrix for grading ripeness
status using FFNN
using CNN

Figure 15. Accuracy curve for image classification Figure 16. Confusion matrix for grading ripeness
(tomato or not) using CNN using FFNN

Published by St. Xavier’s College,

88 Kathmandu, Nepal
SXC JOURNAL
Volume 1

Figure 17. Accuracy curve for type classification Figure 18. Accuracy curve for type classification
using CNN using FFNN

Figure 19. Accuracy curve for grading ripeness Figure 20. Accuracy curve for grading ripeness
using CNN using FFNN

Figure 21. Accuracy curve for grading damage Figure 22. Accuracy curve for grading damage
status using CNN status using FFNN

Published by St. Xavier’s College,

Maitighar, Kathmandu, Nepal 89
SXC JOURNAL
Volume 1

Conclusion classification as well through the utilization of edge

computing.
This paper proposes deep learning neural network
algorithms, namely, Convolutional Neural Networks References
(CNN) and Feed Forward Neural Networks
Activation functions — All you need to know! | by
(FFNN), for grading and classification of tomatoes.
Sukanya Bag | Analytics Vidhya. (n.d.).
The tomato classification program is a vital tool for
Medium. Retrieved August 30, 2023, from
businesses such as food processing, distribution,
https://medium.com/analytics-vidhya/
and restaurants, allowing for effective image-based
activation-functions-all-you-need-to-know-
tomato categorization. CNN is found to be more
355a850d025e
effective in the case of tomato classification as
compared to FFNN algorithm in all the use cases. Agriculture Old. (n.d.). Federation of nepalese
The accuracies for classification of image (tomato or chambers of commerce & industry (FNCCI).
not) using CNN is 95.83%, type classification using https://www.fncci.org/agriculture-148.html
CNN is 81.52% whereas with FFNN it is 66.30%,
ripeness grading for CNN is 92.86% whereas for Amos, Z. (n.d.). Smart farming: The future of
FFNN it is 57.14%, and damage status grading is agriculture. IoT for all. Retrieved August
92.86% using CNN and 67.86% using FFNN. 30, 2023, from https://www.iotforall.com/
smart-farming-future-of-agriculture
CNN’s prowess in pattern recognition and shape
recognition makes it the best choice for complicated Artificial neural network - Applications, algorithms
image classification, such as tomato types. It requires and examples. (n.d.). TechVidvan. Retrieved
minimal preprocessing, has hierarchical feature August 30, 2023, from https://techvidvan.
learning, and is praised for its user-friendliness. com/tutorials/artificial-neural-network/
CNN employs the use of convolutional layers which The complete guide to smart farming &
extracts low level features like edges and textures agriculture. (n.d.). Smarter Technologies.
to recognize patterns and structure in the images. Retrieved August 30, 2023, from https://
These features of CNN resulted in better accuracy smartertechnologies.com/guides/the-
while grading and classifying tomatoes as compared complete-guide-to-smart-agriculture-
to FFNN. farming/
Limitations and Recommendations El-Bendary, N., Hariri, E. E., Hassanien, A. E., &
Potential enhancements in the future may encompass Badr, A. (2015, March). Using machine
precise classification of a wide range of tomato learning techniques for evaluating tomato
varieties, integration of emerging characteristics, ripeness. Expert Systems With Applications,
enlargement of datasets, improvement in adaptability 42(4). 10.1016/j.eswa.2014.09.057
to varying light conditions, and incorporation Gejima, Y., Zhang, H., & Nagata, M. (2004, March
of machine learning frameworks for increased 22). Judgment on level of maturity for
efficiency and accuracy. CNN algorithm was tested tomato quality using L*a*b* color image
with 4 and 6 hidden layers, but 5 hidden layers gave processing. IEEE/ASME lnternational
better results and accuracy. Similarly, the FFNN Conference on Advanced Intelligent
algorithm uses a total of 4 hidden layers, with 3 and Mechatronics. 10.1109/aim.2003.1225540
5 hidden layers also tested for accuracy. Different
numbers of hidden layers could also be tested for Huynh, D. P., Van Vo, M., Van Dang, N., & Truong,
better accuracy for tomato classification. Adam T. Q. (2021, March). Classifying maturity
Optimizer is used for model training whose learning of cherry tomatoes using Deep Transfer
rate is set to 0.001, but different optimizers and Learning techniques. IOP conference series,
learning rates can also be tested for better accuracy. 1109(1). 10.1088/1757-899x/1109/1/012058
Ultimately, there is a prospect to explore real-time Kumar, S., Esakkirajan, S., Bama, S. S.,

Published by St. Xavier’s College,

90 Kathmandu, Nepal
SXC JOURNAL
Volume 1

& Keerthiveena, B. (2020, July). A August 30, 2023, from https://www.turing.

microcontroller based machine vision com/kb/mathematical-formulation-of-feed-
approach for tomato grading and sorting forward-neural-network
using SVM classifier. Microprocessors
Unzueta, D. (2022, October 18). Fully connected
and Microsystems, 76. 10.1016/j.
layer vs convolutional layer: Explained.
micpro.2020.103090
Built In. Retrieved August 30, 2023, from
LeCun, Y., & Benaissa, A. (n.d.). Convolutional https://builtin.com/machine-learning/fully-
neural network definition. DeepAI. connected-layer
Retrieved August 30, 2023, from https://
Wan, P., Toudeshki, A., Tan, H., & Ehsani, R. (2018,
deepai.org/machine-learning-glossary-and-
March). A methodology for fresh tomato
terms/convolutional-neural-network
maturity detection using computer vision.
Momeny, M., Jahanbakhshi, A., Jafarnezhad, K., Computers and Electronics in Agriculture,
& Zhang, Y. (2020, August). Accurate 146. 10.1016/j.compag.2018.01.011
classification of cherry fruit using deep
What is pooling in a convolutional neural
CNN based on hybrid pooling approach.
network (CNN): Pooling layers explained.
Postharvest Biology and Technology, 166.
(2021, December 5). Programmathically.
10.1016/j.postharvbio.2020.111204
Retrieved August 30, 2023, from https://
Praharsha, V. (n.d.). ReLU (Rectified Linear programmathically.com/what-is-pooling-
Unit) activation function. OpenGenus IQ. in-a-convolutional-neural-network-cnn-
Retrieved August 30, 2023, from https:// pooling-layers-explained/
iq.opengenus.org/relu-activation/
Zhu, Q., He, C., Lu, R., Mendoza, F., & Cen, H.
Ratan, P. (2020, October 28). Convolutional neural (2015, May). Ripeness evaluation of ‘Sun
network architecture | CNN Architecture. Bright’ tomato using optical absorption
Analytics Vidhya. Retrieved August 30, and scattering properties. Postharvest
2023, from https://www.analyticsvidhya. Biology and Technology, 103. 10.1016/j.
com/blog/2020/10/what-is-the- postharvbio.2015.02.007
convolutional-neural-network-architecture/
Rouse, M. (2023, June 17). What is an artificial
neural network (ANN)? - Definition from
Techopedia. Techopedia. Retrieved August
30, 2023, from https://www.techopedia.
com/definition/5967/artificial-neural-
network-ann
Softmax activation function: Everything you need
to know. (n.d.). Pinecone. Retrieved August
30, 2023, from https://www.pinecone.io/
learn/softmax-activation/
Tomatoes: Nutrition facts and health benifits.
(2019, December 17). Allure Vanguard.
Retrieved August 30, 2023, from https://
allure.vanguardngr.com/2019/12/tomatoes-
nutrition-facts-and-health-benefits/
Understanding feed forward neural networks in
deep learning. (n.d.). Turing. Retrieved

Published by St. Xavier’s College,

Maitighar, Kathmandu, Nepal 91