ISSN 2347 - 3983

Volume 8. No. 5, May 2020

Aaron Don M. Africa et al., International Journal of Emerging Trends in Engineering Research, 8(5), May 2020, 1845– 1849
International Journal of Emerging Trends in Engineering Research
Available Online at http://www.warse.org/IJETER/static/pdf/file/ijeter60852020.pdf
https://doi.org/10.30534/ijeter/2020/60852020

Ripe Fruit Detection and Classification using

Machine Learning
Aaron Don M. Africa, Anna Rovia V. Tabalan, Mharela Angela A. Tan
De La Salle University, Manila
2401 Taft Ave., Malate, Manila 1004,
Philippines, [email protected]

ABSTRACT are required to have consistent recognition and analysis which

may be hard or tedious when it becomes repetitive.
One of the most important sectors in any country is the
agricultural sector. However, in some countries, farmers and Nowadays, several research studies that focus on the
fishermen have limited technology compared to other agriculture industry are providing various methods for the
developed countries. One of the effects of limited technology detection and classification of fruits and vegetables. Most of
is the low quality of crops, fruits, and vegetables. This is these methods are based on computer vision both for the
because the quality of the products is only assessed depending processing and harvesting analysis of the products. Computer
on external factors like appearance, shape, color, and texture, Vision refers to the analysis of visual information with the use
which can be prone to human error. Determining the quality of different kinds of images. Developments in image
and ripeness level of fruit requires consistency, which can be processing techniques have contributed on the improvement
hard and tedious for humans when it becomes repetitive work. of the computer vision procedures as well. Despite the
This paper aims to present various methods and approaches advancements of technology in the agricultural industry, the
on how ripe fruit detection and classification can be made classification of fruits and vegetables continues to have
easier and more convenient using machine learning and complex challenges such as the wide variety, irregularity, and
machine vision algorithms. Furthermore, this study presents inconsistency in shape, color, and texture [2]. The fruit
systems that can be utilized in pre and post-harvest analysis. classification process is shown in Figure 1.
This paper aims to provide solutions using computer
applications to help farmers have lesser manual labor yet
more accurate data and results in the evaluation of crops.

Key words :Machine Learning, Classification Engineering,

Convolution, Neural Networks. Figure 1: Fruit Classification Process Distribution

1. INTRODUCTION Recent studies suggest that the detection of ethylene levels in

fruits can help in classifying whether the fruits are ripe or not.
The attempts of imitating the abilities of the human brain are Ethylene is a type of plant hormone discovered during the
the bases of several applications such as object detection, face early years of the 20th century. It was observed that gases
recognition, autonomous vehicles, image processing, and composed of ethylene had effects on the growth of plants.
robotics. Moreover, numerous studies and experiments have Furthermore, ethylene is also considered in the majority of the
also been attempting to provide convenience in the quality plant development fields such as fruit ripening, root
assessment of the food industry using computer applications. elongation, seed germination, and flower development. The
ethylene level can determine whether a fruit is ripe and ready
Classification of various kinds of fruits and even vegetables is to be harvested or not [3].
not a simple task due to several similarities in shape, size, and
color [1]. Typically, fruits, vegetables, and crops, before Most of the existing agricultural technologies are utilizing
harvested and released to the market are examined by an machine learning algorithms. Examples of the applications of
expert or trained personnel. Some factors considered by these machine learning are crop yield prediction and intelligent
people in the quality assessment are the color and texture of irrigation systems. Machine learning techniques are classified
the products. However, manual checking and classification into two types: supervised and unsupervised learning.
give rise to several possible human errors. For a classification Examples of supervised machine learning algorithms are
to be successful, the people trained to examine the products Naïve Bayes (NB), Discriminant Analysis (DA), Support
Vector Machines (SVM), and K-Nearest Neighbor (KNN).

1845
Aaron Don M. Africa et al., International Journal of Emerging Trends in Engineering Research, 8(5), May 2020, 1845– 1849

On the other hand, K-Means Clustering, Gaussian Mixture followed for the classification of bananas as well as the other
Models (GMMs), and Fuzzy Clustering are types of harvested products. Having no standards is alarming for errors
unsupervised machine learning algorithms [4]. in the classification of bananas and other products may occur
and contribute to higher postharvest losses, which can
2. LITERATURE REVIEW eventually lead to a decrease in production profit. Several
studies have also focused on fruit sorting. Most of them used
According to one study, the quality of watermelons is only and presented fruits like apples, mangoes, oranges, and
evaluated by assessing factors such as appearance, texture, lemons. The fruits were individually sorted by using machine
and flavor. Nevertheless, the sole examination of these factors learning algorithms. However, fruits similar to bananas and
is not sufficient and reliable enough to indicate the overall grapes are required to be grouped in bunches or tiers, hence,
ripeness and quality of watermelons since the determination external characteristics of the fruit are more difficult to detect
of these qualities is prone to human mistakes. This study and examine by using simple machine learning. This study by
proposes a simple and economical method of watermelon Tuan-Tang Le, et.al. presents a deep learning sorting process
ripeness detection and analysis using wavelet multi-resolution of clustered bananas having only one image feature. Previous
decomposition or WMRD. Two samples, one for ripe implementations of deep learning, a field of convolutional
watermelons and one for unripe watermelons, are determined neural network or CNN which is also known as masked
by the multi-scale decomposition of the acoustic signals of the region-based convolutional neural networks or Mask R-CNN
watermelon for a particular coefficient. To provide an provide unique features and applications through object
estimation of the degree of separation of the two samples, a detection simultaneous generation of masks. Using Mask
discrimination index is also presented to identify the WMRD R-CNN, the detection of bananas provides a prediction of the
coefficient that works best. By implementing the presented class of the banana while simultaneously creates masks
method, the accuracy rates of obtained in tests and pieces of wherein the fruit is moved apart from the background. In the
training can get as far as 91.76% and 91.67%, respectively [5]. experiments and tests conducted in the study, an actual dataset
was utilized having banana bunches as bases. The proposed
Ethylene is an invisible and odorless substance released by system separates the normal tiers from the abnormal tiers. The
fruits once they have already ripened. The amount of ethylene deep learning classification method that was presented in this
emitted is indicative of the ripeness level of fruit. Numerous study had a successful implementation in banana bunches and
methods have been proposed for the detection of ethylene gas. was proven to be also effective for other horticultural products
Two of the most frequently and conveniently used techniques and crops that are clustered. The common deep learning
is the portable electronic nose or PEN and the electronic diagram that shows the several phases is shown in Figure 2.
olfactory system or EOS835. These methods are composed of
a set of metal oxide sensors and can detect several gases at the
same time. One of the best features of these methods is its
commercial availability. Non-dispersive infrared
spectroscopy or NDIR is also another approach used in
ethylene gas detection. This system has a selective feature in
measuring mixtures of substances and gases. For powerful
ethylene gas detection with a lower range of ppmv, gas
chromatographic systems are utilized. One study focuses on
the use of gas chromatographic systems since the ethylene
level must be measured in a selective manner having a
resolution of 50ppbv. Measurements and values of ethylene Figure 2: Common deep learning diagram containing three phases:
gas in synthetic air having 400ppbv and 35ppbv training, testing, evaluating [7].
concentrations were presented first in the paper. Eventually,
ethylene in the air was measured by the researchers from the It has been proven and demonstrated by several studies that
ripening bananas placed on a storage box having 306ppbv. deep learning offers excellent features and capabilities in
The study shows and demonstrates that the procedure is also understanding and learning image processes which is popular
possible for micro gas chromatographic systems which is a and commonly used in object detection. This paper utilizes the
Control System [6]. Mask Region-based Convolutional Neural Network or
Mask-RCNN to boost the performance of machine vision in
One of the most globally traded fruits and leading export the detection of fruits in a strawberry-harvesting robot. The
products in the Philippines is banana. However, a known backbone network used was Resnet50, along with the Feature
challenge for the product is the high percentage of postharvest Pyramid Network or FPN that served as the structure for
loss caused by various reasons like fruit diseases, fruit feature extraction. To produce region proposals for the feature
immaturity, over-ripeness, physical, and chemical damages. maps, and end-to-end training was conducted for the Regional
A study conducted by Aquino-Nuevo and Apaga in 2010 Proposal Network or RPN. According to the experimental
presented that no standards concerning size and quality are results of a hundred test images, the accuracy of detection

1846
Aaron Don M. Africa et al., International Journal of Emerging Trends in Engineering Research, 8(5), May 2020, 1845– 1849

reached a rate of 95.78% and a 95.41% recall rate. In have been implemented for these industries in which the main
comparison to conventional techniques, the presented method applications were object detection and classification. The
in this study shows a more robust environment, which is main advantage of convolutional neural networks over
suitable and effective for hidden and clustered fruits. The artificial neural networks is that they can execute both feature
summary of the R-CNN process is shown in Figure 3. selection and extraction and can combine large values of
parameters [13].

4. DESIGN CONSIDERATION

Images are important components in machine learning

algorithms as well as in several applications like feature
extraction, object detection, and object classification. One
study focused on the proposal of a fruit detection method that
uses machine vision. From an illumination-compensated
image, a modified green and red chromatic diagram was
produced. Illustrated below is an overview of the procedure
proposed in this study, which can also be applied in the
Figure 3: Summary of the Mask R-CNN Process [8]. detection and classification of other kinds of fruits that will
focus on the process of image processing, enhancement, and
3. THEORETICAL CONSIDERATION illumination. The fruit used in this study is citrus. The
proposed detection method diagram is shown in Figure 4.
Physical appearance is one of the most essential
characteristics of a fruit that is indicative of its ripeness state
and quality. The color and texture of fruits change throughout
their maturity. However, assessment and checking of these
factors even by trained personnel are still prone to
inconsistencies and human error. Recent studies involve the
use of machine learning in detecting and classifying objects
[9].

A common pipeline of machine learning is illustrated in Fig.2.

Machine learning composes of two primary parts: the training
and testing phases. For the phase of training, a training dataset
is utilized to generate a model with a machine learning
technique. Some of the most widely known models for these
are support vector machines or SVMs, random forest, and
neural networks. Training verification is typically done to
make sure that the process done to the model is generalized.
The trained model is then examined in the test phase whether
it has the capability of dataset prediction with the use of input
variables. The train and test datasets have several similarities.
Frequently, the test dataset is the outcome of separating the
initial dataset. The performed prediction in the testing phase if
often compared to the performed training process for further
evaluation [10,11,12].

Deep learning or deep convolutional neural network has

Figure 4: Diagram of Proposed Detection Method [14].
become a popular and widely used technique in the field of
computer vision because of its ability to combine
classification and extraction tasks. With the use of deep CNN, Another study emphasized the utilization of Red-Green-Blue
a system can do a prediction on the ripening state of a specific (RGB) Depth images in fruit maturity detection and
fruit. The rise of deep learning algorithms resulted in the classification. Passion fruit was the fruit used in this study.
incorporation of two components in the method of image For the acquiring of images, a Kinect V2.0 device was
processing which are dataset classification and feature employed, and a total of 4000 images were obtained. The
extraction. Deep and other kinds of convolutional neural device contained infrared depth and an RGB camera. The
networks have recently been widely used in food and figures below illustrate the setup of the Kinect V2.0 device.
agriculture technology. Since the year 2015, several studies The sample prototype and Kinect device is shown in Figure 5.

1847
Aaron Don M. Africa et al., International Journal of Emerging Trends in Engineering Research, 8(5), May 2020, 1845– 1849

5. CONCLUSION

The classification of ripe fruits and the evaluation of their

quality before the release in the market is commonly a human
activity. However, recent studies show that having physical
characteristics like shape, color, and texture as the only bases
of quality assessment may be prone to human error since these
factors require consistency during the examination. Several
Figure 5: (a) Prototype, (b) Kinect V2.0 device studies have proposed and presented various methods for
more accurate fruit detection and classification. Programs can
The images captured eventually underwent training and be created to simulate its algorithms like computer vision
testing phases. Below is an example of an RGB image. The systems [18,19,20,21]. This paper was able to enumerate
initial RGB images are shown in Figure 6. some of the most widely used and proven-effective methods
such as deep learning, image illumination, faster-CNN, and
the use of the gas chromatographic system in detecting
ethylene gas.

REFERENCES

[1] J. Steinbrener, K. Posch, and R. Leitner, “Hyperspectral

fruit and vegetable classification using convolutional
neural networks,” Computers and Electronics in
Figure 6: (a) Initial RGB image, (b) Depth Image, (c) Pre-processed Agriculture. Vol. 162, pp. 364-372, 2019.
depth image [15]. https://doi.org/10.1016/j.compag.2019.04.019
[2] K. Hameed, D. Chai, and A. Rassau, “A comprehensive
Another method that is widely used in fruit detection and review of fruit and vegetable classification techniques,”
classification is the Faster R-CNN. A diagram of an example Image and Vision Computing. Vol. 80, pp. 24-44, 2018.
implementation is illustrated below. The faster R-CNN https://doi.org/10.1016/j.imavis.2018.09.016
Diagram for implementation is shown in Figure 7. [3] Y. Chen, J. Grimplet, K. David, S.D. Castellarin, J. Terol,
D.C.J. Wong, Z. Luo, R. Schaffer, J. Celton, M. Talon,
G.A. Gambetta, and C. Chervin, “Ethylene receptors and
related proteins in climacteric and non-climacteric
fruits,” Plant Science, Vol. 276, pp. 63-72, 2018.
[4] A. Africa, “A rough set-based data model for heart
disease diagnostics.” ARPN Journal of Engineering and
Applied Sciences. Vol. 11, No. 15, pp 9350-9357, 2016.
[5] X. Chen, P. Yuan, and X. Deng, “Watermelon ripeness
detection by wavelet multiresolution decomposition of
acoustic impulse response signals,” Postharvest Biology
Figure 7: Faster R-CNN Implementation Diagram [16]. and Technology. Vol. 142, pp. 135-141, 2018.
https://doi.org/10.1016/j.postharvbio.2017.08.018
[6] A. Africa, P. Arevalo, A. Publico, and M. Tan. “A
There may be numerous possible techniques to be used in fruit
comprehensive study of the functions and operations of
detection. However, these techniques still follow a general
control systems.” International Journal of Advanced
process for fruit detection. Below is a block diagram overview Trends in Computer Science and Engineering. Vol. 8, No.
showing the general sensors, classification methods and 3, pp. 922-926, 2019.
processes involved in fruit detection. The fruit selection https://doi.org/10.30534/ijatcse/2019/89832019
detection process is shown in Figure 8. [7] T. Le, C. Lin, and E. Piedad, “Deep learning for
noninvasive classification of clustered horticultural crops
– A case for banana fruit tiers,” Postharvest Biology and
Technology. Vol. 156, 2019.
[8] Y. Yu, K. Zhang, Li Yang, and D. Zhang, “Fruit detection
for strawberry harvesting robot in non-structural
environment based on Mask-RCNN,” Computers and
Electronics in Agriculture. Vol. 163, 2019.
https://doi.org/10.1016/j.compag.2019.06.001
[9] L. Torrizo and A. Africa. “Next-hour electrical load
forecasting using an artificial neural network:
Figure 8: Fruit Detection process block diagram [17]. Applicability in the Philippines.” International Journal of

1848
Aaron Don M. Africa et al., International Journal of Emerging Trends in Engineering Research, 8(5), May 2020, 1845– 1849

Advanced Trends in Computer Science and Engineering. [21] P. Loresco, A. Namdala, A. Culaba, and E. Dadios.
Vol. 8, No. 3, pp 831-835, 2019. “Computer vision performance metrics evaluation of
https://doi.org/10.30534/ijatcse/2019/77832019 object detection based on Haar-like, HOG and LBP
[10] A. Africa, G. Ching, K. Go, R. Evidente, and J. Uy. “A features for scale-invariant lettuce leaf area calculation.”
comprehensive study on application development International Journal of Engineering and Technology.
software systems.” International Journal of Emerging Vol. 7, No. 4, pp. 4866-4872, 2018.
Trends in Engineering Research. Vol. 7, No. 8, pp
99-103, 2019.
https://doi.org/10.30534/ijeter/2019/03782019
[11] A. Africa, P. Arevalo, A. Publico, and M. Tan “Linear
system interconnections, steady-state analysis and
stability theory.” International Journal of Advanced
Trends in Computer Science and Engineering. Vol. 8, No.
4, pp 1395-1398, 2019.
https://doi.org/10.30534/ijatcse/2019/56842019
[12] C. Uy and A. Africa, “Development of a cost-efficient
waste bin management system with mobile monitoring
and tracking.” International Journal of Advanced Trends
in Computer Science and Engineering. Vol. 8, No. 2, pp
319-327, 2019.
https://doi.org/10.30534/ijatcse/2019/35822019
[13] A. Africa, T. Dolores, M. Lim, L. Miguel, and V. Sayoc.
“Understanding logical reasoning through computer
systems.” International Journal of Emerging Trends in
Engineering Research. Vol. 8, No. 4, pp 1187-1191,
2020.
[14] J.J. Zhuang, S.M. Luo, C.J. Hou, Y. Tang, Y. He, and
X.Y. Xue, “Detection of orchard citrus fruits using a
monocular machine vision-based method for automatic
fruit picking applications,” Computers and Electronics in
Agriculture. Vol. 152, pp. 64-73, 2018.
[15] S.Tu, Y. Xue, C. Zheng, Y. Qi, H. Wan, and L. Mao,
“Detection of passion fruits and maturity classification
using Red-Green-Blue Depth images,” Biosystems
Engineering. Vol. 175, pp. 156-167, 2018.
https://doi.org/10.1016/j.biosystemseng.2018.09.004
[16] J. Gené-Mola, V. Vilaplana, J.R. R. Polo, J. Morros, J.
Ruiz-Hidalgo, and E. Gregorio, “Multi-modal deep
learning for Fuji apple detection using RGB-D cameras
and their radiometric capabilities,” Computers and
Electronics in Agriculture. Vol. 162, pp. 689-698, 2019.
[17] A. Gongal, S. Amatya, M. Karkee, Q. Zhang, and K.
Lewis, “Sensors and systems for fruit detection and
localization: A review,” Computers and Electronics in
Agriculture. Vol. 116, pp. 8-19, 2015.
https://doi.org/10.1016/j.compag.2015.05.021
[18] A. Africa, F. Espiritu, C. Lontoc, and R. Mendez. “The
integration of computer systems into the expansive field
of video games.” International Journal of Advanced
Trends in Computer Science and Engineering. Vol. 8, No.
4, pp. 1139-1145, 2019.
https://doi.org/10.30534/ijatcse/2019/22842019
[19] P. Morales, M. Abugan, J. Olisea, A. Aralar and P.
Loresco. “Intelligent Traffic Light System Using
Computer Vision with Android Monitoring and Control.”
TENCON 2018 - 2018 IEEE Region 10, 2018.
[20] P. Loresco, I. Valenzuela, and E. Dadios, “Color Space
Analysis Using KNN for Lettuce Crop Stages
Identification in Smart Farm Setup.” TENCON 2018 -
2018 IEEE Region 10, 2018.

1849