International Journal of Engineering Systems Modelling

and Simulation

ISSN online: 1755-9766 - ISSN print: 1755-9758

https://www.inderscience.com/ijesms

An efficient fruit quality monitoring and classification using

convolutional neural network and fuzzy system

K.D. Mohana Sundaram, T. Shankar, N. Sudhakar Reddy

DOI: 10.1504/IJESMS.2022.10045885

Article History:
Received: 13 October 2021
Accepted: 02 February 2022
Published online: 01 December 2023

Copyright © 2024 Inderscience Enterprises Ltd.

Powered by TCPDF (www.tcpdf.org)

20 Int. J. Engineering Systems Modelling and Simulation, Vol. 15, No. 1, 2024

An efficient fruit quality monitoring and

classification using convolutional neural network
and fuzzy system

K.D. Mohana Sundaram*

Annamalai University,
Chidambaram, Tamil Nadu, India
Email: [email protected]
*Corresponding author

T. Shankar
Department of Electronics and Communication Engineering,
Annamalai University,
Chidambaram, Tamil Nadu, India
Email: [email protected]

N. Sudhakar Reddy
M.J.R. College of Engineering and Technology,
Andhra Pradesh, India
Email: [email protected]

Abstract: Fruit quality monitoring in agro industries is carried out by people who may deviate
from their responsibility due to tiredness, illness, or personal reasons. So, an automatic quality
assessment system is proposed based on convolutional neural network (CNN) and Mamdani
fuzzy logic that estimate quality of a Persian Lemon. The proposed CNN was trained with the
transfer learning method and the results obtained were compared with previous works. The
proposed CNN achieved 94.79% accuracy in the validation process which is 13% higher than the
existing architecture. The proposed fuzzy logic classified each lemon in three ranges based on
rules customised for the estimation of fruit quality standards.

Keywords: fuzzy systems; transfer learning; convolutional neural network; CNN.

Reference to this paper should be made as follows: Sundaram, K.D.M., Shankar, T. and
Reddy, N.S. (2024) ‘An efficient fruit quality monitoring and classification using convolutional
neural network and fuzzy system’, Int. J. Engineering Systems Modelling and Simulation,
Vol. 15, No. 1, pp.20–26.

Biographical notes: K.D. Mohana Sundaram received his BTech from JNTU Hyderabad
University, Andhra Pradesh, India. He received his MTech from JNTU Anantapur University,
Andhra Pradesh, India. Currently, he is pursuing PhD at Annamalai University, Chidambaram,
Tamil Nadu, India. His area of interest is digital circuits, VLSI design and deep learning.

T. Shankar received his BE, ME and Doctoral degree from Annamalai University, Chidambaram,
Tamil Nadu, India. Currently, he is working as an Assistant Professor in the Department of
Electronics and Communication Engineering at Annamalai University, Chidambaram, Tamil
Nadu, India. He published 30 research articles in various international journals. His area of
interest is image processing.

N. Sudhakar Reddy received his BE from Anna University, Tamil Nadu, India. He received his
MTech from SRM University, Chennai, Tamil Nadu, India. He received his Doctoral degree
from VELTECH University, Chennai, Tamil Nadu, India. Currently, he is working as Principal at
M.J.R. College of Engineering and Technology, Piler, Andhra Pradesh, India. He published
25 research articles in various international journals. His area of interest is wireless
communication and signal processing.

Copyright © 2024 Inderscience Enterprises Ltd.

 An efficient fruit quality monitoring and classification using convolutional neural network and fuzzy system 21

1 Introduction the fruit, size, and weight by sensory devices (Kumarganesh

and Suganthi, 2014). The third part is fuzzy systems for
The Automatic quality assessment process is currently
classification. Figure 1 shows the operation of the overall
gaining more importance in agro-industries (Albers et al.,
algorithm. This work is aimed at quality assessment of
2016; Chow et al., 2012; Markkandan et al., 2021). Present
Persian lemons, but this algorithm can be applied to any
this quality assessment process is mostly conducted by
fruits which need quality assessment for its sales.
people who, during the whole day, due to many reasons
such as stress or tiredness may deviate from their work Figure 1 Block diagram of algorithm
which in turn affect the quality of the product selected. This
need arises to develop software that recognises the quality
of the product obtained so that they are reliable and suitable
for the customer. In quality control operation involving
computer vision and logical classification, the combination
of convolutional neural network (CNN) and fuzzy logic is
ideal. CNNs (O’Shea and Nash, 2015) have found wide
application in computer vision areas covering object
detection and image classification. After the entry of
To evaluate input images a CNN architecture known as
AlexNet (Suresh et al., 2021), deep convolutional networks
ThinNet has been used because of its high accuracy. The
have become acceptable in many interesting applications.
transfer learning algorithm is used to train the CNN and
The diversification of similar networks with neural
final fully connected layers (Basha et al., 2019) with
backgrounds has been substantial. From the Inception
2-classifier and softmax layer (Ijomah et al., 2018) were
module to ResNet (Rajkumar et al., 2021), DenseNet
connected for fruit classification. Originally fully connected
(Perumal et al., 2021) and the ThinNet (Leonid and
layer consists of a maximum of 1,000 classifier but in this
Jayaparvathy, 2021), the strides have been quite rapid.
case, it requires only two to evaluate spoil or good fruits.
The focus of this paper is to choose a suitable CNN to
The Parsian Lemon dataset has a total number of 320
determine the physical state of the fruit (Zeiler and Fergus,
images, out of which 50% belongs to good lemon and the
2014). Next, the physical characteristics are extracted by
rest of the 50% were damaged. Each category consists of
image processing techniques and sensors. Later the
160 images out of 110 is for training 30 images are for
classification is done as per Mamdani Fuzzy logic
network testing 20 are for validation to avoid overfitting of
with suitable inputs and outputs. Mexican standard
the network. The total number of images is divided into five
NMX-FF-077-1996 (Banco Nacional de Comercio Exterior,
groups to define the size. There were 30 epochs to perform
1996) is used to focus on the classification of the fruit in
the training of CNN. Overall, 150 iterations had five
which classification parameters are given as in Table 1.
validations out of which every 30 iterations will be checked
Based on the size codes, the lemon quality is classified into
to avoid overfitting.
three categories such as low, medium, and high based on
weight and damage percentage on the surface of the fruit. Figure 2 Physical parameters extraction
Table 1 Classification based on diameter

Size code Diameter range (mm) Average interval (mm)

1 58–67 62.5
2 53–62 57.5
3 48–57 52.5
4 46–52 49.5
5 43–46 44.5
6 38–43 40.5
Source: Zeiler and Fergus (2014) The block diagram of the second stage of the algorithm is
shown in Figure 2. In this stage, the physical parameters of
the fruit can be determined in four steps. In the first step,
2 Methods hue, saturation, value (HSV) filter was used with filter
configurations of 0.125–0.399 for H, 0.2–1 for S, and 0–1
2.1 Quality assessment
for V. After the binarised image is generated an area filter is
There are three parts of the mechanism for the classification applied to eliminate noise in the image. In the second step,
of fruits, among which the first part is a CNN which have the binary image from the HSV filter is inverted and based
four stages such as taking the database, adoption of on the total number of pixels and small areas which
architecture, training process, and verification with test represent defects, the percentage of the defected area can be
images. The second part is for feature extraction by image obtained from equation (1). In the third step, the diameter of
processing to determine the damaged area on the exterior of the fruit can be identified by formulae which calculate 13
22 K.D. Mohana Sundaram et al.

specimens with a certain number of pixels and this must be The details of ThinNet architecture with one front and four
considered for all the images. These results were tabulated tinier modules were given in Table 2. In the final detection,
and a graph is drawn between the number of pixels and a 1 × 1 convolutional layer is used with linear activation.
diameter as shown in Figure 3. To find the weight of the After each Tinier module with the last module, the
fruit, An Arduino card, which gets an analogue voltage from maximum pooling is performed with stride of 2. For the first
the battery, is used. The weight obtained by this process is Tinier module strength of filters is two 3 × 3 convolution
sent to MATLAB for further processing. Once all the above layers of 64 channels. It may be noted that the number of
processes were completed the final stage, which is fuzzy channels is doubling after each pooling. By trial and error in
classification can be applied to assess the quality of the an experimental activity, the number of 1 × 1 filter has
fruit. ‘μ = 0.25’ (Leonid and Jayaparvathy, 2021) from the output
channels of the preceding module per each Tinier module,
Total Area of Defects(i ) *100
% of Defects =  Total Surface Area of the Lemon (1) where µ is the parameter introduced in ThinNet to reduce
the number of convolution parameters.

Figure 3 Determination of diameter of the fruit based on pixel

2.1.2 Front module and tinier module
values (see online version for colours)
The state-of-the-art networks have repeated building blocks
on a module with identical characteristics. Usually,
better-designed blocks attain higher performance with
relativity less theoretical complexity. For taking advantage
of modular design, the Tinier module and Front module
have been designed. Figure 5 shows a Tinier model which
consists of two 1 × 1 convolution layers coupled with two
3 × 3 conventional layers. Convenient usage of 1 × 1 layer
helps in reducing the dimensions. It may be noted that every
convolution layer is linked to batch normalisation and ReLu
(rectified linear unit) operations in successive stages. As in
2.1.1 CNN architecture many current models, batch normalisation results in
In this work, the CNN used to extract physical substantial improvement in convergence and this obviates
characteristics of the fruit is ThinNet. ThinNet is based on what’s the need for other regularisation.
YOLOv2 (Redmon and Farhadi, 2017). It frames object
Figure 5 Front module and tinier module, (a) front module
detection into a regression problem that separated bounding
(b) tinier module
boxes and class probabilities. As the detection pipeline is a
single network, end-to-end optimisation is possible for
detection. The operation of ThinNet can be explained in
three stages such as,
a resizing of input images to dimensions 416 × 416.
b process ThinNet on the image and predict the
probability and sum of multiple bounding boxes and
related boxes
c the test result is selected according to the confidence
level.
The basic architecture of ThinNet comprises of
1 front module
(a) (b)
2 tinier module
Following patterns of inspectionv3 (Narmatha et al., 2020)
3 detector layer as shown in Figure 4. and inseptionv4 (Szegedy et al., 2016) front module as
shown in Figure 5, can be defined as a stair of three 3 × 3
Figure 4 Architecture of existing ThinNet convolution layers after which 2 × 2 layers of max-pooling
are placed. For the 1st convolution layer the stride = 2
whereas for the next two the stride = 1. The single front
module improves the performance of detection. This can be
explained by the fact that in comparison with the DenseNet
with 7 × 7 structure followed by 3 × 3 max pooling, at the
end of the front module down sampling is applied and so the
Source: Redmon and Farhadi (2017) layers of evolution can have huge feature maps. It is
 An efficient fruit quality monitoring and classification using convolutional neural network and fuzzy system 23

pertinent to note that due to delayed down sampling larger response surfaces of the fuzzy system (Thiyaneswaran et al.,
feature maps can improve the performance as the data loss 2020).
from the raw input image is reduced.
Table 3 If-then rules for fuzzy system
2.1.3 Modified ThinNet architecture Surface defects Weight Diameter Output quality
We modified the existing ThinNet architecture by replacing Low High High Good
detector layer with a fully connected layer (Basha et al., Low High Medium Good
2019) with 2-Classifier and a Softmax layer (Ijomah et al., Low High Low Medium
2018) as exposed in Figure 6, to classify the fruit as good or
Low Medium High Medium
damaged.
Low Medium Medium Medium
Table 2 Architecture information of existing ThinNet Low Medium Low Low
Low Low High Medium
Type Output size Filter size/stride N1×1 N3×3
Low Low Medium Low
Input 416 × 416 × 3  3 × 3, 32, 2 
image   Low Low Low Low
 3 × 3, 32, 1  × 2 Medium High High Good
Front 104 × 104 × 128  3 × 3, 64, 1  - -
module  
Medium High Medium Medium
Tinier1 104 × 104 × 64 1 × 1 / 1  16 128 Medium High Low Low
 ×2
 3 × 3 / 1 Medium Medium High Medium
Tinier2 52 × 52 × 128 1 × 1 / 1  32 256 Medium Medium Medium Medium
 ×2 Medium Medium Low Low
 3 × 3 / 1
Medium Low High Medium
Tinier3 26 × 26 × 256 1 × 1 / 1  64 512
 ×2 Medium Low Medium Low
 3 × 3 / 1
Medium Low Low Low
Tinier4 13 × 13 × 1,024 1 × 1 / 1  128 1,024 High High High Good
 ×2
 3 × 3 / 1 High High Low Medium
Detector 13 × 13 × 125 1 × 1, 125, 1 - - High High Medium Medium
layer High Medium High Medium
Note: N1×1 and N3×3 represents number of 1 × 1 and 3 × 3 High Medium Medium Low
filters.
High Medium Low Low
Source: Leonid and Jayaparvathy (2021)
High Low High Medium
Figure 6 Modified ThinNet architecture for fruit classification High Low Medium Low
High Low Low Low

Figure 7 If-then rules for fuzzy system (see online version

for colours)

2.1.4 Fuzzy system interfacing

The fuzzy interface system (FIS) is an important unit of
fuzzy logic (Kumarganesh and Suganthi, 2018). We used
Mamdani fuzzy interface (shown in Figure 8) as it is more
spontaneous and easier to realise. This fuzzy system works
based on obtained inputs and a series of IF-THEN rules as
shown in Table 3 and Figure 7. Membership Functions for
input variables damages, weight, and diameter are shown in
Figure 9. We assigned triangular membership function
(trimf) for input and output variables. Figure 10 shows the
24 K.D. Mohana Sundaram et al.

Figure 8 Mamdani FIS (see online version for colours)

Figure 9 Membership function for input variables damage, weight, diameter and output variable fruit quality (see online version
for colours)

Figure 10 Fuzzy system response surface (see online version for colours)
 An efficient fruit quality monitoring and classification using convolutional neural network and fuzzy system 25

3 Results Figure 13 Confusion matrix obtained from CNN training

(see online version for colours)
The merits of Modified ThinNet architecture (shown in
Table 4) shows that our proposed architecture achieved
better accuracy than previous works. The trained CNN
provided an average of 67.9% accuracy by considering both
Training and Validation. Figure 11 shows the accuracy and
loss of CNN while trained for 150 iterations in five batches.
Input and Output Images of CNN were shown in Figure 12.
A confusion matrix (Figure 13) obtained from training
process of CNN, evaluating 50 images shows the
classification by the categories was 100% correct.

Table 4 Merits of modified ThinNet architecture

Architecture Output layer Purpose Accuracy

Tiny YOLO Detector layer Object detection 63.4%
AlexNet [11] fully connected Image 80.3%
layer classification
ThinNet detector layer Object detection 81.7%
Modified Fully connected Image 94.79% 4 Conclusions
ThinNet layer with classification The proposed CNN designed with ThinNet architecture,
(proposed 2-classifier and
work) softmax layer trained with transfer learning method achieved an average
of 94.79% of accuracy for the classification of good and
damaged fruit. This accuracy can be improved by modifying
Figure 11 Increment of accuracy and decrement of loss while
training CNN (see online version for colours)
feature extraction properties if CNN and with image
enhancement techniques. The proposed system for physical
characteristics extraction of fruit worked properly. The
Mamdani FIS classified the fruits accurately by considering
all the characteristics into account. This work concentrated
on the Persian lemon data set, but, if necessary changes
made in CNN, in the feature extraction and Fuzzy system
then it can be applied to any kind of product to increase the
quality standards of the products.

References
Albers, A., Gladysz, B., Pinner, T., Butenko, V. and Stürmlinger,
T. (2016) ‘Procedure for defining the system of objectives in
the initial phase of an industry 4.0 project focusing on
intelligent quality control systems’, Procedia Cirp, Vol. 52,
pp.262–267, doi: 10.1016/j.procir.2016.07.067.
Figure 12 Input and output images of CNN (see online version Banco Nacional de Comercio Exterior (1996) ‘NMX-FF-077-
for colours) 1996, PRODUCTOS ALIMENTICIOS NO
INDUSTRIALIZADOS PARA CONSUMO HUMANO
FRUTA FRESCA. LIMÓN PERSA (Citrus Latifotia L.)
ESPECIFICACIONES, NORMAS MEXICANAS.
DIRECCIÓN GENERAL DE NORMAS’ [online]
https://www.colpos.mx/bancodenormas/nmexicanas/NMX-
FF-077-1996.PDF (accessed 30 July 2018).
Basha, S.H.S., Dubey, S.R., Pulabaigari, V. and Mukherjee, S.
(2019) ‘Impact of fully connected layers on performance of
convolutional neural networks for image classification’,
Neurocomputing, doi: https://doi.org/10.1016/j.neucom.2019
.10.008.
Chow, Z.S., Ooi, M.P.L., Kuang, Y.C. and Demidenko, S. (2012)
‘Automated visual inspection system for mass production of
hard disk drive media’, Procedia Engineering, Vol. 41,
pp.450–457, doi: 10.1016/j.proeng.2012.07.197.
26 K.D. Mohana Sundaram et al.

Ijomah, W., Nwankpa, C.E., Gachagan, A. and Marshall, S. (2018) Perumal, S., Tabassum, M., Narayana, G., Suresh, P., Chakraborty,
Activation Functions: Comparison of Trends in Practice and C., Mohanan, S., Basit, Z. and Quasim, M.T. (2021) ‘ANN
Research for Deep Learning, 8 November base novel approach to detect node failure in wireless sensor
arXiv:1811.03378v1 [cs.LG]. network’, CMC-Computers, Materials & Continua
Kumarganesh, S. and Suganthi, M. (2014) ‘Efficient lossless (TechScience), Vol. 69, No. 2, pp.1447–1462, ISSN:1546-
medical image compression technique for real world 2226.
applications using JOSE-encoding’, International Journal of Rajkumar, R., Dileepan, D., Chinmay, C. and Suresh, P. (2021)
Applied Engineering Research, Vol. 9, No. 24, ‘Modified Minkowski fractal multiband antenna with
pp.24625–24640. circular-shaped split-ring resonator for wireless applications’,
Kumarganesh, S. and Suganthi, M. (2018) ‘An enhanced medical Measurement, ID. 109766, ISSN 0263-2241, https://doi.org
diagnosis sustainable system for brain tumor detection and /10.1016/j.measurement.2021.109766.
segmentation using ANFIS classifier’, Current Medical Redmon, J. and Farhadi, A. (2017) ‘YOLO9000: better, faster,
Imaging Reviews, Vol. 14, No. 2, pp.271–279. stronger’, in Computer Vision and Pattern Recognition,
Leonid, T.T. and Jayaparvathy, R. (2021) ‘Statistical-model based pp.6517–6525, DOI: 10.1109/CVPR.2017.690.
voice activity identification for human-elephant conflict Suresh, P., Aanandha Saravanan, K., Iwendi, C., Ibeke, E. and
mitigation’, J. Ambient Intell. Human Comput., Vol. 12, Srivastava, G. (2021) ‘An artificial intelligence based quorum
pp.5269–5275, https://doi.org/10.1007/s12652-020-02005-y. system for the improvement of the lifespan of sensor
Markkandan, S., Sharma, A., Singh, S.P. et al. (2021) ‘SVM-based networks’, IEEE Sensors Journal, Vol. 21, No. 15,
compliance discrepancies detection using remote sensing for pp.17373–17385.
organic farms’, Arab Journal of Geosciences, Vol. 14, Szegedy, C., Ioffe, S., Vanhoucke, V. and Alemi, A.A. (2016)
No. 1334, pp.1–8, https://doi.org/10.1007/s12517-021-07700- ‘Inceptionv4, Inception-ResNet and the impact of residual
4. connections on learning’, in National Conference on Artificial
Narmatha, C., Thangamani, M. and Ibrahim, S.J.A. (2020) Intelligence, pp.4278–4284.
‘Research scenario of medical data mining using fuzzy and Thiyaneswaran, B., Anguraj, K., et al. (2020) ‘Early detection of
graph theory’, International Journal of Advanced Trends in melanoma images using gray level co-occurrence matrix
Computer Science and Engineering, February, Vol. 9, No. 1, features and machine learning techniques for effective clinical
pp.349–355. diagnosis’, International Journal of Imaging System and
O’Sheam, K. and Nash, R. (2015) ‘An introduction to Technology, Vol. 31, No. 20, pp.682–694.
convolutional neural networks’, Research Gate, November Zeiler, M.D. and Fergus, R. (2014) ‘Visualizing and understanding
[online] https://www.researchgate.net/publication/285164623 convolutional networks’, in European Conference on
(accessed 20 March 2020). Computer Visión, Springer, Cham., pp.818–833, doi:
10.1007/978-3-319-10590-1_53.