See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/339094628

Food Spoilage Detection Using Convolutional Neural Networks and K Means

Clustering

Conference Paper · October 2019

DOI: 10.1109/RDCAPE47089.2019.8979114

CITATIONS READS

15 2,280

5 authors, including:

Rajesh Kannan Megalingam Monika Reddy

Amrita Vishwa Vidyapeetham - Humanitarian Technology (HuT) Labs Amrita Vishwa Vidyapeetham
237 PUBLICATIONS 1,731 CITATIONS 4 PUBLICATIONS 20 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Rajesh Kannan Megalingam on 21 May 2022.

The user has requested enhancement of the downloaded file.

 2019 3rd International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE)

Food Spoilage Detection Using Convolutional

Neural Networks and K Means Clustering
Rajesh Kannan Megalingam, Gadde Sakhita Sree, Gunnam Monika Reddy, Inti Rohith Sri Krishna
Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
[email protected], [email protected], [email protected],
[email protected]
L.U.Suriya
Department of Electrical and Electronics Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
[email protected]

Abstract—This paper presents the novel idea for of. Having a device that could identify the spoiled
detecting food spoilage using image classification with food by detecting the color change occurred in food
machine learning algorithms and artificial intelligence will be useful. Not only in households but also in the
technology. Food spoilage is detected by using artificial industries having a system which can detect the
intelligence, deep convolutional neural networks, and spoilage in food will be much useful for the business
computer vision and machine learning algorithms like k people to grade them up.
clusters algorithm for color classifications in images and
its HSV values for spoilage detection. This project is Colorblind people are facing many difficulties which
done using the jupyter notebook platform through may not be known by normally sighted people. The
anaconda prompt. In this project, photos of food or people who have color blindness cannot be able to
fruits will be taken which have to be tested and then it recognize the actual color which can be seen by
will be processed through computer which will then normally sighted people. They even find difficulties
perform image classification and machine learning in preparing food, choosing clothes and even driving
algorithms for getting the colors in the image and
a car because they can't recognize the colors of a
traffic signal. For people who are suffering from
spoilage is detected by HSV (Hue Saturation Value)
colorblindness cannot recognize the spoiled food by
values and percentages of each color which we have got
color change. For them, this project will be effective
by using the k cluster algorithms in jupyter notebook.
and will reduce the effort made by them in
Keywords—Jupyter notebook, anaconda prompt,
recognizing the spoilage.
Machine learning, artificial intelligence, deep
The spoilage food detection project on which this
convolutional neural networks, computer vision, image
paper is written is just a software-based program that
classification, k clusters algorithm, HSV values.
detects the spoilage when images or photos of food
I. INTRODUCTION are processed through it in the jupyter notebook. The
spoilage which has fungi formation can be detected
The rapid growth in the food industry and demand for in this project and predicts whether the food is
preservation of food paved a way to generate many spoiled or not by performing image processing and k
methods to detect the spoilage in food due to clusters algorithm. The same process is applicable for
bacteria, fungi and other germs. Nowadays food fruits as well, but an extra step will be added which
poisoning is a serious problem by which many people will predict the name of fruit before predicting its
all around the world are facing many problems. Over spoilage. Because some fruits show abnormal
200 diseases are caused by spoiled food containing appearance when they are even unspoiled. Based on
harmful bacteria such as Salmonella, Shigella. the name of fruits the conditions of spoilage varies.
Globally every year nearly 3-5 billion people are
being affected by infectious diarrhea and nearly 1.8 II. MOTIVATION
million people died due to contamination of food.
Spoiled food must be detected early to prevent Artificial intelligence and machine learning fields are
diseases. Almost in every household, there will be now the most important and emerging fields which
some food hidden in the refrigerator which will get are very useful in our day to day services. Nowadays
spoiled and turns its color and start smelling due to in many industries, these technologies are introduced
the formation of bacteria which will be detected by which makes the human effortless with accurate
people after a long time and it needs to be disposed results in many of the areas by applying technology

978-1-7281-2068-3/19/$31.00 ©2019 IEEE 488

uthorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on May 21,2022 at 08:15:29 UTC from IEEE Xplore. Restrictions app
 2019 3rd International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE)

to systems. There are many applications of artificial meat by Fourier transform infrared spectroscopy and
intelligence and machine learning in fields like image machine learning is presented in paper [9]. In the
processing, robotic process operation, deep learning research paper [10], rapid qualitative and quantitative
platforms, and many other areas. There are many detection of beef fillets spoilage based on Fourier
ways food spoilage can be detected but the common transform infrared spectroscopy data and artificial
way of achieving is using sensors that sense the neural networks are discussed. Fruit detachment and
spoiled food by ammonia, trimethylamine gases and classification methods for strawberry harvesting
by enzyme activities in the food. But in this paper, robots are explained in paper [11]. Computational
spoiled food can be detected using image processing Analysis between Software and Hardware
and machine learning algorithms. Spoiled food Implementation of the Sobel Edge Detection
detection will be an advantage for many people like Algorithm is presented in the research paper [12]. In
who are suffering from color blindness and this type the research paper [13], Image processing based
of camera-based system can be used to notify the classification of grapes after pesticide exposure is
presence of spoiled foods in a refrigerator. It can also presented. A Fruit Detecting and Grading System
be used in many areas such as large industries where Based on Image Processing-Review is discussed in
this type of system is very useful to grade the packed paper [14]. The fungal spore and food spoilage are
food or fruits which are spoiled in it. It can also be explained in paper [15]. In the research paper [16],
used in exports and imports where the ships take a Developing a framework for fruit detection from
long time to reach from one place to another where images is presented. An Active Thermography
there is a chance of food getting spoiled due to Method for Immature Citrus Fruit Detection is
environmental changes like rain. explained in paper [17]. In the research paper [18], A
Modified Canny Edge Detection Algorithm for Fruit
III. RELATED WORKS Detection & Classification is presented.

There are many applications of machine learning and IV. ALGORITHMS EXPLANATION
artificial intelligence which is not restricted to
perform specific tasks, these had played a significant A. DEEP CONVOLUTIONAL NEURAL NETWORKS
roles in many of the fields and also used in medical
diagnosis, electronic platforms, robotics, remote Convolutional Neural Network (CNN) is a type of
controlling, industries, virtual personal assistants, deep neural networks. It is mostly used to compare
video surveillance, social media, email spams or and classify images. A computer understands every
malware filtering, fraud detection, education, image using numbers at every pixel. This method
transportation and even in finance. Tracking food divides an image into pieces and compares the image
spoilage in the smart home using odor monitoring is piece by piece. These pieces or patches are called
presented in paper [1]. Detection of Food Spoilage features or filters. CNN has four layers namely
and Pathogenic Bacteria Based on Ligation Detection Convolution layer, ReLU layer, Pooling layer, Fully
Reaction Coupled to Flow-Through Hybridization on Connected layer. In the convolution layer, we will
Membranes is presented in paper [2]. In the research move each feature or filter to every part of the input
paper [3], determining spoilage level against time and image. There are a few steps in this layer. First, we
temperature of tomato-based Filipino cuisines using line up all the features of the image. Then we
the electronic nose is explained. The identification of multiply each image pixel by the corresponding
food spoilage in the smart home based on neural and feature pixel values. And a new matrix is formed
fuzzy processing of odor sensor responses is after the multiplication. Next, we add and divide the
discussed in the research paper [4]. An electronic total number of pixels in the matrix. The next step is
nose based on metal oxide semiconductor sensors as to create a map and put that value of the filter at that
an alternative technique for the spoilage classification place. Then we move the filter throughout the image
of red meat is proposed in the research article [5]. A and see how the filter matches every area. Then the
CMOS fish resistance to the period converter with map is filled to form an output matrix. In the same
calibration circuits for fish spoilage detecting way, a new matrix is formed for each filter. The next
applications is explained in paper [6]. Recent layer is the ReLU layer (Rectified Linear Unit) which
developments in rapid multiplexed bioanalytical is an activation function. In this layer, all the negative
methods for foodborne pathogenic bacteria detection values in the filtered images are removed and
is proposed in the research article [7]. In the research replaced with zeroes. The next layer is the pooling
paper [8], it is discussed about Multispectral Imaging layer which shrinks the image into a smaller size.
for Spoilage Detection of Pork Meat. Rapid and There will be a window size according to which we
quantitative detection of the microbial spoilage of take portions in the filtered matrix. In that window

489

uthorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on May 21,2022 at 08:15:29 UTC from IEEE Xplore. Restrictions app
 2019 3rd International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE)

area, only the maximum value is taken. When we all the nearest data points to the centroid from the
move the window across the matrix we get all the given data. The data points nearest to another
values of the new matrix whose size will be lesser. centroid will form a different cluster corresponding to
Now all three steps are repeated and we get the lesser that centroid. And after the clusters are formed, the
sized matrix. Then comes the fully connected layer centroid of the clusters is again calculated which is
which is the final part where the actual classification the mean of the data points in the cluster. This
happens. Here all the shrunk images are taken and process repeats until the same cluster and same
put into a single list. This is called the list or vector. centroid values repeat. And those clusters will be the
This list varies for different images. When a new final groups of the given data set. Here the given data
input is given, it passes through all the CNN layers can be anything. In this model, the data given is an
and the final list is obtained. This list compares itself image. This division of an image into groups is
with the previously obtained lists and classifies the known as Image segmentation. Image segmentation
image accordingly. For this to happen, the model can be done by many methods such as threshold,
should be trained with a lot of images of the same clustering, neural networks, etc. Among them, k-
thing which helps the accuracy of the classification. means clustering is the most used method. K-means
This model is chosen as it can predict the fruit by clustering for image segmentation separates the
comparing it with the training images in the model required area from the background or we can say that
which helps in differentiating the fruit which has it divides the clusters based on colors. For example,
dissimilarities with the other fruits in terms of take the k value as 5. That means there will be 5
ripening. clusters or the clustering happens according to 5
dominant colors of the given image. The image will
B. K MEANS CLUSTERING be converted from BGR to RGB and this RGB image
will be resized using interpolation. Here resizing
K-means Clustering is a machine learning algorithm means nothing but zooming the image for a more
that is used to divide the given data into different detailed image. While zooming the image, the
groups or clusters. Here, the quantity of pixels increases. Centroids or means are
selected at random. Next, the clusters are created.
And the mean of the cluster becomes the new
Grouping centroid. Therefore the centroids are updated. These
Generate Initialize each clusters are 5 different colors. Their hexadecimal
objects
several k cluster values can be calculated from their corresponding
clusters k centroid to closest
clusters RGB values. This method is used to get dominant
colors of the image which helps us to predict the
spoilage of the food item.

V. WORKING
Find
Clusters distance At first images of some fruits like mango, banana,
are objects apple, guava are taken and a data set is created for
formed to different fruits which can be used as a training model
centroids for fruit recognition. For food items, there is no
training model because of the complexity in finding
the difference between the different items as no
proper color difference between them. When an
Fig1. K clusters algorithm functionality image is given, if it is a fruit first it will predict the
name of the fruit by training data sets taken above by
several clusters are the value given to the variable k. deep convolutional neural networks and if it is a food
This is an iterative process. item then this step will be skipped and then the image
Fig1 shows the functionality of the k clusters is then reshaped and resized and it will be passed
algorithm. K-means clustering algorithm results in through the k means clustering algorithm which
means and labels for the new data. The number of actually give the colour code values by which we can
mean values will be the same as the value of k. The convert those color codes values to hue, saturation,
mean values can also be called K centroids of the and value and hence we can predict whether the food
cluster which are randomly selected from the given is spoiled or not by these HSV values.
data. A new cluster is formed based on the
corresponding value of the k centroid. A cluster takes

490

uthorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on May 21,2022 at 08:15:29 UTC from IEEE Xplore. Restrictions app
 2019 3rd International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE)

VI. EXPERIMENTAL RESULTS

In this paper, the platform of the jupyter notebook is

used to experiment. Some of the food items and fruits
are predicted whether spoiled or unspoiled. This
experiment was conducted on food items like dal,
bread and many, fruits like mango, apple, guava,
banana. It is mainly based on the color of food which
gets changed after spoiling and the color can be
identified by HSV which was found by using
algorithms like k means clustering method.
Fig5. Unspoiled banana

Fig2. Spoiled dal

Fig6. Spoiled banana

From Fig2 and Fig3, it is seen that Fig2 is the spoiled

dal and fig3 is the unspoiled dal.in this project the
food can be predicted either spoiled or unspoiled by
analyzing the image of the dish. Hereafter analyzing
it is predicted that Fig2 is the image of spoiled and
Fig3 is the image of unspoiled by satisfying certain
conditions.

Fig4 represents the spoiled image of bread which has

some spots which represent spoilage. By HSV ranges
Fig3. Unspoiled dal of colors obtained in each figure, it can be predicted
that it is either spoiled or not. Fig2, Fig3, Fig4
represents the experiments done on food items by
using the k clustering method which can produce the
k number of colors which are the most prominent.

Fig5 and Fig6 represent spoiled and unspoiled banana

images. The name of the fruit is also predicted by the
deep convolutional neural networks method. As it is a
fruit, spoilage properties differ from each other.
Bananas show a different exposer from outside.
Unspoiled bananas have some black spots but that
doesn't affect the interior part of the bananas. If that
spot occurs more than a certain percentage then that
Fig4. Spoiled bread blackness is considered spoilage. In Fig5 the banana
is considered as a spoiled one as it contains more
percentage of black color. By the increasing number
of clusters and finding their HSV values, even small
contamination in both fruits and on foods can be

491

uthorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on May 21,2022 at 08:15:29 UTC from IEEE Xplore. Restrictions app
 2019 3rd International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE)

predicted. Prediction of fruit can be efficient when Based on Ligation Detection Reaction Coupled to
more than 20 images are placed in the training model Flow-Through Hybridization on
Membranes,” BioMed Research International, vol.
of each type of fruit. 2014, Article ID 156323, 11 pages, 2014.
[3] Meo Vincent C. Caya, Febus Reidj G. Cruz, Patricia
Joy R. Blas, Miriam M. Cagalingan, Renalen Grace L.
VII. FUTURE WORKS Malbas, Wen-Yaw Chung, "Determining spoilage
level against time and temperature of tomato-based
Filipino cuisines using electronic nose", Humanoid
This spoilage detection project which has been done Nanotechnology Information Technology
by artificial intelligence and machine learning Communication and Control Environment and
technology has a great scope and has many Management (HNICEM) 2017 IEEE 9th International
applications in areas where the spoiled food can be Conference on, pp. 1-5, 2017.
detected by cameras without using sensors even when [4] G. C. Green, A. D. C. Chan, R. A. Goubran,
"Identification of food spoilage in the smart home
any number of food items are there. By fixing a good based on neural and fuzzy processing of odor sensor
camera in the refrigerator or in any industries which responses", IEEE Engineering in Medicine and
can take photos on its own and predict whether the Biology Society Conference, vol. 1, pp. 2625-2628,
2009.
food is spoiled or not. Making an app that can predict
[5] N. El Barbri, E. Llobet, N. El Bari, X. Correig, B.
whether the food is spoiled or not by taking a picture Bouchikhi, "Electronic nose based on metal oxide
in the camera and processing it can be useful for semiconductor sensors as an alternative technique for
colorblind people. By connecting a small sensor that the spoilage classification of red meat", Sensors, vol.
can detect the spoilage can be attached to it to 8, no. 1, pp. 142-156, 2008.
increase the effectivity in predicting whether fruits or [6] Cheng-Ta Chiang, Yen-Kuei Lu, Lian-Teng Lin, "A
CMOS fish resistance to the period converter with
spoiled or not when a large number of them get calibration circuits for fish spoilage detecting
imported or exported from one place to another in applications", Mechatronics and Automation (ICMA)
ships. 2017 IEEE International Conference on, pp. 51-55,
2017.
[7] Aldo Roda, Mara Mirasoli, Barbara Roda, Francesca
Bonvicini, Carolina Colliva, Pierluigi Reschiglian,
VIII. CONCLUSIONS "Recent developments in rapid multiplexed
bioanalytical methods for foodborne pathogenic
Spoiled food detection project with artificial bacteria detection", Microchimica Acta, vol. 178, pp.
intelligence and machine learning algorithms through 7, 2012.
computer vision and without using any food spoilage [8] Dissing, B., Papadopoulou, O., Tassou, C., Ersbøll, B.,
Carstensen, J., Panagou, E., Nychas, G.-J.: Using
detecting sensor has been successfully built in the Multispectral Imaging for Spoilage Detection of Pork
platform Jupiter notebook in python which produced Meat. Food and Bioprocess Technology 6, 2268–2279
accurate results. Based on the experiments conducted (2013)
spoilage can be detected accurately in both foods and [9] Ellis, D.I., Broadhurst, D., Kell, D.B., Rowland, J.J.,
fruits. Infinitesimal negligence can be obtained when Goodacre, R.: Rapid and quantitative detection of the
microbial spoilage of meat by Fourier transform
taking pictures with different cameras due to the infrared spectroscopy and machine learning. Appl.
difference in camera lenses, sizes and also due to Environ. Microbiol. 68, 2822–2828 (2002)
daylight and tube lights. [10] Argyri, A.A., Panagou, E.Z., Tarantilis, P.A.,
Polysiou, M., Nychas, G.-J.E.: Rapid qualitative and
quantitative detection of beef fillets spoilage based on
Fourier transform infrared spectroscopy data and
ACKNOWLEDGEMENT artificial neural networks. Sensors and Actuators
B 145, 146–154 (2010)
We are grateful to the Humanitarian Technology Lab [11] G. Feng, C. Qixin et al., "Fruit detachment and
and Electronics and Communication Engineering classification method for strawberry harvesting
department at Amrita Vishwa Vidyapeetham, robot", International Journal of Advanced Robotic
Systems, vol. 5, no. 1, pp. 41-48, 2008.
Amritapuri Campus for their constant support and for
[12] Rajesh Kannan Megalingam; Malavika Karath ; P
aiding us in this endeavor. Prajitha; Goutham Pocklassery, “Computational
Analysis between Software and Hardware
REFERENCES Implementation of Sobel Edge Detection Algorithm”,
2019 International Conference on Communication and
[1] Geoffrey C. Green, Adrian D.C. Chan, Rafik A. Signal Processing (ICCSP)
Goubran, "Tracking food spoilage in the smart home DOI: 10.1109/ICCSP.2019.8697929
using odor monitoring", Medical Measurements and [13] Dutta, Malay & Sengar, Namita & Minhas, Navroj &
Applications Proceedings (MeMeA) 2011 IEEE Sarkar, Biplab & Goon, Arnab & Banerjee, Kaushik.
International Workshop on, pp. 284-287, 2011. (2016). Image processing based classification of
[2] K. Böhme, P. Cremonesi, M. Severgnini, et al., grapes after pesticide exposure. LWT - Food Science
“Detection of Food Spoilage and Pathogenic Bacteria and Technology. 72. 10.1016/j.lwt.2016.05.002.

492

uthorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on May 21,2022 at 08:15:29 UTC from IEEE Xplore. Restrictions app
 2019 3rd International Conference on Recent Developments in Control, Automation & Power Engineering (RDCAPE)

[14] Banot, Ms & P.M., Dr. (2016). A Fruit Detecting and

Grading System Based on Image Processing-Review.
IJIREEICE. 4.47-52. 10.17148/IJIREEICE.2016.4112.
[15] Dijksterhuis, Jan. (2017). The fungal spore and food
spoilage. Current Opinion in Food Science. 17.
10.1016/j.cofs.2017.10.006.
[16] Hossen, Md & Arefin, Mohammad. (2017).
Developing a framework for fruit detection from
images. 951-954. 10.1109/ECACE.2017.7913041.
[17] Gan, Hao & Lee, W. S. & Alchanatis, Victor & Abd-
Elrahman, A. (2018). An Active Thermography
Method for Immature Citrus Fruit Detection.
[18] Rabby, Md Khurram Monir & Chowdhury, Brinta &
Kim, Jung. (2018). A Modified Canny Edge Detection
Algorithm for Fruit Detection & Classification. 237-
240. 10.1109/ICECE.2018.8636811.

493

uthorized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on May 21,2022 at 08:15:29 UTC from IEEE Xplore. Restrictions app
View publication stats