Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

Drone based Solid Waste Detection using Deep

Learning & Image Processing
Anjali Pradipbhai Anadkat , B V Monisha, Manasa Puthineedi, Ankit Kumar Patnaik, Dr. Shekhar R, Riyaz Syed*

Computer Science and Engineering , Aerospace Engineering*,Alliance University, Bangalore, India

Abstract— A significant cause of waste generation is littering. Whether intentional or unintentional,

littering has an adverse impact on the environment and it also costs municipalities millions of dollars
annually to clean up the waste. Traditional systems for waste management include regular waste
collection by assigned groups such as the municipal corporation trucks for cities, janitors for an
institution etc. Due to the shortcomings of existing waste monitoring and management systems, this
paper proposes an alternative and efficient approach for waste monitoring in large. This paper
presents a smart solid waste detection system which uses the motion of an Unmanned Aerial Vehicle
(UAV) such as drone for waste detection based on image processing techniques which help identify
places contaminated with waste and sends the location coordinates as a notification. This system uses
the concepts of deep learning, image stitching and image processing through feature extraction. In the
last section, we will discuss the future applications of this smart system which will tackle both
garbage detection and collection. Finally, the future scope and relevance of this system will be
discussed.

Keywords—solid waste detection, waste management, UAV, drone, image processing,

litter, deep learning
I. INTRODUCTION
Solid waste management has been a major concern for a long time. India produces 42.0 million
tons of Municipal Solid Waste (MSW) annually at present. According to a recent consensus, the per
capita waste generation in India is increasing at a rate of 1.3% per annum. Garbage generated
annually across the globe amounts to billions of tons today, with almost one lakh metric tons of
garbage generated in India per day. This can cause a major impact on the environment. With
urbanization in India, the scale of cities and the number of residents is rising and with it is increasing
the amount of urban waste generated [1][2][3].

Often the presence of litter in a given area results in the intentional throwing of litter at that particular
spot as it gives the impression that it is a right place for discarding the waste. Throwing kitchen
generated waste out of the window or chucking chewing gum wrappers along the roadside for
instance, without realizing the harmful effects it might have on the environment have become a
commonplace. It costs residents, local, state and federal governments millions of dollars to pick up
litter, reverse the effects of littering and prevent it. Municipal solid waste (MSW) management
includes collection, storage, transportation and disposal of solid waste. Poor collection and inadequate
transportation leads to heap of MSW at many places, which causes health and environmental
problems. Governments across the world are making efforts to improve solid waste management in
their respective countries. [4] The accumulated waste causes environmental problems such as the
spread of pest species and diseases, marine plastic pollution, damage to wildlife ecosystems and
toxicity in the land/waterways.

While most urban waste management systems in place are designed for regular collection of waste
from fixed locations at regular intervals, there are certain places (remotely located) which go
unnoticed and are therefore, not cleaned. Many solutions have been designed for effective garbage
management. There are applications available on the Google Play Store which allow the user to send
an alert message to the municipal corporation about the status of the garbage container. However, this
process is time-consuming as it relies on human discretion and action and communication via an
intermediary. A better solution designed for this problem is to use sensors for obtaining information

357
Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

about the level of garbage in containers and provide alert messages to the Garbage Collector Truck
about the level of waste in a garbage container. Nonetheless, the problem of waste accumulation in
remote places persists. Thus, it is crucial to detect and manage litter accumulation at remote and
undesired locations with little or no human interference.

Through extensive background study and data collection, it can be concluded that most of the current
automated waste management systems rely on IoT and hardware devices such as infrared sensor,
ultrasonic sensor, metal sensor, etc. for detection of garbage and subsequent communication of the bin
status. Other than this, there are waste sorting systems based on image processing which identify
metal materials or other special types of waste. However, these become dysfunctional in case of paper
and plastic products. Moreover, there are very few existing artificially intelligent systems which are
self-sufficient for waste classification. With a combined approach consisting of the hardware devices
such as a drone, Arduino, GPS and GSM module and image processing software algorithms, this
paper offers a better and innovative solution to the problem of effective waste detection and
management in wide and remote areas.
This system aims to achieve the following:
 To capture Ariel view images using Drone and further process the same to identify solid
waste disposed at inappropriate places
 To notify the respective person-in-charge for taking the necessary action to clean the
litter/solid waste
 Help maintain cleanliness in places such as beaches, institutions, cities etc.

II. BACKGROUND

A. UAV
Drones are Unmanned Aerial Vehicles (UAV) which operate without a human pilot aboard.
They can either be flown using a ground-based remote controller operated by a human or
autonomously by onboard computers. They are mostly used in military, commercial, scientific,
agricultural, peacekeeping and other applications. They have the ability to navigate through large
and remote places which security cameras otherwise fail to cover. The drone that will be used in this
project is an assembled flamewheel 450 mm model equipped with first-person-view camera. The
utilization of the drone will allow for real-time transmission of video image set to be analyzed by the
litter detection deep learning framework.

B. Image Stitching
Image Stitching is a technique used for attaining high-resolution panoramic image from multiple
images combined together. Image stitching techniques can either be direct intensity-based or feature
based. This project focuses on using feature-based image stitching algorithms such as Scale Invariant
Feature Transformation (SIFT) to determine a relationship between the images through distinct
features extracted from the processed images. Snapshots from the live video feed captured with the
help of the drone will be taken at regular intervals and stitched together for further analysis.

C. Deep Learning and Computer Vision

Concepts of Deep Learning and Computer Vision have been used in this project to achieve the
automated detection of litter. Deep learning is a subpart of Artificial Intelligence (AI) which is
concerned with emulating the learning approach that humans use to gain knowledge from data
patterns. In contrast to machine learning algorithms, deep which are supervised and linear, deep
learning algorithms are stacked in a hierarchy of increasing complexity and abstraction. The
advantage of deep learning is that the program builds the feature set by itself without any sort of

358
Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

human supervision and hence are generally faster and more accurate. A Deep Neural Network (DNN)
is a neural network with a certain level of complexity, they have multiple layers between the input
and output layers. The DNN finds the correct mathematical manipulation to turn the input into the
output. Deep learning within the field of Computer Vision is concerned with the automatic derivation
of useful information from visual data. Image Processing can be done using ‘TensorFlow’, a dataflow
framework developed by Google for high performance computation. Tensor Flow object recognition
algorithms are used to classify and identify arbitrary objects within larger panoramic images obtained
from multiple datasets. TensorFlow is well suited for deploying deep learning models.

D. Python
Python is a high-level object-oriented programming language with emphasis on code readability
and built-in libraries to provide various functionalities. Being a general purpose programming
language, it provides powerful implementations to facilitate large data and advanced calculations with
libraries such as TensorFlow, Scipy, Scikit-Learn, and NumPy. Tensorflow and Scikit-Learn are
libraries that provide various machine learning frameworks and algorithms, such as neural networks
and support vector machines. TensorBoard is a suite of visualization tools that make it easier to
understand, debug, and optimize TensorFlow programs. NumPy is a library for applying various
advanced mathematical functions with arrays and matrices, both key components of machine learning
algorithms. Scipy is the optimized core package for scientific routines in Python; it is meant to operate
efficiently on NumPy arrays, so that Numpy and Scipy work hand in hand. To use computer vision,
Python was also implemented with Scikit-Image and OpenCV. These libraries provide important
image manipulation tools computer vision techniques, such as feature extraction and image
classification.

E. Hardware Components

Arduino is a programmable controller which can be used for handling file transfer operations.
It can "talk", (transmit or receive data) via a serial channel, so any other device with serial capabilities
can communicate with an Arduino. The drone will be fixed with an arduino to facilitate the
interfacing of other hardware components such as a Global System for Mobile Communication
(GSM) and Global Positioning System (GPS) modules. The GSM can be integrated with the Arduino
to connect with the computer system over a network and communicate. A GPS module integrated
with the drone will fetch coordinates (latitude and longitude) of the places that the drone will fly
through.

F. Cloud Platform
A cloud platform allows for data storage, remote access and encapsulates various tools for data
analytics and machine learning. A cloud platform such as Amazon Web Services (AWS) Elastic
Cloud 2 (EC2) instance will be used to store the serially transmitted location coordinates (recorded by
the GPS module) sent via the GSM module integrated with the drone. The use of cloud is to allow
remote file transfer and further processing of the data recorded for comparison purposes.

III. EXISTING SYSTEMS

A. Waste Monitoring and Management Systems

In the past decade, many smart solutions for waste detection and management involving the use
of IoT have emerged. These systems use mechanisms to check for the status of garbage containers and
eventually indicate the same to the respective in-charge. RFID technology is used for collection of

359
Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

data regarding garbage container in [5]. RFID tag works to detect within the frequency range and
when any tag comes to the range of RFID reader, it automatically reads data from RFID reader, then
filters collected data and arranges it into specific formatted SMS. Subsequently, the data is sent to
central server sends the information to the web server as well as authorized person’s mobile phone. A
similar system design is described in [6] is which uses RFID technology to avoid overflow of the bin
by sending an alert message. It uses Arduino Uno R3 as a microcontroller for reading data from
sensors. When RFID tag interrupts the RFID reader, the sensor will check the status of the bin and
send it to the web server.

Another paper on waste management [7] uses ultrasonic sensor to detect the level of garbage in the
bin and communicates to control room using GSM system. Four IR sensors are used to detect the
level of the garbage bin. When the bin is full the output of the fourth IR is active low and this output
is given to microcontroller to send a message to control room through GSM. A similar system [8]
consists of waste bins equipped with ultrasonic sensors which are interfaced with Arduino Uno and a
Wi-Fi module which collect the waste fill level status and upload the data to database. This data
appears on the android application which notifies the appropriate collector client based on their
location once the bin gets filled up. [9] introduces an Android app, SpotGarbage that can
automatically detect and localize regions containing garbage in user-clicked unconstrained geo-tagged
real-world images. The app utilizes the proposed fully convolutional network, GarbNet for coarsely
segmenting image regions containing garbage and the locations of the images classified as garbage
are marked and plotted on Google Maps.

In this paper [10] ZigBee, GSM and ARM7 controller is used to monitor the garbage bin level. When
garbage bin is full, this message of garbage level is sent to ARM7 controller. Then ARM7 will send
the SMS through GSM to authority as to which bin is overflowing and requires cleaning up. [11] uses
a Raspberry Pi to capture images and performs image processing using edge detection algorithm to
identify waste. The detection is based on computer vision and is carried out with the help of camera
and opt couplers. Waste is then collected by a vacuum unit which takes all the garbage and cleans the
area. A related framework is described in [12] which consists of two parts, one of which is the
hardware platform with Raspberry Pi as the core module and the other is the software platform based
on SURF-BoW and multi-class SVM algorithm. This paper demonstrates the effectiveness of Bag of
Visual Words algorithm in image processing.

B. Surveillance Systems

Drones are unmanned aerial vehicles which often find their application in surveillance and
monitoring of wide areas and remote places which are otherwise inaccessible. [13] describes a smart
transport infrastructure system based on drones and feature extraction techniques to determine the
level of traffic congestion on roads and also indicates vacant parking slots in a particular area. This
system is an instance of the accuracy and feasibility of using drone based image processing to achieve
useful results. A similar approach is discussed in [14] where a custom-built drone is used to capture
images of large fields, crops in particular, to determine the type of soil present in a specific area.
Image processing is used for classification of images with reference to a predefined train dataset. This
paper suggests the possibility of using a GSM integrated drone for instant processing of captured
images on a system upon retrieval. Using a GSM would be beneficial as it has no range limit of
transmission which is useful for real-time processing. This paper [15] presents the design and
implementation of SkyStitch, a multi-UAV-based video surveillance system that supports real-time
video stitching implemented using commercial off-the-shelf hardware. It addresses two key design
challenges: (i) the high computational cost of stitching and (ii) the difficulty of ensuring good
stitching quality under dynamic conditions. Deriving valuable insights from the past works, we aim to
build a system which takes live video feed recorded by drone as input which can be retrieved by the
system in real-time for image processing and feature extraction to identify and classify images of

360
Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

garbage into various categories. The presence of a GSM (communication) and GPS (georeferencing)
module allows for real time communication and location tracking.

IV. PROPOSED SYSTEM

As part of the proposed solution, the focus of this project is to build a system which takes a live
video feed (or multiple images) captured by a drone as its inputs and extracts information from the
images in order to identify places contaminated with solid waste/litter. The drone would capture a wide
area such as a college campus or a locality and provide an aerial view of the location. The drone will be
made to transfer the visual data recorded by it to the local server which is a computer system
performing the processing operations. Arduino will be used for handling file transfer operations which
will allow for real time retrieval of captured images. GSM and GPS module placed over the drone
would provide the functionality of mobile communication and real time location referencing. With the
help of image stitching techniques, the live video feed will be stitched based on similar features
producing panoramic images of a place. This data can then be processed using Deep-Learning image
processing algorithms coded in Python. The predefined dataset would encompass images of litter such
as waste metal cans, bottles, crumpled paper, plastic bags etc. By testing the input data against the
classes/categories of the predefined dataset in Tensor Flow framework, the algorithm will produce an
output which assigns a distinct class to the given input image. The end goal is to send a notification
which contains geographic coordinates (GPS based location) of the place contaminated with garbage to
the respective person-in-charge prompting him/her to take the necessary action. The place coordinates
corresponding to the timestamp of the output image (where litter was detected) will be fetched and
subsequent message will be sent. The key components for this system include the following:
 Remote controlled Drone for surveillance and capturing images
 Arduino for managing the captured images and handling file operations
 Python IDE to run Python based programs
 Interfacing circuitry for making the connections
 GPS and GSM module for position tracking and communication respectively

Fig. 1. Working of the proposed garbage monitoring system

The general working of the proposed system containing the hardware and software components has
been shown above in fig 1.

V. EXPERIMENTAL PROCEDURE
The popular machine learning techniques such as SIFT and Bag of Visual Words (BoVW) have
been used extensively for image processing through feature extraction over the past decade.

361
Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

TensorFlow based algorithm on the other hand is relatively new as it was developed by Google
recently. Being specialized for numerical computation and large-scale machine learning, TensorFlow
provides a convenient front-end API for building applications with the framework. After performing
image processing using both BoVW (which includes SIFT for feature extraction and BoVW for
vocabulary formation) and TensorFlow, it was concluded that Tensor Flow is more efficient and
performs better among the two for image processing and classification. The algorithms were tested
using a sample dataset of images of trash taken from GitHub. The output images of TensorFlow
algorithm are shown in fig 2.

Fig 2: Output images of TensorFlow algorithm, displaying the classification of the given objects as
trash with the confidence percentage

In a similar manner, Image Stitching algorithm was implemented using sample aerial images taken
using first-person view (FPV) camera. OpenCV is an image and video processing library present in
Python with SIFT functionalities which are used for feature extraction and image classification. Based
on opencv_contrib’s SIFT descriptor, the images are stitched together into a single panorama. The
input images are shown in fig 3. Fig 4 shows the output of the image stitching algorithm which is a
stitched image. Image stitching is needed in order to stitch the images taken by the drone in order to
provide a 180 degree panoramic view of a specific location which will be used for detecting the
presence of waste in the location. Real-time implementation of the algorithms will be done using
images taken from the drone.

362
Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

Fig 3: Input aerial images (left, centre, right) taken from FPV camera

Fig 4: Stitched image output

VI. CONCLUSION
As this system is entirely automated, it will be efficient and applicable to all environments. The
use of a drone adds to effective monitoring of waste as it removes the need for humans to manually
travel to places for surveillance. This system offers the benefits of maintaining cleanliness in the
surroundings and the environment. Waste accumulation in undesirable places can be minimized, even
eliminated and waste management can be improved to a great extent. The implementation of this
system will also assist organizations in effectively managing waste in remote places.

However, despite its benefits, this project specifically focuses on waste monitoring while waste
disposal and cleaning remain out of its scope. Therefore, future works can be based on developing an
automated robotic system to handle the waste detected. A robotic garbage collector can be integrated

363
Alliance International Conference on Artificial Intelligence and Machine Learning (AICAAM), April 2019

with the drone. The collector can be essentially be programmed to receive inputs from the drone about
the type of garbage detected upon which the collector will clean up the litter, segregate the waste into
recyclable and non-recyclable materials. Disposal of this waste will also be looked after by the robotic
garbage collector. Such an integrated system would complete the entire process of waste detection and
collection single handedly without requiring human interference.

REFERENCES
[1] Sharma, S., Shah, K.W., 2005. Generation and disposal of solid waste in Hoshangabad. In: Book of
Proceedings of the Second International Congress of Chemistry and Environment, Indore, India, pp. 749–751.

[2] Central Pollution Control Board (CPCB), 2004. Management of Municipal Solid Waste. Ministry of
Environment and Forests, New Delhi, India

[3] Shekdar, A.V., Krshnawamy, K.N., Tikekar, V.G., Bhide, A.D., 1992. Indian urban solid waste management
systems – jaded systems in need of resource augmentation. Journal of Waste Management 12 (4), 379–387.

[4] Arti Pamnani, Meka Srinivasarao, “Municipal Solid Waste Management in India: a review and some new
results”, vol 5, Issue 2, February (2014), pp. 01-08

[5] Dr.N.Satish Kumar, B.Vijayalakshmi, R. Jenifer Prathana, A.Shankar “IOT Based Smart Garbage alert
system using Arduino UNO”, published in Region 10 Conference (TENCON) on 22-25 Nov 2016.ISBN 978-1-
5090-0751-6/16/2016 IEEE.

[6] Trushali S. Vasagade, Shabanam S. Tamboli, Archana D. Shinde “Dynamic Solid Waste Collection and
Management System Based On Sensors, Elevator and GSMA” in International Conference on Inventive
Communication and Computa-tional Technologies on 17 April 2017

[7] Ruhin Mary Saji, Drishya Gopakumar, Harish kumar, Lakshmi “A Survey on Smart Garbage management in
cities using IOT” in international journal of enginnering and computer science ISSN: 2319-7242 on 11 Nov
2016.

[8] Vaibhav E. Pawar, Mustafa Bhatkar , Omkar Jadhav , Amey Mhatre “Smart Garbage Monitoring System
Using IoT”, Vol 8 Issue 4, IJESC journal

[9] Gaurav Mittal, Kaushal B. Yagnik, Mohit Garg, and Narayanan C. Krishnan, “SpotGarbage: Smartphone
App to Detect Garbage Using Deep Learning”, UBICOMP '16, SEPTEMBER 12–16, 2016, HEIDELBERG,
GERMANY

[10] Pavithra, “Smart Trash system: An Application using ZigBee”, IJISET - International Journal of
Innovative Science, Engineering & Technology, Vol. 1 Issue 8, October 2014

[11] Samruddhi There, Chetan Shinde, Ashish Kumar Nath, Shubhangi A Joshi “Garbage Detection And
Collection Of Garbage Using Computer Vision”

[12] Yijian Liu, King-Chi Fung, Wenqian Ding, Hongfei Guo, Ting Qu & Cong Xiao “Novel Smart Waste
Sorting System based on Image Processing Algorithms: SURF-BoW and Multi-class SVM”

[13] G.Maria, E.Baccaglini, D.Brevi, M.Gavelli, R.Scopigno “A drone-based image processing system for car
detection in a smart transport infrastructure” Proceedings of the 18th Mediterranean Electrotechnical
Conference MELECON 2016, Limassol, Cyprus, 18-20 April 2016

[14]“Drone based image processing through feature extraction”, 2017 2nd IEEE International Conference on
Recent Trends in Electronics Information & Communication Technology (RTEICT), May 19-20, 2017, india

[15] Xiangyun Meng, Wei Wang and Ben Leong, “SkyStitch: a Cooperative Multi-UAV-based Real-time Video
Surveillance System with Stitching”, 2016

364