Waste Management 142 (2022) 29–43

Contents lists available at ScienceDirect

Waste Management
journal homepage: www.elsevier.com/locate/wasman

Computer vision for solid waste sorting: A critical review of

academic research
Weisheng Lu, Junjie Chen *
Department of Real Estate and Construction, The University of Hong Kong, Pokfulam Road, Hong Kong, China

A R T I C L E I N F O A B S T R A C T

Keywords: Waste sorting is highly recommended for municipal solid waste (MSW) management. Increasingly, computer
Municipal solid waste vision (CV), robotics, and other smart technologies are used for MSW sorting. Particularly, the field of CV-
Waste sorting enabled waste sorting is experiencing an unprecedented explosion of academic research. However, little atten­
Computer vision
tion has been paid to understanding its evolvement path, status quo, and prospects and challenges ahead. To
Image recognition
Machine learning
address the knowledge gap, this paper provides a critical review of academic research that focuses on CV-enabled
Deep learning MSW sorting. Prevalent CV algorithms, in particular their technical rationales and prediction performance, are
introduced and compared. The distribution of academic research outputs is also examined from the aspects of
waste sources, task objectives, application domains, and dataset accessibility. The review discovers a trend of
shifting from traditional machine learning to deep learning algorithms. The robustness of CV for waste sorting is
increasingly enhanced owing to the improved computation powers and algorithms. Academic studies were un­
evenly distributed in different sectors such as household, commerce and institution, and construction. Too often,
researchers reported some preliminary studies using simplified environments and artificially collected data.
Future research efforts are encouraged to consider the complexities of real-world scenarios and implement CV in
industrial waste sorting practice. This paper also calls for open sharing of waste image datasets for interested
researchers to train and evaluate their CV algorithms.

1. Introduction construction and demolition (C&D) activities. A typical MSW manage­

ment cycle is comprised of waste generation, collection, treatment, and
Rapid industrialization has caused the skyrocketing increase of disposal as appropriate (Hoornweg and Bhada-Tata, 2012).
municipal solid waste (MSW) generation in the past few decades. In Waste sorting is a practice highly recommended in MSW manage­
2016, for example, 2.01 billion tonnes of MSW were generated globally, ment (Wang et al., 2020a; Xia et al., 2021). The term “waste sorting”
which was projected to surge to 2.59 billion tonnes by 2030 (Kaza et al., usually appears along with two other closely related terminologies:
2018). The definition and composition of MSW vary from case to case. “waste segregation” and “waste separation”. While there is no universal,
This paper accepts the definition used by the 2012 World Bank report explicit definition, it is widely accepted in the field of waste manage­
(Hoornweg and Bhada-Tata, 2012), in which MSW encompasses a wide ment that: (a) waste segregation refers primarily to the grouping of
range of waste generation sources, e.g., residential and municipal ser­ waste materials into different categories when they are generated at
vices (RM), industrial, commercial and institutional sources (ICI), and source (e.g., households, workplace, or construction sites) or at the point

Abbreviations: AI, Artificial intelligence; ANN, Artificial neural network; C&D, Construction and demolition; CCD, Charge-coupled device; CMOS, Complementary
metal oxide semiconductor; CNN, Convolutional neural network; CV, Computer vision; DCNN, Deep convolutional neural network; DL, Deep learning; DT, Decision
tree; ELVs, End-of-life vehicles; GLCM, Gray level co-occurrence matrix; GPC, Gaussian process classification; HOG, Histogram of oriented gradient; HSI, Hyper­
spectral imaging; ICI, Industrial, commercial and institutional; LDA, Linear discriminant analysis; mAP, Mean average precision; ML, Machine learning; MLP,
Multilayer perceptron; MSW, Municipal solid waste; NIR, Near-infrared spectroscopy; NN, Nearest neighbor; PCA, Principal Component Analysis; R-CNN, Region-
based convolutional neural network; RGB, Red, green, and blue; RM, Residential and municipal services; RP, Region proposals; SIFT, Scale-invariant feature
transform; SSD, Single shot multibox detector; SVM, Support vector machine; TACO, Trash annotation in context; TL, Transfer learning; UGV, Unmanned ground
vehicle; VGG, Visual geometry group; WM, Waste management; WoS, Web of Science; YOLO, You only look once.
* Corresponding author.
E-mail address: [email protected] (J. Chen).

https://doi.org/10.1016/j.wasman.2022.02.009
Received 12 August 2021; Received in revised form 12 December 2021; Accepted 6 February 2022
Available online 13 February 2022
0956-053X/© 2022 Elsevier Ltd. All rights reserved.
W. Lu and J. Chen Waste Management 142 (2022) 29–43

of collection/dumping (Christensen and Matsufuji, 2011); (b) waste classification algorithms can be enhanced by feeding DL model with a
sorting, used interchangeably with waste separation, can either occur massive amount of visual data captured in different environments. The
manually at source (Wang et al., 2020a), or be implemented in a rela­ potential benefits of DL drove the surge of relevant research efforts.
tively centralized place (Gundupalli et al., 2017b). In this research, we There have been reviews on the applications of artificial intelligence in
accept this conventional definition, and use the term “waste sorting” to waste management (WM) (Abdallah et al., 2020) or on the general topic
encompass the waste separation behaviors occurring both at source and of automated waste sorting (Gundupalli et al., 2017b). However, to the
in centralized treatment facilities. best of the authors’ knowledge, no previous study has focused specif­
When the waste is generated at source, waste generators are ically on the applications of CV in waste sorting.
encouraged to separate their waste according to its type (Christensen This research aims to conduct a comprehensive literature review of
and Matsufuji, 2011; Gundupalli et al., 2017b), e.g., “wet” food waste academic studies concerning the evolvement path, status quo, and
and “dry” recyclables for household residue in Shanghai (Zuo and Yan, prospects and challenges of applying CV for waste sorting. The primary
2019), inert and non-inert materials for construction waste in Hong focus is the processing of RGB images captured by ordinary digital
Kong (Lu et al., 2015; Lu and Yuan, 2021), or separate collection bins cameras, as it is the area where most of the recent progress has been
used for different waste types in public commercial areas (Keramitso­ achieved, and represents a major trend of future development. The re­
glou and Tsagarakis, 2018). However, with the increasing complication view sets out to address the following questions: Firstly, what are the
of waste taxonomies, it becomes more and more difficult for both citi­ main CV algorithms for waste sorting? How are the DL algorithms
zens and regulators to distinguish among different MSW materials (Chen different from the traditional ones, and what makes them surpass the
et al., 2021). The application of computer vision (CV) can potentially others to drive the recent publication boom? Secondly, what is the
assist the sorting of MSW at source or when it is collected. With sufficient current status of academic research? How is previous research output
data, it is viable to train a CV model to identify various MSW materials. distributed over different application domains and waste generation
Applied to smart devices such as mobile phones, the CV model can then sectors? How was the performance evaluated, and how has it evolved
help users determine the types of their generated waste materials for over the years? Thirdly, by a comprehensive review, what lessons can
proper classification. The CV model can even be deployed to robotic we draw from previous research efforts? What challenges and issues
platforms such as unmanned ground vehicles (UGV), enabling automatic shall we foresee in the future?
waste collection in indoor built environments (Paulraj et al., 2016) or The rest of this paper is organized as follows. Subsequent to this
open construction sites (Wang et al., 2020b; Wang et al., 2019b). introductory section is Section 2 to delineate the material search, se­
Waste sorting can be conducted in a relatively centralized place as lection, and some preliminary analyses. Section 3 is a detailed analysis
well. There, the collected solid waste is transported by a conveyor belt of CV algorithms for waste sorting, and Section 4 is an analysis of
through a series of sorting machinery and robots (Faibish et al., 1997; research outputs from four aspects: waste sources, task objectives,
Gundupalli et al., 2017b; Huang et al., 2010; Mattone et al., 2000). application domains, and dataset accessibility. Section 5 presents the
During this process, two lines of approaches, i.e., direct and indirect prospects and challenges of CV for waste sorting, and conclusions are
sorting, are adopted (Gundupalli et al., 2017b; Huang et al., 2010). drawn in Section 6.
Direct sorting separates waste materials directly by applying forces such
as gravity, magnetic force, or manual picking. In contrast, indirect 2. Bibliographic material collection
sorting first uses sensors (e.g., optical sensor, spectroscope, inductive
sensors, and thermal camera) to detect specific types of waste materials, The materials included in this review were confined to academic
and then sorts the detected materials with machinery or robots (Gun­ literature in English and published between 1997 and 2021. We used
dupalli et al., 2017b; Huang et al., 2010). The potential of using CV for two keyword combinations: “waste sorting AND computer vision” and
indirect waste sorting has long been acknowledged. For example, “waste sorting AND image recognition”, to search for relevant publica­
Faibish et al. (1997) presented a robotic system with stereo vision to tions on the Web of Science (WoS) platform, as they directly reflect the
detect and separate paper objects for recycling. Mattone et al. (2000) two main themes of the review topic, i.e., “waste sorting” and “computer
formalized the problem of sorting items on a moving conveyor belt, and vision”. The searching results were manually screened to exclude irrel­
provided a solution based on optical devices. Compared with other evant papers. After the suitability check, the remaining records were
sensing techniques such as hyperspectral imaging (HSI) and X-Ray, vi­ merged, which resulted in a total of 17 papers. Many other studies might
sual sensors, e.g., CCD (charge-coupled device) cameras, are cost- have been left out as the initial search was only confined to the WoS
effective, easy for maintenance, and versatile for a wide range of database with two keyword combinations. To find the missing literature,
waste (Rahman et al., 2014; Zulkifley et al., 2014). While traditional references cited by the 17 papers were successively checked out to find
approaches rely more on huge investments on expensive sensing hard­ other articles that have addressed the review topic. This “snowball”
ware (e.g., HSI cameras), sorting based on CV only requires simple technique drastically expanded the literature collection to 86 publica­
installation of RGB cameras and harnesses the power of algorithms for tions, which consist of 51 journal articles, 32 conference papers, and 3
waste material detection. In such waste sorting systems, CV serves as the reports/preprints. The expanded literature collection includes not only
“eye” and “brain”, which is used to detect and identify waste materials publications from other notable databases such as Scopus, but also many
on the conveyor belts, enabling robots to execute sorting operations conference papers that are highly regarded, particularly in computer
autonomously. science.
Despite the promising prospects, the role of CV in MSW sorting had Based on the collected literature, the evolvement of research pro­
been limited and remained relatively marginal for a long time. The ductivity over the past two decades was analyzed. As shown by Fig. 1
sluggish development is attributable to the tremendous manual efforts (a), the academic research outputs have experienced an explosion since
required for feature handcrafting and the relatively low robustness in 2016, when the historic triumph of AlphaGo against Lee Sedol further
the early stage. The situation has been improved with the development promoted the concept of DL to the general public. It is observed that the
of deep learning (DL), signaled by AlexNet’s great success in the number of publications related to DL presents an upward trend while the
ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012 numbers of those using other CV algorithms gradually shrink. Note that
(Krizhevsky et al., 2012). The DL techniques can address the limitations the number in 2021 only takes into account literature published before
of traditional CV algorithms via two aspects. Firstly, with DL’s power in July of the year, when this review paper is drafted.
feature extraction, the tiresome feature handcrafting can be avoided Fig. 1 (b) and (c) shows composition of the literature collection by
since visual features displayed by different waste types can be auto­ publication names/types and countries. As shown in Fig. 1 (b), the most
matically learned from big data. Secondly, the robustness of waste productive journals during the past 20 years are Waste Management and

30
W. Lu and J. Chen Waste Management 142 (2022) 29–43

Fig. 1. (a) Yearly number of publications of CV in waste sorting, where DL represents papers that have used or partially used deep learning, and OT represents papers
that used other CV algorithms; Distribution of the literature collection (b) over different publication types/names and (c) over different countries.

Resources, Conservation and Recycling, which have published 14 papers in need to be fed with hand-engineered features, e.g., HOG and intensity
total. As shown in Fig. 1 (c), the three leading countries in the number of histogram. Table 1 listed prevalent traditional ML algorithms used by
relevant publications are, respectively, Malaysia, China, and India, previous studies, which include linear discriminant analysis (LDA),
which, interestingly, are all developing countries. This might be related nearest neighbor (NN), decision tree (DT), Bayesian network, artificial
to the rising concerns on the increasing waste generation brought by the neural network (ANN), support vector machine (SVM), and rule-based
rapid economic development in these countries (Bao et al., 2019, 2021). classifier.

3. Computer vision algorithms for waste sorting 3.1.1. Linear discriminant analysis
LDA is a frequently used linear classification algorithm. Given a
Many CV algorithms have been used for waste sorting, which in­ bunch of data samples, the algorithm seeks to optimize a linear mapping
volves image preprocessing (e.g., denoising, thresholding, and seg­ from the original high dimensional feature space to a lower-dimensional
mentation), feature extraction such as scale-invariant feature transform subspace, based on which a classifier can then be designated to separate
(SIFT), histogram of oriented gradient (HOG) and principal component the samples into different classes (Leitner et al., 2003). Originated from
analysis (PCA), and machine learning (ML) algorithms for classification. Fisher’s research on dichotomous discriminant analysis in 1936, LDA
Among all, ML algorithms are a critical component, for it directly affects has been developed into many variants and extensions, e.g., multiple
the accuracy and efficiency of waste classification. This section sum­ discriminant analysis and quadratic discriminant analysis. As indicated
marizes the basic rationales and application scenarios of prevalent ML by Table 1, six articles have used LDA as classifiers in their studies for the
algorithms, and compares their performance in recognizing different recognition/detection of RM and ICI wastes. The prediction perfor­
waste materials. Note that although computation time is an important mance of the algorithm varies with the specific types of waste, task
part for performance evaluation, it has not been reported by most objectives, and used features, ranging from as low as 53% to over 98%.
studies; thus, accuracy is used as the primary evaluation metric here.
3.1.2. Nearest neighbor
3.1. Traditional ML algorithms NN is a non-parametric ML algorithm that does not make strong
assumptions about the distribution of the mapping between the input
Before the emergence of DL, research efforts sought to use traditional variables and the output class labels (Brownlee, 2016). The rationale of
ML models to classify different waste materials from visual data. These NN algorithm is straightforward: given a set S of sample points from
models usually have simple structures, and lack the ability to automat­ different classes in the feature space and a query point q with unknown
ically extract high-level representations from raw images; thus, they class, the algorithm iteratively calculates the distance (Euclidean or

31
W. Lu and J. Chen Waste Management 142 (2022) 29–43

Table 1
Summary of research based on traditional ML techniques.
No. Algorithms Num. of studies Study Waste materials Task*(R/D) Accuracy

1–1 Linear discriminant analysis 6 Faibish et al. (1997) RM R 90%

1–2 Leitner et al. (2003) RM D 53.89%; 88.16%
1–3 Tachwali et al. (2007) RM R 94.14%
1–4 Ramli et al. (2008) RM R 88.15%; 98.3%
1–5 Koyanaka and Kobayashi (2010) ICI R 88%
1–6 Koyanaka and Kobayashi (2011) ICI R 85%
2–1 Nearest neighbor 11 Faibish et al. (1997) RM R < 90%
2–2 Leitner et al. (2003) RM D 93.41%
2–3 Salmador et al. (2008) RM D /
2–4 Scavino et al. (2009) RM R 90%
2–5 Rahman et al. (2011) RM R 90%; 93%
2–6 Arebey et al. (2012) RM D 97.67%
2–7 Hannan et al. (2012) RM D 89.14%
2–8 Kuritcyn et al. (2015) C&D R 84.8%
2–9 Gundupalli et al. (2017a) RM D 96%; 94%; 85%; 90%
2–10 Gundupalli et al. (2018) RM D 96%; 84%; 87%; 93%
2–11 Wang et al. (2019a) ICI R 96%
3–1 Decision tree 4 Tachwali et al. (2007) RM R 92%
3–2 Kuritcyn et al. (2015) C&D R 88%
3–3 Shaukat et al. (2016) ICI R 98.15%
3–4 Wang et al. (2019a) ICI R 94%
4–1 Bayesian network 5 Brisola et al. (2010) C&D R 73.96%
4–2 Liu et al. (2010) RM R /
4–3 Gokyuu et al. (2011) C&D R 77.2%
4–4 Zulkifley et al. (2014) RM R 79%
4–5 Kuritcyn et al. (2015) C&D R 44.8%
5–1 Artificial neural network 8 Faibish et al. (1997) RM R < 90%
5–2 Mattone et al. (2000) RM R 97%
5–3 Scavino et al. (2009) RM R 95%
5–4 Ramli et al. (2010) RM R 98.8%
5–5 Koyanaka and Kobayashi (2011) ICI R 85%
5–6 Arebey et al. (2012) RM D 87.03%
5–7 Hannan et al. (2012) RM D 90.19%
5–8 Islam et al. (2014) RM D 98.5%
6–1 Support vector machine 10 Nawrocky et al. (2010) RM R 96%
6–2 Aziz et al. (2015) RM D 99.4%
6–3 Özkan et al. (2015) RM R 96%
6–4 Kuritcyn et al. (2015) C&D R 96.5%
6–5 Guttormsen et al. (2016) ICI R 82.5%; 93.4%; 94.5%; 100%
6–6 Paulraj et al. (2016) RM R 94.3%
6–7 Singh et al. (2017) RM R 72%
6–8 Aziz et al. (2018) RM D 99.73%
6–9 Wang et al. (2019a) ICI R 96.64%
6–10 Wang et al. (2019c) RM D 94.7%
7–1 Rule-based classifier 5 Mattone et al. (1998) RM / /
7–2 Mattone et al. (2000) RM R /
7–3 Rahman et al. (2009a) RM R 90.7%
7–4 Pothula et al. (2015) ICI D 97.76%
7–5 Zhu et al. (2018) ICI D 93%
*
The column indicates the task objective a study intended to tackle, which includes two options: (a) R (recognition) is to recognize category of a single waste in an
image; (b) D (detection) is to recognize and locate multiple waste items in an image.

other distance metrics) between q and all the points in S, and designates significant, with an accuracy of near or over 90% in all the studies.
the class label of the closest point in S as the class of q. A variant of NN
algorithm is k-NN, which sets out to find the k closest points, and then 3.1.4. Bayesian network
decide the class label by majority vote. The number of studies based on A Bayesian network refers to a probabilistic graphical model that
NN surpasses all the other traditional ML algorithms. Despite the exploits the causal-effect relationship between variables. In the specific
simplicity of the algorithm, it performed quite well in existing studies, case of waste classification, a Bayesian network gives the conditional
most of which have attained an accuracy of over 85%. probability of a sample belonging to a type of waste with its given visual
features, e.g., color (Gokyuu et al., 2011; Liu et al., 2010; Zulkifley et al.,
3.1.3. Decision tree 2014), shape such as area ratio and aspect ratio (Gokyuu et al., 2011),
DT algorithm transforms given data samples (i.e., the training set) and texture such as gray level co-occurrence matrix (GLCM) (Xiao et al.,
into a representation of a tree structure consisting of nodes, branches, 2020) and Jet (Liu et al., 2010). Naïve Bayes classifier is a subset of
and leaves. A node represents a feature, and a leaf represents a class Bayesian networks, which presumes the strong independence between
label; the branches stretching out from the nodes represent the specific features. The model has primarily been used for C&D and RM waste
values of corresponding features. DT is easy and straightforward to sorting, and the resulting accuracy is <80%.
interpret and explain by human beings (Tachwali et al., 2007). Random
forest is a variant algorithm of DT, which intends to address the pref­ 3.1.5. Artificial neural network
erence of DT for overfitting to the training set. Although the number of An ANN learns the input–output mapping via a structure that mimics
studies that have applied DT is relatively small, its performance is the biological neural networks (Abdallah et al., 2020; Guo et al., 2021).

32
W. Lu and J. Chen Waste Management 142 (2022) 29–43

One of the earliest ANN models is a single-layer perceptron, which is various waste materials, ranging from plastics to metal waste from the
only capable of learning linear separable patterns. A more competent automobile industry. Many studies based on ANN achieved a classifi­
and widely-used ANN structure is multilayer perceptron (MLP), which cation accuracy of more than 95%.
consists of an input layer, one or multiple hidden layers and an output
layer. With such a multilayer structure and the use of non-linear acti­ 3.1.6. Support vector machine
vation functions (e.g., sigmoid and tanh), MLP can distinguish data that SVM can produce significant classification accuracy with a small
is not linearly separable. ANNs have been used for the classification of amount of training data. The objective of SVM training is to find the

Table 2
Summary of research based on end-to-end deep learning.
No. Network Num. of studies Study Waste materials Task a TLb? Accuracy
(R/D) (Y/N)

1–1 AlexNet 6 Mittal et al. (2016)c RM D Y 87.69%

1–2 Sakr et al. (2016) RM R N 83%
1–3 Yang and Thung (2016) RM R N 22%
1–4 Singh et al. (2017) RM R / 87%
1–5 Vrancken et al. (2019) RM R Y 77%
1–6 Bobulski and Kubanek (2019) RM R / 96.41%
2–1 MobileNet 4 Rabano et al. (2018) RM R Y 87.2%
2–2 Bircanoğlu et al. (2018) RM R Y 76%
2–3 Aral et al. (2018) RM R Y 84%
2–4 Srinilta and Kanharattanachai (2019) RM R Y ~88%
3–1 ResNet 6 Bircanoğlu et al. (2018) RM R Y 75%
3–2 Srinilta and Kanharattanachai (2019) RM R Y 91.3%
3–3 Lau Hiu Hoong et al. (2020)d C&D R N 97%
3–4 Huang et al. (2020) RM R Y 86.1%
3–5 Meng and Chu (2020) RM R Y 91 ~ 95%
3–6 Zhang et al. (2021) RM R Y 95.87%
4–1 Inception 3 Bircanoğlu et al. (2018) RM R Y 90%
4–2 Aral et al. (2018) RM R Y 94%
4–3 Huang et al. (2020) RM R Y 80
5–1 DenseNet 5 Bircanoğlu et al. (2018) RM R Y 95%
5–2 Aral et al. (2018) RM R Y 95%
5–3 Srinilta and Kanharattanachai (2019) RM R Y ~89.9%
5–4 Huang et al. (2020) RM R Y 82.2%;
88.6%;
84.2%
5–5 Mao et al. (2021) RM R Y 99.6%
6–1 Xception 3 Bircanoğlu et al. (2018) RM R Y 85%
6–2 Aral et al. (2018) RM R Y 80%
6–3 Huang et al. (2020) RM R Y 84.7%
7–1 VGG 3 Sun et al. (2019) ICI D Y 95 ~ 100%
7–2 Srinilta and Kanharattanachai (2019) RM R Y ~88%
7–3 Huang et al. (2020) RM R Y 89.7%
8–1 R-CNN 1 Ku et al. (2021) C&D D N 90%
9–1 Fast R-CNN 1 Chen et al. (2017) RM D / FNR: 3%;
FPR: 9 %e
10–1 Faster R-CNN 3 Awe et al. (2017) RM D Y 68.3%
10–2 Wang et al. (2019b) C&D D Y mAPf: 0.891
10–3 Nowakowski and Pamuła (2020) RM D / 93.3%;
96.7%
11–1 Mask R-CNN 2 Proença and Simões (2020) RM D / mAPf: 0.194
11–2 Wang et al. (2020b) C&D D Y mAPf: 0.937
12–1 RetinaNet 1 Panwar et al. (2020) RM D Y mAPf: 0.814
13–1 Others g 10 Sudha et al. (2016) / R N 65 ~ 70%
13–2 Rad et al. (2017) RM D Y 62%;
69%
13–3 Anjum and Umar (2018) RM D / Score 4.1 out of 5.0
13–4 Sreelakshmi et al. (2019) RM R N 95.7%; 96.3%
13–5 Liu et al. (2019) RM R / 83.87%
13–6 Kim et al. (2019) RM R / 96%
13–7 Vo et al. (2019) RM R T 94%;
98%
13–8 Yang and Li (2020) RM R Y 96.1%
13–9 Meng and Chu (2020) RM R Y 93.75%
13–10 Liang and Gu (2021) RM D Y 81.5%
a
The column indicates the task objective a study intended to tackle, which includes two options: (a) R (recognition) is to recognize category of a single waste in an
image; (b) D (detection) is to recognize and locate multiple waste items in an image.
b
TL stands for transfer learning.
c
The study only aimed at detecting curb-side garbage in a general sense, instead of classifying it into specific waste types.
d
The study aimed at determining the composition of recycled aggregates.
e
FNR and FPR stand for false negative rate and false positive rate, respectively; the measurements in the study evaluate the accuracy of robotic sorting based on
waste detection results.
f
mAP stands for mean average precision, which is frequently used to evaluate object detection performance.
g
Other network structures, such as Capsule-Net, OverFeat-GoogLeNet, and some self-designed CNNs, etc.

33
W. Lu and J. Chen Waste Management 142 (2022) 29–43

optimal hyperplane that can best separate the data samples of different on the same datasets and manually checking out the level of difficulties
classes. The optimal hyperplane is the one that has the maximum dis­ of relevant datasets, a general pattern can be identified: ResNet,
tance from the nearest points of all classes (support vectors) (Rogers and Inception, DenseNet, and VGG tend to yield higher performance. In a
Girolami, 2016). Despite its original focus on linear binary classification general sense, MobileNet performed poorer than the others; however,
problems, SVM has been widely extended to address multi-classification the model requires less computation power and hence might be a pref­
problems with non-linear hyperplanes by the strategy of one-against-all erable choice in actual industrial deployment.
and kernel operations. SVM is a prevalent technique in CV-based waste The training of DL models relies on a massive amount of data, which
classification. The attained performance is remarkable: all but one of the is usually difficult to collect or have access to in the field of WM. A
studies have achieved an accuracy of over 90%. common practice to address the problem is the use of transfer learning
(TL), a technique that exploits the model structure and parameters
3.1.7. Rule-based classifier learned from the source domain, and adapts them to a new domain
Rule-based classifiers are a type of classification models that decide where only a few data are needed for parameter fine-tuning (Goodfellow
the class of a given example by following a set of “if … then …” rules. et al., 2016). The technique has been used by most of previous studies.
While in some studies (Pothula et al., 2015; Rahman et al., 2009a; Zhu
et al., 2018), the rules were specified as the relationship of a numerical 3.2.2. The R-CNN series and other object detection networks
feature regarding a given threshold (e.g., “if the aspect ratio of the re­ Based on features extracted by the backbones, more advanced tasks
gions of interest is over a threshold, then it is likely to be a slim bottle”), can be accomplished. Object detection is one such task, aiming not only
others (Mattone et al., 2000; 1998) set rules based on empirical obser­ to determine if given images contain objects of interest, but also locate
vations by employing Fuzzy techniques to translate mathematical the objects on images with bounding boxes (Zou et al., 2019). Some CV
numbers to natural languages. tasks such as semantic/instance segmentation go even further to extract
pixel areas corresponding to the objects (Garcia-Garcia et al., 2017). In
3.2. Deep learning algorithms industrial practice, the waste streams are usually in a highly cluttered
state where different materials are overlapped with each other. There­
DL harnesses the power of big data via its deep structure to enable fore, compared with waste classification, waste object detection repre­
the so-called “end-to-end” training. In a typical DL architecture, original sents a more promising research direction by pinpointing the specific
raw images, instead of the handcrafted features, are directly fed to the locations of wastes on images. Row 8-X to 12-X in Table 2 show 5
network consisting of multiple convolutional, pooling and fully- prevalent object detection networks in the field of waste sorting.
connected layers, through which hidden features from the images can R-CNN (region-based convolutional neural network) (Girshick et al.,
be automatically learnt and extracted, and finally be used to predict the 2014) uses an algorithm called “selective search” to find candidate re­
class label. The end-to-end learning mechanism avoids the tiresome gion proposals (RP) from the original image. As the feature extraction is
process of feature handcrafting and thus greatly expands the applica­ repeated many times for all the RP on the image, R-CNN does not
bility of CV-enabled waste sorting. In addition, with a dataset encom­ perform well regarding efficiency (Ku et al., 2021). A series of algo­
passing a wide range of waste samples, the resulting DL models tend to rithms have been developed based on R-CNN. Fast R-CNN improves the
be more robust than those trained with traditional ML algorithms. A original R-CNN in a sense that, instead of extracting features separately
convolutional neural network (CNN) is a state-of-the-art DL algorithm for each individual RP, it produces a unified feature map from the input
that has been successfully applied in a broad spectrum of CV-related image and detect candidate regions from the feature map (Girshick,
tasks (Yang et al., 2020, 2021). Fig. S1 in the Supplementary Material 2015). The improvement significantly reduces the required time to
shows the structure of VGG-16, a well-known CNN architecture with 16 process each image. In Chen et al. (2017), Fast R-CNN was employed to
hidden layers proposed by the Visual Geometry Group (VGG). Table 2 detect and locate waste objects on conveyor belts, which demonstrated a
summarizes previous research that has applied DL in CV-enabled waste false negative rate (FNR) of 3%, a false positive rate (FPR) of 9%, and an
sorting according to their used CNN architectures. computation efficiency of 0.22 s/image. Ren et al. (2015) proposed
Faster R-CNN, for which the computation time is reduced to sub-second
3.2.1. Prevalent backbone networks level. Awe et al. (2017), Wang et al (2019b), and Nowakowski and
In deep learning, a backbone network refers to a CNN structure used Pamuła (2020) applied Faster R-CNN for the detection of RM waste,
to extract features from the input images. On top of the backbone net­ C&D waste, and electronic waste, respectively, which achieved desired
works, the extracted features can be leveraged by subsequent structures performance. Another variant of R-CNN is Mask R-CNN, which, as
to accomplish various tasks. One of such tasks is waste recognition, compared to Faster R-CNN, has an additional branch to extract pixels
which only aims to tell if given images belong to one of the pre­ corresponding to each individual instance of the detected objects (Pro­
determined categories. In such cases, the backbone networks are usually ença and Simões, 2020; Wang et al., 2020b).
followed by a fully connected layer and a Softmax activation layer to There is another stream of networks called single-stage detectors that
output a vector indicating the probabilities of the input being certain treat objection detection as a single regression problem, e.g., YOLO (You
waste materials, e.g., Mao et al. (2021), Yang and Thung (2016), and Only Look Once), SSD (Single Shot MultiBox Detector), and RetinaNet.
Meng and Chu (2020). By integrating patch-based classification or Among them, RetinaNet surpasses the accuracy of many two-stage de­
sliding window, the extracted features can be directly used to locate tectors such as the R-CNN series while still maintains its advantages on
wastes on images as well (Anjum and Umar, 2018; Mittal et al., 2016). efficiency (Lin et al., 2017). Panwar et al. (2020) trained their waste
Rows 1-X to 7-X in Table 2 list backbone networks that are frequently detection model based on RetinaNet to identify contaminants in water
used in waste classification. The number of researches based on AlexNet body, which reached a mean average precision (mAP) of 0.814. Defined
and ResNet exceeds all the other backbone architectures. For AlexNet, as the mean of average precision for each class, mAP is a popular metric
the obtained accuracy varies from case to case, ranging from as low as for object detection evaluation. There are also studies that devised new
22% for the classification of six common MSWs (Yang and Thung, 2016) CNN frameworks according to the domain-specific problem of waste
to the significant 96.41% for the binary classification of plastics/paper sorting. Liang and Gu (2021) developed a CNN-based multi-task
(Bobulski and Kubanek, 2019). As for ResNet, all but one studies achieve learning architecture in order to simultaneously classify and locate
classification accuracy higher than 85%. Direct comparison among the wastes in contexts. The architecture integrated components such as the
reported accuracies is not recommended, as their performances might attention mechanism, multi-level feature pyramids, and joint learning
have been evaluated on different datasets and some datasets might be sub-networks, and reached a mAP of 0.815.
more challenging than the others. However, by comparing studies based

34
W. Lu and J. Chen Waste Management 142 (2022) 29–43

Table 3 presumably higher performance can be obtained. Adedeji and Wang

Summary of research integrating CNN with traditional ML models. (2019) employed such a technique, which extracted features learnt by
No. Study Extractor Classifier Waste Task* Accuracy the ResNet-50 and performed waste classification with SVM. With the
materials (R/D) continuous improvement of CNN, the advantages of such “CNN + SVM”
1 Chu et al. AlexNet MLP RM R 91.6%; is diminishing. Thus, this is no longer the main consideration to decide
(2018) 98.2% whether the integration with tradition ML should be adopted.
2 Adedeji and ResNet-50 SVM RM R 87% Flexibility to fuse with different features. The combined use of CNN and
Wang traditional ML models allows easy integration of deep features extracted
(2019)
3 Toğaçar AlexNet SVM RM R 99.95%
by different networks (Toğaçar et al., 2020) or the fusion with sensing
et al. (2020) ResNet data from other modalities (e.g., weight and magnetism) (Chu et al.,
Inception 2018), leading to accuracy improvement. In order the automate the
4 Xiao et al. / ELM C&D R 95% gauging of C&D waste composition, Chen et al. (2021) developed a
(2020)
hybrid model that integrated visual features extracted by a DenseNet-
5 Chen et al. DenseNet169 SVM C&D R 94%
(2021) 169 network and physical features such as weight and depth collected
6 Yang et al. ResNeXt-101 k-NN RM R 96.96% by other sensors. The hybrid features were input to a SVM for waste
(2021) composition classification, which resulted in an accuracy improvement
*
The column indicates the task objective a study intended to tackle, which of over 20%.
includes two options: (a) R (recognition) is to recognize category of a single Scalability to accommodate new input categories. In industrial practice,
waste in an image; (b) D (detection) is to recognize and locate multiple waste it is not uncommon that new waste categories required to be classified
items in an image. will dynamically increase. In such cases, the original DL model will have
to be re-trained, which is a prolonged process. To address the problem,
Yang et al. (2021) adopted a “ResNeXt + k-NN” structure. When new
3.2.3. Integration with traditional ML algorithms
waste categories are added, retraining of the CNN is not required.
In the above studies, CNN models acted as both feature extractors
Instead, k-NN can classify samples of the new categories according to the
and classifiers to recognize or detect wastes from images. However,
similarity of ResNeXt features.
there are studies where CNNs were only used for feature extraction, and
classification was performed by traditional ML models such as SVM,
4. Analysis of academic research outputs
MLP, and k-NN. Table 3 provides a summary of these studies. The
integration of CNN with tradition ML models has several benefits:
The academic research outputs during 1997 and 2021 are analyzed
Take advantages of different machine learning models. CNNs are good at
from four aspects, i.e., waste sources, task objectives, application do­
learning features via its deep networks while SVM, a traditional ML al­
mains, and dataset accessibility. The pie charts in Fig. 2 show the dis­
gorithm, is a powerful tool for classification. By combining the two,
tribution of publication numbers over these four aspects.

Fig. 2. Distribution of the number of publications over (a) waste sources, (b) task objectives, (c) application domains, and (d) dataset accessibility.

35
W. Lu and J. Chen Waste Management 142 (2022) 29–43

4.1. Waste sources to 40% of the total waste stream in some cities (Hoornweg and Bhada-
Tata, 2012). Traditional sorting methods based on wind selection,
MSW can be generated from three sources, i.e., RM, ICI, and C&D. screening, and manual sorting are inefficient and tend to be inaccurate
Fig. 2 (a) shows the distribution of academic output over the three (Xiao et al., 2020); thus, researchers seek to incorporate CV into the
sectors, among which the RM sector emerges as the most productive one sorting lines for better efficiency and accuracy. Lukka et al. (2014) and
with 61 papers (or reports) published, far outnumbering the ICI sector’s Kujala et al. (2015) presented a robotic system called ZenRobotics
9 papers and the C&D sector’ 11 papers. Recycler for the C&D waste sorting, of which CV is an important module
to tackle the issue of material classification and object grasping. Xiao
4.1.1. Residential and municipal waste et al. (2020) developed an approach to classifying five typical categories
The compelling imbalance of the number of publications implies the of C&D waste, i.e., wood, brick, rubber, rock, and concrete. Chen et al.
major focus on enabling the RM waste sorting with CV. To segregate (2022) proposed a monocular vision approach to estimating composi­
inorganic wastes, Salmador et al. (2008) developed an intelligent tion of C&D wastes loaded by haul trucks. The classification of particles
garbage classifier system that comprises webcam vision, robotic arms, (Brisola et al., 2010) and aggregates also forms a challenging task in the
user interface, and conveyor belt. The CV module of the system first recycling of C&D waste. Lau Hiu Hoong et al. (2020) employed CNN to
employed thresholding and watershed segmentation to extract waste develop a (near) real-time CV-based method to determine the compo­
items, and then performed classification with handcrafted features such sition of recycled aggregates. In Wang et al. (2020b, 2019b), a Faster R-
as Fourier descriptors and moments. Rahman et al. (2011, 2010, 2009a, CNN and a Mask R-CNN were respectively trained on a collection of
b) conducted a line of work for the sorting of waste paper, where co- images showing construction wastes on the ground. The trained models
occurrence features (Rahman et al., 2009b) and window features achieved high performance in detecting C&D wastes (e.g., nails, screws,
(Rahman et al., 2010) were integrated with k-NN, template matching, and residual pipes and cables) scattered around the construction site,
and rule-based classifiers. As an important component in waste streams, which were then used to enable a robot prototype to perform waste
facilitating the separation of plastic with CV has attracted the attention collection.
of many researchers (Rahman et al., 2009b; Ramli et al., 2008; Tachwali Despite the research attempts, the exploitation of CV to empower
et al., 2007; Wang et al., 2019c; Zulkifley et al., 2014). Özkan et al. C&D waste sorting is still limited—only 11 publications over the span of
(2015) investigated five different feature extraction algorithms, and fed more than two decades. The situation seems even worse and puzzling
the extracted features to SVM for plastic classification. The study when considering the huge proportion (up to 40%) of C&D waste in the
attained an average accuracy of 88% based on majority voting. Some total waste streams and the high demand for new technologies for better
researchers focused on using CV to distinguish between common RM recycling practices.
waste, e.g., glass, paper, metal, plastic, and cardboard; examples of such
studies include Bobulski and Kubanek (2019), Liu et al. (2019), and 4.2. Task objectives
Toğaçar et al. (2020).
According to the tasks they aimed to accomplish, existing research
4.1.2. Industrial, commercial and institutional waste can be divided into two categories, namely waste recognition and waste
Research attempts to apply CV for ICI waste sorting scatters among detection (Fig. S2 in the Supplementary Material). The former aims to
the different sectors of the national economy, e.g., the nuclear industry classify images of waste materials into one of the predetermined cate­
(Shaukat et al., 2016; Sun et al., 2019), the automobile industry gories, while the latter not only recognizes the waste categories but also
(Koyanaka and Kobayashi, 2010, 2011; Wang et al., 2019a), and agri­ locates them on the images with bounding boxes or pixelwise labels.
culture and food production (Guttormsen et al., 2016; Pothula et al.,
2015; Verheyen et al., 2016; Zhu et al., 2018). Due to the hazardous 4.2.1. Waste recognition
nature of nuclear radiation, automated robots are in high demand to As shown in Fig. 2 (b), much of existing research attention has been
separate the nuclear waste materials after the decommissioning of nu­ paid to the task of waste recognition. Before the prevalence of DL, early
clear facilities. Vision-based autonomous recognition is deemed as an studies for waste recognition either simplified the problem to allow
indispensable module of such robotic systems. Shaukat et al. (2016) initial extraction of waste areas on images (Nawrocky et al., 2010; Ramli
stressed the superiority of passive cameras with CMOS (complementary et al., 2008; Tachwali et al., 2007), or assumed waste materials on the
metal oxide semiconductor) sensors over other sensors (e.g., X-Ray, laser conveyer belt can only appear in the camera field of view one by one
scanner, and HSI) regarding the cost-effectiveness and the reliability to (Sreelakshmi et al., 2019). Nawrocky et al. (2010) assumed images with
be exposed to the extreme nuclear environment. Sun et al. (2019) in­ bounding boxes around waste items were readily available, and applied
tegrated Gaussian process classification (GPC) and deep convolutional SVM for waste classification. Based on the assumption that the waste
neural network (DCNN) to train a nuclear waste detection model with a items are not overlaid with each other, Tachwali et al. (2007) extracted
limited amount of labeled RGB-D data. bottles from conveyor belt images by using background subtraction
In the automobile industry, the recycling of secondary raw materials techniques.
from end-of-life vehicles (ELVs) is economically beneficial (Wang et al., Recent development of DL makes it possible to directly process im­
2019a). Koyanaka and Kobayashi (2010, 2011) used a 3D imaging ages without prior background subtraction. A major line of such works
camera system to measure shape parameters of metal pieces of ELVs, (Aral et al., 2018; Bircanoğlu et al., 2018; Rabano et al., 2018) originate
which were then fed to a discriminant analyzer and a neural network for from the TrashNet project by Yang and Thung (2016). In the project, the
the sorting of cast aluminum, wrought aluminum, and magnesium. authors simplified waste sorting as a problem to classify a given single-
Wang et al. (2019a) developed a computer vision-based system for the object image into a waste type. The significant amount of research ef­
separation of non-ferrous metals from ELVs, where the performance of forts has resulted in the high accuracy performance on public datasets
multiple ML algorithms, e.g., SVM, k-NN, and DT, were compared. As for such as TrashNet (Yang and Thung, 2016). Yang and Li (2020) devel­
the agriculture and food production industry, several works have oped a lightweight neural network for garbage classification called
focused on enabling effective CV-based sorting of the different compo­ WasNet. The WasNet incorporates attention mechanism to force the
nents of crops and biomass for better usability (Pothula et al., 2015; network to pay more attention to sensitive area related with waste
Verheyen et al., 2016; Zhu et al., 2018). recognition. With data augmentation applied, WasNet realized a 96.10%
classification accuracy on TrashNet. Zhang et al. (2021) proposed a re­
4.1.3. Construction and demolition waste sidual network with a self-monitoring module for recyclable waste
C&D waste is an important component in MSW that accounts for up classification on the same dataset. By the application of a genetic

36
W. Lu and J. Chen Waste Management 142 (2022) 29–43

algorithm for hyperparameter optimization, Mao et al. (2021) improved 4.3. Application scenarios
the classification accuracy on TrashNet to 99.60%.
Despite the remarkable progress, the nature of waste recognition has Waste collection at source and waste sorting at disposal facility are
determined that it will have very limited application scenarios in the two application scenarios where previous research attention has been
WM industry. Firstly, since waste recognition can only classify given paid to. Fig. 2 (c) shows academic publication distribution over the two
images into one of the predefined categories, it is not suitable for scenarios.
automated waste sorting with robotics, which requires not only waste
category information, but also position and geometry of the waste ma­ 4.3.1. Waste collection at source
terials to guide the robot operations. Secondly, waste recognition tends Applications of CV in waste collection stage set out to recognize/
to require individual waste items appearing against a relatively simple detect waste objects from the source, which provides information for
background, which is not the case in most real-life scenarios where the WM departments to make collection plans (Mittal et al., 2016; Now­
waste materials usually scatter or even overlap with each other in akowski and Pamuła, 2020; Yang and Li, 2020), enables initial source
varying contexts. Based on the above analysis, such waste recognition separation with robotic systems (Paulraj et al., 2016; Rad et al., 2017),
techniques should be primarily considered for source separation at the or assists citizens to classify their generated wastes in the household
waste collection stage, where smart phones can be used to assist resi­ (Srinilta and Kanharattanachai, 2019; Yang et al., 2021). Nowakowski
dents to distinguish different household wastes (Srinilta and Kanhar­ and Pamuła (2020) proposed a CNN classifier to identify e-waste (e.g.,
attanachai, 2019; Yang et al., 2021) or be used by the authority to solicit refrigerator, washing machine, and TV set) with smartphones, which
waste collection information from the general public (Singh et al., 2017; can facilitate information exchange between waste generators and
Yang and Li, 2020). collection companies. Mittal et al. (2016) developed an Android app,
SpotGarbage, to automatically detect and localize garbage in uncon­
4.2.2. Waste detection strained real-world images. Singh et al. (2017) developed a system to
Compared with its counterpart, the proportion of research focusing solicit information on uncollected roadside garbage from the general
on waste detection merely exceeds a quarter. In practical engineering public, in which CV was used to determine if the uploaded information is
applications, it is quite common to have multiple waste items appear on valid.
the same image. Hence, sorting operations rely on not only the identified Another noticeable line of research is around the development of
categories but also the exact location and geometry of the wastes (Awe “smart bins”. One prominent feature of such smart bins is that they use
et al., 2017; Ku et al., 2021; Lau Hiu Hoong et al., 2020). Waste detection CV to detect the level of trash bins (e.g., empty, occupied, or full), which
not only recognizes the different waste types, but also identifies their can then inform the municipal departments for waste collection
positions and geometric boundaries on the images, providing critical (Abdallah et al., 2020; Aziz et al., 2018; Islam et al., 2014; Sarc et al.,
information to infer their actual 3D positions for subsequent grasping or 2019). Hannan et al. (2012; 2016) and Aziz et al. (2015; 2018) are
sorting with robot arms (Shaukat et al., 2016). Waste detection presents among the most productive researchers in the field, and have developed
a promising research direction to address the need of waste sorting a series of ML waste level detection methods based on features such as
industry. gray level aura matrix, GLCM and Hough line detection. Some “smart
In the waste sorting process, the bulk of waste materials are bins” research goes further, and intends to automatically segregate trash
randomly thrown onto the conveyor belt, inevitably leading to some in the bins by the integration of CV and robotics (Jacobsen et al., 2020).
waste items cluttered with each other. Overcoming the issue is a chal­ Most of existing researches on smart bins are based on traditional ML
lenging prerequisite for the effective detection of individual waste items. models; research on the applications of DL and its performance com­
Wang et al. (2019c) touched on the problem from the specific case of parison with previous methods is not well established.
plastic bottle classification. The study integrated morphological opera­ In recent years, increasing research attentions are paid to marine
tions, convex hull analysis, and concave points calculation to identify debris, a type of human-created solid waste that is discarded at sea or
“adjacent” or “overlapping” waste items. The development of DL tech­ reach the sea through waterways or domestic and industrial outfalls
niques makes it possible to directly train end-to-end models to identify (Ribic et al., 1992). The collection of marine debris requires information
wastes from complex clustered environments (Awe et al., 2017; Ku et al., on its category and position, which can be obtained with CV technolo­
2021; Mittal et al., 2016; Nowakowski and Pamuła, 2020; Rad et al., gies. Fulton et al. (2019) compared the performance of various DL al­
2017; Sun et al., 2019). Awe et al. (2017) pointed out that previous gorithms in detecting trash in underwater environments, paving the way
works based on TrashNet (Yang and Thung, 2016) can only recognize a for automated waste collection with autonomous underwater vehicles.
single object image, and stressed the importance of waste detection from Hong et al. (2020) proposed a generative approach to augmenting un­
clusters of trash. With a fine-tuned Fast R-CNN, they achieved a mAP of derwater images for visual detection of marine debris. Aside from un­
0.683 for the detection of typical MSW such as paper. derwater trash, a stream of research aims to identify floating or near
In recent years, more and more researchers have realized the limi­ surface marine debris from remote sensing images, such as Mace (2012),
tations of waste recognition, and turned to the problem of waste Taddia et al. (2021), and Hu (2021).
detection (Anjum and Umar, 2018; Liang and Gu, 2021; Panwar et al.,
2020; Proença and Simões, 2020; Wang et al., 2020b; Wang et al., 4.3.2. Waste sorting at disposal facility
2019b). Proença and Simões (2020) realized the importance of image As shown by Fig. 2 (c), research output in automated waste sorting
datasets with wastes in context. Therefore, they created the TACO with robotics (Chu et al., 2018; Gundupalli et al., 2017a, 2018) is
(Trash Annotation in Context) dataset, and implemented Mask R-CNN overwhelmingly higher than that in waste collection. In waste sorting
on the dataset as a benchmark for waste detection. Similarly, Mask R- facilities, a CV-enabled waste classification system consists of both
CNN was employed by Koskinopoulou et al. (2021) for waste detection hardware and software: the former is usually a low-cost camera, which
in industrial scenarios. Panwar et al. (2020) applied RetinaNet to detect acts as “eyes” of the system; the latter is essentially a bunch of computer
wastes in water body. Liang and Gu (2021) released a new dataset with algorithms that serve as the system’s “brain” to enable the identification
bounding box annotations and multiple labels in each image, and of waste objects. In a review of automated sorting techniques, Gundu­
developed a multi-task learning framework that can detect organic, palli et al. (2017b) compared the performance of various sensing tech­
recyclable, hazardous, and other wastes with high performance. niques, and concluded that optical-based sensors are applicable to a
diverse range of materials and can attain acceptable accuracy with very
little time consumption.
As early as the end of 20th century, researchers have already

37
W. Lu and J. Chen Waste Management 142 (2022) 29–43

conceptualized such systems that use machine vision to guide robots for Table 4
waste sorting (Faibish et al., 1997; Mattone et al., 2000; Mattone et al., Information of two notable private datasets of waste images.
1998). The early-year CV algorithms usually require deliberate feature Dataset Num. Waste Background Task Waste types
handcrafting for specific waste materials, which confined their appli­ of img materials a b

cability to a very limited range of wastes with simple features. In (R/

D)
addition, the hand-engineered features are usually not sufficiently
robust to adapt to the complexity of practical sorting tasks. The devel­ VN-trash ( 5,904 RM Simple R 3 categories:
opment of AI, especially DL techniques, significantly improves the Vo et al., medical, organic
2019) and inorganic
robustness of CV algorithms, and expands its applicability to a wide waste
range of waste materials including RM, ICI and C&D wastes. However, WasteRL c ( 57,000 RM In context D 4 categories:
as mentioned in previous section, most existing studies focus on the Liang organic waste,
problem of waste recognition, which deviates from the ultimate goal of and Gu, recyclables,
2021) hazardous waste,
automated waste sorting with robotics. The practical deployment of CV
and other wastes
for automatic waste sorting requires significantly more research efforts
a
to tackle the waste detection problems. Background complexity of wastes on images: (a) “Simple” means the waste
images have a simple and plain background; (b) “In context” means the waste
images were taken in complex real-life context.
4.4. Dataset accessibility b
CV tasks that the datasets are primarily used for, i.e., “R (recognition)” or “D
(detection)”.
c
According to their accessibility, existing waste datasets are either The dataset is available from the corresponding author by reasonable
“public” ones that can be freely accessed by researchers or “private” request.
ones that are only used by their owners. Fig. 2 (d) and Fig. 3 (a) show the
distribution of previous publications by the accessibility of their progressive improvement over time. A fundamental problem behind this
datasets. is the use of private datasets makes it difficult to provide a unified
standard and benchmark for meaningful performance comparison. For
4.4.1. Private datasets example, a collection of waste images acquired from an outdoor and
Most of previous studies evaluated their models on private datasets. cluttered environment is more challenging than one collected in a well-
Table 4 lists details of two notable private datasets. In the table, the controlled lab environment; thus, directly comparing models trained
“Task” column indicates whether the corresponding dataset is used for from the two datasets is meaningless as performance resulting from the
“waste recognition (R)” or “waste detection (D)”. The “Background” latter is presumably higher. The problem calls for the creation and
column reflects the complexity of background in the images: “Simple” sharing of public waste datasets.
means the background is simple and plain while “In context” means the
wastes in the images were captured in real-life contexts that are random 4.4.2. Public datasets
and complex. Fig. 3 (b) demonstrates the evolvement of waste classifi­ Several waste datasets have been collected and publicized by re­
cation accuracy on private datasets over the past two decades. The graph searchers. On Kaggle, a well-known online platform for data science and
presents a fluctuation pattern, which contradicts a common perception machine learning, data scientists, ML engineers and industrial practi­
that a line of research in the same area shall display a pattern of tioners have also published some useful datasets. Table 5 lists

Fig. 3. (a) Numbers of publications based on private and representative public datasets; (b) accuracy evolvement on private dataset: each bin represents the range of
accuracy achieved by studies in the respective year; (c) accuracy evolvement on TrashNet dataset: each bubble represents a study, and location and size of the
bubbles indicate the range of accuracy.

38
W. Lu and J. Chen Waste Management 142 (2022) 29–43

information of representative public datasets. paid to the development of public ICI or C&D waste datasets to facilitate
The most widely used public dataset is TrashNet, collected and relevant research.
publicized by Yang and Thung (2016). The dataset includes 2,527
single-object photos of six waste types (i.e., paper, glass, plastic, metal, 5. Challenges and prospects
cardboard, and trash) with a white poster board as background. Fig. 3
(c) depicts the accuracy evolvement on the TrashNet. A clear trend of 5.1. Challenges ahead
growth can be observed. This demonstrates that, with a unified publicly
available dataset, research efforts can be concentrated to enable Driven by prevailing DL, the application of CV for waste sorting is
continuous performance improvement. However, despite the significant gaining increasing attention. However, our review identifies multiple
performance (99.60% accuracy by 2021), the TrashNet dataset seems lessons from existing research, which might pose challenges to trans­
rather too idealistic to allow for practical applications. As emphasized ferring current academic efforts into practical applications.
by (Meng and Chu, 2020), “it is unrealistic to get a picture of an object First, current studies lack publicly available real-world datasets that
with a clean background“ in reality. Similar concerns have been are oriented to the industrial practices of waste sorting. This has led to
expressed by (Liang and Gu, 2021) and (Proença and Simões, 2020) on the difficulties in rigorous comparison among different studies. Most
the oversimplicity of previous waste images. Therefore, datasets with existing datasets were privately owned by respective research teams.
waste images collected in real-life contexts are required. TACO is a There are open-access datasets, but most of them tend to oversimplify
notable initiative to move towards that direction. It includes 1,500 im­ the industrial need as a waste recognition problem, with waste objects
ages of RM wastes in context, encompassing a wide range of waste items appearing on plain and well-control background (Mohamed, 2021;
(28 categories and 60 subclasses) such as plastic bags, cigarette and Sekar, 2019; x670783915, 2019; Yang and Thung, 2016). Some studies
bottles (Proença and Simões, 2020). The dataset has high-quality have embarked on collecting waste images in contexts. However, they
bounding box annotations and pixel-level waste labels, which can be are either limited in the number (Panwar et al., 2020; Proença and
used for object detection or even instance segmentation. A similar Simões, 2020) or lack high-quality annotations (DataCluster Labs,
research thrust is WasteRL (Liang and Gu, 2021), which includes 57,000 2021). In addition, a predominant proportion of public datasets are for
images of four RM waste categories. The dataset is available by the sorting of RM wastes. The absence of high-quality public datasets
reasonable request to the corresponding author, and thus is not oriented to the diverse MSW types makes it difficult to allow meaningful
considered a “public” one as listed in Table 4. performance evaluation based on a unified standard. This would distract
A noticeable observation from Table 5 is that all but one public research efforts from the core line of the continuous improvement of
datasets are oriented to RM wastes. This is not surprising as a predom­ waste sorting.
inant portion of research efforts have been focused on RM waste sorting Second, existing studies simplified, more or less, the objectives or
(Fig. 2 (a)). However, as the other two wastes types, especially C&D working conditions of their proposed algorithms, which might pose a
wastes, are also important sources of MSW, more attention should be challenge to their future deployment to industrial applications. Before

Table 5
Information of representative public datasets of waste images.
Dataset Num. of img Waste Background Task Waste types
a b
materials

TrashNet (Yang and Thung, 2016) 2,527 RM Simple R 6 types:

Glass, Paper, Cardboard, Plastic, Metal, Trash
Flickr Material Dataset c (Sharan et al., 1,000 / / R 10 types:
2009) Fabric, Foliage, Glass, Leather, Metal, Paper, Plastic, Stone, Water, Wood
Huawei Trash Classification Dataset ( 38,918 RM Simple R 4 types, and 43 classes:
x670783915, 2019) Recyclable: Package, power bank, etc.
Kitchen: fruit peels, leftovers, etc.
Harmful: Dry battery, ointment, etc.
Others: toothpicks, fouled plastics, etc.
Washington d (Lai et al., 2011) 250,000 / In context D 51 categories of common objects, e.g., fruit and vegetable, and device and
container
d
Birmingham (Sun et al., 2019) 217 ICI In Context D 10 categories:
Plastic bottles, cans, chains, cleaning cloths, gloves, metal objects, plastics
pipes, pipe joints, sponges, wooden blocks
The Sekar’s (Sekar, 2019) 25,077 RM Simple R Two classes:
Organic and recyclable wastes
TACO e (Proença and Simões, 2020) 1,500 RM In context D 28 categories and 60 subclasses, e.g., plastic bags, cigarette, bottle, and can
AquaTrash (Panwar et al., 2020) 369 RM In context D 4 categories:
glass, metal, paper, plastic
ReSORT-IT e (Koskinopoulou et al., 21,600 RM In context D 4 categories:
2021) aluminum, paper and cardboard, bottles, nylon
Garbage Classification f (Mohamed, 15,150 RM Simple R 12 classes:
2021) Paper, cardboard, biological, metal, plastic, green-glass, brown-glass, white-
glass, clothes, shoes, batteries, and trash
Domestic Garbage Datasetg f ( greater RM In Context D Paper/plastic cups, razor, plastic bags, etc.
DataCluster Labs, 2021) than9,000
a
Background complexity of wastes on images: (a) “Simple” means the waste images have a simple and plain background; (b) “In context” means the waste images
were taken in complex real-life context.
b
CV tasks that the datasets are primarily used for, i.e., “R (recognition)” or “D (detection).”
c
Material dataset that focuses on relatively micro texture features, and thus the “Waste sources” and “Background” fields are not applicable.
d
Datasets of RGB-D images.
e
The dataset also includes pixel-level annotation that can be used for instance segmentation.
f
Datasets from Kaggle platform.
g
The dataset only includes waste images, but they have not been properly annotated with labels

39
W. Lu and J. Chen Waste Management 142 (2022) 29–43

the popularization of DL, it was a common practice to simplify the than only recognizing the waste type from a single-object image. Hence,
problem of waste recognition within certain system boundary, e.g., waste detection, which aims to detect and classify multiple waste items,
assuming waste materials can only be thrown onto the conveyor belt one should attract more research attention in the future. The efficiency of CV
by one (Sreelakshmi et al., 2019), or cannot overlap with others algorithms is also an indispensable factor to consider in practical ap­
(Tachwali et al., 2007). Most recent DL-related studies confine their plications. While efficiency is comparatively less important in certain
scope in recognizing single objects from images (Aral et al., 2018; Yang scenarios such as smartphone-assisted household waste classification,
and Thung, 2016). Such simplification with tight constraints can make demanding time performance is required in other scenarios like auto­
the developed approaches incompatible with the unstructured envi­ mated waste sorting with robotics. This is especially true when consid­
ronments in actual sorting facilities, where the waste is usually ering that a waste recovery facility can receive thousands of tonnes of
randomly distributed in a cluttered background (Chen et al., 2021; Lu MSW per day, and the sorting speed directly affects the overall
et al., 2022). throughput. However, the great majority of existing studies failed to
Third, while C&D waste accounts for a large proportion in MSW report the time performance of their algorithms. Future researches are
streams, only a limited amount of research has been dedicated to suggested to share the inference time as well as accuracy of their algo­
applying CV for its sorting. C&D waste can represent as much as 40% of rithms. Note that hardware with different computing power can result in
the total waste streams generated by some cities (Hoornweg and Bhada- different time performance of the same algorithm. Hence, the specific
Tata, 2012); In Hong Kong, C&D waste takes up at least one quarter of hardware configuration with which the respective algorithm is imple­
the materials that end up being landfilled (HKEPD, 2020). The dispro­ mented should also be mentioned for readers’ reference.
portionately limited number of studies, as compared to the large amount Computer vision for C&D waste sorting. As a major component of MSW,
of C&D waste, poses a great challenge ahead, which signifies an urgent efficiency improvement of C&D waste sorting is beneficial from both
need of more research attentions. A failure in addressing the need will economic and environmental perspectives. More research efforts are
not only continue to allow the recyclable construction materials not needed to investigate how CV and its relevant algorithms can be inte­
being properly reused, but also cause a series of environmental problems grated into the sorting process given the bulky and heterogeneous
for producing new materials that could have been recycled and taking characteristics of C&D wastes. In fact, stimulated by the economic
up valuable land resources for landfills. benefits and recent technological development, notable change is
Fourth, despite the versatility of CV in distinguishing a wide range of happening. By 2018, only five out of the reviewed studies are relevant to
materials, vision-based methods are inherently incapable of character­ C&D wastes, but since 2019, up to six studies have been published on the
izing physiochemical properties of waste objects. Materials with topic in less than three years. The level of increase reached 120%. The
different physiochemical properties can present similar visual features. publication boom reflects a trend that more and more research is
For example, glass might look similar to a piece of transparent plastic directed to the use of CV in C&D waste sorting. In the near future, the
sheet, but clearly, they are two different types of materials. In such cases, trend is expected to continue.
it would be challenging to rely on CV to implement effective sorting Multi-modal sensing data fusion. Although it is expected the perfor­
without other useful information provided. mance of CV in waste sorting will continue to improve in the near future,
visual sensors have inherent weakness in sensing materials’ physi­
5.2. Prospects and future directions ochemical properties. Fusion of sensing data from different modalities
(Chu et al., 2018; Kuritcyn et al., 2015; Tachwali et al., 2007) can take
In view of the above challenges, several directions are suggested for advantages of both the cost-effectiveness of visual sensors and the
future CV-based waste sorting research. capability of other sensors, e.g., weight meters, near-infrared spectros­
Industry-oriented public datasets for various waste sources. Publicly copy (NIR) and inductive sensor, to detect the physiochemical properties
available datasets with high-quality annotations are required to provide of waste items. It can increase the classification accuracy and improve
benchmarks for rigorous performance evaluation. The datasets should, the system robustness. However, the emerging DL poses new challenges,
on the one hand, orient to the practical needs of industry by providing as the features extracted by CNN can be hundreds of thousands
real-life photos in context, and on the other hand, ensure free and open compared with the sing-digit number of physical features. Hence, future
access to the research community. Some latest studies have realized the research is suggested to further investigate how visual features extracted
limitations of previous oversimplified datasets, and strived to create by CV can be effectively fused with other features from different
datasets that captures the variations of reality, e.g., TACO (Proença and domains.
Simões, 2020) and WasteRL (Liang and Gu, 2021). However, as these
studies were only published recently, their influences remain relatively 6. Conclusions
low. More research efforts should be made to create and share datasets
for ICI and C&D wastes. The access to such waste photos is usually Waste sorting is a critical step towards efficient MSW management.
monopolized by stakeholders of the respective sectors, and thus the The application of CV for waste sorting has been conceptualized and
engagement of industry practitioners is essential. While an ideal and under investigation for over two decades. Driven by the emerging DL
more sustainable way is to construct a centralized database where im­ techniques, the field is currently experiencing an unprecedented
ages of various waste types and their annotations can be solicited by development. Against this background, this paper provides a critical
crowdsourcing (PEER, 2018), it is not a realistic goal in the short term review of academic research to understand the past, present, and future
due to a series of legal and managerial issues such as copy right and of the field of CV-based waste sorting. Prevalent CV algorithms for waste
confidentiality. In the foreseeable future, waste datasets will still be sorting can be categorized into two types: traditional ML and DL algo­
collected, managed and publicized by separate parties, i.e., via a rithms. Traditional ML algorithms require handcrafted visual features as
decentralized manner. Thus, it is important to formulate a common input, while DL algorithms can automatically extract hidden features
protocol to guide how the datasets should be constructed. For examples, from raw images. For its advantages on robustness and automated end-
since different countries/regions and sectors generate different types of to-end training, DL has become the predominant CV algorithms to
waste, it is suggested to include the geolocation (e.g., countries or enable solid waste sorting. It is found that the academic studies are
geographical coordinates) and sources (e.g., RM, ICI, or C&D) as an distributed disproportionately among the RM, ICI, and C&D sectors.
annotation data field or metadata when constructing relevant datasets. While existing studies primarily focused on employing CV for waste
Address the need of engineering practices. Future researches are ex­ sorting at centralized disposal facilities, there were also studies aiming
pected to focus more on the practical problems encountered by the WM to facilitate waste collection by machine vision. Previous studies tend to
industry. For example, sorting waste from conveyor belts requires more confine their research scope to the task of waste recognition with

40
W. Lu and J. Chen Waste Management 142 (2022) 29–43

simplified working conditions (e.g., using a small set of artificially Brisola, D.F., Cunha, B.M., Gomes, O., Lima, P., Paciornik, S., 2010. Automatic
classification of particles from construction and demolition waste through digital
collected images in lab environments). Only a few studies used publicly
image analysis. 65th ABM International Congress, 18th IFHTSE Congress and 1st
available, secondary datasets for model training and evaluation. TMS/ABM International Materials Congress 2010 3046–3052.
The critical review identified several challenges confronting the Brownlee, J., 2016, Parametric and nonparametric machine learning algorithms. https://
further promotion of CV for waste sorting. These include (a) the lack of machinelearningmastery.com/parametric-and-nonparametric-machine-learning-
algorithms/ (Accessed Dec. 6 2021).
comprehensive, sharable datasets that can be used by interested re­ Chen, J., Lu, W., Xue, F., 2021. “Looking beneath the surface”: A visual-physical feature
searchers to train their models, (b) oversimplified working conditions hybrid approach for unattended gauging of construction waste composition.
leading to detached links between research and real-life practice, (c) less J. Environ. Manage. 286, 112233. https://doi.org/10.1016/j.jenvman.2021.112233.
Chen, J., Lu, W., Yuan, L., Wu, Y., Xue, F., 2022. Estimating construction waste truck
attention to C&D waste sorting, and (d) limitation of CV in dis­ payload volume using monocular vision. Resour. Conserv. Recycl. 177, 106013.
tinguishing materials with similar appearance. Future research efforts https://doi.org/10.1016/j.resconrec.2021.106013.
should strive to create and share industry-level datasets oriented to the Chen, Z., Hebin, Z., Yanbo, W., Binyan, L., Yu, L., 2017. A vision-based robotic grasping
system using deep learning for garbage sorting. 2017 36th Chinese Control
diversity of waste sources. Researchers are also suggested to accom­ Conference (CCC) 11223–11226.
modate the actual industrial needs by focusing on the waste detection Christensen, T.H., Matsufuji, Y., 2011. Source segregation and collection of source-
problems while attaching equivalent importance to efficiency of their segregated waste. Solid waste technology and management. Wiley 296–310.
Chu, Y., Huang, C., Xie, X., Tan, B., Kamal, S., Xiong, X., 2018. Multilayer hybrid deep-
algorithms. C&D waste sorting and multimodal feature fusion are two learning method for waste classification and recycling. Computational Intelligence
other promising directions that should gain more attention. Neuroscience 2018, 1–9.
DataCluster Labs, 2021, Domestic trash/garbage dataset. https://www.kaggle.com/
dataclusterlabs/domestic-trash-garbage-dataset (Accessed Aug. 2 2021).
Faibish, S., Bacakoglu, H., Goldenberg, A.A., 1997. An eye-hand system for automated
Declaration of Competing Interest paper recycling, Proceedings of International Conference on Robotics and
Automation, pp. 9-14 vol.11.
The authors declare that they have no known competing financial Fulton, M., Hong, J., Islam, M.J., Sattar, J., 2019. Robotic detection of marine litter using
deep visual detection models. International Conference on Robotics and Automation
interests or personal relationships that could have appeared to influence
(ICRA) 5752–5758.
the work reported in this paper. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Garcia-Rodriguez,
J., 2017. A review on deep learning techniques applied to semantic segmentation.
arXiv preprint arXiv:1704.06857.
Acknowledgment Girshick, R., 2015. Fast R-CNN, Proceedings of the IEEE international conference on
computer vision, pp. 1440-1448.
This research is jointly supported by the Strategic Public Policy Girshick, R., Donahue, J., Darrell, T., Malik, J., 2014. Rich feature hierarchies for
accurate object detection and semantic segmentation. Proceedings of the IEEE
Research (SPPR) Funding Scheme (Project No.: S2018.A8.010.18S) and conference on computer vision and pattern recognition 580–587.
the Environment and Conservation Fund (ECF) (Project No.: ECF 2019- Gokyuu, T., Nakamura, S., Ueno, T., Nakamura, M., Inoue, D., Yanagihara, Y., 2011.
111) of the Government of the Hong Kong Special Administrative Sorting system for recycling of construction byproducts with Bayes’ theorem-based
robot vision. J. Robotics Mechatronics 23 (6), 1066–1072.
Region. Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y., 2016. Deep learning. MIT press
Cambridge.
Gundupalli, S.P., Hait, S., Thakur, A., 2017a. Multi-material classification of dry
Appendix A. Supplementary data
recyclables from municipal solid waste based on thermal imaging. Waste Manage.
70, 13–21.
Supplementary data to this article can be found online at https://doi. Gundupalli, S.P., Hait, S., Thakur, A., 2017b. A review on automated sorting of source-
org/10.1016/j.wasman.2022.02.009. separated municipal solid waste for recycling. Waste Manage. 60, 56–74.
Gundupalli, S.P., Hait, S., Thakur, A., 2018. Classification of metallic and non-metallic
fractions of e-waste using thermal imaging-based technique. Process Saf. Environ.
References Prot. 118, 32–39.
Guo, H.-N., Wu, S.-B., Tian, Y.-J., Zhang, J., Liu, H.-T., 2021. Application of machine
learning methods for the prediction of organic solid waste treatment and recycling
Abdallah, M., Abu Talib, M., Feroz, S., Nasir, Q., Abdalla, H., Mahfood, B., 2020.
processes: A review. Bioresour. Technol. 319, 124114. https://doi.org/10.1016/j.
Artificial intelligence applications in solid waste management: A systematic research
biortech.2020.124114.
review. Waste Manage. 109, 231–246.
Guttormsen, E., Toldnes, B., Bondø, M., Eilertsen, A., Gravdahl, J.T., Mathiassen, J.R.,
Adedeji, O., Wang, Z., 2019. Intelligent waste classification system using deep learning
2016. A machine vision system for robust sorting of herring fractions. Food
convolutional neural network. Procedia Manuf. 35, 607–612.
Bioprocess Technol. 9 (11), 1893–1900.
Anjum, M., Umar, M.S., 2018. Garbage localization based on weakly supervised learning
Hannan, M.A., Arebey, M., Begum, R.A., Basri, H., 2012. An automated solid waste bin
in deep convolutional neural network, Proceedings - IEEE 2018 International
level detection system using a gray level AURA matrix. Waste Manage. 32 (12),
Conference on Advances in Computing, Communication Control and Networking,
2229–2238.
ICACCCN 2018, pp. 1108-1113.
Hannan, M.A., Arebey, M., Begum, R.A., Basri, H., Al Mamun, M.A., 2016. Content-based
Aral, R.A., Keskin, Ş.R., Kaya, M., Hacıömeroğlu, M., 2018. Classification of TrashNet
image retrieval system for solid waste bin level detection and performance
dataset based on deep learning models. 2018 IEEE Int. Conf. Big Data (Big Data)
evaluation. Waste Manage. 50, 10–19.
2058–2062.
HKEPD, 2020, Hong Kong waste treatment and disposal statistics. https://www.epd.gov.
Arebey, M., Hannan, M.A., Begum, R.A., Basri, H., 2012. Solid waste bin level detection
hk/epd/english/environmentinhk/waste/data/stat_treat.html (Accessed April 7
using gray level co-occurrence matrix feature extraction approach. J. Environ.
2021).
Manage. 104, 9–18.
Hong, J., Fulton, M., Sattar, J., 2020. A generative approach towards improved robotic
Awe, O., Mengistu, R., Sreedhar, V., 2017. Smart trash net: Waste localization and
detection of marine litter. 2020 IEEE International Conference on Robotics and
classification, arXiv preprint.
Automation (ICRA) 10525–10531.
Aziz, F., Arof, H., Mokhtar, N., Mubin, M., Abu Talip, M.S., 2015. Rotation invariant bin
Hoornweg, D., Bhada-Tata, P., 2012. What a waste: A global review of solid waste
detection and solid waste level classification. Measurement: Journal of the
management. World Bank, Washington, DC.
International Measurement Confederation 65, 19-28.
Hu, C., 2021. Remote detection of marine debris using satellite observations in the visible
Aziz, F., Arof, H., Mokhtar, N., Shah, N.M., Khairuddin, A.S.M., Hanafi, E., Talip, M.S.A.,
and near infrared spectral range: Challenges and potentials. Remote Sens. Environ.
2018. Waste level detection and HMM based collection scheduling of multiple bins.
259, 112414. https://doi.org/10.1016/j.rse.2021.112414.
PLoS ONE 13.
Huang, G.-L., He, J., Xu, Z., Huang, G., 2020. A combination model based on transfer
Bircanoğlu, C., Atay, M., Beşer, F., Genç, Ö., Kızrak, M.A., 2018. RecycleNet: Intelligent
learning for waste classification. Concurrency and Computation: Practice and
waste sorting using deep neural networks, 2018 Innovations in Intelligent Systems
Experience 32 (19). https://doi.org/10.1002/cpe.v32.1910.1002/cpe.5751.
and Applications (INISTA), pp. 1-7.
Huang, J., Pretz, T., Bian, Z., 2010. Intelligent solid waste processing using optical sensor
Bobulski, J., Kubanek, M., 2019. Waste classification system using image processing and
based sorting technology. 2010 3rd International Congress on Image and Signal
convolutional neural networks, International Work-Conference on Artificial Neural
Processing 1657–1661.
Networks. Springer, pp. 350-361.
Islam, M.S., Hannan, M.A., Basri, H., Hussain, A., Arebey, M., 2014. Solid waste bin
Bao, Z., Lu, W., Chi, B., Yuan, H., Hao, J., 2019. Procurement innovation for a circular
detection and classification using dynamic time warping and mlp classifier. Waste
economy of construction and demolition waste: Lessons learnt from Suzhou, China.
Manage. 34 (2), 281–290.
Waste Manage. 99, 12–21. https://doi.org/10.1016/j.wasman.2019.08.031.
Jacobsen, R.M., Johansen, P.S., Bysted, L.B.L., Skov, M.B., 2020. Waste wizard:
Bao, Z., Lu, W., Hao, J., 2021. Tackling the “last mile” problem in renovation waste
Exploring waste sorting using ai in public spaces. ACM International Conference
management: A case study in China. Sci. Total Environ. 790, 148261 https://doi.
Proceeding Series.
org/10.1016/j.scitotenv.2021.148261.

41
W. Lu and J. Chen Waste Management 142 (2022) 29–43

Kaza, S., Yao, L., Bhada-Tata, P., Van Woerden, F., 2018. What a waste 2.0: A global Panwar, H., Gupta, P.K., Siddiqui, M.K., Morales-Menendez, R., Bhardwaj, P., Sharma, S.,
snapshot of solid waste management to 2050. World Bank Publications. Sarker, I.H., 2020. Aquavision: Automating the detection of waste in water bodies
Keramitsoglou, K., Tsagarakis, K., 2018. Public participation in designing the recycling using deep transfer learning. Case Studies Chem. Environmental Engineering 2,
bins to encourage recycling. Sustainability 10 (4), 1240. https://doi.org/10.3390/ 100026. https://doi.org/10.1016/j.cscee.2020.100026.
su10041240. Paulraj, S.G., Hait, S., Thakur, A., 2016. Automated municipal solid waste sorting for
Kim, J., Nocentini, O., Scafuro, M., Limosani, R., Manzi, A., Dario, P., Cavallo, F., 2019. recycling using a mobile manipulator, Proceedings of the ASME International Design
An innovative automated robotic system based on deep learning approach for Engineering Technical Conferences and Computers and Information in Engineering
recycling objects. ICINCO 2, 613–622. Conference, 2016.
Koskinopoulou, M., Raptopoulos, F., Papadopoulos, G., Mavrakis, N., Maniadakis, M., PEER, 2018, Call for uploading images for phi (peer hub imagenet) challenge. https://
2021. Robotic waste sorting technology: Toward a vision-based categorization apps.peer.berkeley.edu/spo/ (Accessed Aug. 8 2021).
system for the industrial robotic separation of recyclable waste. IEEE Rob. Autom. Pothula, A.K., Igathinathane, C., Kronberg, S., 2015. Profile based image analysis for
Mag. 28 (2), 50–60. identification of chopped biomass stem nodes and internodes. Ind. Crops Prod. 70,
Koyanaka, S., Kobayashi, K., 2010. Automatic sorting of lightweight metal scrap by 374–382.
sensing apparent density and three-dimensional shape. Resour. Conserv. Recycl. 54 Proença, P.F., Simões, P., 2020. Taco: Trash annotations in context for litter detection.
(9), 571–578. arXiv preprint arXiv:2003.06975.
Koyanaka, S., Kobayashi, K., 2011. Incorporation of neural network analysis into a Rabano, S.L., Cabatuan, M.K., Sybingco, E., Dadios, E.P., Calilung, E.J., 2018. Common
technique for automatically sorting lightweight metal scrap generated by elv garbage classification using MobileNet, 2018 IEEE 10th International Conference on
shredder facilities. Resour. Conserv. Recycl. 55 (5), 515–523. Humanoid, Nanotechnology, Information Technology, Communication and Control,
Krizhevsky, A., Sutskever, I., Hinton, G.E., 2012. Imagenet classification with deep Environment and Management, HNICEM 2018. Institute of Electrical and Electronics
convolutional neural networks. Advances in neural information processing systems Engineers Inc.
1097–1105. Rad, M.S., von Kaenel, A., Droux, A., Tieche, F., Ouerhani, N., Ekenel, H.K., Thiran, J.P.,
Ku, Y., Yang, J., Fang, H., Xiao, W., Zhuang, J., 2021. Deep learning of grasping detection 2017. A computer vision system to localize and classify wastes on the streets.
for a robot used in sorting construction and demolition waste. J. Mater. Cycles Waste International Conference on computer vision systems. Springer 195–204.
Manage. 23 (1), 84–95. Rahman, M.O., Hussain, A., Basri, H., 2014. A critical review on waste paper sorting
Kujala, J.V., Lukka, T.J., Holopainen, H., 2015. Picking a conveyor clean by an techniques. Int. J. Environ. Sci. Technol. 11 (2), 551–564.
autonomously learning robot, arXiv preprint arXiv:1511.07608. Rahman, M.O., Hussain, A., Scavino, E., Basri, H., Hannan, M.A., 2011. Intelligent
Kuritcyn, P., Anding, K., Linß, E., Latyev, S.M., 2015. Increasing the safety in recycling of computer vision system for segregating recyclable waste papers. Expert Syst. Appl.
construction and demolition waste by using supervised machine learning. J. Phys. 38 (8), 10398–10407.
Conf. Ser. 588, 012035. https://doi.org/10.1088/1742-6596/588/1/012035. Rahman, M.O., Hussain, A., Scavino, E., Ezlin, N., Basri, H., Hannan, M.A., 2010. Waste
Lai, K., Bo, L., Ren, X., Fox, D., 2011. A large-scale hierarchical multi-view RGB-D object paper grade identification system using window features. J. Comput. Information
dataset. 2011 IEEE International Conf. Robotics Automation 1817–1824. Syst. 2077–2091.
Lau Hiu Hoong, J.D., Lux, J., Mahieux, P.-Y., Turcry, P., Aït-Mokhtar, A., 2020. Rahman, M.O., Hussain, A., Scavino, E., Hannan, M.A., Basri, H., 2009a. Recyclable
Determination of the composition of recycled aggregates using a deep learning-based waste paper sorting using template matching. Recyclable Waste Paper Sorting Using
image analysis. Autom. Constr. 116, 103204. https://doi.org/10.1016/j. Template Matching 467–478.
autcon.2020.103204. Rahman, M.O., Hussain, A., Scavino, E., Hannan, M.A., Basri, H., 2009b. Segregating
Leitner, R., Mairer, H., Kercek, A., 2003. Real-time classification of polymers with nir recyclable waste papers using co-occurrence features. Proceedings of the 9th WSEAS
spectral imaging and blob analysis. Real-Time Imaging 9 (4), 245–251. International Conference on Applied Computer Science ’09..
Liang, S., Gu, Y., 2021. A deep convolutional neural network to simultaneously localize Ramli, S., Mustafa, M.M., Hussain, A., Abdul Waha, D., 2008. Histogram of intensity
and recognize waste types in images. Waste Manage. 126, 247–257. feature extraction for automatic plastic bottle recycling system using machine vision.
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P., 2017. Focal loss for dense object Am. J. Environmental Sci. 4 (6), 583–588.
detection. Proc. IEEE Int. Conference Computer Vision 2980–2988. Ramli, S., Mustafa, M.M., Wahab, D.A., Hussain, A., 2010. Plastic bottle shape
Liu, C., Sharan, L., Adelson, E.H., Rosenholtz, R., 2010. Exploring features in a Bayesian classification using partial erosion-based approach. 2010 6th International
framework for material recognition. 2010 IEEE Computer Society Conference on Colloquium on Signal Processing & its Applications 1–4.
Computer Vision and Pattern Recognition 239–246. Ren, S., He, K., Girshick, R., Sun, J., 2015. Faster R-CNN: Towards real-time object
Liu, H., Owolabi, G.O., Kim, S.H., 2019. Automatic classifications and recognition for detection with region proposal networks. Adv. Neural Information Processing Syst.
recycled garbage by utilizing deep learning technology. Proceedings of the 2019 7th 28, 91–99.
International Conference on Information Technology: IoT and Smart City 1–4. Ribic, C.A., Dixon, T.R., Vining, I., 1992. Marine debris survey manual.
Lu, W., Chen, J., Xue, F., 2022. Using computer vision to recognize composition of Rogers, S., Girolami, M., 2016. A first course in machine learning. CRC Press.
construction waste mixtures: A semantic segmentation approach. Resour. Conserv. Sakr, G.E., Mokbel, M., Darwich, A., Khneisser, M.N., Hadi, A., 2016. Comparing deep
Recycl. 178, 106022. https://doi.org/10.1016/j.resconrec.2021.106022. learning and support vector machines for autonomous waste sorting. 2016 IEEE Int.
Lu, W., Peng, Y.i., Webster, C., Zuo, J., 2015. Stakeholders’ willingness to pay for Multidisciplinary Conf. Eng. Technology (IMCET) 207–212.
enhanced construction waste management: A Hong Kong study. Renew. Sustain. Salmador, A., Pérez Cid, J., Rodríguez Novelle, I., 2008. Intelligent garbage classifier,
Energy Rev. 47, 233–240. International Journal of Interactive Multimedia and Artificial Intelligence.
Lu, W., Yuan, L., Xue, F., 2021. Investigating the bulk density of construction waste: A International Journal of Interactive Multimedia and Artificial. Intelligence.
big data-driven approach. Resour. Conserv. Recycl. 169, 105480. https://doi.org/ Sarc, R., Curtis, A., Kandlbauer, L., Khodier, K., Lorber, K.E., Pomberger, R., 2019.
10.1016/j.resconrec.2021.105480. Digitalisation and intelligent robotics in value chain of circular economy oriented
Lukka, T.J., Tossavainen, T., Kujala, J.V., Raiko, T., 2014. Zenrobotics recycler–robotic waste management - a review. Waste Manage. 95, 476–492.
sorting using machine learning. Proceedings of the International Conference on Scavino, E., Wahab, D.A., Hussain, A., Basri, H., Mustafa, M.M., 2009. Application of
Sensor-Based Sorting (SBS) 1–8. automated image analysis to the identification and extraction of recyclable plastic
Mace, T.H., 2012. At-sea detection of marine debris: Overview of technologies, processes, bottles. J. Zhejiang University-SCIENCE A 10 (6), 794–799.
issues, and options. Mar. Pollut. Bull. 65 (1-3), 23–27. Sekar, S., 2019, Waste classification data| kaggle. https://www.kaggle.com/techsash/
Mao, W.-L., Chen, W.-C., Wang, C.-T., Lin, Y.-H., 2021. Recycling waste classification waste-classification-data (Accessed Aug. 2 2021).
using optimized convolutional neural network. Resour. Conserv. Recycl. 164, Sharan, L., Rosenholtz, R., Adelson, E., 2009. Material perception: What can you see in a
105132. https://doi.org/10.1016/j.resconrec.2020.105132. brief glance? J. Vision 9 (8), 784.
Mattone, R., Campagiorni, G., Galati, F., 2000. Sorting of items on a moving conveyor Shaukat, A., Gao, Y., Kuo, J.A., Bowen, B.A., Mort, P.E., 2016. Visual classification of
belt. Part 1: A technique for detecting and classifying objects. Rob. Comput. Integr. waste material for nuclear decommissioning. Rob. Auton. Syst. 75, 365–378.
Manuf. 16 (2-3), 73–80. Singh, S., Mehta, K.S., Bhattacharya, N., Prasad, J., Lakshmi, S.K., Subramaniam, K.V.,
Mattone, R., Campagiorni, G., Wolf, A., 1998. Fuzzy-based processing of 3d information Sitaram, D., 2017. Identifying uncollected garbage in urban areas using
for items localization in the automated sorting of recyclable packaging, 1998 IEEE crowdsourcing and machine learning, 2017 IEEE Region 10 Symposium (TENSYMP),
International Conference on Fuzzy Systems Proceedings. IEEE World Congress on pp. 1-5.
Computational Intelligence (Cat. No.98CH36228), pp. 1613-1618 vol.1612. Sreelakshmi, K., Akarsh, S., Vinayakumar, R., Soman, K.P., Ieee, 2019. Capsule neural
Meng, S., Chu, W., 2020. A study of garbage classification with convolutional neural networks and visualization for segregation of plastic and non-plastic wastes, 2019
networks, 2020 Indo – Taiwan 2nd International. Conference Computing, Analytics 5th International Conference on Advanced Computing & Communication Systems,
Networks (Indo-Taiwan ICAN) 152–157. pp. 631-636.
Mittal, G., Yagnik, K.B., Garg, M., Krishnan, N.C., 2016. Spotgarbage: Smartphone app to Srinilta, C., Kanharattanachai, S., 2019. Municipal solid waste segregation with CNN,
detect garbage using deep learning. Proceedings of the 2016 ACM International Joint 2019 5th International Conference on Engineering, Applied Sciences and Technology
Conference on Pervasive and Ubiquitous Computing 940–945. (ICEAST), pp. 1-4.
Mohamed, M., 2021, Garbage classification (12 classes). https://www.kaggle.com/ Sudha, S., Vidhyalakshmi, M., Pavithra, K., Sangeetha, K., Swaathi, V., 2016. An
mostafaabla/garbage-classification (Accessed Aug. 2 2021). automatic classification method for environment: Friendly waste segregation using
Nawrocky, M., Schuurman, D.C., Fortuna, J., 2010. Visual sorting of recyclable goods deep learning. 2016 IEEE Technological Innovations in ICT for Agriculture and Rural
using a support vector machine. CCECE 2010, 1–4. Development (TIAR) 65–70.
Nowakowski, P., Pamuła, T., 2020. Application of deep learning object classifier to Sun, L., Zhao, C., Yan, Z., Liu, P., Duckett, T., Stolkin, R., 2019. A novel weakly-
improve e-waste collection planning. Waste Manage. 109, 1–9. supervised approach for rgb-d-based nuclear waste object detection. IEEE Sens. J. 19
Özkan, K., Ergin, S., Işık, Ş., Işıklı, İ., 2015. A new classification scheme of plastic wastes (9), 3487–3500.
based upon recycling labels. Waste Manage. 35, 29–35. Tachwali, Y., Al-Assaf, Y., Al-Ali, A.R., 2007. Automatic multistage classification system
for plastic bottles recycling. Resour. Conserv. Recycl. 52 (2), 266–285.

42
W. Lu and J. Chen Waste Management 142 (2022) 29–43

Taddia, Y., Corbau, C., Buoninsegni, J., Simeoni, U., Pellegrinelli, A., 2021. UAV community. Resour. Conserv. Recycl. 174, 105775. https://doi.org/10.1016/j.
approach for detecting plastic marine debris on the beach: A case study in the po resconrec.2021.105775.
river delta (italy). Drones, 140. Xiao, W., Yang, J., Fang, H., Zhuang, J., Ku, Y., 2020. Classifying construction and
Toğaçar, M., Ergen, B., Cömert, Z., 2020. Waste classification using AutoEncoder demolition waste by combining spatial and spectral features. Proceedings of the
network with integrated feature selection method in convolutional neural network Institution of Civil Engineers - Waste and Resource Management 173 (3), 79–90.
models. Measurement 153, 107459. https://doi.org/10.1016/j. Yang, L., Liu, J., Lu, Y., Ao, Y., Guo, Y., Huang, W., Zhao, R., Wang, R., 2020. Global and
measurement.2019.107459. local associations between urban greenery and travel propensity of older adults in
Verheyen, M., Beckers, W., Claesen, E., Moonen, G., Demeester, E., 2016. Vision-based Hong Kong. Sustainable Cities Soc. 63, 102442 https://doi.org/10.1016/j.
sorting of medium density fibreboard and grade a wood waste. 2016 IEEE 21st scs.2020.102442.
International Conference on Emerging Technologies and Factory Automation (ETFA) Yang, J., Zeng, Z., Wang, K., Zou, H., Xie, L., 2021. GarbageNet: A unified learning
1–6. framework for robust garbage classification. IEEE Transactions on Artificial
Vo, A.H., Hoang Son, L., Vo, M.T., Le, T., 2019. A novel framework for trash classification Intelligence 2 (4), 372–380.
using deep transfer learning. IEEE Access 7, 178631–178639. Yang, M., Thung, G., 2016. Classification of trash for recyclability status, CS229 Project
Vrancken, C., Longhurst, P., Wagland, S., 2019. Deep learning in material recovery: Report.
Development of method to create training database. Expert Syst. Appl. 125, Yang, L., Ao, Y., Ke, J., Lu, Y., Liang, Y., 2021. To walk or not to walk? Examining non-
268–280. linear effects of streetscape greenery on walking propensity of older adults.
Wang, C., Hu, Z., Pang, Q., Hua, L., 2019a. Research on the classification algorithm and J. Transp. Geogr. 94, 103099 https://doi.org/10.1016/j.jtrangeo.2021.103099.
operation parameters optimization of the system for separating non-ferrous metals Yang, Z., Li, D., 2020. WasNet: A neural network-based garbage collection management
from end-of-life vehicles based on machine vision. Waste Manage. 100, 10–17. system. IEEE Access 8, 103984–103993.
Wang, S., Wang, J., Yang, S., Li, J., Zhou, K., 2020a. From intention to behavior: Zhang, Q., Zhang, X., Mu, X., Wang, Z., Tian, R., Wang, X., Liu, X., 2021. Recyclable
Comprehending residents’ waste sorting intention and behavior formation process. waste image recognition based on deep learning. Resour. Conserv. Recycl. 171,
Waste Manage. 113, 41–50. 105636. https://doi.org/10.1016/j.resconrec.2021.105636.
Wang, Z., Li, H., Yang, X., 2020b. Vision-based robotic system for on-site construction Zhu, W., Chen, L., Wang, B., Wang, Z., 2018. Online detection in the separation process
and demolition waste sorting and recycling. Journal of Building Engineering 32, of tobacco leaf stems as biomass byproducts based on low energy x-ray imaging.
101769. https://doi.org/10.1016/j.jobe.2020.101769. Waste Biomass Valorization 9 (8), 1451–1458.
Wang, Z., Li, H., Zhang, X., 2019b. Construction waste recycling robot for nails and Zou, Z., Shi, Z., Guo, Y., Ye, J., 2019. Object detection in 20 years: A survey. arXiv
screws: Computer vision technology and neural network approach. Autom. Constr. preprint arXiv:1905.05055.
97, 220–228. Zulkifley, M.A., Mustafa, M.M., Hussain, A., Mustapha, A., Ramli, S., 2014. Robust
Wang, Z., Peng, B., Huang, Y., Sun, G., 2019c. Classification for plastic bottles recycling identification of polyethylene terephthalate (pet) plastics through bayesian decision,
based on image recognition. Waste Manage. 88, 170–181. PLoS ONE.
x670783915, 2019, Huaweiyun garbage classify learning. https://github.com/ Zuo, M., Yan, A., 2019, Shanghai begins new waste sorting era, as china eyes cleaner
x670783915/huaweiyun_garbage_classify__learning (Accessed Aug. 2 2021). image. https://www.scmp.com/news/china/society/article/3016801/shanghai-
Xia, Z., Zhang, S.i., Tian, X.i., Liu, Y., 2021. Understanding waste sorting behavior and begins-new-waste-sorting-era-china-eyes-cleaner-image (Accessed 2 November
key influencing factors through internet of things: Evidence from college student 2020).

43