Geo-spatial Information Science

ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/tgsi20

Deep learning for change detection in remote

sensing: a review

Ting Bai, Le Wang, Dameng Yin, Kaimin Sun, Yepei Chen, Wenzhuo Li &
Deren Li

To cite this article: Ting Bai, Le Wang, Dameng Yin, Kaimin Sun, Yepei Chen, Wenzhuo Li &
Deren Li (2023) Deep learning for change detection in remote sensing: a review, Geo-spatial
Information Science, 26:3, 262-288, DOI: 10.1080/10095020.2022.2085633

To link to this article: https://doi.org/10.1080/10095020.2022.2085633

© 2022 Wuhan University. Published by

Informa UK Limited, trading as Taylor &
Francis Group.

Published online: 01 Jul 2022.

Submit your article to this journal

Article views: 13572

View related articles

View Crossmark data

Citing articles: 27 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=tgsi20
GEO-SPATIAL INFORMATION SCIENCE
2023, VOL. 26, NO. 3, 262–288
https://doi.org/10.1080/10095020.2022.2085633

Deep learning for change detection in remote sensing: a review

a
Ting Bai , Le Wangb, Dameng Yin c
, Kaimin Sun d
, Yepei Chen e
, Wenzhuo Li d
and Deren Lid,f
a
Project Planning Department, Wuhan Land Arranging Storage Center, Wuhan, China; bDepartment of Geography, University at Buffalo,
The State University of New York, Buffalo, USA; cInstitute of Crop Sciences, Chinese Academy of Agricultural Sciences, Beijing, China; dState
Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China; eSchool of
Computer Science, Hubei University of Technology, Wuhan, China; fCollaborative Innovation Center of Geospatial Technology, Wuhan
University, Wuhan, China

ABSTRACT ARTICLE HISTORY

A large number of publications have incorporated deep learning in the process of remote Received 19 August 2021
sensing change detection. In these Deep Learning Change Detection (DLCD) publications, Accepted 30 May 2022
deep learning methods have demonstrated their superiority over conventional change detec­ KEYWORDS
tion methods. However, the theoretical underpinnings of why deep learning improves the Deep learning; change
performance of change detection remain unresolved. As of today, few in-depth reviews have detection; remote sensing;
investigated the mechanisms of DLCD. Without such a review, five critical questions remain review; information
unclear. Does DLCD provide improved information representation for change detection? If so, representation
how? How to select an appropriate DLCD method and why? How much does each type of
change benefits from DLCD in terms of its performance? What are the major limitations of
existing DLCD methods and what are the prospects for DLCD? To address these five questions,
we reviewed according to the following strategies. We grouped the DLCD information assem­
blages into the four unique dimensions of remote sensing: spectral, spatial, temporal, and
multi-sensor. For the extraction of information in each dimension, the difference between
DLCD and conventional change detection methods was compared. We proposed a taxonomy
of existing DLCD methods by dividing them into two distinctive pools: separate and coupled
models. Their advantages, limitations, applicability, and performance were thoroughly inves­
tigated and explicitly presented. We examined the variations in performance between DLCD
and conventional change detection. We depicted two limitations of DLCD, i.e. training sample
and hardware and software dilemmas. Based on these analyses, we identified directions for
future developments. As a result of our review, we found that DLCD’s advantages over
conventional change detection can be attributed to three factors: improved information
representation; improved change detection methods; and performance enhancements.
DLCD has to surpass the limitations with regard to training samples and computing infra­
structure. We envision this review can boost developments of deep learning in change
detection applications.

1. Introduction 1.1. Background of change detection

Deep learning is often treated as a black box; thus, the Change detection aims to discern both subtle and
theoretical underpinnings of deep learning mechan­ abrupt alternation in a given area as manifested through
isms that improve change detection in remote sensing remote sensing images acquired at different times
are little understood. In this paper, we argue that (Singh 1989). It provides a means to study land use/
change detection performance improvements stem­ cover change (Weng 2002), biodiversity (Newbold et al.
ming from deep learning result from improved infor­ 2015), the urbanization process (Han et al. 2017a), dis­
mation representation and more effective models than aster detection (Saito et al. 2004), and other environ­
those found in statistical, and machine learning meth­ mental changes. Conventionally, two types of changes
ods, more commonly deployed. A comparative review are often examined (Lu et al. 2004): 1) binary changes,
of the related literature shows that Deep Learning and 2) from-to changes. The binary changes only focus
Change Detection (DLCD) provides semantic infor­ on where change has been incurred or not. From-to
mation neglected by other methods, and flexible net­ change detection not only detects variation over time
work configurations that can improve change but also the specific types of change e.g. from buildings
detection methods and performance. Semantic infor­ to vegetation. Lately, a new type of change has been
mation, improved change detection methods, and per­ studied: multi-class changes, i.e. clustering the detected
formance enhancements facilitate improved change changes into different groups regardless of the land
detection outcomes in remote sensing applications. cover types (Saha, Bovolo, and Bruzzone 2019).

CONTACT Le Wang [email protected]

© 2022 Wuhan University. Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits
unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting
of the Accepted Manuscript in a repository by the author(s) or with their consent.
 GEO-SPATIAL INFORMATION SCIENCE 263

Substantial research has been carried out to recog­ easily uncovered from the time series data. 2) The data
nize these three types of change. The definition of sources of change detection are more diverse than
change detection was first introduced by Singh ever. Contemporary data are comprised of multiple
(1989) but more recently, Tewkesbury et al. (2015) spectrums of platforms (satellite and aerial), sensors
summarized all the change detection methods in six (passive and active), spatial resolution (250 m to sub-
categories: 1) Layer arithmetic methods (Howarth, meter), and spectral resolution (multispectral and
and Wickware 1981), 2) Post-Classification Change hyperspectral). This diverse source makes information
Methods (PCCMs) (Silván-Cárdenas, and Wang expression not consistent. What’s more, there una­
2014; Yuan et al. 2005), 3) Direct Classification voidably exists a problem of spectral variability in
Methods (DCMs) (Bovolo, Bruzzone, and more hyperspectral imagery (Hong et al. 2018),
Marconcini 2008), 4) Transformation methods which leads to inaccurate change detection results. 3)
(Gong 1993), 5) Change Vector Analysis (CVA) meth­ The data tend to suffer from various degradation,
ods (Chen et al. 2003; Johnson, and Kasischke 1998), noise effects, or variabilities in the process of imaging
and 6) Hybrid change detection methods (Healey et al. due to atmospheric conditions, illumination, viewing
2018; McDermid et al. 2008). Among these six cate­ angles, soil moisture, etc. These factors are random
gories, the layer arithmetic and transformation meth­ and difficult to be considered precisely in traditional
ods first obtain a Difference Image (DI) and then select change detection methods. Therefore, how address
a threshold to discriminate the DI to obtain binary these three issues have become a pressing challenge
changes. These methods are easy to implement but before change detection can be working for the big
cannot detect from-to changes because of the infor­ remote sensing data.
mation loss in the process of obtaining the DI.
Unlike these two categories, PCCMs and DCMs are
1.2. Deep learning methods used in change
both to classify multi-date images to detect changes.
detection
The difference between these two categories is that
PCCMs conduct change analysis after obtaining the The evolution of deep learning has demonstrated
independent classification map of each image, whereas immense potential for addressing various change
DCMs directly classify stacked features obtained from detection challenges in remote sensing. Deep learning
multi-date images to detect changes. As a result, is a particular kind of machine learning method based
PCCMs can be used for multi-sensor images, and on artificial neural networks with representation
detect both binary changes and from-to changes. learning (LeCun, Bengio, and Hinton 2015).
However, errors stemming from the classification Compared with other machine learning methods, the
maps are compounded in the final change map, thus deep learning model achieves great power and flex­
reducing the accuracy of the final change detection ibility by representing the world as a nested hierarchy
result (Chan, Chan, and Yeh 2001; Dai, and of concepts, with each concept defined about simpler
Khorram 1999; Lillesand, Kiefer, and Chipman concepts, and more abstract representations com­
2015). On the other hand, DCMs can overcome the puted in terms of less abstract ones (LeCun, Bengio,
error propagation problem, but requires training sam­ and Hinton 2015).
ples of all change types. To simultaneously mitigate As of today, deep learning has been intensively used
the error propagation problem and the sampling diffi­ in the remote sensing field (Ma et al. 2019a), such as
culty, the fifth category, CVA, was developed. CVA image fusion (Shao, and Cai 2018), image registration
constructs change vectors, and then utilizes their mag­ (Wang et al. 2018b), image matching (Chen,
nitudes and directions to detect changes. This method Rottensteiner, and Heipke 2021; Zhang et al. 2020;
can detect binary changes, but its ability to detect Cheng et al. 2020), change detection (Zhu et al.
from-to changes is limited (Tewkesbury et al. 2015). 2018), land use and cover classification (Hong et al.
To combine the merits of these five categories, the last 2021, 2020b, 2020a; Li et al. 2022; Zhou et al. 2022),
category, hybrid change detection methods, emerged. semantic segmentation (Kemker, Salvaggio, and
However, this category simultaneously inherits the Kanan 2018), and object-based image analysis (Liu,
weaknesses of the change detection methods used. Yang, and Lunga 2021). Specifically, a new sub-field,
The big data era has further complicated the pro­ known as DLCD, has emerged when deep learning
blems associated with conventional change detection methods are employed to detect changes in multi-
methods (Reichstein et al. 2019). This is manifested in date remote sensing images.
the following three ways: 1) Data volume of change Compared to other topics in remote sensing, DLCD
detection is unprecedentedly large and quickly is still in its infancy. We carried out a search of all the
expanding, compounding the weaknesses. For exam­ published papers that involves DLCD using the search
ple, an error will be further propagated when more query “deep learning AND change detection” in the
data layers are engaged; the required number of train­ Scopus database, similar to Ma et al. (2019a).
ing samples is escalating; from-to changes cannot be Altogether 80 publications (Figure 1) have witnessed
264 T. BAI ET AL.

the early-stage developments of DLCD-relevant appli­ Convolutional Neural Networks (CNNs), Auto-
cations between 2013 and 2019, including peer- Encoders (AEs), Deep Belief Networks (DBNs),
reviewed articles and conference papers. Recurrent Neural Networks (RNNs), and Generative
There is not a lot of DLCD literature as compared Adversarial Networks (GANs). DLNNs may each con­
with the scene classification and object detection, land tain multiple specific models. For example, CNNs can
use and cover classification, and semantic segmenta­ be implemented using ResNet, AlexNet, DenseNet,
tion tasks because DLCD faces additional challenges. etc. To discuss the structures of these specific models
While segmentation and classification tasks work on are beyond the scope of this review; thus, these specific
images at a single time point, change detection works models are not included. For each type of DLNNs,
simultaneously on images at multiple time points. Table 1 lists the first work that proposed the DLNN
Therefore, the problems and limitations (e.g. noise) (deep learning reference) and the first work that used
associated with single-date image information extrac­ the DLNN in change detection (DLCD reference). To
tion are multiplied in DLCD. Furthermore, the multi- further understand the structure of specific DLNNs or
date images may contain different information expres­ how they are used in change detection, readers are
sions (e.g. inconsistent radiometric and geometric referred to these publications.
information, different spatial resolution, and different
sensor), which makes DLCD even more difficult.
Another particular difficulty in DLCD is that 1.3. Unsolved problems in DLCD reviews
change is often minimal in the images. While segmen­ Until now, most existing studies reviewing deep learn­
tation and classification tasks extract the bulk of the ing are general reviews concerning the algorithm devel­
relevant information, change detection aims to find opment of deep learning and remote sensing specific
the minority pixels that have experienced changes. applications (Zhang, Zhang, and Du 2016; LeCun,
Because changes only occur in limited amounts Bengio, and Hinton 2015; Ball, Anderson, and Chan
(<50%) in general, the change information is easily 2018; Ma et al. 2019a; Yuan et al. 2020; Zhu et al. 2017).
confused with noise. The quantity of the related Zhu et al. (2017) and Ma et al. (2019a) have reviewed
DLCD publications, however, is increasing rapidly applications and technology of deep learning remote
(Figure 1) due to the need of change detection sensing from preprocessing and mapping. Yuan et al.
applications. (2020) have compared the use of the traditional neural
Among these studies, the most popular Deep network and deep learning methods to advance the
Learning Neural Networks (DLNNs) (i.e. deep learn­ environmental remote sensing process. There are two
ing models) used in change detection are review papers for DLCD, which provided a technical

Figure 1. Publication numbers per year of DLCD.

Table 1. The first proposed reference and DLCD reference for each DLNN.
DLNN Deep learning Reference DLCD reference
CNN LeCun et al. (1998) Iino et al. (2018)
AE Hinton and Salakhutdinov (2006) Chen, Shi, and Gong (2016)
DBN Hinton, Osindero, and Teh (2006) Argyridis and Argialas (2016)
RNN Bengio, Simard, and Frasconi (1994) Lyu, Lu, and Mou (2016)
GANs Goodfellow et al. (2014) Gong et al. (2017a)
 GEO-SPATIAL INFORMATION SCIENCE 265

review of the advance of deep learning for change better understanding of the difference can help to
detection (Shi et al. 2020; Khelifi, and Mignotte 2020). measure the accuracy vs. computation burden tradeoff
These two reviews are from the perspectives of the when selecting deep learning or conventional methods
implementation process, technical methods, and data in different change detection applications.
sources, providing a good starting point for beginners 4) What are the major limitations of existing DLCD
to understand DLCD. However, the theoretical under­ methods?
pinning of why deep learning improves the perfor­ The incorporation of deep learning introduces two
mance of change detection as compared with major limitations including training sample and hard­
conventional change detection remains unresolved. ware and software dilemmas except greatly benefiting
Such knowledge could enable refinements in the exist­ change detection. The excellent performance of typical
ing deep learning methods and thus supersede current deep learning models relies on extremely large labeled
limitations such as data pollution and opaque network training sample sets (Gong et al. 2019b; Wang et al.
configurations. Specifically, to unveil the black box of 2018a) but geospatial systems usually have particularly
DLCD, the following questions have to be investigated. limited labeled training samples available (Ball,
1) Does DLCD provide improved information Anderson, and Chan 2018; De et al. 2017; Reichstein
representation for change detection? If so, how? et al. 2019), especially change detection. Deep learning
The first step in DLCD begins with data input. In places high demands on computational power in hard­
general, four different types of inputs are engaged in a ware and software. These pose two big challenges for
typical DLCD study, i.e. spectral, spatial, temporal, DLCD, as no research is available. The lack of such
and multi-source information. It is critical to under­ research may become a major hurdle for further
stand whether DLCD has a more powerful capability applying deep learning in change detection.
to represent the four types of information compared to 5) What are the prospects for DLCD?
the conventional change detection methods. If so, how Based on the analysis of the benefits and limitations
such information is represented. Without such under­ of DLCD, we can outline prospects. Although there
standing, it is a challenge to further refine DLCD are some discussions of future directions in the DLCD
methods to achieve the best tradeoff that can max­ literature, these articles have tended to focus on the
imize the authentic information carried over in the disadvantages of methods themselves, or implementa­
input and suppressed the noise at the same time. tion processes, and so there is no overview of direc­
2) How to select an appropriate DLCD method tions for future research based on the theoretical
and why? underpinnings. This paper addresses this gap.
Methodology plays an important role in change
detection. The introduction of deep learning network
1.4. Structure of this review
architectures opens up a new avenue for change detec­
tion. Since 2013, a variety of DLCD methods has To answer the five questions, our review made five
emerged. Each DLCD method has its respective com­ major contributions, as follows:
bination of network layers, training samples’ require­
ments, and applicability. These characteristics result in (1) We compare the difference in how spectral,
different performances. A systematical comparison of spatial, temporal, and multi-sensor information
different advantages, disadvantages, and performance are represented in DLCD and conventional
of DLCD methods would give methodological gui­ change detection methods.
dance in future change detection applications. (2) We propose a taxonomy for DLCD methods by
However, such a comparison is not available. dividing them into two distinctive pools: sepa­
Without such knowledge, it is a challenge to select rate and coupled DLNNs, based on which, a
DLCD methods suitable for actual applications. thorough analysis of their advantages, limita­
3) How much does each type of change benefits tions, applicability, and performance is
from DLCD in terms of their performance? investigated.
There are three types of change detection results (3) We examine the holistic accuracy between
including binary changes, multiclass changes, and DLCD and conventional change detection
from-to changes. The complexity of information pro­ methods by adopting a box plot to analyze four
cessing increases from binary change detection to major land-use types: Urban, Water, Vegetation,
from-to change detection. As a result, the performance and Hazards for the three primary change types:
improvement from conventional change detection to i.e. binary, multiclass, and from-to.
DLCD is different regarding these three types of (4) We discuss two limitations of DLCD including
changes. Such difference is even more varied depend­ training sample and hardware and software
ing on the application. However, this difference has dilemmas.
not been reviewed systematically. Given the fact that (5) We identify four directions for future
deep learning has a larger computation burden, a development.
266 T. BAI ET AL.

2. Improved information representation Technically, such multi-scale integration is implemen­

ted using a large number of neurons and a complex
The success of change detection depends on maximiz­
network structure for deep learning. The advantages
ing real change. Although both conventional change
of deep learning become more prominent as the spec­
detection and DLCD aims to achieve this goal, the key
tral band information increases. DLCD methods can
difference between the two groups of methods is in the
directly handle the high-dimensional spectral infor­
execution. DLCD is optimized for spectral, spatial,
mation and more effectively learn available multi-
temporal, and multi-sensor information representa­
scale spectral information. This information includes
tion. In this section, we will compare how these
detailed spectral information at a low-level scale and
kinds of information are extracted by DLCD as
semantic information at a high-level scale (Wang et al.
opposed to the conventional change detection
2019; Li, Yuan, and Wang 2019).
methods.

2.2. Spatial information

2.1. Spectral information
DLCD is optimized for multi-scale spatial information
DLCD is optimized for abstract spectral information extraction as compared with the conventional change
representation (such as spectral curve) compared with detection methods. This spatial information includes
the conventional change detection methods. texture, location information at low scale, and seman­
Traditional thresholding-based, transformation, spec­ tic information at a high scale, and the most com­
tral analysis, and classification-based methods (Liu et monly used conventional change detection methods
al. 2014) can extract low-dimension spectral informa­ for extracting spatial information are object-based
tion (such as the mean value(R, G, B), the standard (Zhou, Yu, and Qin 2014; Chehata et al. 2014; Tang,
variance(R, G, B), the brightness, and the maximum Huang, and Zhang 2013), which can extract multi-
difference). However, they neglect abstract spectral scale and hierarchical spatial information features.
information, resulting in inaccurate recognition of However, the extraction accuracy is heavily dependent
complicated land use and land cover changes. on the accuracy of the initial segmentation (Cao et al.
DLCD adopts a different strategy to discern real 2014; Tewkesbury et al. 2015; Hussain et al. 2013).
changes based on multi-scale spectral information Another common method is to extract spatial features
differences, as shown in Figure 2. Figure 2 outlines a based on the fixed and limited window such as the
basic scenario of change detection based on spectral gray level co-occurrence matrix (Murray, Lucieer, and
information. T1 and T2 represent the reflectance at two Williams 2010). Thus, these extracted spatial features
different times (i.e. before the change and after are limited-scale features, are not complete and sys­
change). Two rectangular boxes represent the multi- tematic, and thus are not robust enough to detect
scale spectral information difference between T1 and complicated urban land use and land cover changes.
T2. Typical change detection detects changes based on In contrast to the conventional change detection
the spectral information difference presented between methods, DLCD utilizes multi-scale spatial informa­
T1 and T2 reflectance curves. tion differences to discern real changes, presented in
From Figure 2, one example of such multi-scale Figure 3. Figure 3 outlines an example of change
spectral information difference can be understood as detection based on spatial information. A typical
conducting an average of the original spectral infor­ change detection approach detects changes based on
mation curve (e.g. T1 and T2) with a different number the spatial information difference presented between
of spectral bands. Once an average curve is derived, T1 and T2.
the difference between T1 and T2 is calculated. Then all From Figure 3, one example of such multi-scale
the changes at disparate scales will be accumulated spatial information difference can be understood as
before an overall conclusion of change can be derived. extracting the spatial features (e.g. T1 and T2) utilizing

Figure 2. The usage of spectral information in DLCD.

 GEO-SPATIAL INFORMATION SCIENCE 267

Figure 3. An example of the usage of spatial information in DLCD.

CNN. CNN utilizes the convolution kernels of differ­ either based on time-sequential reflectance or reflec­
ent sizes (e.g. 5 × 5) to extract spatial features from a tance rank (Zhu 2017; Shao et al. 2019a); however,
32 × 32 patch in T1 and T2. Once the spatial features time-series change detection uses limited and linear
are derived, the spatial information difference between statistical features to depict continuous temporal
T1 and T2 can be calculated. All the changes at dis­ information, neglecting other types of temporal
parate scales will be accumulated before an overall information.
change determination can be derived. Technically, DLCD adopts a recurrent method (Section 3.2.3) to
such multi-scale integration is implemented using discern real changes based on the differences in the
multi-layer convolutions. Unlike conventional change time-series curves, as shown in Figure 4. Figure 4
detection methods, DLCD can automatically extract outlines a basic scenario of change detection based
multi-scale spatial information (Amit, and Aoki 2017; on temporal information. The blue curve represents
El Amin, Liu, and Wang 2016; Khan et al. 2017). Pixel 1 (Change) and the yellow curve represents Pixel
2 (No change). A typical time-series change detection
detects changes based on the temporal feature differ­
2.3. Temporal information
ences from the time-series curves between T1, T2, . . . ,
DLCD is optimized for extracting nonlinear temporal and TN for different pixels.
information compared with the conventional change From Figure 4, for Pixel 1 and Pixel 2, the reflec­
detection methods. Conventionally, time-series tance {X1, X2, . . . , and XN} in T1, T2, . . . , and TN is the
change detection utilizes statistical metrics such as input of the recurrent method. The recurrent method
individual ranks, means, and regression slopes for learns temporal information by building recurrent
spectral bands and vegetation indices as the input connections between T1, T2, . . . , and TN. Thus, the
(Shao, and Liu 2014). These are generally calculated output feature information of TN depends on input XN

Figure 4. The usage of temporal information in DLCD.

268 T. BAI ET AL.

in TN and feature information in TN-1. Once the tem­ to find the relationship between the unchanged areas
poral feature in TN for Pixels 1 and 2 is derived, the in multi-sensor remote sensing images and they do
feature difference between Pixel 1 and Pixel 2 is calcu­ not consider the influence of changed areas when
lated. An overall conclusion of change can be derived. identifying this relationship.
Technically, such a recurrent method is implemented DLCD adopts a mapping transformation method
using the recurrent connections between the neural (Section 3.1.5.2.2) to discern authentic changes based
activations of RNN at consecutive time steps. In con­ on different sensor information differences between T1
trast to traditional change detection methods, the and T2, as shown in Figure 5. Figure 5 outlines an
DLCD can learn detailed temporal information as example of the usage of multi-sensor (multi-source
well as nonlinear temporal information automatically, and multi-spatial) information in DLCD. A typical
supporting the detection of complicated urban land- multi-sensor change detection is how to detect
use change (Lyu, Lu, and Mou 2016; Mou, Bruzzone, changes from T1 and T2 for different sensors.
and Zhu 2018) and vegetation phenological patterns From Figure 5, one example of such multi-sensor
(Song et al. 2018). The advantages of DLCD become information difference can be understood as detecting
more pronounced as the time series expands. changes between Landsat (T1) and sentinel 1 (T2). The
mapping transformation method transforms T1 and T2
image information into a multi-scale feature space to
2.4. Multi-sensor information (multi-source/ obtain features similar to those found in the other image
multi-spatial) or finds the relationship between two image features.
Once the similar features or the relationships between
DLCD is optimized for extracting multi-scale features
features in two images are derived, the difference
compared with the conventional change detection
between T1 and T2 can be calculated, and an overall
methods. Change detection methods for multi-sensor
determination of change can be derived. Technically,
images can be divided into two categories: 1) compara­
the mapping transformation method is implemented by
tive analysis of independently produced classifications
building transformation neural networks. Unlike tradi­
for different sensors and 2) simultaneous analysis of
tional change detection methods, the DLCD method
multi-temporal data (Singh 1989; Zhang et al. 2019).
learns multi-scale features and transforms inconsistent
The former conducts multi-sensor images classification
information into consistent information. In addition, it
separately for change detection analysis. DLCD also has
can enlarge the distance between changed pixels as the
this type of method, i.e. the post-classification change
constraint rule to find more accurate relationships
method (Section 3.1.1.1), which extracts high-dimen­
between information transformations.
sional feature representations to replace limited and
low-level feature extractions (Iino et al. 2018; Nemoto
et al. 2017). This method obtains higher accuracy than
3. Improved change detection methods
the traditional post-classification methods (Iino et al.
2018; Nemoto et al. 2017). DLCD improves change detection methods as well as
The simultaneous analysis of multi-temporal data information representation for change detection. To pro­
methods includes the copula theory (Mercier, Moser, vide methodological guidance for future applications, we
and Serpico 2008), manifold learning strategy present a taxonomy of these methods and provide a
(Prendes et al. 2015b), kernel canonical correlation systematic comparison of their advantages, limitations,
analysis (Volpi, Camps-Valls, and Tuia 2015), and applicability, and performance. DLCD methods are sepa­
Bayesian nonparametric model associated with a rated into two categories at the first level as either sepa­
Markov random field (Prendes et al. 2015a). rate or coupled DLNNs. At the second level, seven sub-
However, these methods utilize the limited features categories are included. The separate DLNNs contain

Figure 5. An example of the usage of multi-sensor information in DLCD.

 GEO-SPATIAL INFORMATION SCIENCE 269

Figure 6. The structure of DLCD methods. Yellow boxes represent DLNNs while blue boxes denote the interaction of multi-date
information. (a) Post-classification change method (PCCM). (b) Differencing method (DM). (c) Direct classification method (DCM).
(d) Differencing neural network method (DNNM). (e) Mapping transformation method (MTM). (f) Recurrent method (RM). (g)
Adversarial method (AM).

three sub-categories: PCCM, Differencing Methods Recurrent Method (RM), and Adversarial Method
(DM), and DCM. The coupled DLNNs contain four (AM). A diagram of separate DLNNs and coupled
sub-categories: Differencing Neural Network Method DLNNs is shown in Figure 6. The frameworks are pro­
(DNNM), Mapping Transformation Method (MTM), vided for each sub-category.
270 T. BAI ET AL.

From Figure 6, change detection requires the inter­ independent classification maps. In addition, PCCM
action between multi-date information. The main dif­ can detect complete from-to changes. Nevertheless,
ference between separate DLNNs and coupled DLNNs similar to the traditional PCCM, the disadvantage is
is clearly shown as the relationship between DLNNs that the errors from two classification maps can accu­
(yellow boxes) and the interaction (blue boxes). In mulate in the final change detection map. To date, this
separate DLNNs, DLNNs are used to generate deep method has only been used to detect changes in urban
features before interaction or after interaction between land use (Nemoto et al. 2017) (Table 2).
multi-date image information, but DLNNs do not
directly interact. Coupled DLNNs, in contrast, multi-
3.1.2. Differencing method
date interaction occurs between DLNNs. The taxon­
DM detects change by comparing the differences in
omy of DLCD methods is summarized in Table 2. The
image radiance (Li et al. 2017; Liu et al. 2016b; Xiao et
difference, definition, advantages, limitations, and
al. 2018) or deep features (Gong et al. 2017b; Su et al.
application examples are provided for each type of
2016; Xu et al. 2013), and then changes are detected
DLCD.
with a supervised or unsupervised deep learning tech­
From Table 2, separate DLNNs are adapted from
nique. If differences in image radiance are used in DM,
conventional change detection methods by refining
we refer to it as image DM. Image DM has three steps
image features with DLNNs. They share the same
(Figures 6(b)-(1)). First, differencing; a DI is obtained.
principles as the corresponding conventional change
Second, is feature learning; DLNNs are built to extract
detection methods. The coupled DLNNs, on the other
the deep features of this DI. Third, classification; the
hand, are novel strategies where multiple DLNNs
deep features are input into a soft classifier to obtain
interact to maximize the change information. Unlike
the final change map. If differences in deep features are
separate DLNNs, coupled DLNNs require more
used in DM, we refer to it as feature DM. Feature DM
knowledge about deep learning and cannot be directly
also has three steps (Figures 6(b)-(2)). First, is feature
compared to conventional change detection methods.
learning; two images are input into two DLNNs to
A more detailed discussion of each sub-category is
obtain deep features. Second, differencing; different
provided in the following sections.
features are obtained. Third, classification; these fea­
tures are input into a classifier to obtain the final
change detection result.
3.1. Separate DLNNs
The key step in DM is differencing, which aims to
In this sub-section, we introduce three types of meth­ suppress unchanged information and highlight chan­
ods that do not allow DLNNs to interact but only use ged information. Although the techniques that have
DLNNs to extract deep features, i.e. separate DLNNs been used to obtain DI or difference features are
(Figure 6). They are adapted from traditional PCM, limited, more are available. Similar to traditional
layer arithmetic and transformation, and traditional layer arithmetic, operations such as subtracting
DCM, respectively. (Arabi, Karoui, and Djerriri 2018), log rationing
(Gong, Yang, and Zhang 2017), and log-mean ration­
ing (Li et al. 2017) can be applied. Techniques such as
3.1.1. Post-classification change method
Principal Component Analysis (PCA) can serve a
PCCM compares classification maps from different
similar purpose. For example, El Amin, Liu, and
times to identify changes between them. PCCM
Wang (2017) applied PCA to stacked high resolution
includes three steps (Figure 6(a)). First, feature learn­
images in order to identify changed features.
ing; two images are input into separate DLNNs to
In summary, like the PCCM, DM is also easy to
obtain deep feature representations of each image
implement and the subsequent change detection result
respectively. Secondly classification; the deep features
is easy to interpret. In this method, only one classifica­
are classified separately to obtain two classification
tion stage is required and identified changes are the­
maps. Third, is change analysis; the two classification
matically labeled. Nevertheless, DM does not provide
maps are compared to obtain the final change map.
complete from-to changes. To date, this method has
The difference between this method and a traditional
been widely used for urban land use (Xu et al. 2013),
PCCM is that feature learning is through a deep learn­
water and hazard (Zhao et al. 2014), and vegetation
ing model (Nemoto et al. 2017; Iino et al. 2018).
change detections (Li et al. 2017) (Table 2).
PCCM is easy to implement because the DLNN is
only applied for feature learning but no effort is
required for the configuration of the DLNN structure. 3.1.3. Direct classification method
Another advantage of PCCM is that it works on multi- DCM directly classifies stacked multi-date images or
sensor images and does not require radiometric nor­ deep features for change detection. If images are
malization, because the changes are detected from two stacked in DCM, we refer to it as image stacking
 GEO-SPATIAL INFORMATION SCIENCE 271

DCM. Image stacking DCM includes three steps initial DI to design a cost function. Cao et al. (2017)
(Figures 6(c)-(1)). Multi-date images are stacked and extended a backpropagation algorithm to build a cost
the stacked images are input into DLNN to learn the function between two DBNs.
deep features. The resulting deep features are input In summary, these methods enhance the difference
into a soft classifier for change detection. in change areas and effectively suppress the noise as
If deep features are stacked in DCM, we refer to it as compared to traditional DI creation methods (Chu,
feature stacking DCM. Feature stacking DCM also Cao, and Hayat 2016). Compared to the methods in
includes three steps (Figures 6(c)-(2)). Two images separate DLNNs, especially DM, the advantage of this
are input into DLNNs to obtain deep feature repre­ method is that it uses an extra cost function to simul­
sentations of each image respectively. The two sets of taneously adjust the parameters of two DLNNs, which
deep features are stacked. The stacked features are makes the network parameters more accurate.
input into a soft classifier to obtain the final However, the disadvantage is that because of this
change map. additional cost function, the structure of this method
In summary, like the other separate DLNNs is complex. It is difficult to construct training sample
(PCCM and DM), DCM is also easy to implement. sets for from-to change detection. To date, this
Like traditional DCM, this method requires only one method has been used for urban land use (Chu, Cao,
classification stage and can identify changes themati­ and Hayat 2016), water and hazard (Chen, Shi, and
cally (Tewkesbury et al. 2015). However, the disadvan­ Gong 2016), and vegetation change detections (Geng
tage is difficult to construct training samples for from- et al. 2017) (Table 2).
to-change detection. To date, this method has been
widely used for urban land use (Zhang et al. 2016a), 3.2.2. Mapping transformation method
water (Gao et al. 2019a), and hazard and vegetation MTM constructs a transformation function layer
change detections (Gong et al. 2015) (Table 2). between inconsistent deep feature representations for
change detection. For multi-spatial resolution or
3.2. Coupled DLNNs. multi-source remote sensing images, the direct com­
In this sub-section, we introduce four types of meth­ parison between pixel-pair or feature-pair is meaning­
ods that not only allow DLNNs to interact but also use less (Zhan et al. 2018; Zhang et al. 2016b). Therefore,
DLNNs to extract deep features, i.e. coupled DLNNs MTM was proposed to explore the inner relationships
(Figure 6). Compared to the separate DLNNs where between multi-sensor data.
DLNNs are adapted from the traditional change detec­ MTM includes four steps (Figure 6(e)). Two images
tion methods, four types of methods in the coupled are input into DLNNs to obtain deep feature repre­
DLNNs category are novel implementation strategies sentations of each image, respectively. A transforma­
for specific change detection application scenarios. tion function between the two sets of deep features is
DNNM can highlight different information, so it is constructed. The similarity of the transformed features
resistant to noise. MTM can deal with multi-source of the two images is calculated, and similarity features
images and multi-spatial-resolution images. RM can are clustered to obtain the final change map.
be used in change detection applications that are time- The key step in MTM is transformation. The trans­
sensitive (e.g. crop growth). AM can eliminate noise formation function can be constructed based on dif­
and generate high-quality information. ferent principles. A transformation function can be
built based on the principle that unchanged pixels at
3.2.1. Differencing neural network method the same position in two input images have similar
DNNM builds a cost function between two DLNNs to representations (Liu et al. 2016a; Zhang et al. 2016b).
highlight the difference in deep features for change Conversely, the transformation function can be built
detection. DNNM includes three steps (Figure 6(d)). by shrinking the difference between the paired features
Two images are input into DLNNs to pre-train two of unchanged positions while enlarging the difference
DLNN models. A cost function is used to adjust the between the paired features of changed positions
parameters of the two DLNNs to generate deep fea­ (Zhan et al. 2018). The target of these transformation
tures. With these deep features, the difference between functions is to make incomparable information from
changed pixels at T1 and T2 is enlarged. The two sets of different sensors comparable. For instance, Zhang et
deep features are input into a soft classifier to obtain al. (2016b) for the first time proposed a Stacked
the final change detection result. Denoised Auto-Encoder (SDAE) based MTM to
The key step in DNNM is fine-tuning, i.e. the cost detect changes between images of different resolu­
function, which aims to highlight the changed infor­ tions. Liu et al. (2018) established a novel MTM by
mation and suppress the unchanged information. using a convolutional coupling network to detect the
There are different methods to build a cost function. change between optical and radar images. The success
For example, Chen, Shi, and Gong (2016) used the in these examples showed that this method is capable
difference between bi-temporal deep features and an of detecting changes from multi-sensor images.
 272

Table 2. The taxonomy, difference, definition, advantages, limitations, and applications of DLCD methods.
T. BAI ET AL.

Category Difference Sub-category Definition Advantages Limitations Applications

3.1.Separate Adapted from conventional change detection methods, Post-Classification Compares multi-date ● abb3”>Easy to ● Errors from individual Urban land use (Nemoto et al. 2017)
DLNNs using DLNNs to refine features. They share the same Change Method classification maps to implement. classification maps accu­
(No principles with their corresponding conventional (PCCM) identify changes. ● Applicable to mulated in the final
interaction methods. multi-sensor change map.
between images.
DLNNs) ● Provides complete
from-to changes.
Differencing Method Compares the difference of ● Easy to implement. ● Difficult to provide com­ Urban land use (Xu et al. 2013), water and
(DM) image radiance or deep ● Easy to interpret plete from-to changes. hazard (Zhao et al. 2014), and vegetation
features to identify changes. change detection (Li et al. 2017)
results.
Direct Classification Stacks multi-date images or ● Easy to implement. ● Difficult to construct Urban land use (Zhang, Zhang, and Du
Method (DCM) deep features to conduct training samples. 2016), water (Gao et al. 2019a)
classification directly. ● Difficult to provide com­
plete from-to changes.
3.2.Coupled Novel strategies that modify deep learning models. The Differencing Neural Builds a cost function between ● Accurate deep ● Complex to implement. Urban land use (Chu, Cao, and Hayat 2016),
DLNNs processes cannot be directly compared to conventional Network Method two DLNNs to highlight the learning network ● Difficult to construct water and hazard (Chen, Shi, and Gong
(DLNNs change detection methods. (DNNM) difference of deep features. parameters training samples. 2016), and vegetation (Geng et al. 2017)
interact) obtained. ● Difficult to provide com­
plete from-to changes.
Mapping Constructs a transformation ● Applicable to ● Complex to implement. Urban land use (Zhan et al. 2018), water and
Transformation function layer between multi-source or ● Difficult to construct hazard (Zhang, Zhang, and Du 2016), and
Method (MTM) inconsistent deep features. multi-spatial- reso­ training samples. vegetation (Su et al. 2017)
lution images. ● Difficult to provide com­
plete from-to changes.
Recurrent Method (RM) Uses recurrent connections ● Makes full use of ● Complex to implement. Urban land use (Lyu, Lu, and Mou 2016) and
between multi-date images temporal features. ● Difficult to construct vegetation (Song et al. 2018)
to learn deep features. training samples.
● Difficult to provide com­
plete from-to changes.
Adversarial Method Makes generator and ● Generates high- ● Complex to implement. Urban land use and water (Gong et al.
(AM) discriminator neural quality ● Difficult to provide com­ 2017a)
networks play against each information. plete from-to changes.
other.
 GEO-SPATIAL INFORMATION SCIENCE 273

MTM is a powerful tool for transforming informa­ distinguish the feature map from the real data. This
tion from heterogeneous images into consistent infor­ classification method is used to classify a feature map
mation. However, like the DNNM, the disadvantage is to obtain a binary change detection map.
that the additional transformation function makes its The generator is common DLNNs such as CNN
structure more complex. It is also difficult to construct (Gong et al. 2017a). Choosing which DLNN for the
training samples and provide complete from-to generator depends on the specific application (Radford,
changes. To date, this method has been used for Metz, and Chintala 2015). Gong et al. (2017a) used this
detecting changes in urban land use (Zhan et al. AM based on GANs to generate a better DI for change
2018), water and hazard (Zhang et al. 2016b), and detection, which has less noise than a real DI. Gong et al.
vegetation applications (Su et al. 2017) (Table 2). (2019b) used this method to generate training data and
used these additional training data, label data, and unla­
3.2.3. Recurrent method beled data to build a semi-supervised classifier for change
RM uses recurrent connections between multi-date detection. The success of these two examples shows that
images to learn deep features and includes three this method has a powerful ability for generating high-
steps (Figure 6(F)). First, is feature learning. DLNNs quality information.
such as DBN, AE, and CNN are used to extract spec­ Compared with other methods, the advantage of
tral or spatial features, while RNNs extracts temporal AM is that it can eliminate noise and generate high-
features. The temporal features are input into a soft quality information, which provides a new avenue
classifier to obtain the final change detection result. for change detection tasks such as generating train­
This method adds temporal features to traditional ing samples and better change information.
spectral or spatial feature methods. For instance, Lyu, However, the disadvantage is the combination of
Lu, and Mou (2016) used a long short-term memory generator and discriminator neural networks makes
network to learn spectral and temporal features for its structure complex. It is also difficult to provide
change detection. Mou, Bruzzone, and Zhu (2018) complete from-to changes. To date, this method has
proposed a new RM by combining CNN and RNN been used for detecting changes in urban land use
to learn joint spectral-spatial-temporal features for and water applications (Gong et al. 2017a) (Table 2).
change detection. These examples demonstrate that
learning temporal features can be an effective way to
3.3. Performance comparison
detect change. In addition, it can learn the phenologi­
cal characteristics of vegetation. For example, Song et Performance comparisons of the overall accuracy of
al. (2018) proposed an RM by using a 3D fully con­ different DLCD methods, including DM, DCM,
volutional network and a convolutional long short- DNNM, MTM, RM, and AM were made using a
term memory network to learn phenological features boxplot, as shown in Figure 7. The accuracy values
for change detection. These examples show that for each DLCD method are from case studies in the
extracting phenological features can effectively facil­ respective DLCD references. Some of the referenced
itate change detection. methods may use the same dataset. For example, the
As compared to other methods, RM with the help of Ottawa dataset is used for evaluation studies of the
RNN considers the temporal connections in multi-tem­ difference (Li et al. 2017) and direct classification
poral change detection tasks. A disadvantage is that the methods (Gao et al. 2017). We did not include
combination of RNN and other DLNNs makes its PCCM in Figure 7 because the PCCM publications
structure complex. It is also difficult to construct train­ (Lyu, and Lu 2017; Cao, Dragićević, and Li 2019; Iino
ing sample sets and provide complete from-to changes. et al. 2018; Nemoto et al. 2017) did not report the
To date, this method has been used for detecting overall change detection accuracy.
changes in urban land use (Lyu, Lu, and Mou 2016) From Figure 7, using the median of the accuracy
and vegetation applications (Song et al. 2018) (Table 2). values as an indicator of DLCD performance, DLCD
methods can be ranked in descending order as
3.2.4. Adversarial method DNNM, DCM, RM, DM, MTM, and AM. That
AM plays generator neural networks and the discri­ DNNM ranks number one can be attributed to that
minator neural networks against each other (i.e. DNNM is more resistant to noise than the other
GANs) to achieve change detection. AM includes methods. AM can automatically generate training
three steps (Figure 6(g)). The generator is used to samples, so unsupervised and semi-supervised classi­
extract deep features for each image. The deep features fiers are often incorporated into this method. Thus,
are stacked (Gong et al. 2019b) or differenced (Gong et the median accuracy was slightly lower. In addition,
al. 2017a) to obtain a feature map. The discriminator is the accuracy variability of the coupled DLNNs was
used to discriminate the feature map (e.g. generated much lower as compared with the separate DLNNs,
DI) from the real data (e.g. real DI). A final feature as illustrated by the Inter-Quartile Range (IQR) within
map is obtained when the discriminator cannot each DLCD method, i.e. the IQR using the coupled
274 T. BAI ET AL.

Figure 7. Distribution of overall accuracies for DLCD methods (differencing method (DM), direct classification method (DCM),
differencing neural network method (DNNM), mapping transformation method (MTM), recurrent method (RM), and adversarial
method (AM)).

DLNNs is smaller than the IQR using separate DLCD methods is shown in Figure 8. With regards to
DLNNs; this implies that the interaction between the four applications: urban land use, water, vegeta­
DLNNs can make change detection more robust. tion, and hazard, the overall accuracy values of con­
ventional change detection and DLCD methods were
extracted from case studies presented in representative
4. Performance enhancements references where both conventional change detection
DLCD optimized performance in three types of exist­ and DLCD methods are used.
ing change detection (binary, multi-class, and from-to From Figure 8, the DLCD methods improved per­
change detection), but how much each type of change formance in comparison with conventional change
detection in specific applications benefits from DLCD detection methods in urban land use, water, hazard,
in terms of their performance is not yet clear. In the and vegetation change detection applications. This can
following, we review the difference in the overall be evidenced by the fact that the overall median accu­
improvement in accuracy between binary, multi- racy using the DLCD methods was higher than that
class, and from-to change detection for different appli­ using the conventional change detection methods for
cations. Due to space restrictions, not all potential urban land use (92.55% to 96.07%), water (95.35% to
applications and references are included. We explore 96.99%), hazard (96.91% to 98.09%), and vegetation
the most popular applications including urban land (95.18% to 97.23%). The second conclusion is that the
use, water, hazard, and vegetation change detection. accuracy variability of the DLCD methods was lower
than the conventional change detection methods. This
can be evidenced by the respective IQR within each
4.1. Binary changes application, i.e. IQR using the DLCD methods is smal­
The overall accuracy of binary change detection in ler than that using the conventional change detection
urban land use, water, hazard, and vegetation applica­ methods. This is because DLCD yields a more precise
tions using the conventional change detection and the feature representation for binary change detection.

Figure 8. Distribution of overall accuracies of binary changes for urban land use, water, hazard, and vegetation applications using
the conventional change detection (CD) and DLCD methods.
 GEO-SPATIAL INFORMATION SCIENCE 275

In addition, the increase in the median overall accuracy of these four applications when using the
accuracy for urban land use (increased by 3.52%) are DLCD methods was higher than instances using con­
more apparent than the increase in accuracy for water ventional change detection methods (urban land use
(increased by 1.64%), hazard (increased by 1.18%), from 91.28% to 96.04%, water from 91.17% to 95.61%,
and vegetation (increased by 2.05%). It is easy to hazard from 73.95% to 93.79%, and vegetation from
understand why DLCD methods improve perfor­ 94.25% to 97.30%). Variability in the accuracy of the
mance on the urban land-use change detection. On DLCD methods was lower than the conventional
the one hand, the conventional change detection change detection methods for these four applications.
methods used for water (95.35%), hazard (96.91%), This can be evidenced by the respective IQR within
and vegetation (95.18%) applications perform at rela­ each application, i.e. IQR using the DLCD methods is
tively higher accuracy than the urban land use appli­ smaller than that using the conventional change detec­
cations (92.55%); therefore, DLCD methods are more tion methods. This is because DLCD has more power­
difficult to apply when increasing these relatively high ful feature representation capabilities.
accuracies for water, hazard, and vegetation applica­ In addition, the increase in the median overall
tions. On the other hand, urban land-use change is accuracy for hazard (increase by nearly 20%) is more
caused by human activities, and thus more compli­ obvious than that of urban land use (increase by
cated than the water, hazard, and vegetation changes 4.76%), water (increase by 4.45%), and vegetation
caused by natural processes. DLCD performs most (increase by 3.05%) applications. However, this con­
effectively when addressing complex tasks (Ma et al. clusion is not very reliable because the research on
2019a). hazard change detection is sparse with only one case
study. We need to do more research in the future to
reach a definitive conclusion.
4.2. Multiclass changes
The overall accuracy of multiclass changes for urban
4.3. From-To changes
land use, water, hazard, and vegetation applications
using the conventional change detection and the The overall accuracy of from-to changes for urban
DLCD methods is shown in Figure 9. With regards land use, water, and vegetation applications using the
to these four applications: urban land use, water, vege­ conventional change detection and the DLCD meth­
tation, and hazard change detection, the overall accu­ ods is shown in Figure 10. With regards to three
racy values of conventional change detection and applications: urban land use, water, and vegetation,
DLCD methods are from case studies reported in the the overall accuracy values of conventional change
representative references where both conventional detection and DLCD methods are from case studies
change detection and DLCD methods are used. For in the respective references where both conventional
the hazard application, there is only one case study. change detection and DLCD methods are used. For
From Figure 9, in terms of the statistical accuracy, water and vegetation change detection applications,
compared with the traditional change detection meth­ there is only one case study. For hazard detection,
ods, DLCD methods also increase the accuracy of there are no case studies.
these four applications in the same way as binary From Figure 10, DLCD increases the accuracy of
change detection discussed in Section 4.1. This can these three applications in contrast to traditional
be evidenced by the fact that the median overall change detection methods. This can be evidenced by

Figure 9. Distribution of overall accuracies of multi-class changes for urban land use, water, hazard, and vegetation applications
using the conventional change detection (CD) and DLCD methods.
276 T. BAI ET AL.

Figure 10. Distribution of overall accuracies of from-to changes for urban land use, water, hazard, and vegetation applications
using the conventional change detection (CD) and DLCD methods.

the fact that the median the overall accuracy of these change detection applications. From-to changes
three applications using the DLCD methods was far involve many classification types, and are more com­
greater than the accuracy using the conventional plicated than binary and multi-class changes. In the
change detection methods (urban land use from existing literature, it has been shown that DLCD per­
85.22% to 95.68%, water from 85.14% to 98.42%, and forms optimally in complex tasks (Ma et al. 2019a).
vegetation from 71.75% to 91.65%). The second con­ However, there is limited published work on the
clusion is that the variability of the accuracy of the multi-class and from-to change detection applications
DLCD methods was lower than the conventional using DLCD. A larger sample of case studies is needed
change detection methods for these three applications. to confirm this conclusion.
This can be evidenced by the respective IQR within
each application, i.e. IQR using the DLCD methods is
smaller than that using the conventional change detec­ 5. Dilemmas of DLCD
tion methods. This is because of the feature represen­
Although it is evident that the introduction of deep
tation capabilities for from-to change detection in the
learning improves information representation, change
DLCD methods.
detection methods, and performance in change detec­
In addition, the increase in the median overall
tion, the limitations on DLCD are not yet clear. The
accuracy for vegetation (increased by 19.90%) and
reference studies have indicated two major dilemmas
water (increased by 13.28%) change detection is
are often encountered when deep learning is applied in
more apparent than the overall accuracy of urban
a specific application, large sets of labeled training
land-use change detection applications (increased by
samples are required, and DLCD places high demands
10.46%) when using DLCD methods. The most likely
on hardware and software. This is also true in change
cause may be that the urban land use from-to change
detection. Therefore, in the following subsections, we
detection using the DLCD methods has more case
will examine these dilemmas related to DLCD.
studies as compared with vegetation and water change
detection applications, decreasing the median overall
accuracy. From-to change detection case studies how­
5.1. Training sample dilemma
ever are too few, requiring additional research to reach
a more credible conclusion. The training sample dilemma adds complications
According to the existing literature, from-to change when deep learning is incorporated into change detec­
detection accuracy improvement using DLCD tion. Deep learning can execute change detection tasks
increased by nearly 10–20%. Next comes multi-class when there are relatively abundant labeled training
change detection which increased by 3%–20%. The samples (LeCun, Bengio, and Hinton 2015; Gong et
conventional detection methods for binary changes al. 2019b; Wang et al. 2018a). However, the required
were more than 92.5% and performed at a relatively sample size is well beyond what is available for change
higher median accuracy than the multi-class change detection (Ball, Anderson, and Chan 2018; Reichstein
detection from 73.95% to 94.25%, and from-to change et al. 2019; De et al. 2017). This poses a big challenge
detection increased from 71.75% to 85.22% in four for DLCD, for which no research is available. The lack
applications. Therefore, DLCD methods do not of such research may become a major hurdle for the
increase these relatively high accuracies in binary further application of deep learning in change
 GEO-SPATIAL INFORMATION SCIENCE 277

detection. Therefore, in this section, we summarized

Wang et al. (2018a)

Gong et al. (2019b)

Connors and Vatsavai
Gong et al. (2015)
Zhang et al. (2016b)

Khan et al. (2017)

Liu et al. (2016a)

the various solutions that have been proposed to this

Zhan et al. (2017)

Zhao et al. (2014)

Only a few or no training ● Transferability depends on the spectral similarity between Lyu, Lu, and Mou
Examples

● The application scenarios are restrained by unsupervised Su et al. (2016)

dilemma.
Except for the manual method, we grouped the

(2017)

(2016)
remaining solutions into two categories: 1) generating
large training samples, and 2) adapting to small train­
ing samples. The definition, advantages, limitations,

● Accuracy of the extended training samples directly

and examples of each solution are shown in Table 3.
The first published publication and the most cited
publication for each solution are listed as examples.

DLNNs and unsupervised change detection.

● Only a few training sam­ ● Not transferred to other multi-date images.
From Table 3, the first category is to enhance the

the training data and the target image.

● No initial training sam­ ● Inaccurate training samples generated.
size of training samples. The second category is to

depends on initial training samples.

develop DLCD methods to enhance the efficiency of

Limitations
● Initial training samples needed.

● Accurate training sam­ ● Initial training samples needed.

the training samples. The review then continues with a
more detailed discussion of each solution.

● Complex to implement.
5.1.1. Generating large training samples
In this sub-section, we divide the methods for gener­
ating large training samples into three sub-categories:
Data Augmentation Method (DAM); Supervised
Change Detection Method (SCDM); and
Unsupervised Change Detection Method (USCDM).

Transferable to other

No training samples are

DAM is a method of utilizing affine transformations to

samples are needed.

● Easy to implement.
enhance the size and diversity of training samples. It
Advantages

ples are needed.

ples are needed.

Fully automatic.
Table 3. The definition, advantages, limitations, and examples of solutions to the training sample dilemma in DLCD.

ples extended.
includes one step. For this method, four basic trans­
formation operations including rotation (Zhan et al.

needed.
images.
2017; Nemoto et al. 2017; Wang et al. 2018a; Zhu et al.
2018), flip (Zhan et al. 2017), mirror (Zhu et al. 2018),

Uses the pre-trained model from other training data to detect ●

●
●
and cropping (Nemoto et al. 2017; Zhan et al. 2017)
Uses traditional unsupervised change detection methods to
Data Augmentation Method Utilizes affine transformation or data generation method to

are directly used to generate augmented training sam­

Unsupervised DLCD Method Combines unsupervised DLNNs with unsupervised change

Uses traditional supervised change detection methods to

Uses the combination of labeled and unlabeled training

ples. The advantage of this method is that it is easy to

implement and does not change the spectral or topo­
logical information for the training samples. Data changes in the current multi-date images.

augmentation (i.e. flips, mirror, rotations, and crop­

ping) for training samples diversifies its holistic spatial
Definition

detection for change detection.

layout and orientation (Yu et al. 2017), which can
samples for change detection.

effectively avoid the over-fitting problem in deep

generate training samples.
extend training samples.

extend training samples.

learning (Cireşan et al. 2010; Luus et al. 2015;

Simard, Steinkraus, and Platt 2003). However, the
disadvantage is that it needs an initial training set
and the accuracy of the extended training samples
directly depends on initial training samples.
Recently, beyond these basic transformation opera­
tions (i.e. flips, mirror, rotations, and cropping),
Detection Method (SCD)

Gong et al. (2019b) utilized GANs to generate the

Transfer Learning Method
5.1.2.Adapting to small Semi-Supervised Method
Unsupervised Change

new training samples given its advantages of generat­

Detection Method
Sub-category

Supervised Change

ing new information. To extend accurate training

samples, SCDM was developed.
(UDLCDM)
(USCD)

SCDM refers to using traditional SCDM such as

(DAM)

(SSM)

(TLM)

the supervised object-based change detection method

to extend training samples (Zhang et al. 2016b). It
includes two steps. The traditional SCDM is used to
5.1.1.Generating large
training samples

training samples

obtain the initial change detection result and the

training samples are selected from this initial change
detection result. The advantage of this method is that
Category

it can generate more accurate training samples than

that using the traditional USCDM. However, the
278 T. BAI ET AL.

disadvantage is that two typical techniques (i.e. the 2019; El Amin, Liu, and Wang 2017). Fine-tuning
supervised change detection and the subsequent means using small training samples from the current
supervised DLCD) are combined, which makes the multi-date images to fine-tune a pre-trained model
structure of the whole model more complex. USCDM that came from other training datasets for change
was developed to eliminate the need for initial train­ detection (Waldeland, Reksten, and Salberg 2018).
ing samples. Fine-tuning yields more accurate change detection
USCDM refers to using traditional USCDMs to results than direct application. The advantage of
generate change and no change training samples. It TLM is that it only needs a few training samples, and
includes two steps. The traditional USCDMs are used this model can be transferred to new multi-date
to obtain the initial change detection result and the images. However, the disadvantage is that the trans­
training samples are selected from this initial change ferability depends on the spectral similarity between
detection result. Unsupervised pixel-based change the training data and the target image (Yosinski et al.
detection methods include thresholding methods 2014). UDLCDM overcomes the dependency on spec­
(Liu et al. 2016b), level set methods (Liu et al. tral similarity and initial training sample
2016b), CVA (Zhang, and Zhang 2016), and clustering requirements.
methods (Geng et al. 2017) are used to generate train­ UDLCDM refers to methods that combine unsu­
ing samples. Unsupervised object-based methods pervised DLNNs with USCDMs for change detection.
including an ensemble learning method based on It includes three steps. Unsupervised DLNNs such as
objects (Gong et al. 2017b) and an unsupervised DBN (Zhang et al. 2016a) and AE (Su et al. 2016; Liu et
object-based Markov random field (Li, Xu, and Liu al. 2016a) are feature learning tools that are used to
2018) are also utilized to generate training samples. effectively extract high-dimension features, and
Compared with pixel-based methods, the object-based USCDMs such as CVA (Zhang et al. 2016a; Su et al.
methods can include neighboring information and 2016) is used to map these features to characterize
edge information to generate more accurate training change information. Clustering methods are used to
samples. Compared with DAM and SCDM, the advan­ detect changes. As compared with other methods, the
tage of USCDM is that it does not need an initial advantage of UDLCDM is that it is automatic and does
training set. However, the disadvantage is that the not need training samples. However, the disadvantage
generated training samples are less accurate. In addi­ is that the application scenarios are constrained by the
tion to the DLCD methods that generate large training limitations of unsupervised DLNNs and unsupervised
samples, DLCD methods that adapt to small training change detection.
samples have also been developed.
5.2. Hardware and software dilemmas
5.1.2. Adapting to small training samples
We divided the methods that adapt to small training Compared with the conventional change detection
samples into three sub-categories: Semi-Supervised methods, DLCD methods have stricter software and
Methods (SSMs), Transfer Learning Method (TLM), hardware requirements. Considering hardware
and Unsupervised DLCD Methods (UDLCDMs). SSM requirements, conventional change detection methods
refers to methods that use a combination of labeled just need a CPU card, but DLCD often requires a
and unlabeled training samples for change detection computer with a GPU card. In terms of software
(Connors, and Vatsavai 2017; Gong et al. 2019b). It requirements, many conventional change detection
includes two steps. Unlabeled training samples are methods are ready-to-use tools in software such as
used to train an unsupervised deep learning network ENVI, ERDAS, and ArcGIS. DLCD however, not
to extract relevant feature information, and this fea­ only requires open-source deep learning frameworks
ture information is input into the supervised classifier such as Caffe/Caffe2.0, Pytorch, Theano, Tensorflow,
for change detection using the labeled training sam­ Keras, and MATLAB but also requires custom pro­
ples. The advantage of the SSM is that it only needs a gramming. A detailed discussion of popular deep
few training samples, which can reduce the expense of learning frameworks can be found in (De Felice
manually obtaining training samples. The disadvan­ 2017). Fortunately, the difficulty of applying DLCD
tage is that it cannot be transferred to other multi-date is decreasing. For example, ENVI has released the
images; TLM overcomes this shortcoming. ENVI Deep Learning Module.
TLM is a method that uses a pre-trained model
from other data to detect changes in current multi-
6. Future prospects of DLCD
date images. In the DLCD literature, there are two
popular ways to transfer learning, direct application In sections 2, 3, 4, and 5, we compared the difference
and fine-tuning. Direct application means using pre- in how spectral, spatial, temporal, and multi-sensor
trained neural networks from other kinds of data to information were represented between DLCD and
extract deep features (Saha, Bovolo, and Bruzzone conventional change detection methods. We
 GEO-SPATIAL INFORMATION SCIENCE 279

introduced a taxonomy of DLCD methods and pro­ multiclass change detection (specific change types are
vided a systematic comparison of their advantages, unclear), which has 19 studies (Su et al. 2016; Zhang et
limitations, applicability, and performance. We al. 2016a). Only 12 of the 89 studies have detected land
reviewed the difference in the overall improvement use/cover from-to changes (Lyu, Lu, and Mou 2016;
in accuracy between binary, multi-class, and from-to Lyu, and Lu 2017). Although building change detec­
change detection for different applications. We tion is very popular in traditional urban change detec­
reviewed two major limitations in DLCD: training tion, only nine of the 89 urban DLCD studies have
sample and hardware and software dilemmas. In this detected building changes (Argyridis, and Argialas
section, we summarize four future directions: 1) 2016; Nemoto et al. 2017). The limited studies for
DLCD methods, 2) DLCD applications, 3) training land use/land cover from-to and building change
samples, and 4) the implication of remote sensing/ detection can be attributed to the fact that the former
change detection/deep learning for DLCD. involves many change types and the latter involves the
building target, more complicated, compared to bin­
ary and multiclass change detection. It would be inter­
6.1. DLCD methods
esting to see more studies on these two types of change
The DLCD community has made significant progress detection.
in developing DLCD methods. However, there are still For water applications, most water DLCD studies
existing fields to be developed. The future direction of focused on the DLCD algorithm development and
the DLCD methods could be the coupled DLNNs. used public data sets. In addition, the study areas are
This can be attributed to its lower accuracy variability limited. Among the 40 water DLCD studies, 18 were
(Section 3.2) and the ability to solve specific problems along the yellow river (Zhao et al. 2014; Gong et al.
(i.e. resistance to noise, multi-source images and 2019a; Su, and Cao 2018; Zhao et al. 2016), four in San
multi-spatial resolution images change detection, Francisco (Gong, Yang, and Zhang 2017; Gao et al.
time-sensitive change detection, and generating the 2017; Zhao et al. 2016), four in the Sulzberger Ice Shelf
high-quality information). Specifically, in coupled (Gao et al. 2019a, 2019b), six for the Weihe River,
DLNNs, RM and AM can be potential methods for China (Zhang, and Zhang 2016; Gong et al. 2017b;
change detection. For RM, on the one hand, currently, Lei et al. 2019b), five for Hongqi, China (Gong et al.
RM has the second-highest median accuracy perfor­ 2017a, 2017b; Zhao et al. 2017), two in Sardinia, Italy
mance in coupled DLNNs (Section 3.2). On the other (Zhang et al. 2016b; Gong et al. 2019a), one for Lake
hand, time-series change detection is the future direc­ Lotus (Zhang et al. 2016a). In the future, we need to
tion of change detection, and RM can provide tem­ conduct more experiments to make the accuracy of
poral features for it. For AM, although it has the lowest these water studies sufficient to meet the requirements
median accuracy performance in coupled DLNNs, it of real-world applications.
may be the future direction. Compared with other For hazard applications, there are 42 applications of
basic deep learning networks, which only have a dis­ before- and after-hazard change detection. Among
criminator neural network (i.e. (0, 1) classifier), AM these studies, 31 worked on flooding, including 17
not only includes a discriminator, but also includes a using the same public data set in Ottawa (Zhao et al.
generator (i.e. generating new information by a noise 2014; Gong et al. 2015)) and six using the same public
input). The current AM studies also proved that it has data set in Bern (Zhao et al. 2014; Liu et al. 2017)), six
the advantage of generating training samples (Gong et along the yellow river (Ma et al. 2019b; Chen et al.
al. 2019b) and high-quality change information (Gong 2019), and two in Thailand (Amit, and Aoki 2017). Six
et al. 2017a). However, the current studies are limited, worked on the landslide, including two in Japan
so it remains a subject for more research. (Amit, and Aoki 2017) and four in China (Chen et
al. 2018; Lei et al. 2019a). Two studies mapped the
damage caused by the Tohoku tsunami (Sublime, and
6.2. DLCD applications
Kalinicheva 2019). One detected the changes before
The DLCD community has made significant progress and after the Aere typhoon (Li, Yuan, and Wang
for real applications. However, there are still existing 2019). One detected the damage by a forest fire (Cao
fields to be developed. For urban land use applications, et al. 2017). One conducted avalanche detection
in terms of the spatial extent, all studies focused on the (Waldeland, Reksten, and Salberg 2018). However,
local areas (e.g. 14,400 ha (Mou, Bruzzone, and Zhu there are no damaged buildings, civil war, volcanic
2018)) within a city. Therefore, it would be interesting eruptions, and droughts change detection studies. It
to extend study areas to a larger spatial scale (i.e. would be interesting to see DLCD used in more hazard
regional, national, and global scale). In terms of the scenarios.
detected changes, 49 of the 89 case studies have Compared with urban land use, water, and hazard
detected binary changes (El Amin, Liu, and Wang applications, the development of vegetation DLCD
2016; Chu, Cao, and Hayat 2016). Next comes application lags. To date, only 13 publications have
280 T. BAI ET AL.

worked on vegetation DLCD in 22 different areas. 6.4. The implication of remote sensing/change
Among these 22 studies, 20 worked on farmland detection/deep learning for DLCD
(Wang et al. 2019; Li, Yuan, and Wang 2019; Yuan,
In this section, we discuss the implication of remote
Wang, and Li 2018) and the other two on forests
sensing, change detection, and deep learning for DLCD.
(Khan et al. 2017). All farmland studies focused on
The relationship between DLCD, remote sensing,
small areas (e.g. 5670 ha (Yuan, Wang, and Li 2018)).
change detection, and deep learning is shown in
Forest studies are too few to analyze their pattern. It
Figure 11.
would be interesting to see vegetation DLCD in more
From Figure 11, DLCD is the combination of
application scenarios such as mangrove change detec­
remote sensing, change detection, and deep learning.
tion and larger scale (e.g. global scale) in the future.
Thus, we argue that the developments in remote sen­
sing, change detection, and deep learning could be
promising directions for developments in future
6.3. Training samples
DLCD methods.
The DLCD community has made significant progress The incorporation of spatiotemporal information
on the training sample dilemma, with six solutions will play a crucial role in future DLCD developments.
currently available. These solutions, however, created The latest work from Yuan et al. (2020) has indicated
new problems so it remains a subject for more research. that spatiotemporal information is indispensable when
DLCD methods that adapt to small training samples deep learning is applied to remote sensing applications.
may be a potential area to explore, because this category With regards to DLCD, a few attempts have been made
only needs a few or no training samples, which reduces to exploit spatial and temporal information. For exam­
the manual workload. When generating large training ple, Lyu, Lu, and Mou (2016) used the RNN to learn the
samples, USCDM generated inaccurate training sam­ temporal features. Liu et al. (2017) employed a full-
ples, so it remains a subject for more research. In connection layer and a softmax layer to concatenate
adapting to small training samples, for UDLCDM, the the output features of two paralleled CNN channels,
application scenarios were limited by the unsupervised which amplifies the spatial information. However, spa­
DLNNs and change detection methods, so it remains a tial and temporal information are considered sepa­
subject for more research. In the future, we can make rately. Recently, one exception is the work of Mou
use of the flexible and deep network structure of deep and Zhu (2018) who managed to extract joint spatial-
learning to extend the application range of UDLCDM. temporal features for land use change detection in
TLM also has the great potential to solve the training complex urban areas using Landsat images by combin­
sample problem because it only needs a few training ing CNN and RNN. This study is a good example of
samples, but its transferability is influenced by the dis­ simultaneously considering spatial and temporal infor­
tance of the spectral distribution between the training mation by using the same loss function for RNN and
data and the target image, so it also remains a subject CNN. However, this study used shallow CNN and RNN
for more research. To make the spectral distribution of models, which face difficulties in dealing with high-
the training data closer to the target image, the change resolution images and long time-series images. In the
detection model needs to be trained through repetition future, we need to develop a spatiotemporally con­
and variation such as in a never-ending learning model strained deep learning model for high-resolution
(Mitchell et al. 2018). images and long-time-series images.

Figure 11. The relationship between DLCD, remote sensing, change detection, and deep learning.
 GEO-SPATIAL INFORMATION SCIENCE 281

The incorporation of hybrid methods into very difficult. The emergence of deep learning provides
DLNNs is anticipated. In the field of change detec­ an opportunity for change detection. In this paper, we
tion, hybrid methods have been used to successfully review DLCD literature to reveal the theoretical under­
combine the advantages of separate change detec­ pinnings in five ways: improved information represen­
tion methods. Currently, in the field of DLCD, tations, improved change detection methods,
hybrid change detection methods have been only performance enhancements, dilemmas of DLCD, and
used to solve the training samples dilemmas by prospects of DLCD. Compared to conventional change
generating large training samples. Other hybrid detection, DLCD brings advantages in information
strategies such as combining pixel-based and representation and change detection methods which
object-based methods have almost been entirely result in refined performance. Nevertheless, DLCD
neglected. Therefore, more algorithms that combine still faces challenges in lacking training samples and
DLNNs with hybrid change detection methods are requiring more advanced hardware and software.
anticipated. In addition, combining different Finally, we envision the future research of DLCD to
DLNNs may be able to compensate for their single improve DLCD methods from the perspective of
deficiencies, rendering some more reliable results. coupled DLNNs; to widen DLCD applications in
For example, CNN and SDAE can be combined to land-use/land cover from-to, building, civil war, volca­
take advantage of spatial features from CNN and nic eruption, and droughts change detection; to deal
simultaneously off-load the need for training sam­ with small training samples from the perspective of
ples thanks to SDAE (Zhang et al. 2016b). UDLCDM and TLM; and to absorb developments in
The rapid developments in the deep learning field remote sensing, change detection, and deep learning to
open new avenues for DLCD as well. Deep learning the field of DLCD. We hope this review makes it easier
algorithms such as AlexNet (Han et al. 2017b), ResNet for researchers to find DLCD methods that are most
(Zhu et al. 2021), GoogleNet (Bazi et al. 2019), Unet appropriate to their specific applications, to understand
(Jiao et al. 2020), Graph Convolutional Network the benefits and shortcomings of DLCD, and to con­
(GCN) (Hong et al. 2020a; Gao et al. 2021), Spectral tribute to the future development of DLCD Table A1.
Former (Hong et al. 2021), and multimodal deep
learning framework (Hong et al. 2020), which have
been extensively used for remote sensing, are the Acknowledgments
potential models for DLCD. For example, Hong et al.
The authors are grateful to anonymous reviewers whose
(2020) have developed a new supervised version of constructive and valuable comments greatly helped us to
GCNs. This model can jointly use CNNs and GCNs improve the paper.
for extracting more diverse and discriminative feature
representations for the hyperspectral image classifica­
tion task, which can potentially model for change Disclosure statement
detection. To date, a few attempts have been made to No potential conflict of interest was reported by the
exploit these algorithms for DLCD. For example, author(s).
Waldeland, Reksten, and Salberg (2018) used a
ResNet pre-trained from ImageNet data to detect ava­
lanche in SAR images and found that additional train­ Notes on contributors
ing was needed for avalanche detection. Wang et al.
Ting Bai received her BS degree in geographic information
(2018a) used a 50-layer Residual Net to obtain differ­ systems from Huazhong Agricultural University, Wuhan,
encing feature maps of multi-date high-resolution China, in 2014 and her PhD degree in photogrammetry
remote sensing images for change detection. and remote sensing from Wuhan University, Wuhan,
However, this model has only been used to detect China, in 2021. She is an engineer in Wuhan Land
simple land cover changes; this study confronts chal­ Arranging Storage Center. Her current research interests
include remote sensing and feature fusion, machine learn­
lenges for complex urban land use change detections.
ing, ensemble learning, deep learning, and land use and land
Therefore, an immediate need for future studies is to cover change detection.
incorporate AlexNet, ResNet, GoogleNet, Unet, GCN,
Le Wang received his BS degree in Wuhan Technical
Spectral Former, and multimodal deep learning fra­
University of Surveying and Mapping, Wuhan, China, in
mework algorithms in more DLCD applications. 1996. He got his MS degree in remote Sensing from Peking
University, Beijing, China in 1996. He got his PhD degree in
environmental science from University of California,
7. Conclusion Berkeley in 2003. He is a professor in the State University
of New York at Buffalo. His current research interests
Change detection permits more effective management
include remote sensing; geoscience; forest characterization;
and monitoring of natural resources and environmental environment modeling; land cover and land-use change;
change. However, as remote sensing images form big urban population estimation; invasive species modeling;
data, finding the real change in the multi-date images is spatio-temporal analysis and modeling.
282 T. BAI ET AL.

Dameng Yin received her BS degree in physics in 2010 from References

Beijing Normal University, China, her ME degree in remote
sensing in 2013 from Beijing Normal University, China, and Amit, S.N.K.B., and Y. Aoki. 2017. “Disaster Detection from
her PhD degree in 2021 from University at Buffalo, USA. Aerial Imagery with Convolutional Neural Network.”
She is an assistant professor in the Institute of Crop 2017 International Electronics Symposium on Knowledge
Sciences, Chinese Academy of Agricultural Sciences, Creation and Intelligent Computing (IES-KCIC), 239–
Beijing, China. Her current research focuses on the use of 245. doi:10.1109/KCIC.2017.8228593.
UAV remote sensing and LiDAR for crop phenotyping. Dr. Arabi, M.E.A., M.S. Karoui, and K. Djerriri. 2018. “Optical
Yin is a member of the American Association of Remote Sensing Change Detection Through Deep Siamese
Geographers (AAG) and the International Association of Network.” In IGARSS 2018 - 2018 Ieee International
Chinese Professionals in Geographic Information Sciences Geoscience and Remote Sensing Symposium, 5041–5044.
(CPGIS). She was a recipient of the Third-Place Award of doi:10.1109/IGARSS.2018.8518178.
the AAG Remote Sensing Specialty Group (RSSG) Student Argyridis, A., and D.P. Argialas. 2016. “Building Change
Honors Paper Competition in 2017, the Second-Place Detection Through Multi-Scale GEOBIA Approach by
Award of the CPGIS Best Student Paper Competition in Integrating Deep Belief Networks with Fuzzy Ontologies.”
2017, and the Sigma Xi ”Companions in Zealous International Journal of Image and Data Fusion 7 (2): 148–
Research” Award in the University at Buffalo Chapter in 171. doi:10.1080/19479832.2016.1158211.
2020. Ball, J.E., D.T. Anderson, and C.S. Chan. 2018. “Special
Section Guest Editorial: Feature and Deep Learning in
Kaimin Sun received his BS, MS, and PhD degrees in photo­ Remote Sensing Applications.” Journal of Applied Remote
grammetry and remote sensing from Wuhan University, Sensing 11 (4): 042601. doi:10.1117/1.JRS.11.042601.
Wuhan, China, in 1999, 2004, and 2008, respectively. He is Bazi, Y., M.M. Rahhal, H. Alhichri, and N. Alajlan. 2019.
a professor in the State Key Laboratory of Information “Simple Yet Effective Fine-Tuning of Deep Cnns Using
Engineering in Surveying, Mapping, and Remote Sensing, an Auxiliary Classification Loss for Remote Sensing Scene
Wuhan University. His research interests include photo­ Classification.” Remote Sensing 11 (24): 2908.
grammetry, object-oriented image analysis, and image doi:10.3390/rs11242908.
change detection. Bengio, Y., P. Simard, and P. Frasconi. 1994. “Learning
Yepei Chen received her PhD degree in photogrammetry Long-Term Dependencies with Gradient Descent is
and remote sensing from Wuhan University, Wuhan, Difficult.” IEEE Transactions on Neural Networks 5 (2):
China, in 2021. She is a lecturer in School of Computer 157–166. doi:10.1109/72.279181.
Science, Hubei University of Technology.Her current Bovolo, F., L. Bruzzone, and M. Marconcini. 2008. “A Novel
research interests include time series analysis and change Approach to Unsupervised Change Detection Based on a
detection. Semisupervised SVM and a Similarity Measure.” IEEE
Transactions on Geoscience and Remote Sensing 46 (7):
Wenzhuo Li received his BS and PhD degrees in photo­ 2070–2082. doi:10.1109/TGRS.2008.916643.
grammetry and remote sensing from Wuhan University, Cao, G., Y. Li, Y. Liu, and Y. Shang. 2014. “Automatic
Wuhan, China, in 2011 and 2017, respectively. He is a Change Detection in High-Resolution Remote-Sensing
postdoctoral researcher in State Key Laboratory of Images by Means of Level Set Evolution and Support
Information Engineering in Surveying, Mapping and Vector Machine Classification.” International Journal of
Remote Sensing, Wuhan University. His current research Remote Sensing 35 (16): 6255–6270. doi:10.1080/
interests include image segmentation, image classification, 01431161.2014.951740.
land use, and land cover changes detection, and object- Cao, G., B. Wang, H.C. Xavier, D. Yang, and J. Southworth.
oriented image analysis. 2017. “A New Difference Image Creation Method Based
Deren Li received the PhD degree in photogrammetry and on Deep Neural Networks for Change Detection in
remote sensing from the University of Stuttgart, Stuttgart, Remote-Sensing Images.” International Journal of
Germany, in 1984. He is an Academician of the Chinese Remote Sensing 38 (23): 7161–7175. doi:10.1080/
Academy of Sciences, the Chinese Academy of Engineering, 01431161.2017.1371861.
and the Euro-Asia International Academy of Sciences. His Cao, C., S. Dragićević, and S. Li. 2019. “Land-Use Change
research interests are spatial information science and tech­ Detection with Convolutional Neural Network Methods.”
nology represented by RS, GPS, and GIS. Environments 6 (2): 25. doi:10.3390/environments6020025.
Chan, J.C.W., K.P. Chan, and A.G.O. Yeh. 2001. “Detecting
the Nature of Change in an Urban Environment: A
ORCID Comparison of Machine Learning Algorithms.”
Photogrammetric Engineering and Remote Sensing 67
Ting Bai http://orcid.org/0000-0002-7990-5405 (2): 213–226. doi:10.1144/petgeo.7.1.81.
Dameng Yin http://orcid.org/0000-0001-9668-2744 Chehata, N., C. Orny, S. Boukir, D. Guyon, and J.P.
Kaimin Sun http://orcid.org/0000-0002-2664-9479 Wigneron. 2014. “Object-Based Change Detection in
Yepei Chen http://orcid.org/0000-0002-2877-5065 Wind Storm-Damaged Forest Using High-Resolution
Wenzhuo Li http://orcid.org/0000-0002-2920-3202 Multispectral Images.” International Journal of Remote
Sensing 35 (13): 4758–4777. doi:10.1080/01431161.
2014.930199.
Data availability statement (DAS) Chen, J., P. Gong, C. He, R. Pu, and P. Shi. 2003. “Land-Use/
land-Cover Change Detection Using Improved Change-
The data that support the findings of this study are available Vector Analysis.” Photogrammetric Engineering and
from the corresponding author [L. Wang], upon reasonable Remote Sensing 69 (4): 369–379. doi:10.14358/
request. PERS.69.4.369.Chen.
 GEO-SPATIAL INFORMATION SCIENCE 283

Chen, F., J. Shi, and M. Gong. 2016. “Differencing Neural Gao, F., X. Wang, Y. Gao, J. Dong, and S. Wang. 2019a. “Sea
Network for Change Detection in Synthetic Aperture Ice Change Detection in SAR Images Based on
Radar Images.” International Conference on Bio-Inspired Convolutional-Wavelet Neural Networks.” IEEE
Computing: Theories and Applications 431–437. Geoscience and Remote Sensing Letters 16 (8): 1240–
doi:10.1007/978-981-10-3611-8_38.Chen. 1244. doi:10.1109/LGRS.2019.2895656.
Chen, Z., Y. Zhang, C. Ouyang, F. Zhang, and J. Ma 2018. Gao, Y., F. Gao, J. Dong, and S. Wang. 2019b. “Transferred
“Automated Landslides Detection for Mountain Cities Deep Learning for Sea Ice Change Detection from
Using Multi-Temporal Remote Sensing Imagery.” Synthetic-Aperture Radar Images.” IEEE Geoscience
Sensors 18 (3): 821. doi:10.3390/s18030821. Remote Sensing Letters 16 (10): 1655–1659. doi:10.1109/
Chen, H., L. Jiao, M. Liang, F. Liu, S. Yang, and B. Hou 2019. LGRS.2019.2906279.
“Fast Unsupervised Deep Fusion Network for Change Gao, Y., J. Shi, J. Li, and R. Wang. 2021. “Remote Sensing Scene
Detection of Multitemporal SAR Images.” Classification Based on High-Order Graph Convolutional
Neurocomputing 332: 56–70. doi:10.1016/j.neucom. Network.” European Journal of Remote Sensing 54 (sup1):
2018.11.077. 141–155. doi:10.1080/22797254.2020.1868273.
Chen, L., F. Rottensteiner, and C. Heipke. 2021. “Feature Geng, J., H. Wang, J. Fan, and X. Ma. 2017. “Change
Detection and Description for Image Matching: From Detection of SAR Images Based on Supervised
Hand-Crafted Design to Deep Learning.” Geo-Spatial Contractive Autoencoders and Fuzzy Clustering.” In
Information Science 24 (1): 58–74. doi:10.1080/ 2017 International Workshop on Remote Sensing with
10095020.2020.1843376. Intelligent Processing (RSIP), 1–3. doi:10.1109/RSIP.
Cheng, G., X. Xie, J. Han, L. Guo, and G.S. Xia. 2020. 2017.7958819.
“Remote Sensing Image Scene Classification Meets Deep Gong, P. 1993. “Change Detection Using Principal
Learning: Challenges, Methods, Benchmarks, and Component Analysis and Fuzzy Set Theory.” Canadian
Opportunities.” IEEE Journal of Selected Topics in Journal of Remote Sensing 19 (1): 22–29. doi:10.1080/
Applied Earth Observations and Remote Sensing 13: 07038992.1993.10855147.
3735–3756. doi:10.1109/JSTARS.2020.3005403. Gong, M., J. Zhao, J. Liu, Q. Miao, and L. Jiao. 2015.
Chu, Y., G. Cao, and H. Hayat. 2016. “Change Detection of “Change Detection in Synthetic Aperture Radar Images
Remote Sensing Image Based on Deep Neural Networks.” Based on Deep Neural Networks.” IEEE Transactions on
Proceedings of the 2016 2nd International Conference on Neural Networks and Learning Systems 27 (1): 125–138.
Artificial Intelligence and Industrial Engineering 133 (1): doi:10.1109/TNNLS.2015.2435783.
262–267. doi:10.2991/aiie-16.2016.61. Gong, M., H. Yang, and P. Zhang. 2017. “Feature Learning
Cireşan, D.C., U. Meier, L.M. Gambardella, and J. and Change Feature Classification Based on Deep
Schmidhuber. 2010. “Deep, Big, Simple Neural Nets for Learning for Ternary Change Detection in SAR
Handwritten Digit Recognition.” Neural Computation 22 Images.” ISPRS Journal of Photogrammetry and Remote
(12): 3207–3220. doi:10.1162/NECO_a_00052. Sensing 129: 212–225. doi:10.1016/j.isprsjprs.2017.05.001.
Connors, C., and R.R. Vatsavai. 2017. “Semi-Supervised Gong, M., X. Niu, P. Zhang, and Z. Li. 2017a. “Generative
Deep Generative Models for Change Detection in Very Adversarial Networks for Change Detection in
High Resolution Imagery.” In 2017 IEEE International Multispectral Imagery.” IEEE Geoscience and Remote
Geoscience and Remote Sensing Symposium (IGARSS), Sensing Letters 14 (12): 2310–2314. doi:10.1109/
1063–1066. doi:10.1109/IGARSS.2017.8127139. Lgrs.2017.2762694.
Dai, X.L., and S. Khorram. 1999. “Remotely Sensed Change Gong, M., T. Zhan, P. Zhang, and Q. Miao. 2017b. “Superpixel-
Detection Based on Artificial Neural Networks.” Based Difference Representation Learning for Change
Photogrammetric Engineering and Remote Sensing 65 Detection in Multispectral Remote Sensing Images.” IEEE
(10): 1187–1194. doi:10.1117/1.482717. Transactions on Geoscience and Remote Sensing 55 (5): 2658–
De, S., D. Pirrone, F. Bovolo, L. Bruzzone, and A. 2673. doi:10.1109/TGRS.2017.2650198.
Bhattacharya. 2017. “A Novel Change Detection Gong, M., X. Niu, T. Zhan, and M. Zhang. 2019a. “A
Framework Based on Deep Learning for the Analysis of Coupling Translation Network for Change Detection in
Multi-Temporal Polarimetric Sar Images.” In 2017 IEEE Heterogeneous Images.” International Journal of Remote
International Geoscience and Remote Sensing Symposium Sensing 40 (9): 3647–3672. doi:10.1080/01431161.2018.
(IGARSS), 5193–5196. doi:10.1109/IGARSS.2017.8128171. 1547934.
De Felice, M. 2017. “Which Deep Learning Network is Best Gong, M., Y. Yang, T. Zhan, X. Niu, and S. Li. 2019b. “A
for You? IDG Communications.” accessed May 4 20 1 7. Generative Discriminatory Classified Network for
http://www.cio.com/article/3193689/artificial-intelligence/ Change Detection in Multispectral Imagery.” IEEE
which-deep-learning-network-is-best-for-you.html Journal of Selected Topics in Applied Earth Observations
El Amin, A.M., Q. Liu, and Y. Wang. 2016. “Convolutional and Remote Sensing 12 (1): 321–333. doi:10.1109/
Neural Network Features Based Change Detection in JSTARS.2018.2887108.
Satellite Images.” First International Workshop on Pattern Goodfellow, I., J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-
Recognition, no. 10011: 181–186. doi:10.1117/12.2243798. Farley, S. Ozair, A. Courville, and Y. Bengio. 2014.
El Amin, A.M., Q. Liu, and Y. Wang. 2017. “Zoom Out “Generative Adversarial Nets.” Advances in Neural
Cnns Features for Optical Remote Sensing Change Information Processing Systems 27. doi:10.5555/
Detection.” In 2017 2nd International Conference on 2969033.2969125.
Image, Vision and Computing (ICIVC), 812–817. Han, J., X. Meng, X. Zhou, B. Yi, M. Liu, and W.N. Xiang.
doi:10.1109/ICIVC.2017.7984667. 2017a. “A Long-Term Analysis of Urbanization Process,
Gao, F., X. Liu, J. Dong, G. Zhong, and M. Jian. 2017. Landscape Change, and Carbon Sources and Sinks: A
“Change Detection in SAR Images Based on Deep Semi- Case Study in China’s Yangtze River Delta Region.”
NMF and SVD Networks.” Remote Sensing 9 (5): 435. Journal of Cleaner Production 141: 1040–1050.
doi:10.3390/rs9050435. doi:10.1016/j.jclepro.2016.09.177.
284 T. BAI ET AL.

Han, X., Y. Zhong, L. Cao, and L. Zhang. 2017b. “Pre- Khan, S.H., X. He, F. Porikli, and M. Bennamoun. 2017.
Trained Alexnet Architecture with Pyramid Pooling and “Forest Change Detection in Incomplete Satellite Images
Supervision for High Spatial Resolution Remote Sensing with Deep Neural Networks.” IEEE Transactions on
Image Scene Classification.” Remote Sensing 9 (8): 848. Geoscience and Remote Sensing 55 (9): 5407–5423.
doi:10.3390/rs9080848. doi:10.1109/TGRS.2017.2707528.
Healey, S.P., W.B. Cohen, Z. Yang, C.K. Brewer, E.B. Khelifi, L., and M. Mignotte. 2020. “Deep Learning for
Brooks, N. Gorelick, A.J. Hernandez, et al. 2018. Change Detection in Remote Sensing Images:
”Mapping Forest Change Using Stacked Generalization: Comprehensive Review and Meta-Analysis.” IEEE
An Ensemble Approach”. Remote Sensing of Environment Access 8: 126385–126400. doi:10.1109/ACCESS.2020.
204: 717–728. 10.1016/j.rse.2017.09.029. 3008036.
Hinton, G.E., S. Osindero, and Y.W. Teh. 2006. “A Fast LeCun, Y., L. Bottou, Y. Bengio, and P. Haffner. 1998.
Learning Algorithm for Deep Belief Nets.” Neural “Gradient-Based Learning Applied to Document
Computation 18 (7): 1527–1554. doi:10.1162/neco.2006. Recognition.” Proceedings of the IEEE 86 (11): 2278–
18.7.1527. 2324. doi:10.1109/5.726791.
Hinton, G.E., and R.R. Salakhutdinov. 2006. “Reducing the LeCun, Y., Y. Bengio, and G. Hinton. 2015. “Deep
Dimensionality of Data with Neural Networks.” Science Learning.” Nature 521 (7553): 436–444. doi:10.1038/
313 (5786): 504–507. doi:10.1126/science.1127647. nature14539.
Hong, D., N. Yokoya, J. Chanussot, and X.X. Zhu. 2018. “An Lei, T., Q. Zhang, D. Xue, T. Chen, H. Meng, and A.K.
Augmented Linear Mixing Model to Address Spectral Nandi. 2019a. “End-To-End Change Detection Using a
Variability for Hyperspectral Unmixing.” IEEE Symmetric Fully Convolutional Network for Landslide
Transactions on Image Processing 28 (4): 1923–1938. Mapping.” In ICASSP 2019-2019 IEEE International
doi:10.1109/TIP.2018.2878958. Conference on Acoustics, Speech and Signal Processing
Hong, D., L. Gao, J. Yao, B. Zhang, A. Plaza, and J. (ICASSP), 3027–3031. doi: 10.1109/ICASSP.
Chanussot. 2020a. “Graph Convolutional Networks for 2019.8682802.
Hyperspectral Image Classification.” IEEE Transactions Lei, Y., X. Liu, J. Shi, C. Lei, and J. Wang. 2019b. “Multiscale
on Geoscience and Remote Sensing 59 (7): 5966–5978. Superpixel Segmentation with Deep Features for Change
doi:10.1109/TGRS.2020.3015157. Detection.” IEEE Access 7: 36600–36616. doi:10.1109/
Hong, D., L. Gao, N. Yokoya, J. Yao, J. Chanussot, Q. Du, ACCESS.2019.2902613.
and B. Zhang. 2020b. “More Diverse Means Better: Li, Y., L. Zhou, G. Lu, B. Hou, and L. Jiao. 2017. “Change
Multimodal Deep Learning Meets Remote-Sensing Detection in Synthetic Aperture Radar Images Based on
Imagery Classification.” IEEE Transactions on Log-Mean Operator and Stacked Auto-Encoder.” In 2017
Geoscience and Remote Sensing 59 (5): 4340–4354. IEEE International Geoscience and Remote Sensing
doi:10.1109/TGRS.2020.3016820. Symposium (IGARSS), 3090–3306. doi:10.1109/
Hong, D., Z. Han, J. Yao, L. Gao, B. Zhang, A. Plaza, and J. IGARSS.2017.8127652.
Chanussot. 2021. “SpectralFormer: Rethinking Li, Y., L. Xu, and T. Liu. 2018. “Unsupervised Change
Hyperspectral Image Classification with Transformers.” Detection for Remote Sensing Images Based on Object-
IEEE Transactions on Geoscience and Remote Sensing 60: Based MRF and Stacked Autoencoders.” 2016
1–15. doi:10.1109/TGRS.2021.3130716. International Conference on Orange Technologies
Howarth, P.J., and G.M. Wickware. 1981. “Procedures for (ICOT), 64–67. doi:10.1109/ICOT.2016.8278980.
Change Detection Using Landsat Digital Data.” Li, X., Z. Yuan, and Q. Wang. 2019. “Unsupervised Deep
International Journal of Remote Sensing 2 (3): 277–291. Noise Modeling for Hyperspectral Image Change
doi:10.1080/01431168108948362. Detection.” Remote Sensing 11 (3): 258. doi:10.3390/
Hussain, M., D. Chen, A. Cheng, H. Wei, and D. Stanley. rs11030258.
2013. “Change Detection from Remotely Sensed Images: Li, Y., C. Peng, Y. Chen, L. Jiao, L. Zhou, and R. Shang. 2019.
From Pixel-Based to Object-Based Approaches.” ISPRS “A Deep Learning Method for Change Detection in
Journal of Photogrammetry and Remote Sensing 80: 91– Synthetic Aperture Radar Images.” IEEE Transactions
106. doi:10.1016/j.isprsjprs.2013.03.006. on Geoscience and Remote Sensing 57 (8): 5751–5763.
Iino, S., R. Ito, K. Doi, T. Imaizumi, and S. Hikosaka. 2018. doi:10.1109/Tgrs.2019.2901945.
“CNN-Based Generation of High-Accuracy Urban Li, R., S. Zheng, C. Duan, L. Wang, and C. Zhang. 2022.
Distribution Maps Utilising SAR Satellite Imagery for “Land Cover Classification from Remote Sensing Images
Short-Term Change Monitoring.” International Journal Based on Multi-Scale Fully Convolutional Network.”
of Image and Data Fusion 9 (4): 302–318. doi:10.1080/ Geo-Spatial Information Science 1–17. doi:10.1080/
19479832.2018.1491897. 10095020.2021.2017237.
Jiao, L., L. Huo, C. Hu, and P. Tang. 2020. “Refined Unet: Lillesand, T., R.W. Kiefer, and J. Chipman. 2015. Remote
Unet-Based Refinement Network for Cloud and Shadow Sensing and Image Interpretation. New York, NJ, USA:
Precise Segmentation.” Remote Sensing 12 (12): 2001. John Wiley & Sons.
doi:10.3390/rs12122001. Liu, S., L. Bruzzone, F. Bovolo, and P. Du. 2014.
Johnson, R.D., and E.S. Kasischke. 1998. “Change Vector “Hierarchical Unsupervised Change Detection in
Analysis: A Technique for the Multispectral Monitoring of Multitemporal Hyperspectral Images.” IEEE
Land Cover and Condition.” International Journal of Remote Transactions on Geoscience and Remote Sensing 53 (1):
Sensing 19 (3): 411–426. doi:10.1080/014311698216062. 244–260. doi:10.1109/TGRS.2014.2321277.
Kemker, R., C. Salvaggio, and C. Kanan. 2018. “Algorithms Liu, J., M. Gong, K. Qin, and P. Zhang. 2016a. “A Deep
for Semantic Segmentation of Multispectral Remote Convolutional Coupling Network for Change Detection
Sensing Imagery Using Deep Learning.” ISPRS Journal Based on Heterogeneous Optical and Radar Images.” IEEE
of Photogrammetry and Remote Sensing 145: 60–77. Transactions on Neural Networks and Learning Systems 29
doi:10.1016/j.isprsjprs.2018.04.014. (3): 545–559. doi:10.1109/TNNLS.2016.2636227.
 GEO-SPATIAL INFORMATION SCIENCE 285

Liu, J., M. Gong, J. Zhao, H. Li, and L. Jiao. 2016b. Mou, L., and X. Zhu. 2018. “A Recurrent Convolutional Neural
“Difference Representation Learning Using Stacked Network for Land Cover Change Detection in Multispectral
Restricted Boltzmann Machines for Change Detection Images.” In IGARSS 2018 - 2018 IEEE International
in SAR Images.” Soft Computing 20 (12): 4645–4657. Geoscience and Remote Sensing Symposium, 4363–4366.
doi:10.1007/s00500-014-1460-0. doi:10.1109/IGARSS.2018.8517375.
Liu, T., Y. Li, Y. Cao, and Q. Shen. 2017. “Change Detection Murray, H., A. Lucieer, and R. Williams. 2010. “Texture-
in Multitemporal Synthetic Aperture Radar Images Using Based Classification of Sub-Antarctic Vegetation
Dual-Channel Convolutional Neural Network.” Journal Communities on Heard Island.” International Journal of
of Applied Remote Sensing 11 (4): 042615. doi:10.1117/1. Applied Earth Observation and Geoinformation 12 (3):
JRS.11.042615. 138–149. doi:10.1016/j.jag.2010.01.006.
Liu, J., M. Gong, K. Qin, and P. Zhang. 2018. “A Deep Nemoto, K., R. Hamaguchi, M. Sato, A. Fujita, T. Imaizumi,
Convolutional Coupling Network for Change Detection and S. Hikosaka. 2017. “Building Change Detection via a
Based on Heterogeneous Optical and Radar Images.” Combination of Cnns Using Only RGB Aerial
IEEE Transactions on Neural Networks and Learning Imageries.” Remote Sensing Technologies and
Systems 29 (3): 545–559. Applications in Urban Environments II 10431: 104310J.
Liu, T., L. Yang, and D. Lunga. 2021. “Change Detection doi:10.1117/12.2277912.
Using Deep Learning Approach with Object-Based Image Newbold, T., L.N. Hudson, S.L. Hill, S. Contu, I. Lysenko, R.
Analysis.” Remote Sensing of Environment 256: 112308. A. Senior, L. Börger, et al. 2015. ”Global Effects of Land
doi:10.1016/j.rse.2021.112308. Use on Local Terrestrial Biodiversity.” Nature 520 (7545):
Lu, D., P. Mausel, E. Brondizio, and E. Moran. 2004. 45–50. doi:10.1038/nature14324.
“Change Detection Techniques.” International Journal Prendes, J., M. Chabert, F. Pascal, A. Giros, and J.Y.
of Remote Sensing 25 (12): 2365–2401. doi:10.1080/ Tourneret. 2015a. “Change Detection for Optical and
0143116031000139863. Radar Images Using a Bayesian Nonparametric Model
Luus, F.P., B.P. Salmon, F. Van den Bergh, and B.T.J. Coupled with a Markov Random Field.” In 2015 IEEE
Maharaj. 2015. “Multiview Deep Learning for Land-Use International Conference on Acoustics, Speech and Signal
Classification.” IEEE Geoscience and Remote Sensing Processing (ICASSP), 1513–1517. doi:10.1109/
Letters 12 (12): 2448–2452. doi:10.1109/ ICASSP.2015.7178223.
LGRS.2015.2483680. Prendes, J., M. Chabert, F. Pascal, A. Giros, and J.Y.
Lyu, H., H. Lu, and L. Mou. 2016. “Learning a Transferable Tourneret. 2015b. “Performance Assessment of a Recent
Change Rule from a Recurrent Neural Network for Land Change Detection Method for Homogeneous and
Cover Change Detection.” Remote Sensing 8 (6): 506. Heterogeneous Images.” Revue Francaise de
doi:10.3390/rs8060506. Photogrammetrie Et de Teledetection 209: 23–29.
Lyu, H., and H. Lu. 2017. “A Deep Information Based Radford, A., L. Metz, and S. Chintala. 2015. “Unsupervised
Transfer Learning Method to Detect Annual Urban Representation Learning with Deep Convolutional
Dynamics of Beijing and New York from 1984–2016.” Generative Adversarial Networks.” arXiv preprint
In 2017 IEEE International Geoscience and Remote arXiv:1511.06434. doi:10.48550/arXiv.1511.06434.
Sensing Symposium (IGARSS), 1958–1961. doi:10.1109/ Reichstein, M., G. Camps-Valls, B. Stevens, M. Jung, J.
IGARSS.2017.8127363. Denzler, and N. Carvalhais. 2019. “Deep Learning and
Ma, L., Y. Liu, X. Zhang, Y. Ye, G. Yin, and B.A. Johnson. Process Understanding for Data-Driven Earth System
2019a. “Deep Learning in Remote Sensing Applications: Science.” Nature 566 (7743): 195–204. doi:10.1038/
A Meta-Analysis and Review.” ISPRS Journal of s41586-019-0912-1.
Photogrammetry and Remote Sensing 152: 166–177. Saha, S., F. Bovolo, and L. Bruzzone. 2019. “Unsupervised
doi:10.1016/j.isprsjprs.2019.04.015. Deep Change Vector Analysis for Multiple-Change
Ma, W., Y. Xiong, Y. Wu, H. Yang, X. Zhang, and L. Jiao. Detection in VHR Images.” IEEE Transactions on
2019b. “Change Detection in Remote Sensing Images Geoscience and Remote Sensing 57 (6): 3677–3693.
Based on Image Mapping and a Deep Capsule doi:10.1109/TGRS.2018.2886643.
Network.” Remote Sensing 11 (6): 626. doi:10.3390/ Saito, K., R.J. Spence, C. Going, and M. Markus. 2004.
rs11060626. “Using High-Resolution Satellite Images for Post-
McDermid, G.J., J. Linke, A.D. Pape, D.N. Laskin, A.J. Earthquake Building Damage Assessment: A Study
McLane, and S.E. Franklin. 2008. “Object-Based Following the 26 January 2001 Gujarat Earthquake.”
Approaches to Change Analysis and Thematic Map Earthquake Spectra 20 (1): 145–169. doi:10.1193/
Update: Challenges and Limitations.” Canadian Journal 1.1650865.
of Remote Sensing 34 (5): 462–466. doi:10.5589/m08-061. Shao, Z., and C. Liu. 2014. “The Integrated Use of DMSP-
Mercier, G., G. Moser, and S.B. Serpico. 2008. “Conditional OLS Nighttime Light and MODIS Data for Monitoring
Copulas for Change Detection in Heterogeneous Remote Large-Scale Impervious Surface Dynamics: A Case Study
Sensing Images.” IEEE Transactions on Geoscience and in the Yangtze River Delta.” Remote Sensing 6 (10): 9359–
Remote Sensing 46 (5): 1428–1441. doi:10.1109/ 9378. doi:10.3390/rs6109359.
TGRS.2008.916476. Shao, Z., and J. Cai. 2018. “Remote Sensing Image Fusion
Mitchell, T., W. Cohen, E. Hruschka, P. Talukdar, B. Yang, with Deep Convolutional Neural Network.” IEEE Journal
J. Betteridge, A. Carlson, et al. 2018. ”Never-Ending of Selected Topics in Applied Earth Observations and
Learning.” Communications of the ACM 61 (5): 103– Remote Sensing 11 (5): 1656–1669. doi:10.1109/
115. DOI:10.1145/3191513. JSTARS.2018.2805923.
Mou, L., L. Bruzzone, and X. Zhu. 2018. “Learning Spectral- Shao, Z., H. Fu, D. Li, O. Altan, and T. Cheng. 2019a.
Spatial-Temporal Features via a Recurrent Convolutional “Remote Sensing Monitoring of Multi-Scale Watersheds
Neural Network for Change Detection in Multispectral Impermeability for Urban Hydrological Evaluation.”
Imagery.” IEEE Transactions on Geoscience and Remote Remote Sensing of Environment 232: 111338.
Sensing 57 (2): 924–935. doi:10.1109/TGRS.2018.2863224. doi:10.1016/j.rse.2019.111338.
286 T. BAI ET AL.

Shao, Z., Y. Pan, C. Diao, and J. Cai. 2019b. “Cloud Wang, Q., X. Zhang, G. Chen, F. Dai, Y. Gong, and K. Zhu.
Detection in Remote Sensing Images Based on 2018a. “Change Detection Based on Faster R-CNN for High-
Multiscale Features-Convolutional Neural Network.” Resolution Remote Sensing Images.” Remote Sensing Letters
IEEE Transactions on Geoscience and Remote Sensing 57 9 (10): 923–932. doi:10.1080/2150704x.2018.1492172.
(6): 4062–4076. doi:10.1109/TGRS.2018.2889677. Wang, S., D. Quan, X. Liang, M. Ning, Y. Guo, and L. Jiao.
Shi, W., M. Zhang, R. Zhang, S. Chen, and Z. Zhan. 2020. 2018b. “A Deep Learning Framework for Remote Sensing
“Change Detection Based on Artificial Intelligence: State- Image Registration.” ISPRS Journal of Photogrammetry
Of-The-Art and Challenges.” Remote Sensing 12 (10): and Remote Sensing 145: 148–164. doi:10.1016/j.
1688. doi:10.3390/rs12101688. isprsjprs.2017.12.012.
Silván-Cárdenas, J.L., and L. Wang. 2014. “On Quantifying Wang, Q., Z. Yuan, Q. Du, and X. Li. 2019. “GETNET: A
Post-Classification Subpixel Landcover Changes.” ISPRS General End-To-End 2-D CNN Framework for
Journal of Photogrammetry and Remote Sensing 98: 94– Hyperspectral Image Change Detection.” IEEE
105. doi:10.1016/j.isprsjprs.2014.09.018. Transactions on Geoscience and Remote Sensing 57 (1):
Simard, P.Y., D. Steinkraus, and J.C. Platt. 2003. “Best 3–13. doi:10.1109/tgrs.2018.2849692.
Practices for Convolutional Neural Networks Applied to Weng, Q. 2002. “Land Use Change Analysis in the Zhujiang
Visual Document Analysis.” Seventh International Delta of China Using Satellite Remote Sensing, GIS and
Conference on Document Analysis and Recognition, 958– Stochastic Modelling.” Journal of Environmental
963. doi:10.1109/ICDAR.2003.1227801. Management 64 (3): 273–284. doi:10.1006/jema.2001.
Singh, A. 1989. “Review Article Digital Change Detection 0509.
Techniques Using Remotely-Sensed Data.” International Xiao, R., R. Cui, M. Lin, L. Chen, Y. Ni, and X. Lin. 2018.
Journal of Remote Sensing 10 (6): 989–1003. doi:10.1080/ “SOMDNCD: Image Change Detection Based on Self-
01431168908903939. Organizing Maps and Deep Neural Networks.” IEEE
Song, A., J. Choi, Y. Han, and Y. Kim. 2018. “Change Access 6: 35915–35925. doi:10.1109/ACCESS.2018.2849110.
Detection in Hyperspectral Images Using Recurrent 3D Xu, Y., S. Xiang, C. Huo, and C. Pan. 2013. “Change
Fully Convolutional Networks.” Remote Sensing 10 (11): Detection Based on Auto-Encoder Model for VHR
1827. doi:10.3390/rs10111827. Images.” Mippr 2013: Pattern Recognition and Computer
Su, L., J. Shi, P. Zhang, Z. Wang, and M. Gong. 2016. Vision 8919: 891902. doi:10.1117/12.2031104.
“Detecting Multiple Changes from Multi-Temporal Yosinski, J., J. Clune, Y. Bengio, and H. Lipson. 2014. “How
Images by Using Stacked Denosing Autoencoder Based Transferable are Features in Deep Neural Networks?”
Change Vector Analysis.” In 2016 International Joint Advances in Neural Information Processing Systems.
Conference on Neural Networks (IJCNN), 1269–1276. 3320–3328.
doi:10.1109/IJCNN.2016.7727343. Yu, X., X. Wu, C. Luo, and P. Ren. 2017. “Deep Learning in
Su, L., M. Gong, P. Zhang, M. Zhang, J. Liu, and H. Yang. Remote Sensing Scene Classification: A Data
2017. “Deep Learning and Mapping Based Ternary Augmentation Enhanced Convolutional Neural
Change Detection for Information Unbalanced Images.” Network Framework.” GIScience & Remote Sensing 54
Pattern Recognition 66: 213–228. doi:10.1016/j. (5): 741–758. doi:10.1080/15481603.2017.1323377.
patcog.2017.01.002. Yuan, F., K.E. Sawaya, B.C. Loeffelholz, and M.E. Bauer.
Su, L., and X. Cao. 2018. “Fuzzy Autoencoder for Multiple 2005. “Land Cover Classification and Change Analysis
Change Detection in Remote Sensing Images.” Journal of of the Twin Cities (Minnesota) Metropolitan Area by
Applied Remote Sensing 12 (3): 035014. doi:10.1117/1. Multitemporal Landsat Remote Sensing.” Remote
jrs.12.035014. Sensing of Environment 98 (2–3): 317–328. doi:10.1016/
Sublime, J., and E. Kalinicheva. 2019. “Automatic Post- j.rse.2005.08.006.
Disaster Damage Mapping Using Deep-Learning Yuan, Z., Q. Wang, and X. Li. 2018. “ROBUST Pcanet for
Techniques for Change Detection: Case Study of the Hyperspectral Image Change Detection.” In IGARSS 2018-
Tohoku Tsunami.” Remote Sensing 11 (9): 1123. 2018 IEEE International Geoscience and Remote Sensing
doi:10.3390/rs11091123. Symposium, 4931–4934. doi:10.1109/IGARSS.2018.8518196.
Tang, Y., X. Huang, and L. Zhang. 2013. “Fault-Tolerant Yuan, Q., H. Shen, T. Li, Z. Li, S. Li, Y. Jiang, H. Xu, et al.
Building Change Detection from Urban High-Resolution 2020. ”Deep Learning in Environmental Remote Sensing:
Remote Sensing Imagery.” IEEE Geoscience and Remote Achievements and Challenges”. Remote Sensing of
Sensing Letters 10 (5): 1060–1064. doi:10.1109/ Environment 241: 111716. 10.1016/j.rse.2020.111716.
LGRS.2012.2228626. Zhan, Y., K. Fu, M. Yan, X. Sun, H. Wang, and X. Qiu. 2017.
Tewkesbury, A.P., A.J. Comber, N.J. Tate, A. Lamb, and P.F. “Change Detection Based on Deep Siamese
Fisher. 2015. “A Critical Synthesis of Remotely Sensed Convolutional Network for Optical Aerial Images.”
Optical Image Change Detection Techniques.” Remote IEEE Geoscience and Remote Sensing Letters 14 (10):
Sensing of Environment 160: 1–14. doi:10.1016/j. 1845–1849. doi:10.1109/LGRS.2017.2738149.
rse.2015.01.006. Zhan, T., M. Gong, J. Liu, and P. Zhang. 2018. “Iterative
Volpi, M., G. Camps-Valls, and D. Tuia. 2015. “Spectral Feature Mapping Network for Detecting Multiple
Alignment of Multi-Temporal Cross-Sensor Images Changes in Multi-Source Remote Sensing Images.”
with Automated Kernel Canonical Correlation ISPRS Journal of Photogrammetry and Remote Sensing
Analysis.” ISPRS Journal of Photogrammetry and Remote 146: 38–51. doi:10.1016/j.isprsjprs.2018.09.002.
Sensing 107: 50–63. doi:10.1016/j.isprsjprs.2015.02.005. Zhang, H., and P. Zhang. 2016. “Deep Difference
Waldeland, A.U., J.H. Reksten, and A.B. Salberg. 2018. Representation Learning for Multi-Spectral Imagery
“Avalanche Detection in Sar Images Using Deep Change Detection.” Proceedings of the 2016 5th
Learning.” In IGARSS 2018-2018 IEEE International International Conference on Advanced Materials and
Geoscience and Remote Sensing Symposium, 2386–2389. Computer Science, 80: 1008–1014. doi:10.2991/icamcs-
doi:10.1109/IGARSS.2018.8517536. 16.2016.204.
 GEO-SPATIAL INFORMATION SCIENCE 287

Zhang, L., L. Zhang, and B. Du. 2016. “Deep Learning for Zhao, W., Z. Wang, M. Gong, and J. Liu. 2017.
Remote Sensing Data: A Technical Tutorial on the State “Discriminative Feature Learning for Unsupervised
of the Art.” IEEE Geoscience and Remote Sensing Change Detection in Heterogeneous Images Based on a
Magazine 4 (2): 22–40. doi:10.1109/MGRS.2016.2540798. Coupled Neural Network.” IEEE Transactions on
Zhang, H., M. Gong, P. Zhang, L. Su, and J. Shi. 2016a. Geoscience and Remote Sensing 55 (12): 7066–7080.
“Feature-Level Change Detection Using Deep doi:10.1109/TGRS.2017.2739800.
Representation and Feature Change Analysis for Zhou, J., B. Yu, and J. Qin. 2014. “Multi-Level Spatial
Multispectral Imagery.” IEEE Geoscience and Remote Analysis for Change Detection of Urban Vegetation at
Sensing Letters 13 (11): 1666–1670. doi:10.1109/ Individual Tree Scale.” Remote Sensing 6 (9): 9086–9103.
Lgrs.2016.2601930. doi:10.3390/rs6099086.
Zhang, P., M. Gong, L. Su, J. Liu, and Z. Li. 2016b. “Change Zhou, L., Z. Shao, S. Wang, and X. Huang. 2022. “Deep
Detection Based on Deep Feature Representation and Learning-Based Local Climate Zone Classification Using
Mapping Transformation for Multi-Spatial-Resolution Sentinel-1 SAR and Sentinel-2 Multispectral Imagery.”
Remote Sensing Images.” ISPRS Journal of Geo-Spatial Information Science 1–16. doi:10.1080/
Photogrammetry and Remote Sensing 116: 24–41. 10095020.2022.2030654.
doi:10.1016/j.isprsjprs.2016.02.013. Zhu, X., D. Tuia, L. Mou, G. Xia, L. Zhang, F. Xu, and F.
Zhang, H., X. Ning, Z. Shao, and H. Wang. 2019. Fraundorfer. 2017. “Deep Learning in Remote Sensing: A
“Spatiotemporal Pattern Analysis of China’s Cities Comprehensive Review and List of Resources.” IEEE
Based on High-Resolution Imagery from 2000 to 2015.” Geoscience and Remote Sensing Magazine 5 (4): 8–36.
ISPRS International Journal of Geo-Information 8 (5): doi:10.1109/MGRS.2017.2762307.
241. doi:10.3390/ijgi8050241. Zhu, Z. 2017. “Change Detection Using Landsat Time
Zhang, R., Z. Shao, X. Huang, J. Wang, and D. Li. 2020. Series: A Review of Frequencies, Preprocessing,
“Object Detection in UAV Images via Global Density Algorithms, and Applications.” ISPRS Journal of
Fused Convolutional Network.” Remote Sensing 12 (19): Photogrammetry and Remote Sensing 130: 370–384.
3140. doi:10.3390/rs12193140. doi:10.1016/j.isprsjprs.2017.06.013.
Zhao, J., M. Gong, J. Liu, and L. Jiao. 2014. “Deep Learning Zhu, B., H. Gao, X. Wang, M. Xu, and X. Zhu. 2018. “Change
to Classify Difference Image for Image Change Detection Based on the Combination of Improved SegNet
Detection.” In 2014 International Joint Conference on Neural Network and Morphology.” In 2018 IEEE 3rd
Neural Networks (IJCNN), 411–417. doi:10.1109/ International Conference on Image, Vision and Computing
IJCNN.2014.6889510. (ICIVC), 55–59. doi:10.1109/ICIVC.2018.8492747.
Zhao, Q., J. Ma, M. Gong, H. Li, and T. Zhan. 2016. “Three- Zhu, H., M. Ma, W. Ma, L. Jiao, S. Hong, J. Shen, and B.
Class Change Detection in Synthetic Aperture Radar Hou. 2021. “A Spatial-Channel Progressive Fusion
Images Based on Deep Belief Network.” Journal of ResNet for Remote Sensing Classification.”
Computational and Theoretical Nanoscience 13 (6): Information Fusion 70: 72–87. doi:10.1016/j.inffus.
3757–3762. doi:10.1166/jctn.2016.5208. 2020.12.008.
288 T. BAI ET AL.

Appendix

Table A1. Nomenclature.

Abbreviations Meaning
DLCD Deep Learning Change Detection
PCCMs Post-Classification Change Methods
DCMs Direct Classification Methods
CVA Change Vector Analysis
DI Difference Image
DLNNs Deep Learning Neural Networks
CNNs Convolutional Neural Networks
AEs Auto-Encoders
DBNs Deep Belief Networks
RNNs Recurrent Neural Networks
GANs Generative Adversarial Networks
DM Differencing Methods
DNNM Differencing Neural Network Method
MTM Mapping Transformation Method
RM Recurrent Method
AM Adversarial Method
PCA Principal Component Analysis
SDAE Stacked Denoised Auto-Encoder
IQR Inter-Quartile Range
CD Change Detection
DAM Data Augmentation Method
SCDM Supervised Change Detection Method
USCDM Unsupervised Change Detection Method
SSMs Semi-Supervised Methods
TLM Transfer Learning Method
UDLCDMs Unsupervised DLCD Methods
GCN Graph Convolutional Network