Advances in Hyperspectral

Image and Signal Processing

A comprehensive overview of the state of the art

landscape—©istockphoto.com/maximkabb,
background art—all-free-download.com

PEDRAM GHAMISI, NAOTO YOKOYA, JUN LI, WENZHI LIAO,

SICONG LIU, JAVIER PLAZA, BEHNOOD RASTI, AND ANTONIO PLAZA

R ecent advances in airborne and spaceborne hyper-

spectral imaging technology have provided end us-
ers with rich spectral, spatial, and temporal information.
e­ xisting methodologies to effectively process and analyze
the data sets. Hence, rigorous and innovative methodolo-
gies are required for hyperspectral image (HSI) and signal
They have made a plethora of applications feasible for the processing and have become a center of attention for re-
analysis of large areas of the Earth’s surface. However, a searchers worldwide.
significant number of factors—such as the high dimen- This article offers a comprehensive tutorial/overview fo-
sions and size of the hyperspectral data, the lack of train- cusing specifically on hyperspectral data analysis, which is
ing samples, mixed pixels, light-scattering mechanisms categorized into seven broad topics: classification, spectral un-
in the acquisition process, and different atmospheric and mixing, dimensionality reduction (DR), resolution enhance-
geometric distortions—make such data inherently non- ment, HSI denoising and restoration, change detection (CD),
linear and complex, which poses major challenges for and fast computing. For each topic, we provide a synopsis of
the state-of-the-art approaches and numerical results for vali-
Digital Object Identifier 10.1109/MGRS.2017.2762087
dating and evaluating different methodologies, followed by a
Date of publication: 27 December 2017 discussion of future challenges and research directions.

december 2017 ieee Geoscience and remote sensing magazine 0274-6638/17©2017IEEE 37

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 THE PROMISE OF HYPERSPECTRAL IMAGING 0.4–2.5 μm.) Such detailed spectral sampling, making use
Remote sensing involves obtaining information from an of numerous small, commercial, high spatial and spectral
object or a scene without any direct physical contact. This is instruments, has made HSIs a valuable source of informa-
possible because different objects uniquely reflect, absorb, tion for a wide variety of applications, including precision
and emit electromagnetic radiation based on their molecu- agriculture (e.g., monitoring the development and health
lar composition and texture. If the radiation arriving at a of crops), the food industry (e.g., characterizing product
sensor is measured at a detailed wavelength range, the con- quality), environmental monitoring, mineralogy, defense
sequent spectral signature, also known as a spectrum, can and security-based applications (e.g., identification of man-
potentially be used to identify any given object of interest. made materials), chemical imaging, astronomy, ecological
To this end, the intent of hyperspectral imaging technol- sciences, and many others.
ogy is to capture, from the immediate surface of the Earth, A better understanding of HSIs can be gained from Fig-
hundreds of spectral channels (i.e., to shape the spectra) ure 1. A three-dimensional (3-D) hyperspectral data cube
that can precisely characterize the chemical composition of consists of n 1 # n 2 # d pixels, in which n 1 # n 2 is the num-
different materials. ber of pixels in each spectral channel and d represents the
Hyperspectral sensors sample mainly the reflective por- number of spectral channels. An HSI can be characterized
tion of the electromagnetic spectrum, ranging from the vis- using one of the following more detailed definitions.
ible region (0.4–0.7 μm) to the short-wave infrared (SWIR) 1) Spectral perspective (or spectral dimension): From this perspec-
region (almost 2.4 μm) in hundreds of narrow contiguous tive, a hyperspectral data cube is composed of n 1 # n 2 pix-
spectral channels, each of which is 10-nm wide. There are, els, where each pixel is a vector of d values. Each pixel cor-
however, other types of hyperspectral sensors that are able responds to the reflected radiation of the specific region of
to characterize the emissive properties of objects by collect- the earth and has multiple values in spectral bands. This
ing data in the range of the midwave and long-wave infra- detailed spectral information can be used to analyze dif-
red region. (Hyperspectral imaging covers a broad range of ferent materials with precision. Figure 1(c) shows a spec-
imaging systems, such as medical hyperspectral imaging, tral profile of one pixel, with multiple values for each band
atmospheric sounding, close-range hyperspectral imaging, in the spectral dimension.
and so on. Here, we focus solely on airborne or spaceborne 2) Spatial perspective (or spatial dimension): In this con-
remotely sensed HSIs with a spectral coverage ranging text, a hyperspectral data cube consists of d gray-scale

A Single Band
Pixel Vector
Spatial Dimension

0.5
Reflectance

0.4
0.3
0.2
on 0.1
si
en 0
im 0.5 0.6 0.7 0.8
lD
ia Wavelength (µm)
at
Spatial Dimension Sp

(a) (b) (c)

FIGURE 1. An example of a hyperspectral data cube: (a) a gray-scale image, (b) a hyperspectral data cube, and (c) a pixel vector and its
corresponding spectral signature.

38 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 ­images, with a size of n 1 # n 2. The values of all of the is based on using effective and efficient classification
pixels in one spectral band shape a gray-scale image approaches that can handle high dimensionality, even
with two dimensions [as shown in Figure 1(a)], which if a limited number of training samples is available.
are both spatial. In addition, along with the detailed spectral informa-
Although the greater dimensionality of HSIs compared tion provided by HSIs, it is possible to take advantage
with multispectral images improves data information con- of available spatial information (in particular, for very-
tent considerably, it does introduce new challenges to con- high-spatial-resolution HSIs) to further improve the
ventional image analysis techniques, which have been spe- eventual classification map. The “Classification” sec-
cifically designed for multispectral data. Furthermore, it is tion elaborates on advances in HSI classification.
almost impossible for humans to visualize spaces of higher Spectral mixing (including both linear and nonlinear mod-
than three dimensions (e.g., red-green-blue images). A mis- els) is another bottleneck for HSI analysis that occurs for a num-
understanding of high-dimensional spaces and conven- ber of reasons, such as insufficient spatial resolution of the sen-
tional spaces sometimes leads to incorrect interpretations sor and an intimate mixing effect. When mixing takes place, it
of HSIs and the inappropriate choice of the data process- is not possible to directly distinguish the materials available in
ing technique. Bearing this in mind, in the next section, we the pixels from the corresponding measured spectral vectors.
provide an overview of a few common HSI challenges and However, detailed spectral in-
their possible solutions. formation provided by HSIs can
be used to unmix hyperspectral WHEN THE NUMBER OF
MAIN CHALLENGES OF HYPERSPECTRAL pixels. The “Spectral Unmixing” TRAINING SAMPLES
IMAGE ANALYSIS AND POSSIBLE SOLUTIONS section focuses on spectral un- REMAINS CONSTANT, AFTER
Several factors make the analysis and processing of HSIs mixing to address these issues.
A FEW FEATURES,
a challenging task. Figure 2 illustrates the main paths in Spaceborne imaging spec-
CLASSIFICATION ACCURACY
HSI analysis that have been developed primarily to address trometers are usually designed
ACTUALLY DECREASES AS
these factors. In this section, we take a closer look at each to acquire HSIs with a moder-
of the applications shown in Figure 2. The common under- ate spatial resolution—e.g., THE NUMBER OF FEATURES
standing of HSIs is that, because such data contain a rich a ground sampling distance INCREASES.
amount of spectral information, the whole dimensionality (GSD) of 30 m—because of
needs to be used to define precise boundaries in the feature the inevitable tradeoffs among
space for a specific application. The increasing spectral res- spatial resolution, spectral resolution, temporal resolution,
olution of HSIs benefits precision applications (e.g., Earth and signal-to-noise ratio (SNR). Spatial resolution enhance-
observation, precision agriculture, and disease detection). ment of HSIs is a technology essential to expanding the range
However, it challenges conventional signal-­processing of applications for spaceborne hyperspectral missions. In the
techniques and, thus, hampers the abilities of HSIs in many “Resolution Enhancement” section, we discuss techniques for
real applications. the resolution enhancement of HSIs.
Taking classification as an example (because classifica- The degradation mechanisms associated with the
tion is one of the most popular applications for HSIs), we measurement process and atmospheric effects inject un-
found in [1] that, when the number of training samples desirable noise that substantially downgrades the quality
remains constant, after a few features, classification accu- of hyperspectral data. The HSI SNR is usually decreased
racy actually decreases as the number of features increas- during the imaging process, depending on different noise
es. Two solutions have been widely exploited to address sources. In remote-sensing HSIs, highly corrupted bands
this problem. must often be removed before any further processing. Al-
1) Dimension (feature) reduction: As mentioned in several ternatively, HSI restoration can recover those corrupted
studies, such as [2]–[4], a high-dimensional space is al- bands and also improve the HSI SNR, thereby improving
most empty, and multivariate data can be represented the effectiveness of any further processing of the HSI. In
in a lower-dimensional space, where the undesirable ef- this context, the “HSI Denoising and Image Restoration”
fects of high-dimensional geometric characteristics and section is dedicated to HSI denoising and image restora-
the curse of dimensionality are reduced. This fact has led tion techniques that address such effects.
to a chain of research on dimension (feature) reduction, Another emerging research domain in the hyperspectral
which will be detailed in the “Dimensionality Reduc- community, CD is the process of identifying and examining
tion” section. spectral–temporal changes in signals. The detailed spectral
2) Robust classifiers: The imbalance between the number of sampling and representation in HSIs result in the potential
bands and available training samples has a dramatic in- identification of more subtle spectral variations, which are
fluence on supervised classifiers. In this context, HSIs usually not easily detected in traditional multispectral imag-
often demand a vast number of training samples to ef- es. Accordingly, land cover dynamic monitoring can be en-
fectively estimate class parameters. To benefit from the hanced to a finer level. To this end, advanced CD techniques
rich spectral information of HSIs, one possible solution must be designed to address CD issues in multitemporal

december 2017 ieee Geoscience and remote sensing magazine 39

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 40
H X N

= +

Observed HSI Estimated HSI Estimated Noise

(a) (b) (c)

n
sio l
en ctra
m
Di Spe
Dimension

Time 1
CD Reduction
D d

Spatial
D≥d

Dimensions
CD Map

Time 2
(d) (e)

(f) (g)

ieee Geoscience and remote sensing magazine december 2017

FIGURE 2. The different paths of HSI analysis: (a) spectral unmixing, (b) classification, (c) image restoration and denoising, (d) CD, (e) DR, (f) fast computing, and (g) resolution enhancement.

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 HSIs and, at the same time, overcome the challenges caused global coverage. Shalom’s GSDs of 8 m and 10 m in the
by the hyperspectral data set. We elaborate on different CD visible near-infrared (VNIR) and SWIR ranges, respectively,
methods in the “Change Detection” section. are driven by both operational and commercial needs. DE-
Another crucial aspect of HSI analysis to be precisely SIS and HISUI will be mounted on the International Space
taken into account is that hyperspectral remote sensors Station. The launch of these satellites will further accelerate
are now in the era of massive automatic data collection research on HSI processing and its applications.
resulting from the improved spatial, spectral, and tempo- Figure 3 shows statistics on articles related to HSIs and
ral resolutions provided by several hyperspectral instru- signal processing published in IEEE journals during 2009–
ments. As a result, fast computing (detailed in the “Fast 2012 and 2013–2016. All articles were searched via IEEE
Computing” section) is critical to accelerating the efficient Xplore using hyperspectral as
exploitation and analysis of HSIs. the main keyword and then
categorized into broad topics UNMANNED-AERIAL-
MISSIONS AND STATISTICS by analyzing keywords in the VEHICLE-BASED REAL-TIME
Several hyperspectral imaging instruments are currently titles. The size of each pie in
HYPERSPECTRAL IMAGING
available for the purpose of remote-sensing image and Figure 3 is proportional to the
HAS become increasingly
signal analysis, providing a large volume of images for number of articles.
various thematic applications. Airborne hyperspectral The totals returned by this COMMON IN VARIOUS
imaging sensors [e.g., the Airborne Visible/Infrared Imag- search are an indicator of the APPLICATIONS.
ing Spectrometer (AVIRIS), Hyperspectral Digital Imag- hyperspectral community’s
ery Collection Experiment, Compact Airborne Spectro- recent growth. Seven topics
graphic Imager (CASI), Airborne Prism Experiment, and under investigation represented 61.5% of all of the articles
HySpex] play a central role in acquiring data sources for published in 2013–2016. Classification was the most active-
the hyperspectral scientific community. The European ly addressed topic in both periods, while spectral unmixing
Facility for Airborne Research (http://www.eufar.net) has was the second most common. Classification- and unmix-
established standards and protocols in the field of air- ing-related studies accounted for 41.6% of the total. These
borne hyperspectral remote sensing, allowing transna- top two topics were followed by DR and image restoration.
tional access to national infrastructures. In recent years, Image restoration showed a high growth rate, indicating
unmanned-aerial-vehicle-based real-time hyperspectral that the improvement in the quality of HSIs is significant
imaging has become increasingly common in various ap- in subsequent processing. Resolution enhancement received
plications, such as agricultural monitoring, while raising particular attention during 2013–2016, as demonstrated by
new challenges in image processing. the highest growth rate. Although the number of articles re-
Table 1 presents the principal parameters of seven space- lated to CD increased steadily, the overall number was still
borne imaging spectroscopy missions planned for the near small, probably due to limited data sets.
future: the China commercial remote-sensing satellite sys-
tem, DLR Earth sensing imaging spectrometer (DESIS) [5], CONTRIBUTION
environmental mapping and analysis program [6], hyper- This article introduces a detailed and organized overview of
spectral imager suite (HISUI) [7], precursore iperspettrale HSIs and signal processing, categorized into the seven differ-
della missione applicativa (PRISMA) [8], spaceborne hy- ent themes previously mentioned. In each section, we pro-
perspectral applicative land and ocean mission (Shalom) vide some numerical results, illustrations, a critical overview
[9], and hyperspectral infrared imager [10]. Many of those of the state of the art, current challenges, and possible future
satellites are designed to have a GSD of 30 m, aiming at works. It is worth noting that the methodologies described

TABLE 1. THE PARAMETERS OF SEVEN SPACEBORNE IMAGING SPECTROSCOPY MISSIONS.

PARAMETER CCRSS DESIS EnMAP HISUI PRISMA Shalom HyspIRI

Altitude (km) 30 400 653 400 615 600 626
GSD (m) 30 30 30 30 30 10 30
Bandwidth (nm) 5–20 3.3 5.25–12.5 10–12.5 # 12 10 # 10
Spectral coverage ( μm) 0.4–2.5 0.4–1.0 0.42–2.45 0.44–2.5 0.4–2.5 0.4–2.5 0.38–2.5
Number of bands 328 180 228 185 237 241 210
Swath width (km) 30 30.7 30 20 30–60 10 45
Other sensor Pan — — — Pan Pan TIR

TIR: thermal infrared; CCRSS: China commercial remote-sensing satellite system; EnMAP: environmental mapping and analysis program; HyspIRI: ­hyperspectral infrared imager.

december 2017 ieee Geoscience and remote sensing magazine 41

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 Classification
22.4%
Others
45.1%
Classification
450 Papers 27.2%
Unmixing Others
16.2% 38.5%

1,206 Papers

DR 7.8%
Unmixing
Image Restoration 2.0% 14.4%
Fast Computing 3.6%
Resolution Enhancement 1.6%
CD 1.3%
DR
(a) CD 1.6%
Resolution Enhancement 2.7% 9.4%
Fast Computing 3.0%
Image Restoration 3.2%
(b)

FIGURE 3. Some statistics on articles related to HSIs and signal processing published in IEEE journals during (a) 2009–2012 and
(b) 2013–2016. The size of each pie chart is proportional to the number of articles.

or mentioned are rooted mainly in the signal and image pro- Spectrometer (ROSIS)-03 Pavia University, CASI Houston
cessing, statistical inference, and machine-learning fields, University, and Hyperion Umatilla County.
with a particular emphasis on methodologies developed
since 2013, after the publication of a previous survey article PAVIA UNIVERSITY
on hyperspectral remote-sensing image analysis [11]. This data set was captured over the University of Pavia,
Italy, by the ROSIS-03 airborne instrument. The flight
DATA SETS over the city of Pavia, Italy, was operated by the Deutsches
Throughout this article, three benchmark hyperspectral Zentrum für Luft- und Raumfahrt (the German Aerospace
data sets are referenced: Reflective Optics System Imaging Agency) within the context of the HySens project, managed
and sponsored by the European Union. The ROSIS-03 sen-
sor has 115 data channels with a spectral coverage ranging
from 0.43 to 0.86 μm. Twelve channels have been removed
because of the existence of noise, and the remaining 103
spectral channels processed. The data have been corrected
atmospherically, but not geometrically. The spatial resolu-
tion is 1.3 m per pixel. The data set covers the Engineering
School at the University of Pavia and consists of different
classes, including trees, asphalt, bitumen, gravel, metal
sheet, shadow, bricks, meadow, and soil. The subset data set
investigated in this review article comprises 640 # 340 pixels.
Figure 4 presents a false color image of ROSIS-03 Pavia
University data and the corresponding training and test
(a) (b) (c) samples that have already been separated.

Asphalt Trees Bitumen

UNIVERSITY OF HOUSTON
Meadows Metal Sheets Bricks
This data set was captured by the CASI imager over the Univer-
Gravel Bare Soil Shadows
sity of Houston campus and the neighboring urban area in June
2012. The size of the data is 349 # 1,905 pixels, with a spatial
FIGURE 4. ROSIS-03 Pavia University: (a) a false color composite, resolution of 2.5 m. This data set is composed of 144 spectral
(b) training samples, and (c) test samples. bands ranging from 0.38 to 1.05 μm. These data consist of

42 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 15 classes, including grass healthy, grass stressed, grass synthetic,
tree, soil, water, residential, commercial, road, highway, railway,
parking lot 1, parking lot 2, tennis court, and running track.
Parking lot 1 includes parking garages at the ground level and
also in elevated areas, while parking lot 2 corresponds to parked (a)
vehicles. Figure 5 shows a three-band false color image and its
corresponding already-separated training and test samples.

UMATILLA COUNTY
A pair of real bitemporal Hyperion HSIs acquired on 1 May (b)
2004 ^ X 1 h and 8 May 2007 ^ X 2 h were used to test several
selected state-of-the-art CD approaches. This scene covers
irrigated agricultural land in Umatilla County, Oregon. The
images under consideration have a size of 180 # 225 pixels.
(c)
The original Hyperion images contain 242 spectral bands,
ranging from 0.35 to 2.58 μm, i.e., VNIR, and SWIR, with Thematic Classes:
a spectral resolution of 0.01 μm and a spatial resolution of Healthy Grass Stressed Grass Synthetic Grass
30 m. Preprocessing operations, such as the removal of the Tree Soil Water
uncalibrated and noisiest bands, bad stripes repair, atmo- Residential Commercial Road
spheric correction, and coregistration, have been carried Highway Railway Parking Lot 1
Parking Lot 2 Tennis Court Running Track
out. Finally, 159 preprocessed bands (i.e., 8–57, 82–119,
131–164, 182–184, and 187–220) out of the original 242
bands were used in the CD experiment. Changes occurring FIGURE 5. CASI Houston: (a) a false color composite (red: band 70,
in this scenario include the land cover class transitions be- green: band 50, blue: band 20), (b) training samples, and (c) test samples.
tween crops, bare soil, subtle variations in soil moisture,
and water content of vegetation. More detailed descriptions comparison with other pixel-level-based approaches could
of this data set can be found in [12]. Figure 6(a) and (b) be conducted fairly. Thus, six pixel-level changes were con-
shows the false color composite of X 1 and X 2, respectively. sidered, as shown in Figure 6(d).
The false color composite of three spectral change vector
(SCV) channels is shown in Figure 6(c); possible different DIMENSIONALITY REDUCTION
changed pixels are illustrated in different colors, whereas The increasing spectral resolution of hyperspectral data
the unchanged pixels are in gray. The multiclass change benefits precision pattern recognition, but it challenges
reference map is created based on careful image inter- both the memory capacity of ordinary personal computers
pretation, as shown in Figure 6(d). Note that the possible and conventional signal-processing techniques. For an HSI
subtle subpixel-level changes (e.g., the one associated with with a spatial dimension of 600 # 400 pixels at 16 b-per-
the road surrounding the irrigated agricultural land [12]) band-per-pixel, the data volume becomes 240 MB for 500
are not considered in this article, so that the quantitative spectral bands. The data volume can be linearly increased

ωC1
ωC2

ωC3

ωC4
ωC5

ωC6
ωn

(a) (b) (c) (d)

FIGURE 6. Umatilla County: (a) a false color composite (red: 650.67 nm, green: 548.92 nm, blue: 447.17 nm) of the bitemporal EO-1 Hy-
perion images acquired over an irrigated agricultural area in Umatilla County, Oregon, in 2004 (X1) and (b) in 2007 (X 2); (c) a composite of
three SCV channels (red: 823.65 nm, green: 721.90 nm, blue: 620.15 nm); and (d) a multiclass change reference map, in which six changes
are in different colors, whereas the unchanged pixels are in gray.

december 2017 ieee Geoscience and remote sensing magazine 43

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 when time-series hyperspectral data are acquired to moni- extraction compresses the high-dimensional original data
tor environmental changes. The complexities of storing and to generate a small number of new features, where each
processing the data will easily exceed the memory capacity band often contributes to determining f, as shown in Fig-
of ordinary personal computers. Moreover, as previously ure 7. DR methods can be categorized into unsupervised,
discussed, when the ratio between the spectral bands and supervised, and semisupervised approaches, depending
the number of training samples is high, high-dimensional on whether the class label information is being used.
hyperspectral data suffer from the well-known issue of the
curse of dimensionality. UNSUPERVISED DIMENSIONALITY REDUCTION
DR, with the goal of identifying and eliminating statistical Unsupervised DR methods deal with cases where no la-
redundancies of hyperspectral data while keeping as much beled samples are available and aim to find another rep-
spectral information as pos- resentation of the data in the lower-dimensional space by
sible, is widely used in hyper- satisfying some given criterion. A variety of unsupervised
spectral data processing. Rela- DR methods have been introduced in the literature. The
MINIMUM NOISE- tively few bands can represent objective of these methods is not to optimize the accuracy
FRACTION TRANSFORM- most of the information in for a given classification task, because they do not consider
ation OBTAINS THE HSIs [13], making DR very use- class-specific information provided by labeled samples.
REDUCED FEATURES ful for storage, transmission, For example, principal component analysis (PCA) [20] re-
ACCORDING TO THE IMAGE classification, spectral unmix- duces dimensionality by capturing the maximum variance
QUALITY MEASURED ing, target detection [14], and in the data. Independent component analysis (ICA) [21]
visualization of remote-sens- finds the project matrix by maximizing the statistical in-
BY THE SNR.
ing data [13], [15]. Recent work dependence. Minimum noise-fraction (MNF) transforma-
demonstrates the benefits of tion [22] obtains the reduced features according to the im-
using DR when extracting rel- age quality measured by the SNR, and local linear feature
evant information from HSIs for CD [16], forest management extraction (LLFE) [23]–[25] methods seek a projection di-
[17], and urban planning [18]. The applications of DR are of rection in which neighborhood relationships are preserved
interest well beyond hyperspectral data, i.e., for various ap- in the feature spaces. The nonlinear versions of these meth-
plications in signal processing and computer vision [19], and ods, such as kernel methods (e.g., kernel PCA, kernel ICA,
wherever interpretation and analysis of high-dimensional and kernel MNF [26]) and local methods (e.g., locally lin-
data are of interest. ear embedding [27], Laplacian eigenmap, and local tan-
Hyperspectral DR consists of both feature selection gent space alignment [19]) have been widely used to detect
and feature extraction [13]. Feature selection tries to select higher-order statistical redundancies. In the same manner,
a minimal subset of D features S = {S 1, S 2, f, S D} from the conventional unsupervised feature selection methods for
original feature set F = {F1, F2, f, Fd} based on an adopted DR select a subset of features from the original data accord-
selection criterion, where D # d and S 3 F, while aiming ing to a specific criterion, such as linear prediction error
to achieve improved performances for a specific applica- [28], entropy [29], or mutual information (by minimizing
tion (e.g., classification, target detection, and so forth). ­dependency) [30].
The objective of feature extraction is to find a transfor- Recently, fusion-based methods and manifold-learning
mation function f : R d " R D that can transform the high- methods have been widely explored for HSI unsupervised
dimensional data point {x i ! R d} iN= 1 to z i = f (x i), where DR. Graph-based fusion methods couple data fusion and
{z i ! R D} iN= 1 and D # d, such that most information of DR in a unified framework for classification [31], [32].
the high-dimensional data is kept in a much lower-di- Borhani and Ghassemian presented a kernel-based method
mensional subspace. The term f can be a linear or non- to incorporate spectral and spatial information simultane-
linear transformation. Unlike feature selection, feature ously for DR and classification of hyperspectral data [33],
while Zhang et al. represented multiple features in a low-
dimensional feature space where the complementary infor-
mation of each feature was exploited by comanifold learn-
ing and cograph regularization [34]. In the approaches of
[35], manifold learning was exploited for feature extraction
and salient band selection of HSIs. In [36], orthogonal to-
tal variation component analysis (OTVCA) was proposed,
where a nonconvex cost function was optimized to find the
best representation for HSIs in a low-dimensional feature
(a) (b) space while controlling the spatial smoothness of the fea-
tures by using a total variation (TV) regularization. The TV
FIGURE 7. Hyperspectral DR: (a) feature selection and (b) feature penalty promotes piecewise smoothness (homogeneous
extraction. spatial regions) on the extracted features and thus helps

44 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 to extract spatial (local neighborhood) information that is stacked autoencoders [62] and convolutional neural net-
very useful for classification. It was shown that OTVCA is works (CNNs) [63], are used for spectral-spatial feature ex-
highly robust to noise, because it exploits a penalized least- traction for HSI classification [64], [65].
squares minimization framework.
SEMISUPERVISED DIMENSIONALITY REDUCTION
SUPERVISED DIMENSIONALITY REDUCTION In real-world applications, labeled data are usually very lim-
Supervised methods rely on the existence of labeled sam- ited, and labeling a large amount of data may sometimes
ples to infer class separability. Several widely used super- require considerable human resources or expertise. On the
vised DR methods for HSIs are linear discriminant analy- other hand, unlabeled data are available in large quantities at
sis (LDA) [37], nonparametric weighted feature extraction very low cost. For this reason, semisupervised methods [66]–
(NWFE) [38], band selection based on Jeffries–Matsushita [68], which aim at improved
(J–M) distance [39], and mutual information [40]. Many classification by utilizing both
extensions of these methods have been proposed in past unlabeled and limited labeled
decades, including modified Fisher’s LDA [41], regularized data, have gained popularity SOME SEMISUPERVISED
LDA [42], modified NWFE using spatial and spectral infor- in the machine-learning com- FEATURE EXTRACTION
mation [43], kernel NWFE [44], extended J–M to multiclass munity. Some of the represen- METHODS ADD A
cases [40], J–M distance for spatially invariant features [45], tative semisupervised learning REGULARIZATION TERM
minimal-redundancy/maximal-relevance based on mutual methods include cotraining TO PRESERVE CERTAIN
information [46], and normalized mutual information [47]. [66], transductive support vec- POTENTIAL PROPERTIES
Recent supervised DR methods for hyperspectral data tor machines (SVMs) [67], and
OF THE DATA.
exploit the local neighborhood properties of data. Li et al. graph-based semisupervised
[48] employed local Fisher’s LDA [49] to reduce the dimen- learning methods [68].
sionality of the data while preserving the corresponding Some semisupervised fea-
multimodal structure. In [50], local neighborhood infor- ture extraction methods add a regularization term to pre-
mation was exploited in both the spectral and spatial do- serve certain potential properties of the data. For example,
mains to find a discriminative projection for DR of hyper- semisupervised discriminant analysis (SDA) [69] adds a
spectral data. Cao et al. [51] proposed a supervised band regularizer into the objective function of LDA. The resulting
selection, by introducing the local spatial smoothness of method makes use of a limited number of labeled samples
the HSI into the wrapper method. Dong et al. [52] presented to maximize class discrimination and employs both labeled
an ensemble-discriminative local-metric-learning method and unlabeled samples to preserve the local properties of
for DR, where local spatial information was incorporated the data. The approach of [70] proposed a general semisu-
into distance metric learning to learn a subspace, keeping pervised DR framework based on pairwise constraints and
the samples from the same class closer while pushing those employed regularization with sparse representation (SR). A
from different classes farther away. semisupervised pairwise band selection method [71] was
Sparse graph embedding (SGE) explores the sparsity proposed for HSIs, in which an individual band selection
structure of the data for hyperspectral DR. Ly et al. [53] process was performed only on each pair of classes. Other
proposed block sparse-graph-based discriminant analysis, semisupervised feature extraction methods combine su-
which learns a block sparse graph for a supervised DR. Xue pervised methods with unsupervised ones using a tradeoff
et al. [54] proposed a spatially and spectrally regularized lo- parameter, such as semisupervised local Fisher’s (SELF) dis-
cal discriminant embedding method for DR, where spatial criminant analysis [72].
information was integrated into the sparse graph learning It may not be easy, however, to specify the optimal pa-
process. In [55], a discriminative sparse multimodal learn- rameter values in these and similar semisupervised tech-
ing was developed for multiple-feature selection. However, niques, as mentioned in [70] and [72]. Liao et al. [73] pro-
the sparse coding used in SGE is helpful for learning under posed a semisupervised local discriminant (SELD) analysis
conditions where the coding is local [56], which means lo- to overcome this problem by combining unsupervised meth-
cality is more important than sparsity. Unfortunately, the ods (LLFE [23]–[25]) and a supervised method (LDA [37]) in
converse is not true: sparsity does not always guarantee lo- a novel framework without any free parameters. They found
cality [56]. He et al. [57] proposed a weighted sparse graph to an optimal projection matrix that preserves the local neigh-
overcome the drawback of sparse coding in SGE, where both borhood information inferred from unlabeled samples,
the locality and sparsity of the training pixels are integrated. while simultaneously maximizing the class discrimination
Other trends in supervised DR methods exploit various of the data inferred from the labeled samples. The approach
algorithms and learning techniques from soft computing, of [74] improved SELD [73] by better modeling the differ-
artificial intelligence, and machine learning. Genetic algo- ences and similarities between samples. Specifically, this
rithms (GAs) [58], particle swarm optimization (PSO) [59], method built a semisupervised graph where labeled samples
and the combination of GAs and PSO are used to optimize were connected according to their label information and un-
feature selection [60], [61]. Deep learning techniques, e.g., labeled samples by their nearest-neighborhood information.

december 2017 ieee Geoscience and remote sensing magazine 45

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 Graph embedding and manifold-based SR were combined We compared the performance when using raw hyperspectral
in a semisupervised framework for hyperspectral DR [75], data and seven DR methods (including unsupervised, super-
where the sparse coefficients were exploited to construct the vised, and semisupervised DR methods) with three popular
graph. Semisupervised manifold alignment [76] and semisu- classifiers, the parameter settings of which are the same as those
pervised transfer component analysis [77] were proposed to in [74]. The training samples were randomly selected from the
find a transformation matrix to project high-dimensional training set, with the sample size corresponding to different
multimodal images into a lower-dimensional feature space, cases: 20, 40, and 80 samples per class, respectively. The results
where the geometry of each modality could be preserved. were averaged over ten runs on different numbers of extracted
These semisupervised DR methods try to build a similar features from one to 30, and the averaged overall accuracy (OA)
objective function, i.e., maximizing class discrimination of the classification was recorded for each method.
while at the same time preserving the intrinsic geometric The results confirmed that DR can improve classifica-
structure of the data. The optimal solutions are acquired by tion performance on HSIs. As the size of the training sample
solving generalized eigenvalue problems in the same man- increased, classification accuracy increased. Semisupervised
ner [78]. These methods can be further expanded to shape a DR methods (especially those in [73] and [74], designed for
manifold learning method by using the kernel trick, similar hyperspectral data) outperformed both unsupervised and su-
to the approaches in [79]. pervised methods for the one-nearest-neighbor classifier.
DR methods that exploit spatial smoothness produced better
EXPERIMENTAL RESULTS results, even for unsupervised methods; e.g., OTVCA [36]
Table 2 and Figure 8 show the performances of some DR outperformed the other methods for both random forest
methods on the classification of the Pavia University HSIs. (RF) and SVM classifiers in terms of classification accuracy.

TABLE 2. THE DIMENSIONALITY REDUCTION FOR THE CLASSIFICATION OF PAVIA UNIVERSITY DATA, OA%
(OPTIMAL NUMBER OF REDUCED FEATURES).

NUMBER OF LABELED SAMPLES

DR METHODS CLASSIFIER 20 40 80
Unsupervised Raw 1NN 63.14 68.36 69.88
SVM 67.29 70.15 71.51
RF 68.72 71.06 73.39
PCA 1NN 67.38 (11) 69.46 (12) 70.23 (11)
SVM 70.21 (8) 74.77 (8) 76.81 (9)
RF 73.46 (11) 73.88 (11) 76.13 (11)
LPP [24] 1NN 66.97 (18) 69.43 (18) 70.17 (12)
SVM 71.80 (9) 77.14 (12) 77.95 (18)
RF 72.46 (18) 74.20 (19) 75.82 (19)
OTVCA [36] 1NN 71.93 (19) 75.81 (20) 80.39 (23)
SVM 92.74 (19) 96.32 (21) 97.52 (18)
RF 96.24 (23) 97.75 (21) 98.79 (22)
Supervised NWFE [38] 1NN 71.34 (9) 73.34 (9) 74.42 (12)
SVM 72.29 (8) 76.62 (8) 77.18 (9)
RF 75.84 (9) 77.03 (11) 78.74 (8)
Semisupervised SDA [69] 1NN 52.67 (7) 62.64 (8) 70.76 (8)
SVM 51.62 (8) 63.64 (9) 70.96 (10)
RF 55.72 (9) 66.97 (9) 71.34 (9)
SELF [72] 1NN 61.42 (18) 68.75 (18) 69.56 (16)
SVM 63.93 (19) 75.85 (19) 82.74 (18)
RF 67.98 (18) 75.97 (12) 80.56 (12)
SELD [73] 1NN 77.27 (17) 79.03 (10) 81.95 (11)
SVM 75.71 (11) 76.20 (10) 82.03 (11)
RF 75.53 (8) 77.50 (9) 82.26 (9)
SEGL [74] 1NN 79.40 (9) 81.22 (9) 83.57 (10)
SVM 77.57 (8) 78.16 (8) 82.26 (8)
RF 76.67 (8) 79.14 (8) 83.51 (9)

LPP: locality preserving projections; SEGL: semisupervised graph learning; 1NN: one nearest neighbor.

46 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 We can also see from Figure 8 that, as the number of features covers by assigning unknown pixel vectors to one of the
increases, the classification performance does not always in- classes (or clusters). The individual classes are commonly
crease—in fact, some decrease. To achieve optimal clas- differentiated based on the similarity to a certain class or
sification performance, the number of reduced features by defining decision boundaries constructed in the feature
needs to be optimized. space. The initial set of features for classification usually en-
compasses spectral channels [4].
CHALLENGES for DIMENSIONALITY REDUCTION With reference to Figure 1, two types of classification
Recent advances in sensor technologies and processing tech- approaches can be broadly defined: spectral classifiers
niques strongly support the use of hyperspectral data. More- and spectral-spatial classifiers [4], where the former con-
over, global Earth observation missions (e.g., AVIRIS from sider the HSIs to be a list of spectral measurements with
NASA, the PROBA series from the European Space Agency, no spatial organization, while the latter classify the input
and the Gaofen series from China) make such data in- data by taking into account the spatial dependencies of
creasingly accessible. Furthermore, at lower altitudes, air- adjacent pixels.
planes and unmanned aerial vehicles can deliver extremely
high-resolution hyperspectral data from targeted locations. SPECTRAL CLASSIFIERS
In addition, image processing techniques allow us to extract Based on the availability of training samples (also referred
multiple-level features from these big hyperspectral data. to as learning with a teacher) for the training stage, classi-
Two main challenges remain in hyperspectral DR: 1) fication approaches can be grouped into three categories:
mining complementary features (while reducing the di- supervised, unsupervised (also known as clustering), and
mension and redundancy) from multiple levels of big hy- semisupervised approaches.
perspectral data and 2) coupling DR and applications in a Supervised approaches classify input data using train-
unified framework, ensuring that optimal features for ap- ing samples. These samples are usually collected in one of
plications are obtained. Most state-of-the-art research has two ways: 1) by manually labeling a small number of pix-
separated DR and applications into two different steps. els in an image or 2) based on some field measurements.
For example, morphological operators were employed In contrast, unsupervised classification does not consider
in [80] to extract low-level features (such as the size and training samples. The supervised approach classifies input
shape of objects) from remote-sensing images. In [81] and data based only on an arbitrary number of initial cluster
[82], middle-level attribute features were extracted from centers that may be either user-specified or selected quite
HSIs for land cover mapping. High-level features, such arbitrarily. During processing, each pixel is associated with
as object-based [83] and so-called deep learning features one of the cluster centers, usually in an iterative way, based
[84], have been used for CD and classification. on a similarity criterion [85], [86]. In semisupervised ap-
State-of-the-art DR methods typically deal with either proaches [87], the training stage is based not only on la-
lower-level or higher-level features, but not with a combi- beled training samples but also on unlabeled samples.
nation of both. The features extracted at each level have Because the consideration of training samples leads
their own characteristics: high-level ­features are usually to higher classification accuracies than in situations
more powerful but less robust, while low-level ones are where there is no class-specific information, supervised
less informative but more robust.
On the other hand, classification
is taken as one of the most popular 100
applications to validate DR perfor-
mances. Hyperspectral classification 90
typically consists of two steps: 1) DR 80
(via either feature extraction or fea-
ture selection) and 2) a training pro- 70
OA (%)

cedure for designing the classifier.

60
However, it is difficult to ensure that
the best features from the first step 50
will optimize the classification per- Raw PCA LPP NWFE SDA
formance of the f­ ollowing one. 40 SELF SELD SEGL OTVCA

30
CLASSIFICATION
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

HSI classification is a fast-growing Number of Features

and highly active field of research
in the hyperspectral community. A FIGURE 8. The classification performance on the Pavia University data as the number of
classification algorithm is used to features increases. Forty labeled training samples per class were randomly selected from the
distinguish between different land training set with an SVM classifier.

december 2017 ieee Geoscience and remote sensing magazine 47

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 approaches have gained more attention in the hyperspec- [98], [99], where hierarchical features were extracted from
tral community than unsupervised ones. However, the the input data.
curse of dimensionality is a bottleneck for supervised clas- Another deep model, the deep belief network, was pro-
sification techniques. In theory, a large number of training posed for the classification of hyperspectral data through
samples is required to define precise class boundaries in the the learning of spectral-based features [100]. The critical
feature space. This problem intensifies when the number comparison conducted in [88], specifically on supervised
of bands (features) increases. However, in practice, there spectral classifiers, offered tantalizing hints about the logi-
are not enough training samples to train supervised classi- cal selection of an appropriate classifier based on the ap-
fiers, because collecting such plication at hand. One of the main conclusions was that
samples is time consuming there is no classifier that can consistently provide the best
and/or costly. Therefore, clas- performance in terms of classification accuracy when differ-
CLASSIFICATION sification approaches devel- ent data sets or different sets of training and test samples are
APPROACHES DEVELOPED oped for HSIs need to be able considered. Instead, in addition to the resulting classifica-
FOR HSIs NEED TO BE ABLE to handle high-dimensional tion accuracies, the consideration of an appropriate classi-
TO HANDLE HIGH- data with only a limited num- fier should be based on the complexity of the analysis sce-
DIMENSIONAL DATA WITH ber of training samples. nario (e.g., the availability of training samples, processing
ONLY A LIMITED NUMBER The most widely used su- requirements, tuning parameters, algorithm speed, and so
pervised spectral classifiers forth) and on the considered application domain.
OF TRAINING SAMPLES.
have been studied precisely and
compared in [88]. Table 3 dem- SPECTRAL-SPATIAL CLASSIFIERS
onstrates classification accura- Neighboring pixels in HSIs are highly related or correlated
cies—i.e., OA, average accuracy (AA), and kappa coefficient— because remote sensors acquire a significant amount of
obtained on the University of Houston data set by a number of energy from adjacent pixels and homogeneous structures
widely used supervised spectral classifiers in the hyperspectral in the image scene are generally larger than the size of a
community, including SVM [89], RF [90], rotation forest (RoF) pixel. This is especially evident for images of high spatial
[91], canonical correlation forest (CCF) [92], [93], back-propa- resolution. Spatial and contextual data can provide useful
gation (BP) neural network [94], extreme learning machines information about the shape of different structures. In ad-
(ELMs) [95], kernel ELMs (KELMs) [96], one-dimensional (1- dition, such information reduces the labeling uncertainty
D) deep CNNs [84], and multiple linear regression (MLR) [97]. that exists when only spectral information is taken into
For the algorithm setup, see [88] and [93]. account and helps to address the salt-and-pepper appear-
Currently, a major contribution in the hyperspectral ance of the resulting classification map. In general, spec-
community is based on the use of deep learning for HSI tral-spatial classification techniques are composed of three
classification. HSIs are highly influenced by various atmo- main stages:
spheric scattering conditions, complicated light-scattering 1) extracting spectral information (i.e., based on spectral
mechanisms, interclass similarity, and intraclass variabil- classifiers discussed in the “Spectral Classifiers” section)
ity, which make the hyperspectral imaging procedure in- 2) extracting spatial information (to be discussed later in
herently nonlinear [65]. Compared to the so-called shal- this section)
low models, deep learning approaches are expected to 3) combining the spectral information extracted during
potentially extract high-level, hierarchical, and abstract the first stage and the spatial information extracted dur-
features that are, by nature, more robust when handling ing the second.
the nonlinearities of the input hyperspectral data. Al- To extract spatial information, two common strategies
though the use of deep learning in the hyperspectral com- are available: the crisp neighborhood system and the adap-
munity is in its early days, some contributions in the com- tive neighborhood system. While the former considers spa-
munity have focused on the use of deep learning for HSI tial and contextual information in a predefined neighbor-
classification. A stacked autoencoder and an autoencoder hood system, the latter is more flexible and not confined
with sparse constraint were proposed for HSI classification to a given neighborhood system. In the following two

TABLE 3. SUPERVISED SPECTRAL CLASSIFIERS: CLASSIFICATION ACCURACIES OBTAINED from

THE UNIVERSITY OF HOUSTON DATA.

CLASS SVM RF RoF CCF BP ELM KELM 1-D CNN MLR

OA 80.1 72.9 79.1 83.3 80.9 79.5 80.6 78.2 80.6
AA 83.0 76.9 82.0 85.7 83.1 82.4 82.9 81.2 83.0
Kappa 0.786 0.709 0.775 0.820 0.793 0.778 0.790 0.784 0.790

48 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 s­ections, each neighborhood system is briefly explained. approach solves the curse of dimensionality and the lack of
It should be noted that these methods have been elaborated available training samples by iteratively selecting the most
in detail in [4]. Table 4 demonstrates several classification informative bands suitable for the designed network. Table 4
accuracies obtained on the Pavia University data set by demonstrates the classification accuracies obtained by SIC-
different spectral-spatial classification approaches we will NN [65] and 2-D CNN [84] on the Pavia University data set.
now briefly discuss. As can be seen, in all cases, the use of crisp-neighborhood-
system-based spectral-spatial classification can improve the
CRISP NEIGHBORHOOD SYSTEMS classification accuracy of spectral classifiers (e.g., RF and
Markov random fields (MRFs) are a family of probabilistic SVM). However, considering
models that can be described as a two-dimensional (2-D) a set of crisp neighbors has
stochastic process over discrete pixel lattices. MRFs have some disadvantages: TO ADDRESS THE
been widely used to integrate spatial context into image 1) The crisp neighborhood sys- SHORTCOMINGS OF CRISP
classification problems. In this family of approaches, it is as- tem may not contain enough
NEIGHBORHOOD SYSTEMS,
sumed that, for a predefined neighborhood of a given pixel, samples, which downgrades
AN ADAPTIVE
there is a high probability that its closest neighbors belong the effectiveness of the clas-
to the same object. In [101], a classification framework was sifier (particularly when the NEIGHBORHOOD SYSTEM
introduced by integrating an SVM and MRF. The developed input data set is of high reso- CAN BE CONSIDERED.
contextual generalization of an SVMs was achieved by ana- lution and the neighboring
lytically relating the Markovian minimum-energy criterion pixels are highly correlated).
to the application of an SVM. In [102], Ghamisi et al. pro- 2) A larger neighborhood system may lead to intractable
posed a spectral-spatial classification approach based on a computational problems. Unfortunately, the closest fixed
generalization of the MRF called hidden MRF (HMRF). In neighborhoods do not always accurately reflect informa-
that work, spectral and spatial information was e­ xtracted tion about spatial structures. For instance, they provoke
using SVM and HMRF, respectively. Finally, the spectral and assimilation of regions containing only a few pixels with
spatial information was combined via majority voting with- their larger neighboring structures and do not provide ac-
in each object. Xia et al. [103] integrated MRFs with the RoF curate spatial information at the border of regions.
classifier to further improve classification accuracy. 3) In general, the use of a crisp neighborhood system leads
Another way of considering spatial information using to acceptable results for big regions in the scene. Oth-
the crisp neighborhood system is based on 2-D or 3-D deep erwise, it can make small structures in the scene disap-
CNNs [84]. CNNs consider local connections to deal with pear, merging them with larger surrounding objects.
spatial dependencies using sharing weights, which can 4) They may cause oversmoothing on the border of differ-
significantly reduce the number of network parameters ent classes. This problem, however, has been addressed
(compared to its 1-D fully connected version) and extract in [102] using a gradient step.
spatial and contextual information using a predefined crisp
neighborhood system [84]. In [104], an unsupervised ap- ADAPTIVE NEIGHBORHOOD SYSTEMS
proach was introduced to learn feature extraction frame- To address the shortcomings of crisp neighborhood sys-
works from unlabeled hyperspectral imagery. This method tems, an adaptive neighborhood system can be consid-
extracts generalizable features by training on sufficiently ered. One approach is to take advantage of different seg-
large quantities of unlabeled data that are distinct from the mentation methods. Image segmentation is the process of
target data set. The trained network is then able to extract partitioning a digital photo into multiple nonoverlapping
features from smaller, labeled target data sets and address regions or objects. In image segmentation, a label is as-
the curse of dimensionality. signed to each pixel in the image such that pixels with the
In [65], a self-improving CNN (SICNN)-based approach same label share certain visual characteristics [105]. These
was proposed for classifying hyperspectral data. This objects provide more information than individual pixels,

TABLE 4. SUPERVISED SPECTRAL–SPATIAL CLASSIFIERS: CLASSIFICATION ACCURACIES OBTAINED

FROM the PAVIA ­UNIVERSITY HYPERSPECTRAL DATA.

CLASS RF SVM 2-D CNN SICNN FODPSO MSF EMP EMAP APDAFE DBAPDA RF-EMEP RoF-EMEP GCK MFL
OA 71.3 78.8 78.8 83.4 88.1 91.1 77.7 90.7 97.0 98.0 96.1 96.3 98.0 97.5
AA 82.2 87.0 79.7 83.0 92.0 94.8 82.5 91.4 96.7 98.1 96.6 97.9 97.4 97.1
Kappa 0.648 0.735 0.734 0.778 0.848 0.880 0.710 0.877 0.960 0.974 0.949 0.952 0.974 0.967

MSF: minimum spanning forest; APDAFE: attribute profiles with discriminant analysis feature extraction; DBAPDA: decision boundary feature extraction and attribute profiles and
discriminant analysis feature extraction.

december 2017 ieee Geoscience and remote sensing magazine 49

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 because the interpretation of images based on objects is ◗◗ SEs are unable to characterize information about the
more meaningful than interpretations based on individ- gray-level characteristics of the regions, such as spectral
ual pixels. homogeneity, contrast, and so on.
For spectral-spatial classification of HSIs using segmen- To address these MP shortcomings, [112] introduced a
tation approaches, there are usually two methods to consid- morphological attribute profile (AP) that provides a multi-
er: 1) segmentation and classification maps can be integrat- level characterization of an image by using the sequential
ed using majority voting within each object by assigning application of morphological attribute filters. A comprehen-
the whole object to the most frequent classification label sive survey on the use of APs for HSI classification can be
within that particular object [106] (such majority voting is found in [113] and [4]. The classification accuracy obtained
described in [107]) and 2) segments can be considered to by the extension of APs on HSIs, known as extended multi-AP
be input vectors for supervised classification [108]. In [109], (EMAP), is given in Table 4. There are two main difficulties of
however, a reverse view was employed, where markers for using the AP, however: not knowing 1) which attributes lead
spatial regions were automat- to a better discrimination ability for different classes and 2)
ically obtained from classifi- which threshold values should be considered to initialize
EPs SOLVE THE MAIN ISSUE cation results and then used each AP. To solve these issues, several articles, such as [114]–
as seeds for region-growing [116], have tried to introduce automatic techniques for the
OF CONVENTIONAL APs,
in the segmentation step. The use of APs. In [116] and [115], automatic spectral-spatial clas-
THE INITIALIZATION OF THE
classification accuracy of this sification methods were proposed based on the use of EMAP
THRESHOLD VALUES. segmentation method with and supervised/unsupervised feature extraction approaches.
an extra step (where the clas- The classification accuracy of the APs with discriminant
sification map is refined us- analysis feature extraction [116] and decision boundary fea-
ing the results of a pixelwise classification and a majority ture extraction and APs and discriminant analysis feature
voting within the spatially connected regions) is shown in extraction [115] are shown in Table 4.
Table 4 as the minimum spanning forest. MPs and APs produce extremely redundant features. To
One common way to segment an image is based on his- address this issue, a sparse classification using both spec-
togram thresholding. A commonly used exhaustive search tral and spatial information was investigated in [117]. In
for optimal thresholds in terms of between-class distances [118], the performance of different feature extraction ap-
is based on the Otsu criterion [110]. The approach is easy proaches, including linear, nonlinear, and manifold
to implement, but it has the disadvantage of being compu- approaches, was investigated to generate base images for
tationally expensive. An exhaustive search for an n-level constructing EMAPs.
segmentation (i.e., n - 1 optimal thresholds) involves To further improve the conceptual capability of the AP
evaluations of the fitness of n (L - n + 1) n -1 combina- and the corresponding classification accuracies, Ghamisi et
tions of thresholds, where L shows the number of intensity al. proposed extinction profiles (EPs) in 2016 [119] by con-
values. Therefore, this method is not suitable from a com- sidering a set of connected idempotent filters and extinc-
putational cost perspective. The task of determining n - 1 tion filters. In contrast to the AP, the EP preserves the height
optimal thresholds for n-level image thresholding could of the extrema [119] and, as a result, shows better capabil-
be formulated as a multidimensional optimization prob- ity than the AP in terms of simplification for recognition.
lem. In [106], a thresholding-based segmentation method This advantage leads to higher classification accuracy for
was proposed, where an evolutionary-based optimization EPs than for APs. In addition, the EPs’ parameters can be
technique, called fractional order Darwinian PSO (FODPSO), set automatically, independent of the kind of the attribute
sought to find the best set of thresholds with the highest be- being used (e.g., area, volume, and so on). In other words,
tween-class distance. The classification accuracy obtained EPs solve the main issue of conventional APs, the initializa-
by this segmentation method is given in Table 4. This meth- tion of the threshold values [119]. In [120], the concept of
od is very fast, even for large data sets, because it works on EPs was generalized to extract spatial and contextual infor-
the image histogram instead of the image space. mation from HSIs, known as extended multi-EP (EMEP). The
Morphological profiles (MPs) are another set of ap- classification accuracy of EMEP using RF (RF-EMEP) and
proaches based on adaptive neighborhood pixels. MPs RoF (RoF-EMEP) is presented in Table 4.
comprise a number of features constructed by applying a
set of openings and closings by reconstruction with a struc- COMPOSITE KERNELS
turing element (SE) of increasing size [111]. The result of The main problem associated with the concept of spectral-
the basic extension of the MP, extended MP (EMP), that is spatial feature extraction approaches is that they usually
applicable to HSIs is shown in Table 4. Although MPs are a increase the number of features, while the number of train-
powerful approach for extracting spatial information, the ing samples remains the same. This can lead to the curse of
concept suffers from some limitations: dimensionality and high executable processing time. This
◗◗ The SE shape is fixed, which imposes a constraint on problem has partially been addressed by combining different
model spatial structures within a scene. kernels for spectral and spatial information (i.e., composite

50 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 kernels) [121] in the SVM classification process. However, SPARSE REPRESENTATION CLASSIFICATION
classification using composite kernels and SVMs demands SR classification (SRC)-based approaches with dictionary-
convex combination of kernels and a time-consuming op- based generative models [129], [130] have received consid-
timization process. Therefore, the approach has been modi- erable attention in the hyperspectral community. In this
fied to deal with convex combinations of kernels through context, an input signal is represented by a sparse linear com-
generalized composite kernels (GCK) [122], the results of bination of samples (atoms) from a dictionary [129], where
which are shown in Table 4, and multiple-kernel learning the training data are generally used as the dictionary. The
[123]. In [124], a classification framework was introduced main advantages of such approaches are that SRC avoids the
that combines multiple features with the linear and nonlin- heavy training procedure usually conducted by a supervised
ear class boundaries present in the data without requiring classifier and that the classi-
any regularization parameters to control the weights of the fication is performed directly
considered features (the results are shown as MFL in Table 4). on the dictionary. Classifica-
tion can be improved by in- THE RICH SPECTRAL
SEMISUPERVISED AND ACTIVE LEARNING corporating contextual infor- RESOLUTION AVAILABLE
As previously discussed, the number of training samples mation from the neighboring IN HYPERSPECTRAL DATA
is usually limited because the collection of such samples pixels into the classifier. This CUBES CAN BE USED TO
is either expensive or time consuming. In such situations, can be performed indirectly UNMIX HYPERSPECTRAL
the limited number of training samples available may not by exploiting the spatial cor- PIXELS.
be representative of the statistical distribution of the data, relation through a structured
which can downgrade the quality of the classification map sparsity imposed earlier in the
obtained by supervised classifiers. To partially address this optimization process. If an ad-
issue, active learning, which aims to find the most infor- equate number of training samples is available, discrimina-
mative training set, has gained popularity in the hyper- tive as well as compact class dictionaries can also be devel-
spectral community. oped to improve classification performance [131].
Active learning starts an iterative process with a small and
suboptimal initial training set and then selects a few addi- CHALLENGES IN CLASSIFICATION
tional samples from a large quantity of unlabeled samples. The main challenges for HSI classification are not particu-
Active learning considers the result of the current model, larly related to methodology. They are, rather, related to
ranking the unlabeled samples according to a criterion that the lack of appropriate benchmark data sets and the cor-
allows selection of the most informative samples to improve responding training and test samples. As can be seen in
the model, thus minimizing the number of training samples Figure 3, most published contributions in the hyperspec-
while preserving discrimination capabilities as much as pos- tral community are dedicated to HSI classification. The ap-
sible [125]. For a complete survey on the use of active learn- proaches are often capable of producing very accurate clas-
ing for remote-sensing image analysis, see [126] and [127]. sification maps on the widely used Indian Pines and Pavia
Active learning and semisupervised learning share a data sets, which makes real comparison of the approaches
similar conceptual background as both types of learning almost impossible. In other words, the existing data sets
try to address the issue of limited labeled samples. In this have already been saturated in terms of classification ac-
manner, both approaches start with a small set of labeled curacies. Therefore, our community is in urgent need of
samples and a large set of unlabeled data. Active learning more complex data sets to share (e.g., highly nonlinear
usually requires a labor-intensive labeling process, while data sets with greater area coverage that are composed of
semisupervised learning, although avoiding manual label- many classes). In addition, a standard set of training and
ing by assigning pseudolabels to unlabeled data, may intro- test samples should be defined for each particular data set,
duce incorrect pseudolabels and consequently downgrade to make the proposed approaches fully comparable with
classification performance [128]. Although active learning each other.
and semisupervised learning follow different work flows,
they both aim to make the most of unlabeled data while SPECTRAL UNMIXING
reducing manual labeling efforts [125]. Therefore, it is com- Spectral unmixing has been an alluring exploitation goal
mon to use both of these strategies to make the most of since the early days of HSI processing [132]. Mixed pixels
these two paradigms for HSI classification. In [128], active are common in remotely sensed HSIs because of the imag-
learning and semisupervised learning were collaboratively ing spectrometer’s insufficient spatial resolution or due to
integrated to form an approach called collaborative active intimate mixing effects. However, the rich spectral resolu-
and semisupervised learning that improves pseudolabeling tion available in hyperspectral data cubes can be used to
accuracy and thus facilitates semisupervised learning. This unmix hyperspectral pixels. In fact, mixed pixels can also
method was based on spectral information. In [125], active be obtained with high-spatial-resolution data because of
learning and hierarchical segmentation were combined for intimate mixtures. This means that increasing the spatial
spectral-spatial classification of HSIs. resolution often does not solve the problem.

december 2017 ieee Geoscience and remote sensing magazine 51

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 In other words, the mixture problem can be approached variance is not unitary in different directions, and the con-
in a macroscopic fashion, which means that only a few tribution from signals may be smaller than from noise.
macroscopic components and their associated abundances HySime addresses this issue by using subspace projection
should be derived. However, intimate mixtures happen techniques, thus contributing an additional feature with
at microscopic scales, thus complicating the analysis with regard to VD: the modeling of noise before the estimation.
nonlinear mixing effects [133]. In addition to spectral mix- The eigenvalue likelihood maximization (ELM) method
ing effects, there are many other interfering factors that can [137], in turn, implements a modification of the VD con-
significantly affect the analysis of remotely sensed hyper- cept based on the following observations: 1) the eigenval-
spectral data. For instance, ues corresponding to the noise are identical in the covari-
atmospheric interferers are a ance and the correlation matrices, and 2) the eigenvalues
potential source of errors in corresponding to the signal (the endmembers) are larger in
THE PIXEL PURITY INDEX spectral unmixing. Multiple the correlation matrix than in the covariance matrix. The
IS PERHAPS THE MOST scattering effects can also lead ELM takes advantage of this fact and provides a fully au-
POPULAR ENDMEMBER to model inaccuracies. tomatic method that does not need an input parameter (as
EXTRACTION ALGORITHM In linear spectral unmix- does VD) or estimation of the noise (as does HySime).
BECAUSE OF ITS ing, the macroscopically pure Finally, the normal compositional model (NCM) [138]
AVAILABILITY IN components are assumed to addresses the possibility that, in real images, there may not
be homogeneously distribut- be any pure pixels. To address this issue, NCM assumes
SOFTWARE PACKAGES.
ed in separate patches within that the HSI pixels are linear combinations of an unknown
the field of view. In nonlin- number of random endmembers (the opposite of the de-
ear spectral unmixing, the terministic approach). This model provides more flexibility
microscopically pure components are intimately mixed. A with respect to the observed pixels and the endmembers,
challenge is how to derive the nonlinear function, because which are allowed to be a greater distance from the ob-
nonlinear spectral unmixing requires detailed a priori served pixels.
knowledge about the materials. Responding to this limita-
tion, a vast majority of techniques have ­focused on linear ENDMEMBER EXTRACTION
spectral unmixing, where the goal is to find a set of mac- The identification of endmembers is a challenging prob-
roscopically pure spectral components (called endmembers) lem, for which many different strategies have been pro-
that can be used to unmix all the other pixels in the data. posed [134]. To categorize algorithms, we consider three
Unmixing thus amounts to finding the fractional coverage different scenarios.
(abundance) of each endmember in each pixel of the scene, 1) The data contain at least one pure pixel per endmember,
which can be approached as a geometrical problem [134]. In i.e., there is at least one spectral vector in each vertex of
the following section, we focus on the most relevant parts of the data simplex (pure pixel assumption).
the linear spectral unmixing chain. We also summarize the 2) The data do not contain pure pixels but contain enough
main efforts in nonlinear spectral unmixing. spectral vectors on each facet. In this case, we may fit a
minimum volume simplex to the data.
ESTIMATION OF THE NUMBER OF ENDMEMBERS 3) The data are highly mixed, with no spectral vectors near
Determining the number of pure spectral endmembers in the facets. In this case, minimum volume algorithms
HSIs is a challenging problem. One of the most commonly fail, and we need to resort to a statistical framework. We
used approaches to this problem is the virtual dimension- also consider algorithms that include spatial informa-
ality (VD) method [135], which follows the pigeon-hole tion in addition to spectral information for this purpose.
principle. If we represent a signal source as a pigeon and
a spectral band as a hole, we can use a spectral band to ac- PURE PIXEL ASSUMPTION
commodate one source. Thus, if a signal source is present in Pure pixel methods assume a classic spectral unmixing
our remotely sensed hyperspectral data set, we should be chain with three stages: DR, endmember selection, and
able to detect this particular source in the relevant spectral abundance estimation. Here, the endmembers are directly
band. This can be accomplished by calculating the eigenval- derived from the original hyperspectral scene. The pixel
ues of both the data-correlation and covariance matrices. A purity index (PPI) [139] is perhaps the most popular end-
source is present if their difference is positive. member extraction algorithm because of its availability
Another popular approach is hyperspectral signal iden- in software packages. PPI has many parameters involved
tification with minimum error (HySime) [136]. The idea of and is not an iterative algorithm. Manual intervention is
HySime is to find the first k eigenvectors that contain the required to select a final set of endmembers, which makes
most data information, i.e., to find k such that the mean- it unattractive for automatization purposes.
square error (MSE) between the original data and their An alternative is the N-FINDR [140], which assumes the
projection onto the eigenvector subspace is minimized. presence of pure pixels in the original hyperspectral scene
Subspace k is ranked in terms of data variance, but noise and further maximizes the volume that can be formed

52 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 with pixel vectors in the data cube. Orthogonal subspace [149] also exploits this concept, allowing a certain number
projection (OSP) [141], in turn, uses the concept of orthog- of outliers when estimating the minimum volume that en-
onal projections. Vertex component analysis (VCA) [142] compasses the HSI.
iteratively projects data on a direction orthogonal to the
subspace spanned by the endmembers, which have already HIGHLY MIXED DATA
been determined. In this regard, the algorithm is similar When the spectral mixtures are highly mixed, geometrical-
to OSP; the main difference is that VCA applies a noise based methods yield poor results because there are not enough
characterization process to reduce the sensitivity to noise. spectral vectors in the simplex facets. Statistical methods are a
This is accomplished by using singular value decomposi- powerful alternative that usu-
tion (SVD) to obtain the projection that best represents the ally comes with a price: higher
data in the maximum-power sense. computational complexity than ONCE THE ENDMEMBER
Another important concept in this category is the end- with geometrical methods. Be- SIGNATURES HAVE BEEN
member bundle, explored by algorithms such as multiple cause, in most cases, the num-
DERIVED, DIFFERENT
endmember spectral mixture analysis (MESMA) [143]. ber of substances and their
STRATEGIES CAN BE USED
Although the shape of an endmember is fairly consistent, reflectances are not known, the
its amplitude generally varies because of illumination con- problem can be approached TO ESTIMATE THEIR
ditions, spectral variability, topographic modulation, and as a blind-source separation FRACTIONAL ABUNDANCES.
other circumstances. MESMA addresses this issue using problem, with some statistical
endmember bundles, which incorporate variability by rep- unmixing approaches propos-
resenting each endmember by a set or bundle of spectra, ing variations on the ICA [150]. However, ICA applicability is
each of which could reasonably be the reflectance of an in- compromised by the statistical dependence existing among
stance of the endmember. abundances. This has been addressed (among other strate-
gies presented in the recent literature) by the dependent com-
MINIMUM VOLUME ALGORITHMS ponent analysis algorithm [151]. Bayesian approaches have
If the data do not contain any pure signatures, we can fit also been used because they can model statistical variability
a simplex of minimum volume in cases where we have and impose priors to constrain solutions to physically mean-
enough spectral vectors on the facets. This idea is the op- ingful ranges.
posite of the concept of maximum volume adopted by
N-FINDR; here, the goal is to find the simplex with the INCLUSION OF SPATIAL INFORMATION
minimum volume that encloses the data. From an optimi- Most available algorithms for endmember identifica-
zation point of view, these algorithms are formulated by tion do not consider information about spatial-contex-
including a data term that minimizes the reconstruction tual information. In certain scenarios, it is important
error and a volume term that promotes mixing matrices to include the spatial information in the analysis. Auto-
of minimum volume. This is the case for the iterative con- matic morphological endmember extraction [152] uses
strained endmember (ICE) [144] and minimum volume extend­e d morphological transformations to integrate
constrained nonnegative matrix factorization approaches spatial and spectral information. However, spatial-spectral
[145], the main differences of which are related to the way end­­member extraction [153] uses a different approach.
they define the data volume term. First, it processes the image using a local search window
The sparsity-promoting ICE approach [146] is an exten- and applies SVD to determine a set of eigenvectors that
sion of the ICE algorithm, which incorporates sparsity-pro- describe most of the spectral variance in the window.
moting priors aimed at finding the number of endmembers. Then, it performs a projection of all of the image data
The minimum volume estimation (MVES) algorithm [147] onto eigenvectors to determine candidate endmember
integrates concepts of convex analysis and volume mini- pixels. Finally, it uses spatial constraints to combine and
mization to provide a solution similar to that of previously average spectrally similar candidate endmember pixels
mentioned algorithms but using cyclic minimization with (preserving similar but distinct endmembers that occupy
linear programming. Again, the assumption is that the en- unique image regions).
closing simplex with minimum volume should coincide To avoid modifying spectral-based algorithms for
with the true endmember simplex (MVES uses hard positiv- endmember extraction, spatial information can also be
ity constraints). included as a preprocessing module, such as the spatial
The minimum volume simplex analysis algorithm [148] preprocessing algorithm [154]. A region-based approach
follows a similar strategy but allows violations of the [155] has also been developed to adaptively include spa-
positivity constraint. This is because, due to the presence tial information. Finally, a spatial-spectral preprocessing
of noise or perturbations, spectral vectors may lie outside approach [156] has been developed to derive a spatial ho-
the true simplex, and this may introduce errors in the char- mogeneity index that uses Gaussian filtering and is thus
acterization. The minimum-volume enclosing algorithm relatively insensitive to the noise present in the hyper-
for endmember identification and abundance estimation spectral data.

december 2017 ieee Geoscience and remote sensing magazine 53

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 ABUNDANCE ESTIMATION of ground materials. An advantage is that it sidesteps the
Once the endmember signatures have been derived, differ- endmember extraction step (including the estimation of the
ent strategies can be used to estimate their fractional abun- number of endmembers).
dances [157]. The idea is to find the abundances that mini- To incorporate spatial information into the spectral un-
mize the reconstruction error obtained after approximating mixing formulation, a TV regularizer has been developed
the original hyperspectral scene using a linear mixture mod- to enforce spatial homogeneity by including this term in
el. However, this is generally an unconstrained solution that the original objective function [162]. It produces spatially
does not satisfy the abundance nonnegativity constraints smooth abundance fractions that improve sparse unmixing
(ANCs) and the abundance sum-to-one constraints (ASCs). performance, even in very high-noise conditions. Further-
Whether or not abundance constraints should be imposed more, because it is generally observed that spectral libraries
depends on the practical application. It has been argued that, are organized in the form of groups with different variations
if the linear mixture model is of the same component (e.g., different mineral alterations),
accurate, the two constraints exploiting the inherent group structure present in spectral
should be satisfied automati- libraries can improve the results of sparse unmixing by se-
SPECTRAL UNMIXING cally. In any event, the ANC is lectively enforcing groups. For this purpose, a group-based
ALGORITHMS WITH THE more important than the ASC. formulation of sparse unmixing has been introduced [163].
PURE PIXEL ASSUMPTION Due to noise and spectral vari- A further development has also been recently introduced
REQUIRE THE PRESENCE ability, reinforcing the ASC based on the concept of collaborativity, which promotes
OF PURE PIXELS IN THE may be prone to introducing solutions with a minimum number of active endmembers
SCENE FOR ENDMEMBER additional estimation error. (the number of endmembers in a scene is generally low).
When endmembers are This allows the number of endmembers participating in the
EXTRACTION.
unknown, endmember sig- final solution to be minimized [164], while also partially
natures should be extracted circumventing the need to estimate the number of end-
or estimated first. Some end- members in the scene [165].
member extraction algorithms can provide abundance es-
timates simultaneously (e.g., algorithms without the pure NONLINEAR UNMIXING
pixel assumption) [134]. Another group of abundance es- Apart from the linear spectral unmixing algorithms, sev-
timation approaches based on blind-source separation, eral alternatives dealing with the mixture problem can be
which does not require endmember signatures to be known found in the literature [166]–[168]. The bilinear mixing
a priori, has been developed [134]. Widely used matrix-fac- model (BMM) considers secondary illumination sources.
torization-based blind-source separation methods include This model represents a simplification of reality, as it only
ICA and nonnegative matrix factorization (NMF), which considers bilinear interactions (objects that can be illumi-
have been mostly used in the context of unsupervised (soft) nated by light reflected first by another object) [133]. BMMs
classification. If the linear assumption does not hold, then can be generalized to deal with multiple endmembers, by
nonlinear unmixing techniques should be used. In addi- trying to model bilinear interactions as new endmembers.
tion, as mentioned in the previous section, if spectral vari- The generalized bilinear model (GBM) provides more flex-
ability or endmember variability is being considered, the ible solutions that can avoid some overfitting problems as-
mixture model must be modified accordingly, which is tra- sociated with bilinear models, but it still assumes that there
ditionally accomplished by generating modified/extended are no self-bilinear interactions [133].
linear mixture models [158]–[160]. The BMM and GBM have inspired several posterior
methodologies: some use polynomial functions to model
SPARSE UNMIXING the nonlinearities provided by layer partitions and scatter-
Spectral unmixing algorithms with the pure pixel assump- ing properties in multilayered scenarios [169], while others
tion require the presence of pure pixels in the scene for end- use different approaches to solve the proposed model [170]–
member extraction. Due to spatial resolution and mixing [173]. In [174], the authors proposed a modification of the
phenomena, this assumption cannot always be guaranteed. BMM in which the transformation of the spectral informa-
Spectral unmixing algorithms without the pure pixel as- tion is based on a two-degree polynomial, thus defining the
sumption generate endmember signatures that often do not polynomial postnonlinear model (PPNM), a main advan-
relate to real physical signatures. A possible solution is to tage of which is that it is able to deal with self-interactions;
use ground spectral libraries to perform spectral unmixing, the PPNM has been recently extended to deal with all poly-
but libraries are very large, and hence the problem becomes nomial degrees [175]. In [169], a p-linear polynomial model
sparse and difficult to solve. Another problem is the dif- is used to characterize nonlinearities in combination with a
ference between the ground library and the image data. To supervised artificial neural network (ANN) and polytope de-
address these issues, sparse unmixing [161] expresses pixel composition scheme. When the interactions are considered
vectors as linear combinations of a few pure spectral signa- to exist at the photon level, nonlinear unmixing methods try
tures obtained from a potentially very large spectral library to model the optical characteristics of the intimate mixture

54 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 from a theoretical analysis of the reflectance behavior at- RESOLUTION ENHANCEMENT
tending the specific geomorphical, chemical, and physical HSI resolution enhancement has attracted increasing at-
properties of the observed data. Several models have been tention in recent years, as shown in the statistical analysis
proposed in this context, but the Hapke model [176] is still of the trend described in the introduction. Resolution en-
the most widely adopted approach. hancement techniques for HSIs can be broadly categorized
In addition, supervised techniques based on the use of into four classes, as shown in Figure 9:
methods such as kernel-based ones and ANNs have been 1) hyperspectral superresolution (multi-image/single-
proposed to perform unmixing on a microscopic scale [177]– image)
[186]. The main advantage of these techniques is that they 2) subpixel mapping (or superresolution mapping)
can create a preliminary model of every nonlinear behavior. 3) hyperspectral pan-sharpening
However, the need for reliable ground truth information 4) hyperspectral–multispectral (HS–MS) data fusion.
about the training samples represents a major shortcoming. The first class, hyperspectral superresolution, is an
extension to HSIs of ordinary superresolution in com-
CHALLENGES IN SPECTRAL UNMIXING puter vision. A high-resolution HSI is reconstructed from
Despite the availability of several consolidated techniques multiple low-resolution HSIs (or a single HSI) acquired
for linear spectral unmixing, as well as a suite of incipient by a single sensor. The second, subpixel mapping, is a
techniques for nonlinear spectral unmixing, an important resolution enhancement technique processed at a classi-
challenge remains related to the nonlinearity of mixing fication level using a single HSI as input. The remaining
phenomena in HSIs. The inherently nonlinear nature of two classes, hyperspectral pan sharpening and HS–MS
the process and the dependence on the observed objects fusion, are multisensor superresolution techniques, in
create the need to incorporate detailed information about which an HSI is fused with an auxiliary higher-resolu-
the observed objects to properly model the multiple scat- tion data source (panchromatic or multispectral images)
tering phenomena occurring in the nonlinear case. The es- taken over the same area at the same time (or a similar
timation of participating endmembers in the mixture also period) to create a high-resolution HSI. The following
remains a challenge, despite the availability of some tech- sections provide an overview of recent advances in these
niques that can provide reasonable approximations. four classes of techniques.
Another important hurdle for the automatic execution
of sparse unmixing algorithms is the heterogeneity in avail- HYPERSPECTRAL SUPERRESOLUTION
able spectral libraries, although important efforts have been A variety of superresolution image reconstruction tech-
made toward the development of open-source libraries of niques have been intensively pursued over the past three de-
spectral materials that can alleviate the need to estimate the cades in computer vision. Superresolution techniques can be
number and the signatures of spectral endmembers in ad- roughly divided into two types [187]: 1) classical multi-image
vance. In this regard, the availability of open libraries such superresolution, which obtains a high-resolution image (or
as SPECCHIO (http://www.specchio.ch) provides an impor- sequence) from multiple low-resolution images for the same
tant first step toward the general use of open spectral librar- scene with different subpixel shifts, and 2) learning-based
ies for spectral unmixing purposes. superresolution, which learns ­correspondences between

Low-Resolution HSIs with Subpixel Shifts High-Resolution HSI Low-Resolution HSI

High-Resolut
High-Resolution
Classification Map
...

(a ) (b)
Low-Resolution Multispectral High-Resolution
Low-Resolution HSI High-Resolution HSI HSI Image HSI
Panchromatic
Image

+
+

(c) (d)

FIGURE 9. The four classes of resolution enhancement techniques for HSIs: (a) multi-image hyperspectral superresolution, (b) subpixel
mapping, (c) hyperspectral pan-sharpening, and (d) HS–MS fusion.

december 2017 ieee Geoscience and remote sensing magazine 55

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 low- and high-resolution image patches from an external SUBPIXEL MAPPING
training database. Subpixel mapping is a technique for enhancing the spatial
Multi-image superresolution techniques have been ex- resolution of spectral images by dividing a mixed pixel into
tended to HSIs. Depending on the type of subpixel shifts in subpixels and assigning land cover classes to these subpix-
the low-resolution image, these techniques can be divided els [197]. Subpixel mapping techniques have been actively
into two approaches: the first uses multiple HSIs acquired studied using multispectral images and extended to HSIs
by the same sensor over the same scene, similar to classi- by exploiting rich spectral information. Subpixel mapping
cal multi-image superresolution; the second takes advan- is made up of two steps: 1) estimating fractional abun-
tage of band-to-band misregistration (so-called keystone) dances of classes (or endmembers) at a coarse resolution by
in a single HSI. Akgum et al. first applied the multi-image soft classification or spectral unmixing and 2) determining
superresolution technique to the subpixel location of each class within a pixel, assum-
multiple HSIs based solely on ing spatial dependence. Many recent advances in subpixel
simulations [188]. mapping of HSIs aim at improving the accuracy of inverse
SUBPIXEL MAPPING
In the remote-sensing com- problems involving these two steps.
TECHNIQUES HAVE BEEN munity, multi-image hyper- Tong et al. proposed a method that exploits not only
ACTIVELY STUDIED USING spectral superresolution has attraction but also repulsion between subpixels to better
MULTISPECTRAL IMAGES been mainly studied using retrieve spatial dependence [198]. Zhang et al. integrated
AND EXTENDED TO HSIs multi-angular HSIs obtained by learning-based superresolution into subpixel mapping,
BY EXPLOITING RICH the Compact High-Resolution requiring an external training set [199]. The estimation of
SPECTRAL INFORMATION. Imaging Spectrometer (CHRIS). unknown spatial details of classes at a high resolution from
In [189], conventional super- a single low-resolution image is a typical ill-posed inverse
resolution techniques were ap- problem. Different techniques for spatial regularization
plied to a set of multi-angular that have been recently studied to mitigate the ill-posed-
CHRIS images. Here, Chan et al. developed a new superreso- ness have assumed spatial prior models, such as Laplacian,
lution technique for multi-angular HSIs based on a thin-plate total variation, bilateral total variation, and nonlocal total
spline nonrigid transform model. This approach was intended variation [200], [201].
to improve the image registration procedure and demonstrat- Obviously, the accuracy of the abundance maps ob-
ed its effectiveness with experiments using the CHRIS images tained in the first step greatly affects the final performance
[190]. The impact of multi-angular superresolution on classi- of subpixel mapping. To address this issue, Tong et al. pro-
fication and unmixing ­applications was further investigated in posed a GA that can correct possible errors in the initial
[191] and [192]. estimation of abundances using a mutation operator [202].
Qian and Chen developed a technique for the second Xu et al. introduced a method that improves the accuracy of
approach that enhances the spatial resolution of HSIs by ex- spectral unmixing in subpixel mapping by taking endmem-
ploiting the keystone characteristics [193]. Keystone is band- ber variability into consideration, based on the assumption
to-band misregistration in the cross-track direction caused by that different representative spectra for each endmember
optical aberrations and misalignments in pushbroom systems. are available [201].
Different band images, including different subpixel miregistra-
tions, can be used as input images in the multi-image super- HYPERSPECTRAL PAN-SHARPENING
resolution framework. The advantage of this method is that it Pan sharpening is a technique that enhances the spatial
requires only a single HSI, but the limitation is that the spatial resolution of multispectral imagery by fusing this imagery
resolution can be enhanced only in the cross-track direction. with a higher-resolution panchromatic image. Hyperspec-
Zhao et al. proposed SR-based algorithms for learn- tral pan-sharpening is an extension to HSIs of conventional
ing-based hyperspectral superresolution [194], [195]. The pan-sharpening and is also a special case of HS–MS fusion.
high-resolution version of a given low-resolution patch is Naturally, there are two main approaches: 1) extensions of
recovered by solving the sparse linear inverse problem with pan-sharpening methods and 2) subspace-based methods
spectral regularization based on spectral unmixing, in (originally developed for HS–MS fusion).
which the patch dictionary is learned from a set of high-res- Hyperspectral pan-sharpening is motivated by space-
olution panchromatic images or HSIs. Patel et al. developed borne imaging spectroscopy missions that mount both
a learning-based superresolution method for HSIs using an hyperspectral and panchromatic imaging sensors, such
adaptive wavelet designed from training HSIs [196]. These as EO-1/Hyperion-advanced land imager (ALI) [203],
learning-based superresolution techniques do not require PRISMA [8], and Shalom [9]. The main advantage of HSIs
multiple images over the same scene but instead require an on multispectral images is the rich spectral information,
external training database with target resolution. For the which enables the discrimination and identification of
upcoming hyperspectral satellites, it is realistic to use im- spectrally similar materials. In other words, any spectral
ages obtained by operational multispectral satellites (e.g., distortions will lead to inaccurate analysis results in the
Sentinel-2) for the training database. subsequent data processing. Therefore, the challenge of

56 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 hyperspectral pan-sharpening is to enhance the spatial algorithms in the last decade. Quite recently, a compara-
resolution of hyperspectral data while minimizing spec- tive review of HS–MS fusion was reported in [215]. Most
tral distortion. of the HS–MS fusion algorithms can be categorized into
Typical pan-sharpening techniques include the compo- at least one of the following six classes of approaches:
nent substitution (CS) [204]–[206], multiresolution analy- 1) CS, 2) MRA, 3) SR, 4) unmixing, 5) Bayesian, and 6)
sis (MRA) [207], [208], and SR [209], [210] algorithms. hybrid. The CS, MRA, and SR methods are extensions of
Aiazzi et al. [206] proposed a general CS-based pan-sharp- pan-sharpening techniques, whereas the unmixing and
ening scheme in which, a multispectral image is sharpened Bayesian approaches fall into the same broader category
through the addition of spatial details obtained when mul- of subspace-based techniques.
tiplying the difference between a panchromatic image and CS-based methods can be adapted to HS–MS fusion by
a synthetic intensity component by a band-wise modula- splitting the fusion problem into several pan-sharpening
tion coefficient. Gram–Schmidt adaptive (GSA) is one of subproblems and applying CS-based pan-sharpening to
the benchmark CS-based pan-sharpening algorithms; here, these subproblems. A key procedure for the CS-based ap-
the synthetic intensity component is computed via linear proach is to divide the HSI into several groups and assign
regression between a panchromatic image and lower-reso- a high-resolution image selected or synthesized from the
lution bands. multispectral image to each group. Chen et al. proposed a
In the MRA-based pan-sharpening scheme, spatial framework that solves the HS–MS fusion problem by divid-
details of each multispectral band are obtained by use of ing the spectrum of hyperspectral data into several regions
MRA, which calculates the difference between a panchro- and fusing hyperspectral and multispectral band images in
matic image and its low-pass version multiplied by a gain each region by conventional pan-sharpening techniques
factor. The gain factor can be computed either locally or [216]. The hypersharpening proposed by Selva et al. is a
globally. Representative MRA pan-sharpening algorithms framework that effectively adapts MRA-based pan-sharp-
include the smoothing filtered-based intensity modula- ening methods to HS–MS fusion by synthesizing a high-
tion (SFIM) method [207], the additive wavelet luminance resolution image for each hyperspectral band as a linear
proportional method [211], and the generalized Laplacian combination of multispectral band images via linear re-
pyramid (GLP) method [208]. gression [217]. An SR-based pan-sharpening technique was
The SR-based pan-sharpening approach can be regard- adapted to HS–MS fusion with spectral grouping and joint
ed as a special case of learning-based superresolution, i.e., sparse recovery in [218].
learning correspondences between low- and high-resolu- Subspace-based methods have been actively developed
tion image patches from a panchromatic image [210] or for HS–MS fusion problems. The HS–MS fusion task is for-
an external database, including multiple high-resolution mulated as the inverse problem of estimating the subspace
multispectral images [212]. In [213], Alparone et al. dem- basis and coefficients of the high-resolution HSI from the
onstrated the fusion of Hyperion and ALI panchromatic two input images [219]. The resolution-enhanced HSI can
images using CS- and MRA-based algorithms [213]. be reconstructed as the product of the basis matrix and the
The subspace-based approach exploits the intrinsic spec- high-resolution coefficient matrix.
tral characteristics of the scene via a subspace spanned by In recent years, a perspective of spectral unmixing has
a set of basis vectors, enhancing the spatial resolution of been attracting considerable attention in the context of
subspace coefficients. Because most subspace-based meth- subspace-based HS–MS fusion, owing to its straightforward
ods were developed for HS–MS fusion, we discuss them in interpretation of the fusion process. The basis matrix is de-
the next section. fined as a set of spectral signatures of intrinsic materials
Eleven state-of-the-art algorithms from the CS-, MRA-, (endmembers), and the subspace coefficients correspond to
and subspace-based approaches were applied to hyper- the fractional abundances. Several unmixing-based meth-
spectral pan-sharpening and compared in [214]. MRA- and ods have been proposed for HS–MS fusion with various op-
subspace-based algorithms demonstrated relatively high- timization formulations [220]–[227]. In [215], it was noted
quality and stable results. However, it was evident that large that the high-resolution abundances estimation step has a
spectral distortion was inevitable for all the algorithms un- significant influence on fusion performance. Outstanding
der comparison, implying room for further technology and stable performance could be achieved by minimizing
development [214]. the unmixing reconstruction errors with respect to both the
hyperspectral and multispectral images rather than only the
HYPERSPECTRAL AND MULTISPECTRAL DATA FUSION multispectral image, particularly when the overlap of spec-
HS–MS fusion is a technique that fuses an HSI with a tral response functions between hyperspectral and multi-
higher-resolution multispectral image to create high- spectral sensors was small. Such algorithms include coupled
resolution hyperspectral data. Unlike hyperspectral pan- NMF (CNMF) [224], coupled spectral unmixing with a pro-
sharpening, this technique yields higher spectral quality jected gradient method [228], and HySURE [229].
owing to spectral information from the high-resolution One important aspect of developing subspace-based
data source. Enormous efforts have made to develop methods has been determining how to mitigate the

december 2017 ieee Geoscience and remote sensing magazine 57

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 ill-posedness of inverse problems involving the estimation The reconstruction performances of algorithms has
of high-resolution subspace coefficients. Regularization on posed the greatest concern for researchers; however, their
subspace coefficients has been extensively explored. Sparsi- general versatility, computational costs, and impacts on
ty-promoting regularization has been commonly adopted applications are also important for users. Because each
for subspace-based methods [222], [225], [226], assum- method has advantages and disadvantages, it is essential
ing that there is a limited number of endmembers at each to choose a method based on the fusion and analysis sce-
high-resolution pixel. HySURE solves a convex optimization narios. The impact of HS–MS fusion on applications has
problem with vector-total-variation-based regularization been recently investigated via classification and unmixing
of the spatial distribution of subspace coefficients, leading [233], [215]. Further research on real data is still neces-
to implicit denoising effects in the fusion results [229]. Wei sary to verify the practicability of HS–MS fusion for future
et al. developed a Bayesian HS–MS fusion methodology in ­hyperspectral satellites.
which different types of regularization terms on subspace
coefficients can be designed flexibly, based on information CHALLENGES IN RESOLUTION ENHANCEMENT
from the prior distribution in the observed scene [230]. A The main challenges in the resolution enhancement of HSIs
Sylvester equation-based explicit solution was further inte- relate to practical issues. The development of multisensor
grated into the Bayesian methodology to speed it up, lead- superresolution algorithms has recently surged; however,
ing to fast fusion based on Sylvester equation (FUSE) [231]. very few publications in the literature discuss experiments
Veganzones et al. [232] introduced local image processing on real data. Studies regarding the impact of HSI resolution
into an unmixing-based approach, where well-posed inverse enhancement on applications are still lacking. It is neces-
problems are solved for each small patch, assuming that the sary to clarify benefits and address practical issues of reso-
number of endmembers in each patch is smaller than the lution-enhancement technology to promote its operational
number of multispectral bands. application for future hyperspectral satellite missions. For
One interesting finding of the comparative study in instance, temporal mismatches included in the input im-
[215] was that hypersharpening methods and unmixing- ages raise challenges for HSI resolution enhancement. Fur-
based methods (CNMF and HySURE), which are entirely thermore, a no-reference image-quality assessment for HSIs
different approaches, showed comparably high numerical needs to be developed to provide reliability information
performances, although the characteristics of the resolu- regarding resolution-enhanced hyperspectral data products
tion-enhanced HSIs are different. This finding implies that for end users.
hybrid methods combining different approaches can be ex- In multisensor superresolution, the problem remains
pected to further improve fusion performance. that large spectral distortions are inevitable when the mis-
Table 5 shows the overall quantitative assessment results match between the two imaging sensors is large in either
of 12 HS–MS fusion algorithms for eight simulated HS–MS the spatial or spectral domain (e.g., a large GSD ratio or hy-
data sets used in [215], including those based on the Pavia perspectral pan-sharpening). To address this issue, a pos-
University and University of Houston data sets. The quantita- sible future direction for performance improvement lies in
tive assessment of fusion performance was carried out based developing algorithms that exploit a spectral library or spa-
on a version of Wald’s protocol presented in [232] and [215]. tial information of a high-resolution image.

TABLE 5. THE AVERAGE QUALITY MEASURES FOR EIGHT DATA SETS FROM [215].

CATEGORY METHOD PSNR SAM ERGAS Q2n

CS GSA [206] 39.221 2.063 1.885 0.8796
MRA SFIM-HS [207], [217] 41.074 1.707 1.789 0.8867
GLP-HS [208], [217] 41.322 1.664 1.733 0.8984
Unmixing CNMF [224] 42.092 1.607 1.681 0.9060
ECCV ‘14 [225] 38.101 2.658 4.561 0.8431
ICCV ‘15 [227] 40.470 1.672 1.902 0.8847
HySURE [229] 42.336 1.602 1.766 0.9089
Bayesian MAP-SMM [219] 40.008 1.822 1.997 0.8635
FUSE [231] 40.568 1.881 1.908 0.8693
FUSE-S [231] 41.177 1.803 1.804 0.8764
Ideal 3 0 0 1

Bold numbers indicate the best performances. HS: hyperspectral; ECCV: European Conference on Computer Vision; ICCV: International Conference on Computer Vision; FUSE-S:
sparse FUSE.

58 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 HSI DENOISING AND IMAGE RESTORATION SIGNAL-INDEPENDENT NOISE
Image restoration generally refers to the reconstruction of the Thermal noise and quantization noise in HSIs are modeled
true image based on its corrupted version. The true image is by signal-independent Gaussian additive noise [234], [235].
unknown and, therefore, is estimated through the observed Usually, noise is assumed to be uncorrelated spectrally, i.e.,
image, which has been degraded by different sources. Deg- the noise covariance matrix is diagonal [236], [235]. The
radation sources depend on the imaging technology, sys- Gaussian assumption has been broadly used in hyperspec-
tem, environment, and other factors. When the observed tral analysis because it considerably simplifies the analysis.
signal is degraded by noise sources, the estimation task is In addition, noise parameter
usually called denoising. Image restoration generally (variance) estimation is sim-
refers to a broader set of methods that also include appli- pler under this assumption. IMAGE RESTORATION
cations such as deconvolution, deblurring, and inpainting. GENERALLY REFERS TO THE
However, in HSI analysis, the term restoration is often used SIGNAL-DEPENDENT NOISE
RECONSTRUCTION OF THE
to refer to denoising. Shot (photon) noise in HSIs is
TRUE IMAGE BASED ON ITS
In hyperspectral imaging the received radiance at the modeled by a Poission distri-
sensor is degraded by different sources, such as atmospheric bution because the noise vari- CORRUPTED VERSION.
haze and instrumental noise. The atmospheric effects are ance is dependent on the sig-
usually compensated for by applying atmospheric correc- nal level. The noise parameter
tions. Instrumental (sensor) noise includes thermal (John- (variance) estimation is more challenging under this assump-
son) noise, quantization noise, and shot (photon) noise. tion compared to the signal-independent case because it varies
Spectral bands are often corrupted to some degree. The with respect to the signal level.
presence of corrupted bands (also called junk bands) could
degrade the efficiency of the image analysis technique, and, SPARSE NOISE
therefore, they are usually removed from the data before any Impulsive noise, missing pixels, missing lines, and other out-
further processing. The information lost by removing those liers often exist in the acquired HSI, usually due to the mal-
bands can be substantial; hence, an alternative approach is functioning of the sensor. In this review, we categorize these
to recover those bands and improve the SNR of the HSI. As a as sparse noise because of their sparse characteristics. Sparsity
result, HSI restoration can be considered an important pre- techniques or sparse and low-rank decomposition techniques
processing step in HSI analysis. have been used to remove sparse noise from the signal. In
An HSI can be modeled by [237], impulsive noise was removed using an , 1-norm for both
penalty and data fidelity terms in the minimization prob-
X = H + N, (1) lem suggested.

where X is an n 12 # d matrix ^n 12 = n 1 # n 2 h containing the STRIPING NOISE

vectorized observed image at band i in its ith columns, H Hyperspectral imaging systems might also induce artifacts in
is the true unknown signal to be estimated and is repre- HSIs, usually referred to as pattern noise. For instance, in push-
sented as an n 12 # d matrix containing the unknown vec- broom imaging systems, the target is scanned line by line,
torized image at band i in its ith columns, and N is an and the image line is acquired in different wavelengths by an
n 12 # d matrix containing the vectorized noise at band i area-array detector (usually a charged coupled device). This
in its ith columns. Note that all of the previously men- line-by-line scanning causes an artifact called striping noise,
tioned noises can be assumed to be additive noise. The which is often due to calibration error and the sensitivity
restoration task is to estimate the original (unknown) sig- variation of the detector [238]. Striping noise reduction (also
nal H. Penalized (regularized) least squares is one of the referred to as destriping in the literature) for pushbroom scan-
most popular and common minimization frameworks ning techniques has been widely studied in the remote-sens-
used for estimation in HSI restoration. Penalized least ing literature [239], [240], including work on HSI remote
squares is usually composed of a fidelity term and a pen- sensing [238], [241]–[243].
alty term. The penalty term is often chosen based on the
prior knowledge of the signal and might be a combination EVOLUTION OF HYPERSPECTRAL IMAGE
of penalties. Penalized least squares might also be solved RESTORATION APPROACHES
subject to some constraints. HSI restoration has been developed considerably over the
past few years. Conventional restoration methods based
NOISE SOURCE ASSUMPTIONS IN on 2-D modeling and convex optimization techniques
HYPERSPECTRAL IMAGES were not effective enough for HSIs because of the lack of
The presence of different noise sources in HSIs makes their understanding of spectral information. The highly corre-
modeling and restoration highly challenging. Therefore, lated spectral bands in HSIs have been found very useful
HSI restoration often employs one or a mixture of the fol- for HSI restoration. This is the main reason for the success
lowing ­approaches. in using MLR as an estimation technique [244]. In [244],

december 2017 ieee Geoscience and remote sensing magazine 59

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 it is assumed that each band is a linear combination of APPROACHES THAT USE LOW-RANK MODELS
the other bands; therefore, the ith band is estimated us- Because of the redundancy along the spectral direction,
ing least squares estimation. Note that this technique has low-rank modeling has been widely used in HSI analyses
been used for noise parameter estimation in several HSI and applications such as DR, feature extraction, unmixing,
restoration approaches [245], [246]. Many restoration ap- and compression. PCA was used for hyperspectral restora-
proaches have been suggested in the literature to exploit tion in [257] to spectrally decorrelate the noise and signal.
the spectral information, and they can be categorized into A low-rank representation method is a three-mode factor
three main groups. analysis called Tucker3 decomposition [258] used for HSI res-
toration [259]. The HSI is assumed to be a third-order tensor,
APPROACHES THAT USE 3-D and the best lower rank of the decomposition is chosen by
MODELS INSTEAD OF 2-D ONES minimizing a Frobenius norm. A similar idea was exploited
In [247], the discrete Fourier transform was used to decor- for HSI restoration by applying more reduction spectrally
relate the signal in the spectral domain, and the 2-D discrete [260]. A GA was developed to choose the rank of the Tuck-
wavelet transform (DWT) was investigated to denoise the er3 decomposition [261]. This work was followed by [262],
signal in the spatial domain. In [248], the HSI was treated in which the kernel function (Gaussian radial basis) was
as a 3-D data cube, and an HSI restoration technique was applied for each spectral band, with the idea of efficiently
proposed based on sparse analysis regularization and un- using multilinear algebra.
decimated wavelet transform. The advantages of 3-D wave- Multidimensional Wiener filtering that exploits Tuck-
lets (orthogonal and undecimated) over 2-D ones for HSI er3 decomposition was used in [263], where the flatten-
restoration were also discussed in [248] and [236]. ing of the HSI was achieved by estimating the main direc-
tion that gives the smallest rank. Parallel factor analysis is
APPROACHES THAT PROPOSE NEW PENALTIES also a low-rank modeling used in [264] for HSI denoising.
FOR PENALIZED LEAST SQUARES THAT ALSO A new 3-D linear model was proposed for HSI in [265],
TAKE INTO ACCOUNT SPECTRAL INFORMATION where 2-D wavelets were used for spatial projection and
An algorithm given in [249] uses the 2-D DWT and a sparse spectral-singular vectors of the observed HSI were used
restoration criterion based on penalized least squares hav- for spectral projection. A convex optimization was used
ing a group of , 2 penalties on the wavelet coefficients. This for the restoration task based on the 3-D linear model and
method was improved on in [250] for HSI restoration by , 1 penalty. Additionally, SURE was used for regulariza-
taking into account the spectral noise variance in the mini- tion parameter selection.
mization problem and solving it using the alternating di- Low-rank modeling has been used in synthesis and
rection method of multipliers. Subsequently, because of analysis penalized least squares [266], [250] and also in
the redundancy and high correlation in the spectral bands TV regularization [267], [250]. A 3-D low-rank model in
in HSIs, penalized least squares using a first-order spectral the form of model (2) in the next section was proposed in
roughness penalty (FOSRP) was proposed for HSI restora- [246], where 2-D wavelets were used as the spatial basis
tion [251]. The new cost function was formulated in the while the spectral basis was assumed to be an unknown
wavelet domain to exploit the MRA property of wavelets. low-rank orthogonal matrix. Therefore, an orthogonal-
The Stein unbiased risk estimator (SURE) was used to ity constraint was added to the optimization problem for
­automatically select the tuning parameters. It was shown the simultaneous estimation of the two unknown matri-
that FOSRP outperforms sparsity penalties for HSI resto- ces in the minimization problem, which led to a noncon-
ration. This method was improved upon using a combi- vex optimization.
nation of a spectral roughness penalty and a group lasso
penalty [252]. HYPERSPECTRAL IMAGE MODEL SELECTION
Cubic TV (CTV), proposed in [253], exploits the gradi- Further studies in HSI modeling and restoration [250] con-
ent in the spectral axis to improve the restoration results firmed that capturing spectral redundancy by low-rank
compared to TV denoising band by band. In [254], an modeling is more appropriate than full-rank modeling for
adaptive version of CTV was applied for preserving both HSI modeling and restoration. In [250], a model selection
texture and edges simultaneously. In [255], a spatial-spec- criterion was given for a general model of the form
tral HSI denoising approach was developed. The spectral
derivation was proposed to concentrate the noise in the X = AW r M Tr + N, (2)
low frequency. Then, noise was removed by applying the
2-D DWT on the spatial domain and the 1-D DWT on the whe re A (n 12 # n 12   m at r i x) a nd M r (d # r mat r i x ,
spectral domain. A spatial-spectral prior for maximum a r # min ^n 12, d h) could be 2-D and 1-D orthogonal (known)
posteriori information was proposed in [256]. The prior bases, respectively, and W r (n 12 # r matrix) contains the
was based on five derivatives, one along the spectral direc- corresponding coefficients for the unknown hyperspectral
tion and the rest applied on the spatial domain for four data, H. The term W r is estimated using , 1 penalized least
neighborhood pixels. t = AW
squares, and the signal is restored by H t r M Tr . Ideally, the

60 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 model that gives the lowest MSE is the best choice among the spectral axis (v 2i ), like a Gaussian shape centered at the
the candidates. However, the MSE is uncomputable in prac- middle band (d/2), as
tice because it depends on the true (uncorrupted) data.
^ i -d/2 h2
Therefore, SURE is suggested for use as an estimator of MSE e- 2h 2
for HSIs. The results have confirmed that low-rank models v 2i = v 2 d ^ j -d/2 h2
, (3)
give lower MSE (estimated by SURE) compared with full-
/e -
2h 2
j =1
rank ones.
where the power of the noise is controlled by v and h
HYPERSPECTRAL IMAGE RESTORATION behaves like the standard deviation for the Gaussian bell
WITH MIXED-NOISE ASSUMPTION curve [244]. To evaluate the restoration results for the simu-
A mixed-noise assumption has also been taken into lated data set, the SNR in decibels is used as
consideration in HSI modeling and restoration. In a
mixed-noise assumption, the HSI (X) in model (1) is as- SNR out = 10 log 10 _ H 2F / H - H
t 2F i,
sumed to be corrupted by a mixture of the noise sources
described in the “Noise Source Assumptions in Hyper- where . F is the Frobenius norm and the noise input level
spectral Images” section. A mixture of the signal-de- for the whole cube is defined as
pendent noise (N SD) and signal-independent noise (N SI)
models has been taken into account in [245], [268], and SNR in = 10 log 10 ^ H 2F / H - X 2F h.
[269] as N = N SI + N SD. Therefore, two parameters need
to be estimated: the variances of N SI and N SD, which In Figure 10, the results are shown when SNR in varies
have Gaussian distribution and Poission distribu- from 5 to 40 dB in increments of 5 dB. Note that the re-
tion, respectively. In [268], a 3-D (block-wise) nonlo- sults shown are means-over-ten experiments (adding ran-
cal sparse restoration method was suggested for HSIs. dom Gaussian noise) and the error bars show the standard
The minimization problem, which uses a group lasso deviations. In this experiment, six restoration methods are
penalty and a dictionary consisting of a 3-D discrete compared based on SNR: , 1 penalized least squares using
cosine transform and a 3-D DW T, was solved by using 2-D wavelet modeling (2-D wavelet) [274] and 3-D wavelet
the accelerated proximal gradient method. In [269], N SI modeling (3-D wavelet) [248]; penalized least squares using
and N SD were removed sequentially. Maximum likeli- FOSRP (FOSRPDN) [251]; , 1 penalized least squares using
hood was used to estimate the two parameters of the a wavelet-based low-rank model called sparse wavelet-based
noise model where MLR was investigated for an earlier model and rank selection (SPAWMARS) [250]; low-rank
estimation of the noise. In [241], a subspace-based ap- matrix recovery (LRMR) [270]; and noise-adjusted iterative
proach was given to restore HSI corrupted by striping low-rank matrix approximation [272]. Note that 2-D wave-
noise and N ID. let, 3-D wavelet, FOSRPDN, and SPAWMARS all exploit
A widely used mixed-noise assumption for HSI res- SURE as a parameter selection technique and, therefore,
toration is N = N SI + N S, where N S represents the sparse are parameter-free methods. Also, note that SPAWMARS is a
noise defined in the “Noise Source Assumptions in Hy-
perspectral Images” section. This mixture assumption
was used in [270], where low-rank and sparse matrix
45
­decomposition was taken into account to restore HSIs.
This method was improved on by augmenting the to- 40
tal variation penalty to the restoration criterion [271]. 35
Also, the noise-adjusted iterative low-rank matrix appro­
SNRout (dB)

30
xi­mation given in [272] approximates an HSI with a
low-rank matrix, while taking into account the changes 25 Noise
of the noise variance through the spectral bands. The 2-D Wavelet
20 3-D Wavelet
authors of [273] proposed using a weighted Schatten p-
15 FOSRPDN
norm as a nonconvex low-rank regularizer for low-rank LRMR
and sparse decomposition of HSIs degraded by sparse 10 NAILRMA
and Gaussian noises. SPAWMARS
5
5 10 15 20 25 30 35 40
EXPERIMENTAL RESULTS SNRin (dB)
In this section, we present some experimental results for HSI
restoration. Figure 10 shows the evolution of HSI restoration FIGURE 10. A comparison of the performance of different HSI
techniques based on SNR. The experiments were applied on restoration methods for different levels of input Gaussian noise
a portion (128 # 128 # 96) of the Pavia University data set; in decibels. NAILRMA: noise-adjusted iterative low-rank matrix
here, the variance of the added Gaussian noise varies along approximation.

december 2017 ieee Geoscience and remote sensing magazine 61

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 TABLE 6. THE CPU PROCESSING TIME IN SECONDS FOR DIFFERENT RESTORATION APPROACHES USED IN THE EXPERIMENT.

2-D WAVELET 3-D WAVELET FOSRPDN LRMR NAILRMA SPAWMARS

1.16 1.72 1.09 25.70 4.79 0.62

fully automatic version of an SVD-based sparse regulariza- swift implementation of HSI restoration approaches in
tion with reduction [236] with rank selection. The MATLAB the future.
codes for FOSRPDN and SPAWMARS are available online in
[275] and [276], respectively. CHANGE DETECTION
The blue line in Figure 10 indicates the noise levels, and, In this section, we define a CD problem in a pair of bitem-
therefore, the effectivity of the HSI restoration techniques poral HSIs (those in the multitemporal domain can be ad-
can be compared with respect to the noise levels. It can be dressed straightforwardly, pair by pair). Let X 1 and X 2 be
seen that 3-D wavelet restoration considerably improves two HSIs acquired over the same geographical area at times
on conventional 2-D wavelet techniques. Also, FOSRPDN, t 1 and t 2, respectively. The hyperdimensional difference
which is based on a 2-D wavelet and an FOSRP, outperforms image X D, i.e., the SCVs, can be computed by subtracting
3-D wavelet restoration, which confirms the importance of the bitemporal images pixel by pixel, i.e.,
the spectral correlations. Finally, SPAWMARS, which utilizes
a wavelet-based low rank, outperforms the other techniques X D = X 2 - X 1. (3)
used in this experiment. Note that Wavelab Fast (a fast wave-
let toolbox), which is provided in [277], was considered for Let X = {~ n, X c} be the set of all classes in X D, where ~ n
the implementation of wavelet transforms. A Daubechies is the no-change class and X c = {~ C 1, ~ C 2, ..., ~ C k} be the
wavelet with two and ten coefficients for spectral and spatial set of K possible change classes. The considered binary CD
bases, respectively, in five decomposition levels was used in problem can be formalized to separate the ~ n and X c class-
all the experiments. The central processing unit (CPU) pro- es without distinguishing different classes in X c ; the objec-
cessing time in seconds for different restoration approaches tive of the multiclass CD task is to detect the changed pixels
used in the experiment confirms that SPAWMARS is the in X c and identify their classes in " ~ C 1, ~ C 2, ..., ~ C k ,.
most efficient method (Table 6). All the experiments in this Continuous satellite observation resulted in the acqui-
section were performed in MATLAB on a computer hav- sition of a large number of multitemporal remote-sensing
ing an Intel Core i7-4710HQ CPU at 2.5 GHz, with 12 GB images. By analyzing these images, a better understanding
of memory, and a 64-b operating system. of the changes and evolutions on Earth’s surface can be
gained. CD is a technique that enables the land cover chang-
CHALLENGES IN HYPERSPECTRAL IMAGE es occurring over a geographical area at different observa-
DENOISING AND IMAGE RESTORATION tion times to be identified [278]. In past decades, CD has
HSI restoration and modeling face several future challenges. played important roles in various multitemporal remote-
1) The HSI model selection and noise parameter estima- sensing applications, such as urban sprawl analysis, disaster
tion need more attention. For instance, a model selec- loss evaluation, and forest and environmental monitoring
tion criterion that is not restricted to the Gaussian noise [279]–[281]. For optical remote-sensing images, CD based
model would be very useful for HSI modeling and res- on multispectral images has been intensively investigated
toration. The main advantage of a model selection crite- because of the availability of multispectral sensors onboard
rion is that it provides an instrument to compare the res- the last-generation Earth observation satellites. With the
toration techniques without using a simulated (noisy) launch of new-generation Earth observation satellites carry-
HSI but rather using the observed HSI itself. ing hyperspectral sensors, there are further opportunities to
2) It is of interest to investigate the contributions of the var- implement CD in multitemporal HSIs.
ious HSI restoration approaches, such as CD, unmixing, Unlike multispectral images, detailed spectral sam-
and resolution enhancement, as preprocessing steps for pling in HSIs allows the potential detection of more spec-
further HSI analysis. tral variations, especially those with subtle, spectrally in-
3) In mixed-noise scenarios, investigating the dominant significant changes [282]. Compared with the abrupt changes
noise type within HSIs should be considered. that present in coarse multispectral images, changes in
4) HSI restoration approaches, which are computationally HSIs are more sophisticated, implicit, and structurally
efficient and parameter free (automatic), can simply be complex. One emerging problem within this context is
used as a preprocessing step in real-world applications. the detection of small and subtle changes compared
Fast computing techniques may be considered for the to the large unchanging background in multitemporal

62 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 HSIs. This is related to the definition of an anomaly CD This section reviews several recent CD techniques for
(ACD) problem, which is not easily addressed by classical HSIs, discussing and analyzing their properties. In ad-
CD techniques. dition, a quantitative analysis comparing the CD results
Traditional CD identifies large land cover changes char- obtained by some state-of-the-art CD techniques on the
acterized by significant spectral variations. An empirical Umatilla County Hyperion data set is provided. A sum-
definition of the change concept in multitemporal HSIs mary of several classical and recent techniques for multi-
from the global and local spectrum discriminability can be temporal HSI CD techniques is provided in Table 7, along
found in [282]. For CD techniques in multispectral images, with some details such as their categories and principal
exhaustive investigations have been made in the past few characteristics. Note that this summary is not exhaustive.
decades. However, we could find relatively few works cov-
ering CD in HSIs. Recently, a book chapter [283] analyzed ANOMALY CHANGE DETECTION
this challenging task and provided a literature review and ACD in HSIs is intended to distinguish anomalous changes
comparison between CD in hyperspectral and multispec- from the nonchanges and pervasive changes in multitem-
tral images. Problems and challenges in the existing meth- poral HSIs [284]. The main idea of ACD is to design a ro-
ods were also discussed. bust detector able to maximize the difference between the

TABLE 7. A SUMMARY OF SOME CLASSICAL AND RECENT TECHNIQUES FOR

MULTITEMPORAL HSI CD (NOT EXHAUSTIVE).

APPLICATION SUPERVISED/SEMISUPERVISED/
­PURPOSE UNSUPERVISED TECHNIQUES MAIN CHARACTERISTICS
ACD Unsupervised Chronochrome [285], covariance-­ An anomaly change estimation based on
equalization [286] linear predictors.
Unsupervised Elliptically contoured distributions-based Anomalous change detectors based on
methods [287] elliptically contoured distributions.
Unsupervised SFA-based RX algorithm [284] SFA is used to construct the change
residual image.
Unsupervised CKRX [288] Background pixels are clustered, and the
cluster centers are used in ACD.
Unsupervised Eliminating external factors (e.g., image Enhancing the CD performance.
parallax errors [289], vegetation and
illumination variations [290], diurnal and
seasonal variations [291])
Binary CD Unsupervised MAD [294], IR-MAD [296], T-PCA [297], Changes are represented and highlighted
ICA [298] in transformed and reduced feature space.

Supervised Subspace-based approach [299] Changes are identified as anomalous

­pixels according to the subspace distance.
Semisupervised Distance metric-based method [300] The distance metric is learned in a
­L aplacian regularized metric framework.
Unsupervised Slight CD method [16] Using block processing and locally linear
embedding.
Unsupervised/supervised Unmixing-based methods [301]–[304] Unmixing single-time or stacked
­multitemporal images to investigate
the subpixel-level change.
Multiple CD Supervised PCC [305] Independently classifying each single-time
image; detailed land cover transitions
(i.e., from–to information) can be
­obtained.
Supervised 3-D spectral modeling approach [306] A spatial/spectral/temporal joint modeling
and unmixing of multitemporal data.
Unsupervised HSCVA [282] Hierarchical clustering and detecting of spec-
tral variations at different significant levels.
Semisupervised S2CVA [307] The adaptive and sequential projection
of SCVs to discover and detect multiple
changes.
Unsupervised MSU [12] Spectral–temporal unmixing to identify
the unique multitemporal endmembers
for representing multiclass changes.
Unsupervised/supervised Band-selection-based method [309] The most informative band subset of the
different images is selected to enhance
CD performance.

december 2017 ieee Geoscience and remote sensing magazine 63

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 anomaly changes and the unchanged scene background metric framework was exploited to learn a distance met-
but to minimize the possible class difference within the ric for detecting change under noisy conditions. In [16], an
background. In this context, multivariate statistical tech- unsupervised CD method was developed for slight change
niques based on linear predictors, e.g., cross covariance extraction in a multitemporal HSI sequence. The feature
(chronochrome) [285] and covariance-equalization [286], space was built using block processing and locally linear
are typical ACD and binary CD detectors. Recently, algo- embedding, and then a final CD map was generated by
rithms were proposed to detect anomalous changes in HSIs clustering the change class and the no-change class. In
by modeling the data with elliptically contoured distribu- [301] and [302], CD in HSIs using unmixing was investi-
tions [287]. A new ACD method was proposed in [284] gated on a single-time image and stacked multitemporal
that constructed the change residual image based on slow images, respectively. Also, in [303] and [304], sparse un-
feature analysis (SFA) and de- mixing and a decision-fusion-based spectral-mixture ap-
tected anomaly changes by proach were exploited, respectively, to detect the subpixel-
using the Reed–Xiaoli (R X) level change information.
FOR BINARY CD, THE algorithm. A cluster kernel
AIM IS TO SEPARATE THE RX (CKRX) algorithm was de- MULTIPLE CHANGE DETECTION
CHANGED AND UNCHANGED veloped in [288] that clustered The aim of multiple CD is more complex and challenging
PIXELS IN THE IMAGES. the background pixels, then than binary CD. Changed pixels are detected while differ-
used the c­ luster centers in the ent classes of change are distinguished. If multitemporal
anomaly detection. Other in- ground reference samples are available, this task can be
vestigations have also focused addressed according to a typical supervised postclassifica-
on various specific changes—e.g., eliminating image paral- tion comparison (PCC) method [305] by classifying inde-
lax errors [289], vegetation and illumination variation [290], pendently two (or more) images at different times and then
and diurnal and seasonal variations [291]. comparing the pixel class label to detect changes. The main
advantage of PCC is that detailed land cover transitions are
BINARY CHANGE DETECTION obtained (i.e., from–to information). However, the accura-
For binary CD, the aim is to separate the changed and un- cy of the CD performance depends greatly on the accuracy
changed pixels in the images. In this case, classical tech- of a single-time image classification result.
niques in multispectral imaging such as thresholding In [306], a new approach for modeling the temporal
[292] or clustering [293] can still be considered, based on variations of the reflectance response as a function of time
the computed magnitude of the SCVs. Transform-based period and wavelength was developed. In this approach, a
methods constitute an important class. Such techniques library of known endmembers that depends mainly on the
represent the original data in a feature space where the sig- 3-D surface reconstruction quality and similarity measure
nificant change information is concentrated in a few trans- is used to perform a classification task. Changes are detect-
formed components. This not only reduces data dimen- ed, and the approach provides better modeling of seasonal
sionality and noise but also focuses on the specific changes variations. However, in practical applications, comprehen-
of interest in specific components. sive multitemporal ground reference samples are usually
The multivariate alteration detection (MAD) method not available. Accordingly, unsupervised techniques that
originally proposed for multispectral images in [294] do not rely on reference samples are more valuable.
was applied to HSIs to detect vegetation change based on In [282], an unsupervised coarse-to-fine hierarchical SCV
­canonical correlation analysis [295]. It was extended to an analysis (HSCVA) approach was proposed. It was designed
iterative reweighted version (IR-MAD) [296] to better em- to cluster and detect changes that include spectral variations
phasize and detect changes. A temporal PCA (T-PCA) was at different significant levels according to their discrim-
proposed in [297] that exploits the temporal variances in inable spectral behaviors. A semisupervised sequential SCV
the combined multitemporal HSIs. The no-change infor- analysis (S2CVA) technique for discovering, identifying,
mation is represented in the first principal component and and discriminating multiple changes in HSIs was proposed
the change information in the second, which is orthogonal in [307]. This method detects different kinds of changes ac-
to the first one. cording to the adaptive and sequential projection of SCVs at
Recently, in [298], ICA was applied, combined with the each level of the hierarchy. S2CVA successfully extends the
uniform feature design strategy, by following a hierarchi- compressed change vector analysis method [308], which is
cal framework to identify specific vegetation changes. In suitable for dealing only with multispectral CD cases hav-
[299], a subspace-based CD approach was designed using ing few spectral channels. To identify the subpixel-level
the undesired land cover class spectral signature as prior spectral changes so that a more accurate multiple CD result
knowledge, where the subspace distance was computed to can be produced, an unsupervised multitemporal spectral
determine the anomalous pixels as change with respect to unmixing (MSU) model was proposed in [12]. The MSU
the background subspace. A semisupervised CD method investigates in detail the spectral-temporal mixture proper-
was proposed in [300], where a Laplacian regularized ties in multitemporal HSIs, and the considered multiple CD

64 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 problem is addressed by analyzing the abundances of differ-
ent distinct multitemporal endmembers at a subpixel level. Table 8. THE CD ACCURACIES AND ­ERRORS OBTAINED
BY CONSIDERED STATE-OF-THE-ART TECHNIQUES ON THE
In addition, both unsupervised and supervised band- UMATILLA COUNTY HYPERION DATA.
selection-based CD approaches were designed to evaluate
the potential detectability of the change detector in a re- METHODS HSCVA [282] S2CVA [307] MSU [12]

duced feature space [309]. Experimental results confirmed OA (%) 95.26 95.36 95.37
that the most informative band subset selected can help to kappa 0.8800 0.8822 0.8826
improve CD performance over the use of the original full- Omission (pixels) 1,715 1,878 1,875
dimensional HSIs. A multiclass CD experimental compari- Commission (pixels) 1,918 1,676 1,239
son was carried out on the Umatilla County Hyperion data
Total errors (pixels) 3,633 3,554 3,114
set using the state-of-the-art CD techniques introduced
above, including HSCVA [282], S2CVA [307], and MSU
[12]. Numeric experimental results are provided in Table 8,
which includes the obtained OA, kappa coefficient, and
omission and commission errors. Learning from the limited prior change information trans-
From Table 8, we can see that the MSU resulted in the ferred between multitemporal HSIs could be another inter-
best performance in this data set, with respect to the high- esting research direction.
est OA (95.37%) and kappa (0.8826) values and lowest
number of total errors (3,114 pixels). This indicates the su- FAST COMPUTING
perior detection of spectral variations at a fine level in the The high-dimensional nature of hyperspectral data sets,
MSU approach. The other two state-of-the-art methods together with the complexity of the processing algorithms,
also obtained a high level of OA and kappa values, which calls for advanced processing techniques to accelerate hyper-
demonstrates the effectiveness of the considered methods. spectral-related computations [310]. Traditionally, software
In practical applications, two pixel-level multiple CD tech- for hyperspectral analysis has been written for serial com-
niques are sufficient to address the challenging multiple CD putation, i.e., to be run on a single computer having a single
task, especially in an unsupervised/semisupervised fashion, CPU in which a problem is broken down into a discrete se-
without using ground reference samples. ries of instructions. Instructions are then executed one af-
ter another so that only one may execute at any moment
CHALLENGES IN CHANGE DETECTION in time. In turn, parallel com-
For CD tasks in multitemporal HSIs, challenges come from puting allows the simultane-
the intrinsic properties of the hyperspectral data and the ous use of multiple computer PARALLEL COMPUTING
design of sophisticated CD techniques to handle the com- resources. In other words, a
ALLOWS THE
plexities in the CD process. From the hyperspectral data problem is run on multiple
SIMULTANEOUS USE
perspective, the high dimension inevitably leads to infor- CPUs by being broken down
OF MULTIPLE
mation redundancy and high computational cost. More- into discrete parts so that each
over, changes are to be more implicitly represented and de- part can be executed simulta- COMPUTER RESOURCES.
tected. In other words, from the spectral signature point of neously. Load balancing refers
view, similar changes are more likely to overlap, especially to the practice of distributing
subtle changes. Thus, the difficulty in discriminating their work among tasks so that all tasks are kept busy all the time.
classes in high-dimensional feature space increases. This practice is considered to minimize task idle time.
From the CD methodology perspective, advanced ap- Taking advantage of these concepts, several techniques
proaches need to be designed that 1) can more adaptively have been developed to accelerate hyperspectral imaging
detect multiple complex changes (i.e., those having dif- computation. In this section, we summarize some of the
ferent spectral significance), 2) can be implemented more available strategies, which vary according to the adopted
automatically, and 3) can be computationally effective. platform for fast computing and acceleration.
In particular, the development of unsupervised CD tech-
niques for HSIs is more important for real applications. CLUSTER COMPUTERS
In fact, several subproblems should be considered within Perhaps the most widely used high-performance computing
a CD process, such as the identification of the number architecture for accelerating hyperspectral-related computa-
of multiple changes, the separation of changed and un- tions is cluster computing. A cluster is a collection of com-
changed pixels, and the discrimination of multiclass modity computers interconnected by a computer network. To
changes. Each of these subproblems deserves to be investi- efficiently execute a parallel problem in a cluster, partitioning
gated in detail, independently or simultaneously, to gener- strategies can be used so that the original problem is broken
ate more accurate CD results. In addition, the subpixel and down into subtasks allocated to the different computers.
superpixel implementation of CD techniques at different In the case of hyperspectral imaging, two kinds of strate-
detection scales is expected to enhance CD performance. gies have been used to partition the original hyperspectral

december 2017 ieee Geoscience and remote sensing magazine 65

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 data set into subsets for efficient cluster-based processing. architectures. In this sense, cloud computing offers ad-
In spectral-domain partitioning, a single pixel vector (spec- vanced capabilities for service-oriented and high-perfor-
tral signature) may be stored in different processing units, mance computing [336]. Furthermore, using cloud com-
and communications would be required for individual puting to analyze large repositories of hyperspectral data
pixel-based calculations, such as spectral angle computa- can be considered a natural solution resulting from the
tions [311]. In spatial-domain partitioning, every pixel vec- evolution of techniques previously developed for other
tor (spectral signature) is stored in the same processing unit types of computing platforms [311]. In particular, using
[312]. These concepts have GPUs within distributed scenarios has been radically ex-
been explored not only in tended worldwide, thanks in part to the increasing devel-
clusters [313], [314] but also opment of deep learning-based frameworks (e.g., Apache
RECONFIGURABILITY IS in heterogeneous workstation Spark, Caffe, Theano, Torch, and TensorFlow), which also
NO LONGER A PROMISE networks [315]. find application in the hyperspectral analysis community
[64], [65], [84], [337], [338].
BUT A REALITY, AND IT IS
HARDWARE However, the recent literature still provides few exam-
FEASIBLE TO HAVE
ACCELERATORS ples of the use of cloud infrastructures to implement hy-
SEVERAL ALGORITHMS
In addition to clusters, other perspectral analysis techniques in general and to perform
IMPLEMENTED ON kinds of high-performance supervised classification of hyperspectral data in partic-
THE SAME BOARD. computing architectures have ular. This may be due to the lack of open repositories of
been used. Specifically, on- HSIs available for public use, a situation that is expected to
board processing of HSIs is change in the near future, as large distributed repositories
required in certain contexts. of hyperspectral data for open use become available to the
This is particularly the case in time-critical applications scientific community.
[316], [317]. For this purpose, other solutions have been
explored. In particular, reconfigurable computing provides CHALLENGES IN FAST COMPUTING
higher performance (throughput and processing power) The most important challenge related to consolidating fast-
compared to other specialized hardware processors [318]. computing techniques to analyze hyperspectral data (par-
Reconfigurability is no longer a promise but a reality, and ticularly in the context of real-time platforms) is still the
it is feasible to have several algorithms implemented on the high energy consumption required by high-performance
same board and dynamically select one out of a pool of al- computing architectures, which reduces their applicabil-
gorithms from a control station [319]. A field-programma- ity in real scenarios for onboard operation. Currently, the
ble gate array (FPGA) is a chip in which there is a matrix of power consumption required by devices such as GPUs is too
blank cells called configurable logic blocks. This device can be high for their incorporation into satellite platforms. Another
used to implement any circuit (provided there is a sufficient shortcoming is the fact that these platforms are often sub-
number of logic blocks). FPGAs have been widely used to ject to radiation tolerance issues. Future developments in
accelerate hyperspectral imaging algorithms for onboard hardware instruments for onboard operation are required
processing [320]–[322]. for efficient real-time processing of HSIs, particularly in the
Another high-performance computing architecture that context of satellite missions.
has provided excellent performance when accelerating hy-
perspectral imaging computations is the graphical process- CONCLUSIONS
ing unit (GPU) [323]. GPUs are now fully programmable The role of HSI analysis cannot be underestimated for a
using high-level languages such as NVIDIA CUDA (http:// plethora of applications, especially those related to CD and
developer.nvidia.com). The GPU specializes in computer- scene classification. Without a doubt, the use of such valu-
intensive, massive-data-parallel computation (which ex- able data has been well established in the remote-sensing
actly describes graphics rendering). Therefore, more transis- community, and the precise investigation of such data is in-
tors can be devoted to data processing rather than to data creasing significantly. To this end, the considerable number
caching and flow control. The fast-growing video game of airborne and spaceborne hyperspectral missions as well
industry exerts strong economic pressure for constant in- as the increasing number of scientific publications on this
novation. This has motivated the extended use of GPUs for particular subject demonstrate that the area of HSI analysis
accelerating many different hyperspectral imaging-related is substantial, dynamic, and vibrant.
tasks [316], [324]–[335]. The field of hyperspectral imagery is extremely broad,
and it is impossible to investigate it comprehensively in
CLOUD COMPUTING one literature review. This article focuses particularly on
Recently, more sophisticated high-performance architec- algorithmic approaches that have been developed, adapt-
tures have been used to process hyperspectral data. For ed, or proposed since 2013, covering a number of key re-
instance, cloud computing platforms are increasingly em- search areas, such as DR, classification, spectral unmixing,
ployed for processing hyperspectral data in distributed resolution enhancement, image restoration, CD, and fast

66 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 computing. It is certainly of significant interest to sum- Sicong Liu ([email protected]) is with the Col-
marize other survey articles from the application point of lege of Surveying and Geoinformatics, Tongji University,
view, where the usefulness of hyperspectral imagery can Shanghai, China. He is a Member of the IEEE.
be demonstrated through different practical aspects such Javier Plaza is with the Department of Technology of
as mineralogy, environmental mapping and monitoring, Computers and Communications, University of Extremad-
geology, and so on. ura, Spain. He is a Senior Member of the IEEE.
In addition to the material presented in this article, Behnood Rasti ([email protected]) is with the Keilir In-
hyperspectral data preprocessing plays a vital role in fos- stitute of Technology, Reykjanesbaer, Iceland, and the De-
tering application-oriented tasks. This important subject partment of Electrical and Computer Engineering, Uni-
lies beyond the scope of this investigation, which mainly versity of Iceland, Reykjavik, Iceland. He is a Member of
focuses on the development of algorithms for HSI analysis the IEEE.
and processing. To this end, we see the need for some tu- Antonio Plaza is with the Department of Technology of
torials and survey articles with a primary focus on hyper- Computers and Communications, University of Extremad-
spectral data preprocessing and preparation designed for ura, Spain. He is a Fellow of the IEEE.
atmospheric corrections, geometric and radiometric cor-
rections, coregistration, and quality assessment. REFERENCES
As indicated several times throughout this article, al- [1] D. A. Landgrebe, Signal Theory Methods in Multispectral Remote
though the area of HSI analysis is well established, there are Sensing. Hoboken, NJ: Wiley, 2003.
still many doors open for further investigation. We hope [2] N. Keshava, J. Kerekes, D. Manolakis, and G. Shaw, “An algorithm
that our work here will raise new possibilities for research- taxonomy for hyperspectral unmixing,” in Proc. SPIE AeroSense
ers to further investigate the remaining issues by develop- Conf. Algorithms for Multispectral and Hyperspectral Imagery VI,
ing fast, accurate, and automatic methodologies suitable for vol. 4049, 2000, pp. 42–63.
applications at hand. [3] E. J. Wegman, “Hyperdimensional data analysis using parallel
coordinates,” J. Amer. Statist. Assoc., vol. 85, no. 411, pp. 664–675,
ACKNOWLEDGMENTS 1990.
We extend our sincere appreciation to Prof. Paolo Gamba [4] J. A. Benediktsson and P. Ghamisi, Spectral-Spatial Classification
of the University of Pavia, Italy, and Prof. D. Landgrebe of of Hyperspectral Remote Sensing Images. Norwood, MA: Artech
Purdue University, West Lafayette, Indiana, for providing House, 2015.
the Pavia University and Indian Pines data sets, respective- [5] A. Eckardt, J. Horack, F. Lehmann, D. Krutz, J. Drescher,
ly. In addition, the authors thank the National Center for M. Whorton, and M. Soutullo, “DESIS (DLR Earth sensing
Airborne Laser Mapping at the University of Houston, Tex- ­imaging spectrometer for the ISS-MUSES platform),” in Proc.
as, for providing the CASI Houston data set and the IEEE IEEE Int. Geoscience and Remote Sensing Symp., July 2015, pp.
Geoscience and Remote Sensing Society Image Analysis 1457–1459.
and Data Fusion Technical Committee for organizing the [6] L . Guanter, H. Kaufmann, K. Segl, S. Förster, C. Rogaß,
2013 Data Fusion Contest. S. Chabrillat, T. Küster, A. Hollstein, G. Rossner, C. Chlebek,
C. Straif, S. Fischer, S. Schrader, T. Storch, U. Heiden, A. Mueller,
AUTHOR INFORMATION M. Bachmann, H. Mühle, R. Müller, M. Habermeyer, A. Ohndorf,
Pedram Ghamisi ([email protected]) is with the Ger- J. Hill, H. Buddenbaum, P. Hostert, S. van der Linden, P. J. Leitão,
man Aerospace Center, Remote Sensing Technology Insti- A. Rabe, R. Doerffer, H. Krasemann, H. Xi, W. Mauser, T. Hank,
tute, and the Technische Universität München, Signal Pro- M. Locherer, M. Rast, K. Staenz, and B. Sang, “The EnMAP space-
cessing in Earth Observation, Munich, Germany. He is a borne imaging spectroscopy mission for earth observation,” Re-
Member of the IEEE. mote Sens., vol. 7, no. 7, pp. 8830–8857, 2015.
Naoto Yokoya ([email protected]) is [7] A. Iwasaki, N. Ohgi, J. Tanii, T. Kawashima, and H. Inada, “Hy-
with the Department of Advanced Interdisciplinar y perspectral imager suite (HISUI): Japanese hyper-multi spec-
Studies, the University of Tokyo, Japan. He is a Member tral radiometer,” in Proc. IEEE Int. Geoscience and Remote Sensing
of the IEEE. Symp., July 2011, pp. 1025–1028.
Jun Li ([email protected]) is with the Guang- [8] P. Stefano, P. Angelo, P. Simone, R. Filomena, S. Federico,
dong Provincial Key Laboratory of Urbanization and Geo- S. Tiziana, A. Umber to, C. Vincenzo, N. Acito, D. Marco,
simulation, Center of Integrated Geographic Information M. Stefania, C. Giovanni, C. Raffaele, D. B. Roberto, L. Giovanni,
Analysis, School of Geography and Planning, Sun Yat-sen and A. Cristina, “The PRISMA hyperspectral mission: Science ac-
University, Guangzhou, China. He is a Senior Member of tivities and opportunities for agriculture and land monitoring,”
the IEEE. in Proc. IEEE Int. Geoscience and Remote Sensing Symp., July 2013,
Wenzhi Liao ([email protected]) is with the De- pp. 4558–4561.
partment of Telecommunications and Information Process- [9] T. Feingersh and E. B. Dor, “Shalom: A commercial hyperspectral
ing, Ghent University, Belgium. He is a Senior Member of space mission,” in Optical Payloads for Space Missions, S.-E. Qian,
the IEEE. Ed. Chichester, U.K.: Wiley, 2015, ch. 11, pp. 247–263.

december 2017 ieee Geoscience and remote sensing magazine 67

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 [10] R. Green, G. Asner, S. Ungar, and R. Knox, “NASA mission to [27] G. Chen and S.-E. Qian, “Dimensionality reduction of hyper-
measure global plant physiology and functional types,” in Proc. spectral imagery using improved locally linear embedding,” J.
Aerospace Conf., Mar. 2008, pp. 1–7. Appl. Remote Sens., vol. 1, pp. 1–10, 2007.
[11] J. M. Bioucas-Dias, A. Plaza, G. Camps-Valls, P. Scheunders, [28] P. Mitra, C. A. Murthy, and S. K. Pal, “Unsupervised feature se-
N. Nasrabadi, and J. Chanussot, “Hyperspectral remote sensing lection using feature similarity,” IEEE Trans. Pattern Anal. Mach.
data analysis and future challenges,” IEEE Geosci. Remote Sens. Intell., vol. 24, no. 3, pp. 301–312, 2002.
Mag., vol. 1, no. 2, pp. 6–36, June 2013. [29] P. Bajcsy and P. Groves, “Methodology for hyperspectral band
[12] S. Liu, L. Bruzzone, F. Bovolo, and P. Du, “Unsupervised mul- selection,” Photogramm. Eng. Remote Sens. J., vol. 70, no. 7,
titemporal spectral unmixing for detecting multiple changes in pp. 793–802, 2004.
hyperspectral images,” IEEE Trans. Geosci. Remote Sens., vol. 54, [30] J. M. Sotoca, F. Pla, and J. S. Sanchez, “Band selection in mul-
no. 5, pp. 1–16, 2016. tispectral images by minimization of dependent informa-
[13] X. Jia, B. Kuo, and M. Crawford, “Feature mining for hyper- tion,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 37, no. 2,
spectral image classification,” Proc. IEEE, vol. 101, no. 3, pp. 258–267, 2007.
pp. 676–697, Mar. 2013. [31] C. Debes, A. Merentitis, R. Heremans, J. Hahn, N. Frangiadakis,
[14] D. M. O. Kuybeda and M. Barzohar, “Rank estimation and re- T. van Kasteren, W. Liao, R. Bellens, A. Pizurica, S. Gautama,
dundancy reduction of high dimensional noisy signals with W. Philips, S. Prasad, Q. Du, and F. Pacifici, “Hyperspectral and
preservation of rare vectors,” IEEE Trans. Signal Process., vol. 55, LiDAR data fusion: Outcome of the 2013 GRSS data fusion
no. 12, pp. 5579–5592, 2007. ­contest,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens.,
[15] J. Ren, J. Zabalza, S. Marshall, and J. Zheng, “Effective feature vol. 7, no. 6, pp. 2405–2418, June 2014.
extraction and data reduction in remote sensing using hy- [32] W. Liao, M. D. Mura, J. Chanussot, and A. Pizurica, “Fusion of
perspectral imaging,” IEEE Signal Process. Mag., vol. 31, no. 4, spectral and spatial information for classification of hyperspec-
pp. 149–154, June 2014. tral remote-sensed imagery by local graph,” IEEE J. Sel. Topics Appl.
[16] J. Yang and W. Sun, “Automatic analysis of the slight change im- Earth Observ. in Remote Sens., vol. 9, no. 2, pp. 583–594, 2016.
age for unsupervised change detection,” J. Appl. Remote Sens., [33] M. Borhani and H. Ghassemian, “Kernel multivariate spec-
vol. 9, no. 1, pp. 1–24, 2015. tralspatial analysis of hyperspectral data,” IEEE J. Sel. Topics
[17] B. Somers and G. P. Asnerb, “Multi-temporal hyperspectral Appl. Earth Observ. in Remote Sens., vol. 8, no. 6, pp. 2418–2426,
mixture analysis and feature selection for invasive species map- 2015.
ping in rainforests,” Remote Sens. Environ., vol. 136, pp. 14–27, [34] L. Zhang, Q. Zhang, B. Du, X. Huang, Y. Tang, and D. Tao, “Si-
May 2013. multaneous spectral-spatial feature selection and extraction for
[18] L. Liu, N. C. Coops, N. W. Aven, and Y. Pang, “Mapping ur- hyperspectral images,” IEEE Trans. Cybern., 2017. doi: 10.1109/
ban tree species using integrated airborne hyperspectral and TCYB.2016.2605044.
LiDAR remote sensing data,” Remote Sens. Environ., vol. 200, [35] Q. Wang, J. Lin, and Y. Yuan, “Salient band selection for hy-
pp. 170–182, 2017. perspectral image classification via manifold ranking,” IEEE
[19] L. van der Maaten, E. Postma, and J. van den Herik, “Dimension- Trans. Neural Netw. Learn. Syst., vol. 27, no. 6, pp. 1279–1289,
ality reduction: A comparative review,” Tilburg University, The 2016.
Netherlands, Tech. Rep., TiCC-TR 2009-005, 2009. [36] B. Rasti, M. O. Ulfarsson, and J. R. Sveinsson, “Hyperspectral fea-
[20] I. Jolliffe, Principal Component Analysis. New York: Springer-Ver- ture extraction using total variation component analysis,” IEEE
lag, 1986. Trans. Geosci. Remote Sens., vol. 54, no. 12, pp. 6976–6985, Dec.
[21] A. Hyvrinen, J. Karhunen, and E. Oja, Independent Component 2016.
Analysis. New York: Wiley, 2001. [37] C.-I. Chang and H. Ren, “An experiment-based quantitative and
[22] A. Green, M. Berman, P. Switzer, and M. Craig, “A transforma- comparative analysis of target detection and image classification
tion for ordering multispectral data in terms of image quality algorithms for hyperspectral imagery,” IEEE Trans. Geosci. Remote
with implications for noise removal,” IEEE Trans. Geosci. Remote Sens., vol. 38, no. 2, pp. 1044–1063, 2000.
Sens., vol. 26, no. 1, pp. 65–74, Jan. 1988. [38] B. Kuo and D. Landgrebe, “Nonparametric weighted feature
[23] X. F. He, D. Cai, S. C. Yan, and H. J. Zhang, “Neighborhood pre- extraction for classification,” IEEE Trans. Geosci. Remote Sens.,
serving embedding,” in Proc. 10th IEEE Int. Conf. Computer Vision, vol. 42, no. 5, pp. 1096–1105, May 2004.
vol. 2, 2005, pp. 1208–1213. [39] P. H. Swain and S. M. Davis, Remote Sensing: The Quantitative Ap-
[24] X. F. He and P. Niyogi, Locality Preserving Projections. Cambridge, proach. New York: McGraw-Hill, 1978.
MA: MIT Press, 2004. [40] R. Battiti, “Using mutual information for selecting features in
[25] L. Zhang, Y. Zhong, B. Huang, J. Gong, and P. Li, “Dimensionality supervised neural net learning,” IEEE Trans. Neural Netw., vol. 5,
reduction based on clonal selection for hyperspectral imagery,” no. 4, pp. 537–550, July 1994.
IEEE Trans. Geosci. Remote Sens., vol. 45, no. 12, pp. 4172–4186, [41] Q. Du, N. Younan, R. King, and V. Shah, “On the performance
Dec. 2007. evaluation of pan-sharpening techniques,” IEEE Geosci. Remote
[26] A. Nielsen, “Kernel maximum autocorrelation factor and mini- Sens. Lett., vol. 4, no. 4, pp. 518–522, Oct. 2007.
mum noise fraction transformations,” IEEE Trans. Image Process., [42] T. V. Bandos, L. Bruzzone, and G. Camps-Valls, “Classification
vol. no. 203, pp. 612–624, Mar. 2011. of hyperspectral images with regularized linear discriminant

68 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 a­ nalysis,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 3, pp. 862– [58] D. Whitley, “A genetic algorithm tutorial,” Statist. Computing,
873, 2009. vol. 4, no. 2, pp. 65–85, 1994.
[43] B. C. Kuo, C. W. Chang, C. C. Hung, and H. P. Wang, “A modi- [59] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in
fied nonparametric weight feature extraction using spatial and Proc. IEEE Int. Conf. Neural Networks, vol. IV, 1995, pp. 1942–
spectral information,” in Proc. Int. Geoscience and Remote Sensing 1948.
Symp., 2006, pp. 172–175. [60] M. Pedergnana, P. R. Marpu, M. D. Mura, J. A. Benediktsson, and
[44] B. Kuo, C. Li, and J. Yang, “Kernel nonparametric weighted L. Bruzzone, “A novel technique for optimal feature selection
feature extraction for hyperspectral image classification,” IEEE in attribute profiles based on genetic algorithms,” IEEE Trans.
Trans. Geosci. Remote Sens., vol. 47, no. 4, pp. 1139–1155, Apr. Geosci. Remote Sens., vol. 51, no. 6, pp. 3514–3528, 2013.
2009. [61] P. Ghamisi, M. S. Couceiro, and J. A. Benediktsson, “A novel
[45] L. Bruzzone and C. Persello, “A novel approach to the selection feature selection approach based on FODPSO and SVM,” IEEE
of spatially invariant features for the classification of hyperspec- Trans. Geosci. Remote Sens., vol. 53, no. 5, pp. 2935–2947, 2015.
tral images with improved generalization capability,” IEEE Trans. [62] G. Hinton and R. Salakhutdinov, “Reducing the dimensional-
Geosci. Remote Sens., vol. 47, no. 9, pp. 3180–3191, 2009. ity of data with neural networks,” Science, vol. 313, no. 5786,
[46] H. Peng, F. Long, and C. Ding, “Feature selection based on mutu- pp. 504–507, 2006.
al information: Criteria of max-dependency, max-relevance, and [63] K. Fukushima, “Neocognitron: A self-organizing neural net-
min-redundancy,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, work model for a mechanism of pattern recognition unaffected
no. 8, pp. 1226–1238, 2005. by shift in position,” Biol. Cybern., vol. 36, no. 4, pp. 193–202,
[47] P. A. Estevez, T. Michel, C. A. Perez, and J. M. Zurada, “Normal- 1980.
ized mutual information feature selection,” IEEE Trans. Neural [64] W. Zhao and S. Du, “Spectral-spatial feature extraction for hy-
Netw., vol. 20, no. 2, pp. 189–201, 2009. perspectral image classification: A dimension reduction and
[48] W. Li, S. Prasad, J. E. Fowled, and L. M. Bruce, “Locality-preserv- deep learning approach,” IEEE Trans. Geosci. Remote Sens., vol. 54,
ing dimensionality reduction and classification for hyperspec- no. 8, pp. 4544–4554, 2016.
tral image analysis,” IEEE Trans. Geosci. Remote Sens., vol. 50, [65] P. Ghamisi, Y. Chen, and X. Zhu, “A self-improving convolution
no. 4, pp. 1185–1198, 2012. neural network for the classification of hyperspectral data,”
[49] M. Sugiyama, “Dimensionality reduction of multimodal labeled IEEE Geosci. Remote Sens. Lett., vol. 13, no. 10, pp. 1537–1541,
data by local Fisher discriminant analysis,” J. Mach. Learn. Res., 2016.
vol. 8, no. 5, pp. 1027–1061, 2007. [66] A. Blum and T. Mitchell, “Combining labeled and unlabeled
[50] Y. Zhou, J. Peng, and C. L. P. Chen, “Dimension reduction using data with co-training,” in Proc. 11th Annu. Conf. Computational
spatial and spectral regularized local discriminant embedding Learning Theory, 1998, pp. 92–100.
for hyperspectral image classification,” IEEE Trans. Geosci. Remote [67] L. Bruzzone, M. Chi, and M. Marconcini, “A novel transduc-
Sens., vol. 53, no. 2, pp. 1082–1095, 2015. tive SVM for semisupervised classification of remote-sens-
[51] X. Cao, T. Xiong, and L. Jiao, “Supervised band selection using ing images,” IEEE Trans. Geosci. Remote Sens., vol. 44, no. 11,
local spatial information for hyperspectral image,” IEEE Geosci. pp. 3363–3373, 2006.
Remote Sens. Lett., vol. 13, no. 3, pp. 329–333, 2016. [68] G. Camps-Valls, T. Bandos, and D. Zhou, “Semisupervised
[52] Y. Dong, B. Du, L. Zhang, and L. Zhang, “Dimensionality reduc- graph-based hyperspectral image classification,” IEEE Trans.
tion and classification of hyperspectral images using ensemble Geosci. Remote Sens, vol. 45, no. 10, pp. 3044–3054, 2007.
discriminative local metric learning,” IEEE Trans. Geosci. Remote [69] D. Cai, X. F. He, and J. W. Han, “Semi-supervised learning litera-
Sens., vol. 55, no. 5, pp. 2509–2524, May 2017. ture survey,” in Proc. IEEE 11th Int. Conf. Computer Vision, 2008,
[53] N. H. Ly, D. Qian, and J. E. Fowler, “Sparse graph-based discrimi- pp. 1–7.
nant analysis for hyperspectral imagery,” IEEE Trans. Geosci. Re- [70] G. Chen and S.-E. Qian, “Denoising of hyperspectral imagery us-
mote Sens., vol. 52, no. 7, pp. 3872–3884, 2014. ing principal component analysis and wavelet shrinkage,” IEEE
[54] Z. Xue, P. Du, J. Li, and H. Su, “Simultaneous sparse graph em- Trans. Geosci. Remote Sens., vol. 49, no. 3, pp. 973–980, 2011.
bedding for hyperspectral image classification,” IEEE Trans. [71] J. Bai, S. Xiang, L. Shi, and C. Pan, “Semisupervised pair-wise
Geosci. Remote Sens., vol. 53, no. 11, pp. 1–20, 2015. band selection for hyperspectral images,” IEEE J. Sel. Topics
[55] Q. Zhang, Y. Tian, Y. Yang, and C. Pan, “Automatic spatial-spec- Appl. Earth Observ. in Remote Sens., vol. 8, no. 6, pp. 2798–2813,
tral feature selection for hyperspectral image via discriminative 2015.
sparse multimodal learning,” IEEE Trans. Geosci. Remote Sens., [72] M. Sugiyama, T. Ide, S. Nakajima, and J. Sese, “Semi-supervised
vol. 53, no. 1, pp. 261–279, 2015. local Fisher discriminant analysis for dimensionality reduction,”
[56] K. Yu, T. Zhang, and Y. Gong, “Nonlinear learning using local Mach. Learn., vol. 78, pp. 35–61, 2010.
coordinate coding,” in Proc. Advances in Neural Information Pro- [73] W. Liao, A. Pizurica, P. Scheunders, W. Philips, and Y. Pi, “Semi-
cessing Systems, 2009, pp. 2223–2231. supervised local discriminant analysis for feature extraction in
[57] W. He, H. Zhang, L. Zhang, W. Philips, and W. Liao, “Weight- hyperspectral images,” IEEE Trans. Geosci. Remote Sens., vol. 51,
ed sparse graph based dimensionality reduction for hyper- no. 1, pp. 184–198, 2013.
spectral images,” IEEE Geosci. Remote Sens. Lett., vol. 13, no. 5, [74] R. Luo, W. Liao, X. Huang, Y. Pi, and W. Philips, “Feature ex-
pp. 686–690, 2016. traction of hyperspectral images with semi-supervised graph

december 2017 ieee Geoscience and remote sensing magazine 69

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 l­ earning,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., [89] F. Melgani and L. Bruzzone, “Classification of hyperspectral re-
vol. 9, no. 9, pp. 4389–4399, 2016. mote sensing images with support vector machines,” IEEE Trans.
[75] F. Luo, H. Huang, Z. Ma, and J. Liu, “Semisupervised sparse Geosci. Remote Sens., vol. 42, no. 8, pp. 1778–1790, 2004.
manifold discriminative analysis for feature extraction of hyper- [90] P. O. Gislason, J. A. Benediktsson, and J. R. Sveinsson, “Random
spectral images,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 10, forests for land cover classification,” Patt. Recog. Lett., vol. 27,
pp. 6197–6211, 2016. no. 4, pp. 294–300, 2006.
[76] D. Tuia, M. Volpi, M. Trolliet, and G. Camps-Valls, “Semisu- [91] J. Xia, P. Du, X. He, and J. Chanussot, “Hyperspectral remote
pervised manifold alignment of multimodal remote sens- sensing image classification based on rotation forest,” IEEE
ing images,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 12, Geosci. Remote Sens. Lett., vol. 11, no. 1, pp. 239–243, Jan. 2014.
pp. 7708–7720, 2014. [92] T. Rainforth and F. Wood. (2017, Aug.). Canonical correlation for-
[77] G. Matasci, M. Volpi, M. Kanevski, L. Bruzzone, and D. Tuia, ests. arXiv. [Online]. Available: https://arxiv.org/abs/1507.05444.
“Semisupervised transfer component analysis for domain adap- [93] J. Xia, N. Yokoya, and A. Iwasaki, “Hyperspectral image classi-
tation in remote sensing image classification,” IEEE Trans. Geosci. fication with canonical correlation forests,” IEEE Trans. Geosci.
Remote Sens., vol. 53, no. 7, pp. 3550–3564, 2015. Remote Sens., vol. 55, no. 1, pp. 421–431, Jan. 2017.
[78] K. Uto, Y. Kosugi, and G. Saito, “Semi-supervised hyperspectral [94] J. A. Benediktsson, P. H. Swain, and O. K. Ersoy, “Neural network
subspace learning based on a generalized eigenvalue problem approaches versus statistical methods in classification of mul-
for regression and dimensionality reduction,” IEEE J. Sel. Topics tisource remote sensing data,” IEEE Trans. Geosci. Remote Sens.,
Appl. Earth Observ. in Remote Sensing, vol. 7, no. 6, pp. 2583– vol. 28, no. 4, pp. 540–552, 1990.
2599, 2014. [95] M. Pal, “Extreme-learning-machine-based land cover classifica-
[79] F. Morsier, M. Borgeaud, V. Gass, J. Thiran, and D. Tuia, “Kernel tion,” Int. J. Remote Sens., vol. 30, no. 14, pp. 3835–3841, 2009.
low rank and sparse graph for unsupervised and semi-super- [96] M. Pal, A. E. Maxwell, and T. A. Warner, “Kernel-based extreme
vised classification of hyperspectral images,” IEEE Trans. Geosci. learning machine for remote-sensing image classification,” Re-
Remote Sens., vol. 54, no. 6, pp. 3410–3420, 2016. mote Sens. Lett., vol. 4, no. 9, pp. 853–862, 2013.
[80] J. Benediktsson, J. Palmason, and J. Sveinsson, “Classification [97] J. Li, J. Bioucas-Dias, and A. Plaza, “Semi-supervised hyperspec-
of hyperspectral data from urban areas based on extended mor- tral image segmentation using multinomial logistic regression
phological profiles,” IEEE Trans. Geosci. Remote Sens., vol. 43, with active learning,” IEEE Trans. Geosci. Remote Sens., vol. 48,
no. 3, pp. 480–490, Mar. 2005. no. 11, pp. 4085–4098, 2010.
[81] W. Liao, M. D. Mura, J. Chanussot, R. Bellens, and W. Philips, [98] Y. Chen, Z. Lin, X. Zhao, G. Wang, and Y. Gu, “Deep learning-
“Morphological attribute profiles with partial reconstruction,” based classification of hyperspectral data,” IEEE J. Sel. Topics
IEEE Trans. Geosci. Remote Sens., vol. 54, no. 3, pp. 1738–1756, Appl. Earth Observ. in Remote Sens., vol. 7, no. 6, pp. 2094–2107,
2016. 2014.
[82] W. Liao, J. Chanussot, M. D. Mura, X. Huang, R. Bellens, [99] C. Tao, H. Pan, Y. Li, and Z. Zou, “Unsupervised spectralspatial
S. Gautama, and W. Philips, “Promoting partial reconstruction feature learning with stacked sparse autoencoder for hyper-
for the morphological analysis of very high resolution urban spectral imagery classification,” IEEE Geosci. Remote Sens. Lett.,
remote sensing images,” IEEE Geosci. Remote Sens. Mag., vol. 5, vol. 12, no. 12, pp. 2438–2442, 2015.
no. 2, pp. 8–28, 2017. [100] Y. Chen, X. Zhao, and X. Jia, “Spectral-spatial classification
[83] T. Blaschke, “Object based image analysis for remote sensing,” of hyperspectral data based on deep belief network,” IEEE
ISPRS J. Photogramm. Remote Sens., vol. 65, no. 1, pp. 2–16, J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 8, no. 6,
2010. pp. 2381–2292, 2015.
[84] Y. Chen, H. Jiang, C. Li, X. Jia, and P. Ghamisi, “Deep feature [101] G. Moser and S. B. Serpico, “Combining support vector ma-
extraction and classification of hyperspectral images based on chines and Markov random fields in an integrated framework
convolutional neural networks,” IEEE Trans. Geosci. Remote Sens., for contextual image classification,” IEEE Trans. Geosci. Remote
vol. 54, no. 10, pp. 6232–6251, Oct. 2016. Sens., vol. 51, no. 5, pp. 2734–2752, 2013.
[85] P. Ghamisi, A.-R. Ali, M. Couceiro, and J. Benediktsson, “A novel [102] P. Ghamisi, J. A. Benediktsson, and M. O. Ulfarsson, “Spectral–
evolutionary swarm fuzzy clustering approach for hyperspectral spatial classification of hyperspectral images based on hidden
imagery,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., Markov random fields,” IEEE Trans. Geosci. Remote Sens., vol. 52,
vol. 8, no. 6, pp. 2447–2456, 2015. no. 5, pp. 2565–2574, 2014.
[86] Y. Chen, S. Ma, X. Chen, and P. Ghamisi, “Hyperspectral data [103] J. Xia, J. Chanussot, P. Du, and X. He, “Spectral-spatial clas-
clustering based on density analysis ensemble,” Remote Sens. sification for hyperspectral data using rotation forests with lo-
Lett., vol. 8, no. 2, pp. 194–203, 2017. cal feature extraction and markov random fields,” IEEE Trans.
[87] Q. Jackson and D. Landgrebe, “Adaptive Bayesian contextual Geosci. Remote Sens., vol. 53, no. 5, pp. 2532–2546, 2015.
classification based on Markov random fields,” IEEE Trans. [104] R. Kemker and C. Kanan, “Self-taught feature learning for hy-
­Geosci. Remote Sens., vol. 40, no. 11, pp. 2454–2463, 2002. perspectral image classification,” IEEE Trans. Geosci. Remote
[88] P. Ghamisi, J. Plaza, Y. Chen, J. Li, and A. Plaza, “Advanced spec- Sens., vol. 55, no. 5, pp. 2693–2705, May 2017.
tral classifiers for hyperspectral images: A review,” IEEE Geosci. [105] P. G ha m isi, M . S. C ouce i ro, J. A . B e ned i k t s son , a nd
Remote Sens. Mag., vol. 5, no. 1, pp. 8–32, Mar. 2017. N. M. F. Ferreira, “An efficient method for segmentation of

70 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 i­ mages based on fractional calculus and natural selection,” Ex- [119] P. Ghamisi, R. Souza, J. A. Beneiktsson, X. X. Zhu, L. Rittner,
pert Syst. Applicat., vol. 39, no. 16, pp. 12 407–12 417, 2012. and R. Lotufo, “Extinction profiles for the classification of re-
[106] P. Ghamisi, M. S. Couceiro, F. M. Martins, and J. A. Benediktsson, mote sensing data,” IEEE Trans. Geosci. Remote Sens., vol. 54,
“Multilevel image segmentation approach for remote sensing no. 10, pp. 5631–5645, 2016.
images based on fractional-order Darwinian particle swarm [120] P. Ghamisi, R. Souza, J. A. Benediktsson, L. Rittner, R. Lotufo,
optimization,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 5, and X. X. Zhu, “Hyperspectral data classification using extend-
pp. 2382–2394, 2014. ed extinction profiles,” IEEE Geosci. Remote Sens. Lett., vol. 13,
[107] M. Fauvel, Y. Tarabalka, J. A. Benediktsson, J. Chanussot, no. 11, pp. 1641–1645, 2016.
and J. C. Tilton, “Advances in spectral-spatial classification [121] G. Camps-Valls, L. Gomez-Chova, J. Munoz-Mari, J. Vila-Frances,
of hyperspectral images,” Proc. IEEE, vol. 101, no. 3, pp. 652– and J. Calpe-Maravilla, “Composite kernels for hyperspectral
675, 2013. image classfication,” IEEE Geosci. Remote Sens. Lett., vol. 3, no. 1,
[108] P. Ghamisi, M. Couceiro, M. Fauvel, and J. A. Benediktsson, pp. 93–97, 2006.
“Integration of segmentation techniques for classification of [122] J. Li, P. Marpu, A. Plaza, J. Bioucas-Dias, and J. A. Benediktsson,
hyperspectral images,” IEEE Geosci. Remote Sens. Lett., vol. 11, “Generalized composite kernel framework for hyperspectral
no. 1, pp. 342–346, Jan. 2014. image classification,” IEEE Trans. Geosci. Remote Sens., vol. 51,
[109] Y. Tarabalka, M. Fauvel, J. Chanussot, and J. A. Benediktsson, no. 9, pp. 4816–4829, 2013.
“Segmentation and classification of hyperspectral images us- [123] D. Tuia, G. Camps-Valls, G. Matasci, and M. Kanevski, “Learn-
ing minimum spanning forest grown from automatically se- ing relevant image features with multiple-kernel classification,”
lected markers,” IEEE Trans. Syst. Man, Cybern. B, Cybern., 2010. IEEE Trans. Geosci. Remote Sens., vol. 48, no. 10, pp. 3780–3791,
[110] N. Otsu, “A threshold selection method from gray-level histo- Oct. 2010.
gram,” IEEE Trans. Syst., Man, Cybern., vol. 9, pp. 62–66, 1979. [124] J. Li, X. Huang, P. Gamba, J. M. Bioucas-Dias, L. Zhang,
[111] J. A. Benediktsson, J. A. Palmason, and J. R. Sveinsson, “Clas- J. A. Benediktsson, and A. Plaza, “Multiple feature learning for
sification of hyperspectral data from urban areas based on hyperspectral image classification,” IEEE Trans. Geosci. Remote
extended morphological profiles,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 3, pp. 1592–1606, Mar. 2015.
Sens., vol. 43, no. 3, pp. 480–491, 2005. [125] Z. Zhang, E. Pasolli, M. M. Crawford, and J. C. Tilton, “An
[112] M. Dalla Mura, J. A. Benediktsson, B. Waske, and L. Bruzzone, active learning framework for hyperspectral image classifica-
“Morphological attribute profiles for the analysis of very high tion using hierarchical segmentation,” IEEE J. Sel. Topics Appl.
resolution images,” IEEE Trans. Geosci. Remote Sens., vol. 48, Earth Observ. in Remote Sens., vol. 9, no. 2, pp. 640–654, Feb.
no. 10, pp. 3747– 3762, 2010. 2016.
[113] P. Ghamisi, M. Dalla Mura, and J. A. Benediktsson, “A survey [126] M. M. Crawford, D. Tuia, and H. L. Yang, “Active learning: Any
on spectral–spatial classification techniques based on attri- value for classification of remotely sensed data?” Proc. IEEE,
bute profiles,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 5, vol. 101, no. 3, pp. 593–608, Mar. 2013.
pp. 2335–2353, 2015. [127] D. Tuia, M. Volpi, L. Copa, M. Kanevski, and J. Munoz-Mari,
[114] M. Pedergnana, P. R. Marpu, M. Dalla Mura, J. A. Benediktsson, “A survey of active learning algorithms for supervised remote
and L. Bruzzone, “A novel technique for optimal feature selec- sensing image classification,” IEEE J. Sel. Topics Signal Process.,
tion in attribute profiles based on genetic algorithms,” IEEE Trans. vol. 5, no. 3, pp. 606–617, June 2011.
Geosci. Remote Sens., vol. 51, no. 6, pp. 3514–3528, June 2013. [128] L. Wan, K. Tang, M. Li, Y. Zhong, and A. K. Qin, “Collaborative
[115] P. Ghamisi, J. A. Benediktsson, G. Cavallaro, and A. Plaza, “Au- active and semisupervised learning for hyperspectral remote
tomatic framework for spectral-spatial classification based on sensing image classification,” IEEE Trans. Geosci. Remote Sens.,
supervised feature extraction and morphological attribute pro- vol. 53, no. 5, pp. 2384–2396, May 2015.
files,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, [129] Y. Chen, N. M. Nasrabadi, and T. D. Tran, “Hyperspectral image
no. 6, pp. 2147–2160, 2014. classification using dictionary-based sparse represenation,”
[116] P. Ghamisi, J. A. Benediktsson, and J. R. Sveinsson, “Automatic IEEE Trans. Geosci. Remote Sens., vol. 49, no. 10, pp. 3973–3985,
spectral-spatial classification framework based on attribute Oct. 2011.
profiles and supervised feature extraction,” IEEE Trans. Geosci. [130] Z. Lai, W. K. Wong, Y. Xu, C. Zhao, and M. Sun. (2014, Oct.).
Remote Sens., vol. 52, no. 5, pp. 5771–5782, 2014. Sparse alignment for robust tensor learning. IEEE Trans. Neural
[117] B. Song, J. Li, M. Dalla Mura, P. Li, A. Plaza, J. M. Bioucas-Dias, Netw. Learn. Syst. [Online]. 25(10), pp. 1779–1792. Available:
J. A. Benediktsson, and J. Chanussot, “Remotely sensed image http://dx.doi.org/10.1109/TNNLS.2013.2295717
classification using sparse representations of morphological at- [131] A. Castrodad, Z. Xing, J. Greer, E. Bosch, L. Carin, and G. Sapiro,
tribute profiles,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 8, “Learning discriminative sparse representations for modeling,
pp. 5122–5136, 2014. source separation, and mapping of hyperspectral imagery,”
[118] X. Huang, X. Guan, J. A. Benediktsson, L. Zhang, J. Li, A. Plaza, IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp. 4263–4281,
and M. Dalla Mura, “Multiple morphological profiles from Dec. 2011.
multicomponent base images for hyperspectral image classifi- [132] A. Goetz, G. Vane, J. Solomon, and B. Rock, “Imaging spec-
cation,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, trometry for Earth remote sensing,” Science, vol. 228, no. 4704,
no. 12, pp. 4653–4669, Dec. 2014. pp. 1147–1153, June 1985.

december 2017 ieee Geoscience and remote sensing magazine 71

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 [133] R. Heylen, M. Parente, and P. Gader, “A review of nonlinear for hyperspectral unmixing,” IEEE Trans. Signal Process., vol. 57,
hyperspectral unmixing methods,” IEEE J. Sel. Topics Appl. no. 11, pp. 4418–4432, Nov. 2009.
Earth Observ. in Remote Sens., vol. 7, no. 6, pp. 1844–1868, [148] J. Li, A. Agathos, D. Zaharie, J. M. Bioucas-Dias, A. Plaza, and
June 2014. X. Li, “Minimum volume simplex analysis: A fast algorithm
[134] J. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, for linear hyperspectral unmixing,” IEEE Trans. Geosci. Remote
P. Gader, and J. Chanussot, “Hyperspectral unmixing over- Sens., vol. 53, no. 9, pp. 5067–5082, Sept. 2015.
view: Geometrical, statistical, and sparse regression-based ap- [149] E. M. T. Hendrix, I. Garcia, J. Plaza, G. Martin, and A. Plaza, “A
proaches,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., new minimum-volume enclosing algorithm for endmember
vol. 5, no. 2, pp. 354–379, 2012. identification and abundance estimation in hyperspectral data,”
[135] C.-I. Chang and Q. Du, “Estimation of number of spectrally IEEE Trans. Geosci. Remote Sens., vol. 50, no. 7, pp. 2744–2757,
distinct signal sources in hyperspectral imagery,” IEEE Trans. July 2012.
Geosci. Remote Sens., vol. 42, no. 3, pp. 608–619, 2004. [150] C. Chen and X. Zhang, “Independent component analysis for
[136] J. Bioucas-Dias and J. Nascimento, “Hyperspectral subspace remote sensing study,” in Proc. SPIE Image and Signal Processing
identification,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 8, Remote Sensing V, vol. 3871, 1999, pp. 150–158.
pp. 2435–2445, 2008. [151] J. Nascimento and J. Bioucas-Dias, “Hyperspectral unmix-
[137] B. Luo, J. Chanussot, S. Doute, and L. Zhang, “Empirical au- ing based on mixtures of Dirichlet components,” IEEE Trans.
tomatic estimation of the number of endmembers in hyper- Geosci. Remote Sens., vol. 50, no. 3, 2012.
spectral images,” IEEE Geosci. Remote Sens. Lett., vol. 10, no. 1, [152] A. Plaza, P. Martinez, R. Perez, and J. Plaza, “Spatial/spectral
pp. 24–28, Jan. 2013. endmember extraction by multidimensional morphological
[138] O. Eches, N. Dobigeon, and J. Y. Tourneret, “Estimating the operations,” IEEE Trans. Geosci. Remote Sens., vol. 40, no. 9,
number of endmembers in hyperspectral images using the pp. 2025–2041, 2002.
normal compositional model and a hierarchical Bayesian algo- [153] D. M. Rogge, B. Rivard, J. Zhang, A. Sánchez, J. Harris, and
rithm,” IEEE J. Sel. Topics Signal Process., vol. 4, no. 3, pp. 582–591, J. Feng, “Integration of spatial–spectral information for the
June 2010. improved extraction of endmembers,” Remote Sens. Environ.,
[139] J. Boardman, F. A. Kruse, and R. O. Green, “Mapping target vol. 110, no. 3, pp. 287–303, 2007.
signatures via partial unmixing of AVIRIS data,” in Proc. JPL [154] M. Zortea and A. Plaza, “Spatial preprocessing for endmember
Airborne Earth Science Workshop, 1995, pp. 23–26. extraction,” IEEE Trans. Geosci. Remote Sens., vol. 47, pp. 2679–2693,
[140] M. E. Winter, “N-FINDR: An algorithm for fast autonomous 2009.
spectral end-member determination in hyperspectral data,” [155] G. Martin and A. Plaza, “Region-based spatial preprocessing
in Proc. Society of Photographic Instrumentation Engineers, Imaging for endmember extraction and spectral unmixing,” IEEE
Spectrometry V, 2003, vol. 3753, pp. 266–277. Geosci. Remote Sens. Lett., vol. 8, no. 4, pp. 745–749, July
[141] J. C. Harsanyi and C.-I. Chang, “Hyperspectral image classifi- 2011.
cation and dimensionality reduction: An orthogonal subspace [156] G. Martin and A. Plaza, “Spatial-spectral preprocessing prior to
projection approach,” IEEE Trans. Geosci. Remote Sens., vol. 32, endmember identification and unmixing of remotely sensed
no. 4, pp. 779–785, 1994. hyperspectral data,” IEEE J. Sel. Topics Appl. Earth Observ. in Re-
[142] J. Nascimento and J. Bioucas-Dias, “Vertex component analy- mote Sensing, vol. 5, no. 2, pp. 380–395, Apr. 2012.
sis: A fast algorithm to unmix hyperspectral data,” IEEE Trans. [157] C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detec-
Geosci. Remote Sens., vol. 43, no. 4, pp. 898–910, 2005. tion and Classification. Norwell, MA: Kluwer, 2003.
[143] B. Somers, S. Delalieux, J. Stuckens, W. W. Verstraeten, and [158] A. Halimi, P. Honeine, and J. M. Bioucas-Dias, “Hyperspectral
P. Coppin, “A weighted linear spectral mixture analysis approach unmixing in presence of endmember variability, nonlinear-
to address endmember variability in agricultural production sys- ity, or mismodeling effects,” IEEE Trans. Image Process., vol. 25,
tems,” Int. J. Remote Sens., vol. 30, no. 1, pp. 139–147, 2009. no. 10, pp. 4565–4579, 2016.
[144] M. Berman, H. Kiiveri, R. Lagerstrom, A. Ernst, R. Dunne, and [159] P.-A. Thouvenin, N. Dobigeon, and J.-Y. Tourneret, “Hyper-
J. F. Huntington, “ICE: a statistical approach to identifying end- spectral unmixing with spectral variability using a perturbed
members in hyperspectral images,” IEEE Trans. Geosci. Remote linear mixing model,” IEEE Trans. Signal Process., vol. 64, no. 2,
Sens., vol. 42, no. 10, pp. 2085–2095, 2004. pp. 525–538, 2016.
[145] L. Miao and H. Qi, “Endmember extraction from highly mixed [160] L . Dr umet z, M.-A . Vegan zones, S. Henrot, R . Phly po,
data using minimum volume constrained nonnegative matrix J. Chanussot, and C. Jutten, “Blind hyperspectral unmixing us-
factorization,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 3, ing an extended linear mixing model to address spectral vari-
pp. 765–777, 2007. ability,” IEEE Trans. Image Process., vol. 25, no. 8, pp. 3890–3905,
[146] A. Zare and P. Gader, “Sparsity promoting iterated con- 2016.
strained endmember detection in hyperspectral imagery,” [161] M. D. Iordache, J. Bioucas-Dias, and A. Plaza, “Sparse unmix-
IEEE Geosci. Remote Sens. Lett., vol. 4, no. 3, pp. 446–450, ing of hyperspectral data,” IEEE Trans. Geosci. Remote Sens.,
July 2007. vol. 49, no. 6, pp. 2014–2039, 2011.
[147] T. H. Chan, C. Y. Chi, Y. M. Huang, and W. K. Ma, “A convex [162] M.-D. Iordache, J. Bioucas-Dias, and A. Plaza, “Total variation
analysis-based minimum-volume enclosing simplex algorithm spatial regularization for sparse hyperspectral unmixing,”

72 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 IEEE Trans. Geosci. Remote Sens., vol. 50, no. 11, pp. 4484–4502, fluctuation mode,” IEEE Trans. Signal Process., vol. 61, no. 2,
Nov. 2012. pp. 480–492, 2013.
[163] M.-D. Iordache, J. Bioucas-Dias, and A. Plaza, “Music-CSR: [178] L. Zhang, B. Wu, B. Huang, and P. Li, “Nonlinear estimation of
Multiple signal classification collaborative sparse regression subpixel proportion via kernel least square regression,” Int. J.
for hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., Remote Sens., vol. 28, no. 18, pp. 4157–4172, 2007.
vol. 52, no. 7, pp. 4364–4382, 2013. [179] M. Brown, H. G. Lewis, and S. R. Gunn, “Linear spectral mix-
[164] M.-D. Iordache, J. Bioucas-Dias, and A. Plaza, “Collaborative ture models and support vector machines for remote sensing,”
sparse regression for hyperspectral unmixing,” IEEE Trans. IEEE Trans. Geosci. Remote Sens., vol. 38, no. 5, pp. 2346–2360,
Geosci. Remote Sens., vol. 52, no. 1, pp. 341–354, 2014. 2000.
[165] J. Li, J. M. Bioucas-Dias, A. Plaza, and L. Liu, “Robust collab- [180] L. Wang and X. Jia, “Integration of soft and hard classifications
orative nonnegative matrix factorization for hyperspectral using extended support vector machines,” IEEE Trans. Geosci.
unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 10, Remote Sens., vol. 6, no. 3, pp. 543–547, 2009.
pp. 6076–6090, Oct. 2016. [181] G. Foody, “Relating the land-cover composition of mixed pixels
[166] N. Dobigeon, J.-Y. Tourneret, C. Richard, J. C. M. Bermudez, to artificial neural network classification output,” Photogramm.
S. McLaughlin, and A. O. Hero, “Nonlinear unmixing of hyper- Eng. Remote Sens., vol. 62, no. 5, pp. 491–499, 1996.
spectral images: Models and algorithms,” IEEE Signal Process. [182] G. Foody, R. Lucas, P. Curran, and M. Honzak, “Non-linear
Mag., vol. 31, no. 1, pp. 82–94, 2014. mixture modeling without end-members using an artificial
[167] J. Chen, C. Richard, and P. Honeine, “Nonlinear estimation neural network,” Int. J. Remote Sens., vol. 18, no. 4, pp. 937–953,
of material abundances in hyperspectral images with l1-norm 1997.
spatial regularization,” IEEE Trans. Geosci. Remote Sens., vol. 52, [183] A. Baraldi, E. Binaghi, P. Blonda, P. A. Brivio, and A. Rampini,
no. 5, pp. 2654–2665, 2014. “Comparison of the multilayer perceptron with neuro-fuzzy
[168] C. Févotte and N. Dobigeon, “Nonlinear hyperspectral unmix- techniques in the estimation of cover class mixture in remotely
ing with robust nonnegative matrix factorization,” IEEE Trans. sensed data,” IEEE Trans. Geosci. Remote Sens., vol. 39, no. 5,
Image Process., vol. 24, no. 12, pp. 4810–4819, 2015. pp. 994–1005, 2001.
[169] A. Marinoni, J. Plaza, A. Plaza, and P. Gamba, “Nonlinear [184] J. Plaza, A. Plaza, R. Perez, and P. Martinez, “On the use of
hyperspectral unmixing using nonlinearity order estimation small training sets for neural network-based characterization
and polytope decomposition,” IEEE J. Sel. Topics Appl. Earth of mixed pixels in remotely sensed hyperspectral images,” Pat-
Observ. in Remote Sens., vol. 8, no. 6, pp. 2644–2654, June tern Recogn., vol. 42, no. 11, pp. 3032–3045, Nov. 2009.
2015. [185] J. Plaza, A. Plaza, R. Perez, and P. Martinez, “Joint linear/non-
[170] A. Halimi, Y. Altmann, N. Dobigeon, and J.-Y. Tourneret, “Non- linear spectral unmixing of hyperspectral image data,” in Proc.
linear unmixing of hyperspectral images using a generalized Int. Geoscience and Remote Sensing Symp., Barcelona, Spain, July
bilinear model,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, 2007, p.4037–4040.
pp. 4153–4162, Nov. 2011. [186] G. Licciardi and F. Del Frate, “Pixel unmixing in hyperspectral
[171] Y. Altmann, N. Dobigeon, S. McLaughlin, and J.-Y. Tourneret, data by means of neural networks,” IEEE Trans. Geosci. Remote
“Nonlinear spectral unmixing of hyperspectral images using Sens., vol. 49, no. 11, pp. 4163–4172, Nov. 2011.
Gaussian processes,” IEEE Trans. Signal Process., vol. 61, no. 10, [187] D. Glasner, S. Bagon, and M. Irani, “Super-resolution from
pp. 2442–2453, May 2013. a single image,” in Proc. Int. Conf. Computer Vision, 2009,
[172] N. Yokoya, J. Chanussot, and A. Iwasaki, “Nonlinear unmix- pp. 349–356.
ing of hyperspectral data using semi-nonnegative matrix [188] T. Akgun, Y. Altunbasak, and R. M. Mersereau, “Super-resolu-
factorization,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 2, tion reconstruction of hyperspectral images,” IEEE Trans. Image
pp. 1430–1437, 2014. Process., vol. 14, no. 11, pp. 1860–1875, 2005.
[173] Q. Qu, N. M. Nasrabadi, and T. D. Tran, “Abundance estima- [189] J. C.-W. Chan, J. Ma, and F. Canters, “A comparison of superreso-
tion for bilinear mixture models via joint sparse and low-rank lution reconstruction methods for multi-angle CHRIS/Proba im-
representation,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 7, ages,” in Proc. SPIE, Image and Signal Processing for Remote Sensing
pp. 4404–4423, 2014. XIV, 710904, vol. 7109, Oct. 1, 2008. doi: 10.1117/12.800256
[174] Y. Altmann, A. Halimi, N. Dobigeon, and J.-Y. Tourneret, “Su- [190] J. C.-W. Chan, J. Ma, P. Kempeneers, and F. Canters, “Super-
pervised nonlinear spectral unmixing using a post-nonlinear resolution enhancement of hyperspectral CHRIS/Proba images
mixing model for hyperspectral imagery,” IEEE Trans. Image with a thin-plate spline nonrigid transform model,” IEEE Trans.
Process., vol. 21, no. 6, pp. 3017–3025, 2012. Geosci. Remote Sens., vol. 48, no. 6, pp. 2569–2579, 2010.
[175] R. Heylen and P. Scheunders, “A multilinear mixing model for [191] J. C.-W. Chan, J. Ma, T. Van de Voorde, and F. Canters, “Pre-
nonlinear spectral unmixing,” IEEE Ttrans. Geosci. Remote Sens., liminary results of superresolution-enhanced angular hyper-
vol. 54, no. 1, pp. 240–251, 2016. spectral (CHRIS/Proba) images for land-cover classification,”
[176] B. Hapke, “Bidirectional reflectance spectroscopy: 1. Theory,” J. IEEE Geosci. Remote Sens. Lett., vol. 8, no. 6, pp. 1011–1015,
Geophys. Res., vol. 86, no. B4, pp. 3039–3054, Apr. 1981. 2011.
[177] J. Chen, C. Richard, and P. Honeine, “Nonlinear unmixing [192] L. Demarchi, J. C.-W. Chan, J. Ma, and F. Canters, “Mapping
of hyperspectral data based on a linear-mixture/nonlinear- impervious surfaces from superresolution enhanced CHRIS/

december 2017 ieee Geoscience and remote sensing magazine 73

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 Proba imagery using multiple endmember unmixing,” ISPRS J. [207] J. G. Liu, “Smoothing filter-based intensity modulation: A spec-
Photogramm. Remote Sens., vol. 72, pp. 99–112, Aug. 2012. tral preserve image fusion technique for improving spatial
[193] S. Qian and G. Chen, “Enhancing spatial resolution of hyper- details,” Int. J. Remote Sens., vol. 21, no. 18, pp. 3461–3472,
spectral imagery using sensor’s intrinsic keystone distortion,” Jan. 2000.
IEEE Trans. Geosci. Remote Sens., vol. 50, no. 12, pp. 5033–5048, [208] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva,
2012. “MTF-tailored multiscale fusion of high-resolution MS and
[194] Y. Zhao, J. Yang, Q. Zhang, L. Song, Y. Cheng, and Q. Pan, “Hy- Pan imagery,” Photogramm. Eng. Remote Sens., vol. 72, no. 5,
perspectral imagery super-resolution by sparse representation pp. 591–596, May 2006.
and spectral regularization,” EURASIP J. Adv. Signal Process., [209] S. Li and B. Yang, “A new pan-sharpening method using a com-
vol. 2011, no. 87, 2011. pressed sensing technique,” IEEE Trans. Geosci. Remote Sens.,
[195] Y. Zhao, J. Yang, and J. C.-W. Chan, “Hyperspectral imagery vol. 49, no. 2, pp. 738–746, Feb. 2011.
super-resolution by spatial–spectral joint nonlocal similarity,” [210] X. X. Zhu and R. Bamler, “A sparse image fusion algorithm
IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 7, no. 6, with application to pan-sharpening,” IEEE Trans. Geosci. Remote
pp. 2671–2679, 2014. Sens., vol. 51, no. 5, pp. 2827–2836, May 2013.
[196] R. C. Patel and M. V. Joshi, “Super-resolution of hyperspectral [211] X. Otazu, M. González-Audícana, O. Fors, and J. Núñez, “In-
images: Use of optimum wavelet filter coefficients and sparsity troduction of sensor spectral response into image fusion meth-
regularization,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 4, ods: Application to wavelet-based methods,” IEEE Trans. Geosci.
pp. 1728–1736, 2015. Remote Sens., vol. 43, no. 10, pp. 2376–2385, 2005.
[197] P. M. Atkinson, “Mapping sub-pixel boundaries from remotely [212] B. Li, R. Yang, and H. Jiang, “Remote-sensing image compres-
sensed images,” Innovations in GIS, vol. 4, pp. 166–180, Apr. sion using two-dimensional oriented wavelet transform,” IEEE
1997. Trans. Geosci. Remote Sens., vol. 49, no. 1, pp. 236–250, Jan.
[198] X. Tong, X. Zhang, J. Shan, H. Xie, and M. Liu, “Attraction-re- 2011.
pulsion model-based subpixel mapping of multi-/hyperspec- [213] L. Alparone, B. Aiazzi, S. Baronti, and A. Garzelli, Remote Sens-
tral imagery,” IEEE Trans. Geosci. Remote Sens., vol. 51, no. 5, ing Image Fusion. Boca Raton, FL: CRC, 2015.
pp. 2799–2814, 2013. [214] L. Loncan, L. B. Almeida, J. B. Dias, X. Briottet, J. Chanussot,
[199] Y. Zhang, Y. Du, F. Ling, S. Fang, and X. Li, “Example-based N. Dobigeon, S. Fabre, W. Liao, G. A. Licciardi, M. Simões,
super-resolution land cover mapping using support vector re- J. Y. Tourneret, M. A. Veganzones, G. Vivone, Q. Wei, and
gression,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., N. Yokoya, “Hyperspectral pansharpening: A review,” IEEE
vol. 7, no. 4, pp. 1271–1283, 2014. Geosci. Remote Sens. Mag., vol. 3, no. 3, pp. 27–46, Sept.
[200] Y. Zhong, Y. Wu, X. Xu, and L. Zhang, “An adaptive subpixel 2015.
mapping method based on map model and class determination [215] N. Yokoya, C. Grohnfeldt, and J. Chanussot, “Hyperspectral
strategy for hyperspectral remote sensing imagery,” IEEE Trans. and multispectral data fusion: A comparative review of the
Geosci. Remote Sens., vol. 53, no. 3, pp. 1411–1426, 2015. ­recent literature,” IEEE Geosci. Remote Sens. Mag., vol. 5, no. 2,
[201] R. Feng, Y. Zhong, Y. Wu, D. He, X. Xu, and L. Zhang, pp. 29–56, June 2017.
“Nonlocal total variation subpixel mapping for hyperspec- [216] Z. Chen, H. Pu, B. Wang, and G.-M. Jiang, “Fusion of hy-
tral remote sensing imagery,” Remote Sens., vol. 8, no. 3, p. perspectral and multispectral images: A novel framework
250, 2016. based on generalization of pan-sharpening methods,” IEEE
[202] X. Tong, X. Xu, A. Plaza, H. Xie, H. Pan, W. Cao, and D. Lv, “A Geosci. Remote Sens. Lett., vol. 11, no. 8, pp. 1418–1422, Aug.
new genetic method for subpixel mapping using hyperspectral 2014.
images,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., [217] M. Selva, B. Aiazzi, F. Butera, L. Chiarantini, and S. Baronti,
vol. 9, no. 9, pp. 4480–4491, 2016. “Hyper-sharpening: A first approach on SIM-GA data,” IEEE
[203] J. Pearlman, S. Carman, C. Segal, P. Jarecke, P. Clancy, and J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 8, no. 6,
W. Browne, “Overview of the Hyperion imaging spectrometer pp. 3008–3024, June 2015.
for the NASA EO-1 mission,” in Proc. IEEE Int. Geoscience and [218] C. Grohnfeldt, “Multi-sensor data fusion for multi- and hyper-
Remote Sensing Symp., vol. 7, 2001, pp. 3036–3038. spectral resolution enhancement based on sparse representa-
[204] W. Carper, T. M. Lillesand, and P. W. Kiefer, “The use of inten- tions,” Ph.D. dissertation, Technical University of Munich,
sity-hue-saturation transformations for merging spot panchro- Germany, 2017.
matic and multispectral image data,” Photogramm. Eng. Remote [219] M. T. Eismann and R. C. Hardie, “Hyperspectral resolution en-
Sens., vol. 56, no. 4, pp. 459–467, Apr. 1990. hancement using high-resolution multispectral imagery with
[205] C. A. Laben and B. V. Brower, “Process for enhancing the spatial arbitrary response functions,” IEEE Trans. Geosci. Remote Sens.,
resolution of multispectral imagery using pan-sharpening,” vol. 43, no. 3, pp. 455–465, Mar. 2005.
U.S. Patent 6 011 875 A, 2000. [220] O. Berné, A. Helens, P. Pilleri, and C. Joblin, “Non-negative
[206] B. Aiazzi, S. Baronti, and M. Selva, “Improving component sub- matrix factorization pansharpening of hyperspectral data: An
stitution pansharpening through multivariate regression of MS application to mid-infrared astronomy,” in Proc. IEEE Work-
+ Pan data,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 10, shop Hyperspectral Image and Signal Processing (WHISPERS), June
pp. 3230–3239, Oct. 2007. 2010, pp. 1–4.

74 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 [221] N. Yokoya, T. Yairi, and A. Iwasaki, “Hyperspectral, multispec- [236] B. Rasti, J. R. Sveinsson, and M. O. Ulfarsson, “SURE based
tral, and panchromatic data fusion based on non-negative ma- model selection for hyperspectral imaging,” in Proc. IEEE Int.
trix factorization,” in Proc. IEEE Workshop Hyperspectral Image Geoscience and Remote Sensing Symp., pp. 4636–4639, July 2014.
and Signal Processing (WHISPERS), June 2011, pp. 1–4. [237] H. K. Aggarwal and A. Majumdar. (2015). Exploiting spa-
[222] R. Kawakami, J. Wright, Y. Tai, Y. Matsushita, M. Ben-Ezra, and tiospectral correlation for impulse denoising in hyperspectral
K. Ikeuchi, “High-resolution hyperspectral imaging via matrix images. J. Electron. Imaging. [Online]. 24(1). Available: http://
factorization,” in Proc. IEEE Conf. Computer Vision and Pattern dx.doi.org/10.1117/1.JEI.24.1.013027
Recognition, June 2011, pp. 2329–2336. [238] L. Gomez-Chova, L. Alonso, L. Guanter, G. Camps-Valls,
[223] J. Bieniarz, D. Cerra, J. Avbelj, P. Reinartz, and R. Müller, “Hy- J. Calpe, and J. Moreno, “Correction of systematic spatial
perspectral image resolution enhancement based on specral noise in push-broom hyperspectral sensors: Application
unmixing and information fusion,” in Proc. ISPRS Archives of to CHRIS/PROBA images,” Appl. Optics, vol. 47, no. 28,
the Photogrammetry, Remote Sensing and Spatial Information Sci- pp. f46–f60, 2008.
ences, Sept. 2011, pp. 33–37. [239] M. di Bisceglie, R. Episcopo, C. Galdi, and S. L. Ullo, “Destrip-
[224] N. Yokoya, T. Yairi, and A. Iwasaki, “Coupled nonnegative ma- ing modis data using overlapping field-of-view method,” IEEE
trix factorization unmixing for hyperspectral and multispec- Trans. Geosci. Remote Sens., vol. 47, no. 2, pp. 637–651, Feb. 2009.
tral data fusion,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 2, [240] P. Rakwatin, W. Takeuchi, and Y. Yasuoka, “Stripe noise reduc-
pp. 528–537, Feb. 2012. tion in modis data by combining histogram matching with
[225] N. Akhtar, F. Shafait, and A. Mian, “Sparse spatio-spectral repre- facet filter,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 6,
sentation for hyperspectral image super-resolution,” in Proc. Euro- pp. 1844–1856, June 2007.
pean Conf. Computer Vision, 2014, pp. 63–78. [241] N. Acito, M. Diani, and G. Corsini, “Subspace-based striping
[226] B. Huang, H. Song, H. Cui, J. Peng, and Z. Xu, “Spatial and spec- noise reduction in hyperspectral images,” IEEE Trans. Geosci.
tral image fusion using sparse matrix factorization,” IEEE Trans. Remote Sens., vol. 49, no. 4, pp. 1325–1342, Apr. 2011.
Geosci. Remote Sens., vol. 52, no. 3, pp. 1693–1704, Mar. 2014. [242] X. Lu, Y. Wang, and Y. Yuan, “Graph-regularized low-rank
[227] C. Lanaras, E. Baltsavias, and K. Schindler, “Hyperspectral su- representation for destriping of hyperspectral images,” IEEE
per-resolution by coupled spectral unmixing,” in Proc. IEEE Int. Trans. Geosci. Remote Sens., vol. 51, no. 7, pp. 4009–4018, July
Conf. Computer Vision, Dec. 2015, pp. 3586–3594. 2013.
[228] C. Lanaras, E. Baltsavias, and K. Schindler, “Advances in hy- [243] P. Meza, J. E. Pezoa, and S. N. Torres, “Multidimensional strip-
perspectral and multispectral image fusion and spectral un- ing noise compensation in hyperspectral imaging: Exploiting
mixing,” in Proc. ISPRS Archives of the Photogrammetry, Remote hypercubes’ spatial, spectral, and temporal redundancy,” IEEE
Sensing and Spatial Information Sciences, vol. XL-3/W3, 2015, J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 9, no. 9,
pp. 451–458. pp. 4428–4441, Sept. 2016.
[229] M. Simões, J. B. Dias, L. Almeida, and J. Chanussot, “A convex [244] J. Bioucas-Dias and J. Nascimento, “Hyperspectral subspace
formulation for hyperspectral image superresolution via sub- identification,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 8,
space-based regularization,” IEEE Trans. Geosci. Remote Sens., pp. 2435–2445, Aug. 2008.
vol. 53, no. 6, pp. 3373–3388, June 2015. [245] N. Acito, M. Diani, and G. Corsini, “Signal-dependent noise
[230] Q. Wei, N. Dobigeon, and J.-Y. Tourneret, “Bayesian fusion modeling and model parameter estimation in hyperspectral
of multiband images,” IEEE J. Sel. Topics Signal Process., vol. 9, images,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 8,
no. 6, pp. 1117–1127, Sept. 2015. pp. 2957–2971, 2011.
[231] Q. Wei, N. Dobigeon, and J.-Y. Tourneret, “Fast fusion of multi- [246] B. Rasti, J. Sveinsson, and M. Ulfarsson, “Wavelet-based sparse
band images based on solving a Sylvester equation,” IEEE Trans. reduced-rank regression for hyperspectral image restoration,”
Image Process., vol. 24, no. 11, pp. 4109–4121, Nov. 2015. IEEE Trans. Geosci. Remote Sens., vol. 52, no. 10, pp. 6688–6698,
[232] M . Ve g a n z one s , M . Si mõe s , G . L icc ia rd i, N. Yokoya , Oct. 2014.
J. Bioucas-Dias, and J. Chanussot, “Hyperspectral super- [247] I. Atkinson, F. Kamalabadi, and D. Jones, “Wavelet-based hy-
resolution of locally low rank images from complementary perspectral image estimation,” in Proc. IEEE Int. Geoscience and
multisource data,” IEEE Trans. Image Process., vol. 25, no. 1, Remote Sensing Symp., vol. 2, July 2003, pp. 743–745.
pp. 274–288, Jan. 2016. [248] B. Rasti, J. R. Sveinsson, M. O. Ulfarsson, and J. A. Benediktsson,
[233] N. Yokoya, J. C. W. Chan, and K. Segl, “Potential of resolution- “Hyperspectral image denoising using 3D wavelets,” in Proc.
enhanced hyperspectral data for mineral mapping using simu- IEEE Int. Geoscience and Remote Sensing Symp., pp. 1349–1352,
lated EnMAP and Sentinel-2 images,” Remote Sens., vol. 8, no. 3, 2012.
p. 172, 2016. [249] A. Zelinski and V. Goyal, “Denoising hyperspectral imagery
[234] J. P. Kerekes and J. E. Baum, “Hyperspectral imaging system and recovering junk bands using wavelets and sparse approxi-
modeling,” Lincoln Laboratory, vol. 14, no. 1, pp. 117–130, mation,” in Proc. IEEE Int. Geoscience and Remote Sensing Symp.,
2003. pp. 387–390, Aug 2006.
[235] D. Landgrebe and E. Malaret, “Noise in remote-sensing sys- [250] B. Rasti, “Sparse hyperspectral image modeling and resto-
tems: The effect on classification error,” IEEE Trans. Geosci. Re- ration,” Ph.D. dissertation, Univ. Iceland, Reykjavik, Dec.
mote Sens., vol. GE-24, no. 2, pp. 294– 300, Mar. 1986. 2014.

december 2017 ieee Geoscience and remote sensing magazine 75

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 [251] B. Rasti, J. R. Sveinsson, M. O. Ulfarsson, and J. A. Benediktsson, [265] B. Rasti, J. R. Sveinsson, M. O. Ulfarsson, and J. A. Benediktsson,
“Hyperspectral image denoising using first order spectral “A new linear model and sparse regularization,” in Proc.
roughness penalty in wavelet domain,” IEEE J. Sel. Topics Appl. IEEE Int. Geoscience and Remote Sensing Symp., July 2013,
Earth Observ. in Remote Sens., vol. 7, no. 6, pp. 2458–2467, pp. 457–460.
June 2014. [266] B. Rasti, J. R. Sveinsson, M. O. Ulfarsson, and J. A. Benediktsson,
[252] B. Rasti, J. R. Sveinsson, M. O. Ulfarsson, and J. A. Benediktsson, “Hyperspectral image restoration using wavelets,” Proc. Society
“Wavelet based hyperspectral image restoration using spatial of Photographic Instrumentation Engineers, vol. 8892, p. 889. 2013.
and spectral penalties,” in Proc. Society of Photographic Instru- doi: 10.1117/12.2029240.
mentation Engineers, vol. 8892, 2013, pp. 88 920I–88 920I–8. [267] B. Rasti, J. R. Sveinsson, and M. O. Ulfarsson, “Total variation
doi: 10.1117/12.2029257. based hyperspectral feature extraction,” in Proc. IEEE Int. Geo-
[253] H. Zhang, “Hyperspectral image denoising with cubic total science and Remote Sensing Symp., July 2014, pp. 4644–4647.
variation model,” ISPRS Ann. Photogramm., Remote Sens. and [268] Y. Qian and M. Ye, “Hyperspectral imagery restoration using
Spatial Inform. Sci., vol. I-7, pp. 95–98, 2012. doi: 10.5194/ nonlocal spectral-spatial structured sparse representation with
isprsannals-I-7-95-2012. noise estimation,” IEEE J. Sel. Topics Appl. Earth Observ. in Re-
[254] Q. Yuan, L. Zhang, and H. Shen, “Hyperspectral image de- mote Sens., vol. 6, no. 2, pp. 499–515, Apr. 2013.
noising employing a spectral-spatial adaptive total varia- [269] X. Liu, S. Bourennane, and C. Fossati, “Reduction of signal-de-
tion model,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 10, pendent noise from hyperspectral images for target detection,”
pp. 3660–3677, 2012. IEEE Trans. Geosci. Remote Sens., vol. 52, no. 9, pp. 5396–5411,
[255] H. Othman and S.-E. Qian, “Noise reduction of hyperspectral Sept. 2014.
imagery using hybrid spatial-spectral derivative-domain wave- [270] H. Zhang, W. He, L. Zhang, H. Shen, and Q. Yuan, “Hyperspec-
let shrinkage,” IEEE Trans. Geosci. Remote Sens., vol. 44, no. 2, tral image restoration using low-rank matrix recovery,” IEEE
pp. 397–408, Feb. 2006. Trans. Geosci. Remote Sens., vol. 52, no. 8, pp. 4729–4743,
[256] S. L. Chen, X. Y. Hu, and S. L. Peng, “Hyperspectral imagery de- Aug. 2014.
noising using a spatial-spectral domain mixing prior,” J. Com- [271] W. He, H. Zhang, L. Zhang, and H. Shen, “Total-variation-reg-
put. Sci. Technol., vol. 27, no. 4, pp. 851–861, 2012. ularized low-rank matrix factorization for hyperspectral im-
[257] G. Chen and S. E. Qian, “Denoising of hyperspectral imagery age restoration,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 1,
using principal component analysis and wavelet shrinkage,” pp. 178–188, Jan. 2016.
IEEE Trans. Geosci. Remote Sens., vol. 49, no. 3, pp. 973–980, [272] W. He, H. Zhang, L. Zhang, and H. Shen, “Hyperspectral image
Mar. 2011. denoising via noise-adjusted iterative low-rank matrix approxi-
[258] L. R. Tucker, “Some mathematical notes on three-mode fac- mation,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens.,
tor analysis,” Psychometrika, vol. 31, no. 3, pp. 279–311, Sept. vol. 8, no. 6, pp. 3050–3061, June 2015.
1966. [273] Y. Xie, Y. Qu, D. Tao, W. Wu, Q. Yuan, and W. Zhang, “Hyper-
[259] L. D. Lathauwer, B. D. Moor, J. Vandewalle, “On the best spectral image restoration via iteratively regularized weighted
Rank-1 and Rank-(R1,R2,…,Rn) approximation of higher- Schatten p-norm minimization,” IEEE Trans. Geosci. Remote
order tensors,” SIAM J. Matrix Anal. Applicat., vol. 21, no. 4, Sens., vol. 54, no. 8, pp. 4642–4659, Aug. 2016.
pp. 1324–1342, Mar. 2000. [274] D. Donoho and I. M. Johnstone, “Adapting to unknown
[260] N. Renard, S. Bourennane, and J. Blanc-Talon, “Denoising and smoothness via wavelet shrinkage,” J. Amer. Statist. Assoc.,
dimensionality reduction using multilinear tools for hyper- vol. 90, no. 432, pp. 1200–1224, Dec. 1995.
spectral images,” IEEE Geosci. Remote Sens. Lett., vol. 5, no. 2, [275] B. Rasti. (2016, May). FORPDN_SURE. [Online]. Available:
pp. 138–142, Apr. 2008. https://w w w.researchgate.net/publication/303445288_­
[261] A. Karami, M. Yazdi, and A. Asli, “Best rank-r tensor selection FORPDN_SURE
using genetic algorithm for better noise reduction and com- [276] B. Rasti. (2016, May). HySURE. [Online]. Available: https://
pression of hyperspectral images,” in Proc. 5th Int. Conf. Digital www.researchgate.net/publication/303784304_HySURE
Information Management (ICDIM), July 2010, pp. 169–173. [277] B. Rasti. (2016, May). Wavelab_fast. [Online]. Available: https://
[262] A. Karami, M. Yazdi, and A. Zolghadre Asli, “Noise reduction www.researchgate.net/publication/303445667_Wavelab_fast
of hyperspectral images using kernel non-negative tucker de- [278] L. Bruzzone and F. Bovolo, “A novel framework for the de-
composition,” IEEE J. Sel. Topics Signal Process., vol. 5, no. 3, sign of change-detection systems for very-high-resolution
pp. 487–493, June 2011. remote sensing images,” Proc. IEEE, vol. 101, no. 3, pp. 609–
[263] D. Letexier and S. Bourennane, “Noise removal from hy- 630, 2013.
perspectral images by multidimensional filtering,” IEEE [279] P. Du, S. Liu, P. Gamba, and K. Tan, “Fusion of difference im-
Trans. Geosci. Remote Sens., vol. 46, no. 7, pp. 2061–2069, ages for change detection over urban areas,” IEEE J. Sel. Topics
July 2008. Appl. Earth Observ. in Remote Sens., vol. 5, no. 4, pp. 1076–
[264] X. Liu, S. Bourennane, and C. Fossati, “Denoising of hyperspec- 1086, 2012.
tral images using the PARAFAC model and statistical perfor- [280] P. Coppin, I. Jonckheere, K. Nackaerts, B. Muys, and E. Lambin,
mance analysis,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 10, “Digital change detection methods in ecosystem monitoring: A
pp. 3717–3724, Oct. 2012. review,” Int. J. Remote Sens., vol. 25, no. 9, pp. 1565–1596, 2004.

76 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 [281] S. Liu, M. Chi, Y. Zou, A. Samat, J. A. Benediktsson, and [295] M. Frank and M. Canty, “Unsupervised change detection for
A. Plaza, “Oil spill detection via multitemporal optical remote hyperspectral images,” in Proc. 12th JPL Airborne Earth Science
sensing images: a change detection perspective,” IEEE Geosci. Workshop, 2013, pp. 63–72.
Remote Sens. Lett., vol. 14, no. 3, pp. 324–328, 2017. [296] A. A. Nielsen, “The regularized iteratively reweighted mad
[282] S. Liu, L. Bruzzone, F. Bovolo, and P. Du, “Hierarchical un- method for change detection in multi- and hyperspectral data,”
supervised change detection in multitemporal hyperspec- IEEE Trans. Image Process., vol. 16, no. 2, pp. 463–478, 2007.
tral images,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 1, [297] V. Ortizrivera, M. Vlezreyes, and B. Roysam, “Change de-
pp. 244–260, 2015. tection in hyperspectral imagery using temporal principal
[283] L. Bruzzone, S. Liu, F. Bovolo, and P. Du, “Change detection components,” in SPIE: Algorithms and Technologies for Multi-
in multitemporal hyperspectral images,” in Multitemporal spectral, Hyperspectral, and Ultraspectral Imagery XII, 2006. doi:
Remote Sensing Remote Sensing and Digital Image Processing, 10.1117/12.667961.
vol. 20, Y. Ban, Ed. Cham, Switzerland: Springer, 2016, pp. [298] S. Liu, L. Bruzzone, F. Bovolo, and P. Du, “Unsupervised hier-
63–88. archical spectral analysis for change detection in hyperspectral
[284] C. Wu, L. Zhang, and B. Du, “Hyperspectral anomaly change images,” in Proc. 4th Workshop Hyperspectral Image and Signal Pro-
detection with slow feature analysis,” Neurocomput., vol. 151, cessing (WHISPERS), 2012, pp. 1–4.
pp. 175–187, Mar. 2015. [299] C. Wu, B. Du, and L. Zhang, “A subspace-based change de-
[285] A. Schaum and A. Stocker, “Long-interval chronochrome tar- tection method for hyperspectral images,” IEEE J. Sel. Topics
get detection,” in Proc. Int. Symp. Spectral Sensing Research, 1998, Appl. Earth Observ. in Remote Sens., vol. 6, no. 2, pp. 815–830,
pp. 1760–1770. 2013.
[286] A. P. Schaum and A. Stocker, “Hyperspectral change detection [300] Y. Yuan, H. Lv, and X. Lu, “Semi-supervised change detection
and supervised matched filtering based on covariance equal- method for multi-temporal hyperspectral images,” Neurocom-
ization,” in Proc. SPIE 5425, Algorithms and Technologies for Multi- put., vol. 148, pp. 363–375, Jan. 2015.
spectral, Hyperspectral, and Ultraspectral Imagery X, The Int. Society [301] Q. Du, L. Wasson, and R. King, “Unsupervised linear unmixing
for Optical Engineering, vol. 5425, 2004, pp. 77–90. for change detection in multitemporal airborne hyperspectral
[287] J. Theiler, C. Scovel, B. Wohlberg, and B. R. Foy, “Elliptically imagery,” in Proc. 2005 Int. Workshop Analysis Multi-Temporal Re-
contoured distributions for anomalous change detection in mote Sensing Images, 2005, pp. 136–140.
hyperspectral imagery,” IEEE Geosci. Remote Sens. Lett., vol. 7, [302] A. Erturk and A. Plaza, “Informative change detection by un-
no. 2, pp. 271–275, Apr. 2010. mixing for hyperspectral images,” IEEE Geosci. Remote Sens.
[288] J. Zhou, C. Kwan, B. Ayhan, and M. T. Eismann, “A novel cluster Lett., vol. 12, no. 6, pp. 1252–1256, 2015.
kernel RX algorithm for anomaly and change detection using [303] A. Ertrk, M. D. Iordache, and A. Plaza, “Sparse unmixing-based
hyperspectral images,” IEEE Trans. Geosci. Remote Sens., vol. 54, change detection for multitemporal hyperspectral images,”
no. 11, pp. 6497–6504, 2016. IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 9, no. 2,
[289] K. Vongsy, M. J. Mendenhall, M. T. Eismann, and G. L. Peterson, pp. 708–719, 2015.
“Removing parallax-induced changes in hyperspectral change [304] P. Du, S. Liu, P. Liu, K. Tan, and L. Cheng, “Sub-pixel change
detection,” in Proc. IEEE Int. Geoscience and Remote Sensing detection for urban land-cover analysis via multi-temporal
Symp., 2012, pp. 2012–2015. remote sensing images,” Geo-spatial Inf. Sci., vol. 17, no. 1,
[290] J. Meola, M. T. Eismann, K. J. Barnard, and R. C. Hardie, “Anal- pp. 26–38, 2014.
ysis of hyperspectral change detection as affected by vegetation [305] G. M. Foody, “Monitoring the magnitude of land-cover change
and illumination variations,” in Proc. SPIE 6565, Algorithms around the southern limits of the Sahara,” Photogramm. Eng. Re-
and Technologies for Multispectral, Hyperspectral, and Ultra- mote Sens., vol. 67, no. 7, pp. 841–848, 2001.
spectral Imagery XIII, 2007. doi: 10.1117/12.716324. [306] S. Hemissi, I. R. Farah, K. S. Ettabaa, and B. Solaiman, “Multi-
[291] M. T. Eismann, J. Meola, and R. C. Hardie, “Hyperspectral spectro-temporal analysis of hyperspectral imagery based on
change detection in the presence of diurnal and seasonal 3-D spectral modeling and multilinear algebra,” IEEE Trans.
variations,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 1, Geosci. Remote Sens., vol. 51, no. 1, pp. 199–216, 2013.
pp. 237–249, 2008. [307] S. Liu, L. Bruzzone, F. Bovolo, and M. Zanetti, “Sequential
[292] L. Bruzzone and D. F. Prieto, “Automatic analysis of the dif- spectral change vector analysis for iteratively discovering
ference image for unsupervised change detection,” IEEE Trans. and detecting multiple changes in hyperspectral images,”
Geosci. Remote Sens., vol. 38, no. 3, pp. 1171–1182, 2000. IEEE Trans. Geosci. Remote Sens., vol. 53, no. 8, pp. 4363–
[293] T. Celik, “Unsupervised change detection in satellite images 4378, 2015.
using principal component analysis and k-means clustering,” [308] F. Bovolo, S. Marchesi, and L. Bruzzone, “A framework for au-
IEEE Geosci. Remote Sens. Lett., vol. 6, no. 4, pp. 772–776, 2009. tomatic and unsupervised detection of multiple changes in
[294] A. A. Nielsen, K. Conradsen, and J. J. Simpson, “Multivari- multitemporal images,” IEEE Trans. Geosci. Remote Sens., vol. 50,
ate alteration detection (MAD) and MAF postprocessing in no. 6, pp. 2196–2212, 2012.
multispectral, bitemporal image data: New approaches to [309] S. Liu, Q. Du, and X. Tong, “Band selection for change de-
change detection studies,” Remote Sens. Environ., vol. 64, no. 1, tection from hyperspectral images,” vol. 9, no. 10, p. 1008,
pp. 1–19, 1998. 2017.

december 2017 ieee Geoscience and remote sensing magazine 77

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.
 [310] A. Plaza, J. Plaza, A. Paz, and S. Sánchez, “Parallel hyperspec- tion of an opencl implementation of the simplex growing al-
tral image and signal processing,” IEEE Signal Process. Mag., gorithm for hyperspectral unmixing,” IEEE Geosci. Remote Sens.
vol. 28, no. 3, pp. 119–126, 2011. Lett., vol. 14, no. 3, pp. 304–308, 2017.
[311] A. Plaza, Q. Du, Y.-L. Chang, and R. King, “High perfor- [325] S. Bernabe, S. Lopez, A. Plaza, and R. Sarmiento, “GPU imple-
mance computing for hyperspectral remote sensing,” IEEE mentation of an automatic target detection and classification
J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 4, no. 3, algorithm for hyperspectral image analysis,” IEEE Geosci. Re-
pp. 528–544, 2011. mote Sens. Lett., vol. 10, no. 2, pp. 221–225, 2013.
[312] A. Plaza, D. Valencia, J. Plaza, and P. Martinez, “Commodity [326] S. Sánchez, A. Paz, G. Martin, and A. Plaza, “Parallel unmixing
cluster-based parallel processing of hyperspectral imagery,” J. of remotely sensed hyperspectral images on commodity graph-
Parallel and Distributed Comput., vol. 66, pp. 345–358, 2006. ics processing units,” Concurrency and Computation: Practice and
[313] S. Sánchez, A. Paz, G. Martin, and A. Plaza, “Parallel unmixing Experience, vol. 23, no. 13, pp. 1538–1557, 2011.
of remotely sensed hyperspectral images on commodity graph- [327] X. Wu, B. Huang, A. Plaza, Y. Li, and C. Wu, “Real-time imple-
ics processing units,” Concurrency and Computation: Practice and mentation of the pixel purity index algorithm for endmember
Experience, vol. 23, no. 13, pp. 1538–1557, 2011. identification on GPUs,” IEEE Geosci. Remote Sens. Lett., vol. 11,
[314] A. Plaza, D. Valencia, J. Plaza, and P. Martinez, “Commodity no. 5, pp. 955–959, 2014.
cluster-based parallel processing of hyperspectral imagery,” J. [328] S. Bernabé, P. R. Marpu, and A. Plaza, “Spectral-spatial clas-
Parallel Distrib. Comput., vol. 66, no. 3, pp. 345–358, 2006. sification of multispectral images using kernel feature space
[315] A. Plaza, J. Plaza, and A. Paz, “Parallel heterogeneous CBIR sys- representation,” IEEE Geosci. Remote Sens. Lett., 2013.
tem for efficient hyperspectral image retrieval using spectral [329] J. M. Nascimento, J. M. Bioucas-Dias, J. M. R. Alves, V. Silva,
mixture analysis,” Concurrency and Computation: Practice and and A. Plaza, “Parallel hyperspectral unmixing on GPUs,” IEEE
Experience, vol. 22, no. 9, pp. 1138–1159, 2010. Geosci. Remote Sens. Lett., vol. 11, no. 3, pp. 666–670, 2014.
[316] S. Sánchez, R. Ramalho, L. Sousa, and A. Plaza, “Real-time [330] Z. Wu, Q. Wang, A. Plaza, J. Li, J. Liu, and Z. Wei, “Parallel
implementation of remotely sensed hyperspectral image un- implementation of sparse representation classifiers for hyper-
mixing on GPUs,” J. Real-Time Image Process., vol. 10, no. 3, spectral imagery on GPUs,” IEEE J. Sel. Topics Appl. Earth Observ.
pp. 469–483, Sept. 2015. in Remote Sens., vol. 8, no. 6, pp. 2912–2925, 2015.
[317] C. Gonzalez, S. Sánchez, A. Paz, J. Resano, D. Mozos, and [331] J. Sevilla, S. Bernabe, and A. Plaza, “Unmixing-based content
A. Plaza, “Use of FPGA or GPU-based architectures for remote- retrieval system for remotely sensed hyperspectral imagery on
ly sensed hyperspectral image processing,” INTEGRATION: The GPUs,” J. Supercomput., vol. 70, no. 2, pp. 588–599, 2014.
VLSI J., vol. 46, no. 2, pp. 89–103, 2013. [332] J. Delgado, G. Martín, J. Plaza, L. I. Jiménez, and A. Plaza, “Fast
[318] S. Lopez, T. Vladimirova, C. Gonzalez, J. Resano, D. Mozos, spatial preprocessing for spectral unmixing of hyperspectral
and A. Plaza, “The promise of reconfigurable computing for data on graphics processing units,” IEEE J. Sel. Topics Appl. Earth
hyperspectral imaging onboard systems: A review and trends,” Observ. in Remote Sens., vol. 9, no. 2, pp. 952–961, 2016.
Proc. IEEE, vol. 101, no. 3, pp. 698–722, 2013. [333] E. Torti, G. Danese, F. Leporati, and A. Plaza, “A hybrid
[319] L. Santos, L. Berrojo, J. Moreno, J. F. López, and R. Sarmien- CPU–GPU real-time hyperspectral unmixing chain,” IEEE
to, “Multispectral and hyperspectral lossless compressor for J. Sel. ­Topics Appl. Earth Observ. in Remote Sens., vol. 9, no. 2,
space applications (HyLoC): A low-complexity FPGA imple- pp. 945–951, 2016.
mentation of the CCSDS 123 standard,” IEEE J. Sel. Topics [334] L. Jiménez, S. Sánchez, G. Martín, J. Plaza, and A. Plaza, “Paral-
Appl. Earth Observ. in Remote Sens., vol. 9, no. 2, pp. 757–770, lel implementation of spatial-spectral endmember extraction
2016. on graphics processing units,” IEEE J. Sel. Topics Appl. Earth Ob-
[320] A. Plaza and C.-I. Chang, “Clusters versus FPGA for parallel serv. in Remote Sens., vol. 10, no. 4, pp. 1247–1255, 2017.
processing of hyperspectral imagery,” Int. J. High Performance [335] L. Riha, J. L. Moigne, and T. El-Gazawhi, “Optimization of se-
Comput. Applicat., vol. 22, no. 4, pp. 366–385, 2008. lected remote sensing algorithms for many-core architectures,”
[321] D. Keymeulen, N. Aranki, A. Bakhshi, H. Luong, C. Sarture, IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 9,
and D. Dolman, “Airborne demonstration of FPGA implemen- no. 12, pp. 5576–5587, 2016.
tation of fast lossless hyperspectral data compression system,” [336] C. Lee, S. Gasster, A. Plaza, C.-I. Chang, and B. Huang, “Recent
in Proc. NASA/ESA Conf. Adaptive Hardware and Systems (AHS), developments in high performance computing for remote sens-
2014, pp. 278–284. ing: A review,” IEEE J. Sel. Topics Appl. Earth Observ. in Remote
[322] C. Gonzalez, S. Lopez, D. Mozos, and R. Sarmiento, “FPGA im- Sens., vol. 4, no. 3, pp. 508–527, 2011.
plementation of the HySime algorithm for the determination [337] W. Li, G. Wu, F. Zhang, and Q. Du, “Hyperspectral image clas-
of the number of endmembers in hyperspectral data,” IEEE sification using deep pixel-pair features,” IEEE Trans. Geosci. Re-
J. Sel. Topics Appl. Earth Observ. in Remote Sens., vol. 8, no. 6, mote Sens., vol. 55, no. 2, pp. 844–853, 2017.
pp. 2870–2883, 2015. [338] E. Aptoula, M. Ozdemir, and B. Yanikoglu, “Deep learning
[323] J. Nickolls and W. J. Dally, “The GPU computing era,” IEEE Mi- with attribute profiles for hyperspectral image classification,”
cro, vol. 30, no. 2, pp. 56–69, Mar. 2010. IEEE Geosci. Remote Sens. Lett., vol. 13, no. 12, pp. 1970–1974,
[324] S. Ber nabé, G. Botella, J. Navar ro, C . Or ueta, F. Ig ual, 2016.
M. Prieto-Matias, and A. Plaza, “Performance-power evalua-  grs

78 ieee Geoscience and remote sensing magazine december 2017

Authorized licensed use limited to: Indian Institute of Technology Hyderabad. Downloaded on July 03,2023 at 10:49:51 UTC from IEEE Xplore. Restrictions apply.