remote sensing

Review
Digital Elevation Model Quality Assessment
Methods: A Critical Review
Laurent Polidori 1, * and Mhamad El Hage 2
1 CESBIO, Université de Toulouse, CNES/CNRS/INRAE/IRD/UPS, 18 av. Edouard Belin, bpi 2801,
31401 Toulouse CEDEX 9, France
2 Laboratoire d’Etudes Géospatiales (LEG), Université Libanaise, Tripoli, Lebanon; [email protected]
* Correspondence: [email protected]




Received: 31 May 2020; Accepted: 23 October 2020; Published: 27 October 2020


Abstract: Digital elevation models (DEMs) are widely used in geoscience. The quality of a DEM is
a primary requirement for many applications and is affected during the different processing steps,
from the collection of elevations to the interpolation implemented for resampling, and it is locally
influenced by the landcover and the terrain slope. The quality must meet the user’s requirements,
which only make sense if the nominal terrain and the relevant resolution have been explicitly specified.
The aim of this article is to review the main quality assessment methods, which may be separated
into two approaches, namely, with or without reference data, called external and internal quality
assessment, respectively. The errors and artifacts are described. The methods to detect and quantify
them are reviewed and discussed. Different product levels are considered, i.e., from point cloud to
grid surface model and to derived topographic features, as well as the case of global DEMs. Finally,
the issue of DEM quality is considered from the producer and user perspectives.

Keywords: digital elevation model; nominal terrain; quality; accuracy; error; autocorrelation; landforms

1. Introduction
Terrestrial relief is essential information to represent on a map. For centuries, it has been displayed
symbolically to indicate the presence of landforms like mountains or valleys (Figure 1a). By definition,
a landform is “any physical, recognizable form or feature on the Earth’s surface having a characteristic
shape and produced by natural causes” [1] and is the result of endogenous processes such as tectonic
motion and exogenous ones such as climate [2]. During the 19th and the first half of 20th century,
when topographic methods became more operational and geodetically more rigorous in terms of
cartographic projection and physical altitude definition, a new relief representation appeared on
topographic maps with elevation value indications and contour lines (Figure 1b).
To produce these maps, elevation values must be collected point by point in field surveys,
and contour lines are then generated by interpolation. While this type of primitive survey has
evolved towards more efficient techniques, it has long been supplemented by photogrammetry.
Although terrestrial photogrammetry was first proposed as early as the 1860s by Captain Laussedat [3],
this technique began to be seriously implemented at the end of the 19th century, initially in mountainous
areas where it brought clear advantages, like in the French Alps by Henri and Joseph Vallot, and in
western Canada by Gaston Deville. Although the stereoplotter industry began in the early 20th
century, photogrammetry became fully operational after World War I with the development of aerial
photography, and it became the standard method for elevation mapping worldwide after World War II.

Remote Sens. 2020, 12, 3522; doi:10.3390/rs12213522 www.mdpi.com/journal/remotesensing

Remote Sens. 2020, 12, x FOR PEER REVIEW 2 of 36

aerial Sens.
Remote photography,
2020, 12, 3522 and it became the standard method for elevation mapping worldwide 2after
of 36
World War II.

(a) (b)
1. Cartographic representation of terrestrial relief in maps in Southern
Figure 1. Southern France:
France: Cassini map,
18th century
18th century (a);
(a); IGN
IGN map,
map, 1957
1957 (b).
(b).

Several
Several major
majorinnovations
innovationstook
tookplace during
place thethe
during lastlast
quarter of the
quarter of20th
the century: black black
20th century: and white
and
photography gave way to color, analytical photogrammetry became the standard
white photography gave way to color, analytical photogrammetry became the standard forfor three-dimensional
(3D) mapping, digital
three-dimensional (3D)databases
mapping,were implemented
digital by mapping
databases were agencies,
implemented and a growing
by mapping variety
agencies, andofa
platforms, sensors, and processing methods extended the possibility of photogrammetry,
growing variety of platforms, sensors, and processing methods extended the possibility of which became
an increasingly powerful
photogrammetry, toolbox an
which became [4]:increasingly powerful toolbox [4]:

•• New
New platforms
platformsextended
extendedthethe capabilities
capabilities of the
of aircraft: satellites
the aircraft: covering
satellites largerlarger
covering areas (first
areasTIROS
(first
observation satellitesatellite
TIROS observation in 1960, across-track
in 1960, across-trackstereoscopy fromfrom
stereoscopy different orbits
different with
orbits SPOT-1
with SPOT-1 in
1986, along-track stereoscopy with SPOT-5 in 2002), UAVs (fixed-wing
in 1986, along-track stereoscopy with SPOT-5 in 2002), UAVs (fixed-wing for large for large area surveys,
multirotor
multirotor for
for small
small area
area surveys
surveys requiring
requiring agility
agility and precision), and mobile mapping systems
on
on land
land vehicles.
vehicles.
•• processing methods
New processing methods were were developed,
developed, such as as aerotriangulation
aerotriangulation and automatic automatic image
image
matching that
matching thatmade
made it possible
it possible to process
to process large quantities
large quantities of images offorimages for theproduction
the industrial industrial
production
of orthophoto ofmosaics
orthophoto mosaics and
and altimetric altimetric
databases withdatabases
few ground with few ground
control control
points [5], points
and the [5],
synergy
and the synergy
between between digital photogrammetry
digital photogrammetry and computer vision and computer
led to very vision led to3D
powerful very powerful 3D
reconstruction
reconstruction
methods such as methods such as from
SfM (structure SfM (structure
motion). from motion).
•• sensors broadened
New sensors broadened the the imaging
imaging possibilities,
possibilities, like spaceborne
spaceborne and airborne SAR (synthetic
aperture radar)
aperture radar)[6–8]
[6–8]andandlidar
lidar[9,10],
[9,10],i.e.,
i.e., active
active sensors
sensors thatthat
mademade it possible
it possible to acquire
to acquire datadata
day
day and night, even in cloudy areas and
and night, even in cloudy areas and beneath forests. beneath forests.
These innovations
These innovationscontributed
contributed torapid
to the the development
rapid development of geographic
of geographic information
information technologies
technologies
by providingby providing
a vast amount a vast amount
of data, amongof data,
whichamong whichelevation
the digital the digitalmodel
elevation
(DEM)model (DEM)
is one is
of the
one of the most widely utilized. Both public and private initiatives have led to an
most widely utilized. Both public and private initiatives have led to an increasing availability of digitalincreasing
availability
elevation of digital
models, eitherelevation
produced models,
on demand either produced
or stored on demand
in multi-user or stored
databases. in stimulated
This has multi-user
databases.
the This has
development of a stimulated
wide rangethe development
of applications in of a wide
which range of description
a quantitative applicationsofinthewhich
terraina
quantitative description of the terrain geometry is needed, such as geoscientific studies
geometry is needed, such as geoscientific studies (flood or landslide hazard assessment) [11,12] and (flood or
landslide hazard assessment) [11,12] and those based on spatial analysis (soil properties,
those based on spatial analysis (soil properties, location of a telecommunication tower or of a new location of
a telecommunication
company) [13,14], etc.,tower
as wellorasofimage
a neworthorectification
company) [13,14], etc., as
[15,16]. wellofasthose
Some image orthorectification
applications require
shape and topology description based on quantitative terrain descriptors extracted fromon
[15,16]. Some of those applications require shape and topology description based quantitative
DEMs, such as
terrainmaps,
slope descriptors
drainageextracted
networks,from DEMs, such
or watershed as slopeleading
delineation, maps, to drainage networks,
the emergence of a or
new watershed
scientific
delineation,
discipline leading
called to the emergence[12,17–19].
“geomorphometry” of a new scientific discipline called “geomorphometry” [12,17–
19]. As with any other product, the ability of a DEM to fulfil user requirements is characterized by a
number of quality criteria. Once the general characteristics of the DEM have been agreed (resolution,
spatial coverage, date etc.), the main requirements relate to the quality of the data themselves, which can
Remote Sens. 2020, 12, 3522 3 of 36

be divided into two main types, namely, elevation quality (generally defined in terms of absolute or
relative accuracy) and shape and topologic quality (related to the accuracy of DEM derivatives such
as slope, aspect, curvature etc. that are quantitative descriptors of the relief [11,20–23]). The quality
of these descriptors is decisive for the applications that use them [24]. If we express these two types
of quality criteria in natural language, they mean that the model must be as close as possible to
the real terrain position, and that it must look as much as possible like the real terrain, respectively.
These two types of requirements are not equivalent, and it seems relevant to consider both of them.
Indeed, a small position error can induce erroneous shape representations [25–27]. On the other
hand, a high autocorrelation of the elevation error tends to preserve terrain shape quality despite high
positional errors [28].
The terrestrial relief surface has specific characteristics, different from those of most surfaces
represented in geographic information, which make DEM quality assessment a challenging task:

• DEM is multi-user information, so that multiple quality criteria must be met, and although a
particular user may specify application-driven requirements, the case of multi-user databases is
obviously more complex.
• The Earth’s surface is a material object that speaks to our senses, so that one can complain about
unrealistic relief modelling even in an unknown region; this leads to strong requirements including
aesthetic ones.

The aim of this article is to discuss the issue of DEM quality assessment and to propose a review
of the main approaches. Defining the nominal terrain and specifying the characteristics of the digital
product are two prerequisites of DEM quality assessment that are addressed in Sections 2 and 3,
respectively. Section 4 is the core of the article, with a review of relevant quality criteria for DEMs.
This review encompasses a variety of quality assessment methods, based or not on reference data.
Section 5 shows that these criteria can be considered at different levels, i.e., from point cloud to grid
surface model, from grid surface model to topographic features, and at the global level. Section 6 shows
that the resolution of a DEM should also be considered in its quality assessment. Finally, an overall
discussion is proposed in Section 7.

2. Prerequisite: Definition of the Nominal Terrain

Assessing the quality of a DEM requires a clear and explicit definition of the nominal surface,
i.e., the physical surface which is supposed to be modelled. Two nominal surfaces are often considered,
namely the ground surface (represented in a DTM—digital terrain model) and the upper surface above
the trees, buildings and all other natural or manmade objects (represented in a DSM—digital surface
model, provided by most DEM production techniques such as photogrammetry and short wavelength
radar technologies). It should be noted that DEM is a generic term that applies to both cases.
The choice between DTM and DSM depends on the foreseen application. For instance, a DSM
is well suited for orthorectification of imagery as it requires information on the elevation of the top
of buildings and trees, i.e., objects visible in the images, whereas a DTM is required for hydrological
modelling that needs information on the level of the ground where surface water runs off.
As illustrated in Figure 2, the DSM is defined over any area, including in the presence of buildings
(roofs) or trees (canopy). On the other hand, the DTM is defined in bare terrains as well as in forested
areas (the ground surface exists and it controls the surface runoff although the presence of trees
modifies the process), but not in the presence of a building, since in this case the ground surface does
not physically exist (it can be interpolated to generate a continuous surface but this is meaningless for
many applications, like hydrological modelling).
The nuance between DSM and DTM does not exist on other planets, nor on the seafloor, while it
is relevant for the modelling of Earth continental surfaces due the abundance of trees and buildings.
In many landscapes, the reality is even more complex, for example in the presence of grass vegetation
whose height would be comparable to the precision of the topographic method: validating a DEM
Remote Sens. 2020, 12, 3522 4 of 36

obtained by photogrammetry or even lidar, where the measured elevation lies at the top of the grass,
by comparison with control points measured on the ground, can lead to an erroneous conclusion on the
accuracy of the DEM and therefore of the mapping method, since the reference data do not represent
the same surface as the DEM. Consequently, it would be meaningless to evaluate the quality of a DEM
Remote Sens.
without 2020, definition
a clear 12, x FOR PEER REVIEW
of the physical surface it is supposed to represent. 4 of 36

Figure 2. DTM vs DSM in the presence of trees and buildings.

Figure 2. DTM vs DSM in the presence of trees and buildings.
Given the variety of relief mapping techniques, which are sensitive to different aspects of the
The nuance between DSM and DTM does not exist on other planets, nor on the seafloor, while it
terrain surface, it is highly recommended that the most appropriate technique be chosen according
is relevant for the modelling of Earth continental surfaces due the abundance of trees and buildings.
to the selected nominal terrain. In the presence of forests, photogrammetry and short wavelength
In many landscapes, the reality is even more complex, for example in the presence of grass
radar techniques are more appropriate for DSM than for DTM, while lidar and long wavelength radar
vegetation whose height would be comparable to the precision of the topographic method:
techniques are suitable for both surface models [29,30]. However, it is not always possible to access
validating a DEM obtained by photogrammetry or even lidar, where the measured elevation lies at
the ideal technique. For example, it is very common to use photogrammetry to produce a DTM
the top of the grass, by comparison with control points measured on the ground, can lead to an
(i.e., the nominal terrain is the ground surface). In order to prevent users from making gross errors by
erroneous conclusion on the accuracy of the DEM and therefore of the mapping method, since the
using such information in forested areas, postprocessing strategies must be employed to remove tree
reference data do not represent the same surface as the DEM. Consequently, it would be
height so that the DEM can theoretically be considered as a DTM.
meaningless to evaluate the quality of a DEM without a clear definition of the physical surface it is
In the case of sand or ice deserts, two possible nominal surfaces can be considered, namely,
supposed to represent.
the bedrock and the upper sand or ice surface which is in contact with the atmosphere. It is relevant to
Given the variety of relief mapping techniques, which are sensitive to different aspects of the
define which of these surfaces is supposed to be represented in a DEM since the expectations will be
terrain surface, it is highly recommended that the most appropriate technique be chosen according
different. Indeed, the bedrock is stable over time and its shape is most often generated by tectonics and
to the selected nominal terrain. In the presence of forests, photogrammetry and short wavelength
long-term hydric erosion, while the shape of the upper sand or ice surface is constantly changing due
radar techniques are more appropriate for DSM than for DTM, while lidar and long wavelength
to climatic processes and gravity. No operational technique, except field survey, is currently available
radar techniques are suitable for both surface models [29,30]. However, it is not always possible to
to map the substratum in such areas for the Earth, but the future Biomass mission could bring new
access the ideal technique. For example, it is very common to use photogrammetry to produce a
opportunities thanks to the penetrating capabilities of its P-band radar [31].
DTM (i.e., the nominal terrain is the ground surface). In order to prevent users from making gross
The case of shallow water bathymetry (i.e., rivers or coastal sea water) raises the question of
errors by using such information in forested areas, postprocessing strategies must be employed to
the nominal terrain as well. Indeed, the surface and the bottom of the water have different physical
remove tree height so that the DEM can theoretically be considered as a DTM.
behaviours with regards to existing imaging sensors and the image of this double surface must be
In the case of sand or ice deserts, two possible nominal surfaces can be considered, namely, the
interpreted with caution. For example, photogrammetry and green lidar can survey the bottom
bedrock and the upper sand or ice surface which is in contact with the atmosphere. It is relevant to
topography in the case of clear water [32,33] but the signal is attenuated by the presence of the water,
define which of these surfaces is supposed to be represented in a DEM since the expectations will be
whereas radar imagery only surveys the surface but the image is influenced by the bottom topography.
different. Indeed, the bedrock is stable over time and its shape is most often generated by tectonics
At low tide, only the ground appears and it can be reconstructed in 3D [34,35].
and long-term hydric erosion, while the shape of the upper sand or ice surface is constantly
The nominal terrain considered in a DEM, which is supposed to be defined in agreement with
changing due to climatic processes and gravity. No operational technique, except field survey, is
the user’s requirements, should be clearly indicated for users’ information. Nevertheless, important
currently available to map the substratum in such areas for the Earth, but the future Biomass mission
altimetric products such as SRTM and TanDEM-X global DEMs do not clearly specify the nominal
could bring new opportunities thanks to the penetrating capabilities of its P-band radar [31].
terrain. Although X- and C-band SAR interferometry rather provides a DSM and no filtering is applied
The case of shallow water bathymetry (i.e., rivers or coastal sea water) raises the question of the
to the native products to transform them into DTMs, all information about these products suggests that
nominal terrain as well. Indeed, the surface and the bottom of the water have different physical
they are specified as ground surface models, and many users regard them as such [36–38]. Considering
behaviours with regards to existing imaging sensors and the image of this double surface must be
that the nominal surface is the ground implies that elevation discrepancies due to tree height must be
interpreted with caution. For example, photogrammetry and green lidar can survey the bottom
topography in the case of clear water [32,33] but the signal is attenuated by the presence of the water,
whereas radar imagery only surveys the surface but the image is influenced by the bottom
topography. At low tide, only the ground appears and it can be reconstructed in 3D [34,35].
The nominal terrain considered in a DEM, which is supposed to be defined in agreement with
the user’s requirements, should be clearly indicated for users’ information. Nevertheless, important
Remote Sens. 2020, 12, 3522 5 of 36

interpreted in terms of DEM error. In other words, they contribute to the fact that the DEM does not
fulfil user requirements for geoscientific studies.
The need for a clear specification of the nominal terrain in the metadata has become even more
critical with the growing development of high resolution and high precision products, which have
stimulated applications based on the modelling of small objects, such as buildings or parts of buildings,
in synergy with topographic methods like terrestrial or mobile laser scanning of urban scenes. In the
case of buildings that are very complex objects, the need for simplification has led to the definition
of level of detail (LOD) standards that can help to specify user requirements [39,40]. Moreover,
undesirable objects such as vehicles or street furniture may appear in the input data and the need
to include them depends on the DEM application. Similarly, when a bridge spans a road or a river,
the product specification must clearly indicate whether the bridge is modeled (which could lead to
large errors in hydrological modeling) or not (which could generate artifacts in the orthophotos created
using the DEM).

3. DEM as a Cartographic Product

The altimetric information collected to represent the terrain surface is generally represented in
the form a cartographic product, which is defined by a number of specifications. Traditional paper
maps are specified with a scale, a cartographic projection and a legend that indicates how the map
describes the terrain. Similarly, the DEM is characterized by metadata, i.e., data about the DEM itself,
which make possible the access to the altimetric data in the digital file and provide information such
as the producer, the date, the accuracy etc. In this section, we show that the Earth’s surface has very
special properties (Section 3.1) which influence the way a DEM must be specified, i.e., the structure of
the planimetric grid (Section 3.2) and the numerical representation of the elevation (Section 3.3).

3.1. A Very Peculiar Surface

Unlike most physical surfaces represented by 3D models for robotics, architecture, industry etc.,
the Earth’s relief has a very particular property: it can be considered as a 2.5D surface. Indeed, once the
nominal terrain has been defined (e.g., the ground surface), a unique altitude z can be assigned to each
horizontal position (x, y). This is due to the gigantic mass of the Earth (M ~6.1024 kg) and the intensity
of the associated gravitational field (g ~9.8 m·s−2 ), where the attraction of all material particles towards
the mass center gives the Earth’s spherical shape. As a result, the Earth’s surface has almost no hidden
part when seen from above. It is comparable to a bas-relief which has no hidden part when seen from
the front. This property is not cancelled out by the centrifugal force produced by the Earth’s rotation,
which transforms the sphere into an ellipsoid.
A few exceptions exist, like overhang cliffs where more than one elevation value can be assigned
to a given horizontal position. But they are rare enough to justify the general assumption that any
vertical axis (from the Earth’s center to the zenith) has a unique intersection with the topographic
surface (Figure 3). This is the definition of a 2.5D surface and it has a very interesting advantage since
the elevation can be considered as a bivariate function z = f(x, y). Indeed, most cartographic systems
consist in separating two planimetric coordinates (latitude and longitude or equivalent) on the one
hand, and a vertical coordinate (altitude above sea level) on the other hand. Therefore, the specification
of a DEM as a cartographic product consists in specifying two aspects: the planimetric grid structure
and the numerical representation of elevation values, beyond the classical specifications of all maps
like datum and map projection. These specification data are essential. They are generally available as
metadata with the DEM file. Our aim is not to recall this well-known information in detail, since it has
been widely addressed in GIS literature [41], but to stress the need for an explicit specification as a
condition for DEM quality assessment.
 x FOR PEER REVIEW
Remote Sens. 2020, 12, 3522 6 of 36

Figure 3. 2.5D representation of the Earth’s surface: general case (a) and exception (b).

3.2. Planimetric Grid Sructure

3.2. Planimetric Grid Sructure
Most relief mapping techniques consist of two steps, namely, altitude computation for a large
Most relief mapping techniques consist of two steps, namely, altitude computation for a large
number of terrain points, and resampling to fit the specifications of the output product when necessary.
number of terrain points, and resampling to fit the specifications of the output product when
The technique implemented to compute terrain altitudes with specific acquisition and processing
necessary. The technique implemented to compute terrain altitudes with specific acquisition and
parameters has an impact on the positional accuracy of each point and on the density of the resulting
processing parameters has an impact on the positional accuracy of each point and on the density of
point cloud. However, the DEM quality is not limited to these aspects since the subsequent resampling
the resulting point cloud. However, the DEM quality is not limited to these aspects since the
step could also have a major impact on the quality of the output DEM. Two important specifications
subsequent resampling step could also have a major impact on the quality of the output DEM. Two
are set up to define the resampling procedure, i.e., the grid structure and the interpolation algorithm.
important specifications are set up to define the resampling procedure, i.e., the grid structure and
the interpolation
3.2.1. algorithm.
Grid Structure

3.2.1.The
Gridgrid structure defines the planimetric positions in which the altimetric values are to be stored.
Structure
Two main grid structures may be considered, namely, regular (e.g., a square mesh raster grid) and
The grid structure defines the planimetric positions in which the altimetric values are to be
irregular structures (e.g., a triangular irregular network).
stored. Two main grid structures may be considered, namely, regular (e.g., a square mesh raster
The regular grid has clear advantages in terms of data handling. Indeed, the horizontal coordinates
grid) and irregular structures (e.g., a triangular irregular network).
are implicit, and they theoretically do not need to be stored provided that they are logically defined by
The regular grid has clear advantages in terms of data handling. Indeed, the horizontal
metadata, and the square mesh grid can be displayed directly on a screen or a printout with no need
coordinates are implicit, and they theoretically do not need to be stored provided that they are
for further resampling. The regular grid structure is defined a priori (i.e., regardless of the distribution
logically defined by metadata, and the square mesh grid can be displayed directly on a screen or a
of the input point cloud), but most data acquisition methods provide irregularly distributed points so
printout with no need for further resampling. The regular grid structure is defined a priori (i.e.,
that an interpolation step is required to resample the data in order to fit the regular grid. Conversely,
regardless of the distribution of the input point cloud), but most data acquisition methods provide
no resampling is required for the irregular structure, which is built a posteriori over previously acquired
irregularly distributed points so that an interpolation step is required to resample the data in order
points. It is typically the case of the well-known TIN (triangular irregular network): this structure
to fit the regular grid. Conversely, no resampling is required for the irregular structure, which is
consists of a network of non-overlapping planar triangles built on the preliminary point cloud [42,43].
built a posteriori over previously acquired points. It is typically the case of the well-known TIN
Intermediate grid structures have been proposed, like for progressive or composite sampling [44,45],
(triangular irregular network): this structure consists of a network of non-overlapping planar
in which a regular grid is iteratively densified as long as the curvature exceeds a given threshold,
triangles built on the preliminary point cloud [42,43]. Intermediate grid structures have been
leading to a semi-regular grid of variable density [46–48]. This multiresolution modeling approach has
proposed, like for progressive or composite sampling [44,45], in which a regular grid is iteratively
gained interest for interactive terrain visualization [49].
densified as long as the curvature exceeds a given threshold, leading to a semi-regular grid of
The size and shape of the grid mesh must be specified with care due to their impact on the
variable density [46–48]. This multiresolution modeling approach has gained interest for interactive
DEM quality.
terrain visualization [49].
• The size
mesh size
and (andoftherefore
shape the grid
the grid mesh mustdensity) has anwith
be specified impact
care on
duethe DEMimpact
to their absolute
on accuracy,
the DEM
but also on the ability of the DEM to describe landforms as shown below in Section 4.2. This ability
quality.
is intuitive, since a smaller mesh is expected to allow a better rendering of small topographic
• The mesh size (and therefore the grid density) has an impact on the DEM absolute accuracy, but
objects, but a signal processing approach may also provide criteria to optimize resampling and to
also on the ability of the DEM to describe landforms as shown below in Section 4.2. This ability is
detect DEM errors [50]. The link between mesh size and resolution, and the way DEM quality
intuitive, since a smaller mesh is expected to allow a better rendering of small topographic
assessment should consider these concepts, are addressed in Section 6.
objects, but a signal processing approach may also provide criteria to optimize resampling and
• The mesh shape has an influence on the adequacy of the surface modelling to local landforms.
to detect DEM errors [50]. The link between mesh size and resolution, and the way DEM quality
Between the two most common grid structures, namely constant size squares and variable size
assessment should consider these concepts, are addressed in Section 6.
triangles, the latter is clearly better suited to describe a surface morphology in which drainages,
• The mesh shape has an influence on the adequacy of the surface modelling to local landforms.
ridges and other slope breaks have variable orientations. However, there are several solutions to
Between the two most common grid structures, namely constant size squares and variable size
build a network of triangles from a given point cloud. For example, the Delaunay triangulation
triangles, the latter is clearly better suited to describe a surface morphology in which drainages,
is advantageous to avoid very elongated triangles (which are less suitable for spatial analysis),
ridges and other slope breaks have variable orientations. However, there are several solutions to
Remote Sens. 2020, 12, 3522 7 of 36

but it does not necessarily give the most accurate DEM nor the most respectful one of terrain
shapes: it must therefore be constrained so that the edges of the triangles are placed along slope
discontinuities such as drainages or other slope break lines. This is an advantage of triangle
networks, which are able to adapt to the topography.

3.2.2. Interpolation Algorithm

Between the grid points, the surface model is built by interpolation so that the elevation and
its derivatives can be determined at any planimetric position. The interpolation is based on the
input points on the one hand, and on a generic mathematical surface model on the other hand.
There are different interpolation algorithms and they can affect the quality of the resulting DEM [51].
The interpolator has less influence on the quality of the DEM if the initial point cloud is dense or if
the grid is irregular with selected points on the slope breaks. Indeed, in these cases the mesh of the
surface model remains close to the ground and a smooth interpolator can be used. On the other hand,
the interpolator has a great influence on the quality of the DEM if the grid points have a low density,
or in the presence of holes (missing information) which can occur as a result of the presence of a cloud
in photogrammetry or a forest in repeat-pass SAR interferometry.
A great variety of interpolators are used in geoscience for interpolating thematic data to produce
raster maps [52], but they are not always relevant for DEMs, since the performances achieved for
moisture or geochemical concentrations, for instance, have no reason to be effective for elevation.
Some algorithms generate altimetric variability, like kriging or fractal zoom. These two different
mathematical formalisms aim at calculating and analyzing the variability around the interpolated
mesh and propagating it at a finer scale within the mesh [53]. It is a way to guarantee geomorphologic
realism as compared to usual interpolators [54].

3.3. Numerical Representation of Elevation Values

Elevation values are generally defined as altitudes, i.e., measured relative to a horizontal surface
that models the Earth at sea level. It is a gravimetric definition of elevation, suitable for many
applications based on hydrologic or hydraulic modelling. With the space age, a global reference
was sought, such as a global geoid model, and the fact of locating points by satellite (e.g., GNSS)
led to prefer a geometric definition of the reference surface, typically an ellipsoid such as WGS84
(World Geodetic System), easier to handle than a geoid model. The coexistence of geometric and
gravimetric references requires conversions between systems, and it is a source of confusion [55–57].
Even if some software tools assign elevation values to surface elements (i.e., pixels), this information
is only relevant when assigned to a Euclidean point, defined by its coordinates which are easy to
store and used to compute distances, angles and other metrics. However, except high precision
topometric techniques (e.g., total station and GNSS), all techniques compute a mean elevation over a
local neighborhood. In a DEM generated by digital photogrammetry for example, the image matching
process is based on the correlation between small windows of a given size, typically several pixels,
and the resulting elevation, though assigned to a point position (x, y), is influenced by the elevation
distribution over a terrain area [58].
The bit depth is also an important product specification for a DEM, since it has an impact on
the data volume and on the vertical quantization. Early generation DEMs were generally coded on
8 bits because they were not very precise and computers had limited storage capacities. It was a
limitation, mainly in plain areas where step-like artifacts could appear on the ground surface. Since the
1990s, the floating-point representation has been enabled by 16 or 32-bit encoding to represent decimal
numbers, increasing the precision and the elevation range for DEM storage. A 16-bit depth is good
enough for global DEMs such as SRTM, with a 5–10 m range of accuracy and a 1-m quantization
unit [59–61]. However, a 32-bit depth is preferable for high accuracy DEM as obtained by aerial or
terrestrial photogrammetry or by laser scanning.
All these specifications are essential to guarantee the quality of a digital elevation model.
Remote Sens. 2020, 12, 3522 8 of 36
Remote Sens. 2020, 12, x FOR PEER REVIEW 8 of 36

4.4.Main
MainDEM
DEMQuality
QualityAssessment
AssessmentApproaches
Approaches
The validation
The validationof of aa DEM
DEM consists
consists inin assessing
assessing its
its ability
ability to
to fulfil
fulfil user
user requirements
requirements based
based onon
adapted quality criteria. For each criterion, a quality assessment procedure can
adapted quality criteria. For each criterion, a quality assessment procedure can be implemented to be implemented to
decidewhether
decide whetheraaDEMDEMisisacceptable
acceptablefor
foraagiven
givenuseuse[62].
[62].In Inthis
thissection,
section,we wereview
reviewthe
themain
mainquality
quality
assessment approaches. Although this review is not exhaustive, it illustrates the variety
assessment approaches. Although this review is not exhaustive, it illustrates the variety of existing of existing
methodscommonly
methods commonlyused usedby byproducers
producers andand users.
users. We
Weshowshowthat
thatDEM DEM quality
quality assessment
assessmentcan
can be
be
consideredwith
considered withoror without
without ground
ground reference
reference data.
data. InInthe
thefirst
firstcase,
case,the theDEM
DEMisiscompared
comparedwith
withaa
reference data set (external validation), whereas in the second case, inconsistencies are
reference data set (external validation), whereas in the second case, inconsistencies are sought within sought within
theDEM
the DEMitself
itselfwith
withnonoreference
reference data
data (internal
(internal validation).
validation).

4.1. External Quality Assessment

4.1. External Quality Assessment
4.1.1. Comparison with Ground Control Data
4.1.1. Comparison with Ground Control Data
The goal of the external validation is to assess the quality of a DEM using external reference data,
whichThe goal
can be aofsparse
the external
cloud of validation is to assess
points, contour lines,the quality ofprofiles,
topographic a DEM using external
or a much morereference
accurate
data, which can be a sparse cloud of points, contour lines, topographic profiles,
DEM. A comprehensive overview of external validation methods can be found in [63]. As mentioned or a much more
accurate DEM. A comprehensive overview of external validation methods can be
in Section 3, a DEM is a digital 2.5D representation of the Earth’s surface, in which the elevation found in [63]. As
mentioned in Section 3, a DEM is a digital 2.5D representation of the Earth’s surface,
is a bivariate function z = f(x, y). Thus, the quality of a DEM includes planimetric and altimetric in which the
elevation
components. is a Most
bivariate function
of the studieszaim= f(x, y). Thus, the
at evaluating thequality
verticalofquality
a DEM in includes planimetric
a DEM, i.e., and
the altimetric
altimetric components. Most of the studies aim at evaluating the vertical quality in
quality rather than the planimetric quality. The usual approach consists in computing some statistical a DEM, i.e., the
altimetric quality rather than the planimetric quality. The usual approach consists
indicators based on the altimetric discrepancies between the DEM and the reference. As a rule of in computing
some
thumb, statistical indicators
the reference based
dataset mustonfulfil
the altimetric
two maindiscrepancies
requirements:between the DEM and the reference.
As a rule of thumb, the reference dataset must fulfil two main requirements:
• It must be much more accurate than the evaluated DEM.
• It must be much more accurate than the evaluated DEM.
• It must be dense enough and well distributed over the area to allow meaningful statistical analysis,
• It must be dense enough and well distributed over the area to allow meaningful statistical
and even a spatial analysis of the error.
analysis, and even a spatial analysis of the error.
The altimetric error of a DEM is composed of three main components, namely, gross, systematic
and The altimetric
random errorserror of aasDEM
[64,65] shown is composed
in Figure of4. three main components,
A systematic namely,
error is a bias betweengross,
thesystematic
modelled
and random
surface and theerrors [64,65]
ground truthasand
shown in Figure
it depends 4. A
on the systematic
production error is aespecially
technique, bias between the acquisition
the data modelled
surface and thebut
configuration, ground
also ontruth and it depends
the interpolation methodon [66].
the production technique,
It can be calculated usingespecially the value
the average data
acquisition configuration, but also on the interpolation method [66]. It can
of the elevation difference between the DEM and the reference data. Random errors are mainly duebe calculated using the
to
average value of the elevation difference between the DEM and the reference data.
the production technique and they are mostly influenced by the quality of the raw data, the processing Random errors
are mainly due
parameters, to the
as well asproduction technique and
the terrain morphology andthey
theare mostly influenced
vegetation by the
[67]. They are qualitythrough
evaluated of the raw
the
data,
calculation of the standard deviation of the elevation difference. Gross errors are outliers They
the processing parameters, as well as the terrain morphology and the vegetation [67]. are
resulting
evaluated
from faultsthrough
during the
the calculation
productionof ofthe
the standard
DEM. deviation of the elevation difference. Gross errors
are outliers resulting from faults during the production of the DEM.

Figure 4. Main behaviours of DEM error.

Figure 4. Main behaviours of DEM error.
As any other geographic data, the elevation in a DEM has a spatial behaviour and so does its
AsIndeed,
error. any other geographic
although a partdata, theelevation
of the elevationerror
in a is
DEM has a itspatial
random, behaviour
is often spatiallyand so does its
autocorrelated,
error. Indeed, although a part of the elevation error is random, it is often spatially autocorrelated,
which means that neighboring points tend to have comparable errors [68,69]. Thus, the random error
which means that neighboring points tend to have comparable errors [68,69]. Thus, the random error
is a combination of two types of random variables: one is spatially autocorrelated whereas the other
Remote Sens. 2020, 12, 3522 9 of 36

is a combination of two types of random variables: one is spatially autocorrelated whereas the other is
a pure noise [70]. This autocorrelation can also be anisotropic [71]. Then, the elevation of a particular
point in a DEM is the sum of its true elevation and the aforementioned errors:

ẑi = zi + µ + ε0 i + ε00 i (1)

where ẑi is the altitude in the DEM, zi is the true altitude on the terrain, µ is the systematic error (bias),
ε0 i is the spatially autocorrelated random error, and ε00 i is the spatially non-autocorrelated random
error (pure noise).
External quality assessment consists in analyzing the whole set of elevation discrepancies
(µ + ε0 i + ε00 i ), calculated between the DEM and a set of ground control points (GCPs), based on
simple statistical indicators such as the mean, the standard deviation, the root mean square error
(RMSE), the maximum error etc. [69]. These indicators are used to assess the DEM error in terms
of altimetric accuracy and precision [67]. Note that the terms related to data uncertainty are often
used in a confusing way [72]. The third edition of the Photogrammetric Terminology [73] defines
accuracy as “the closeness of the result of a measurement, calculation or process to the true, intended
or standard value”, and precision as “the repeatability of a result, including the number of significant
figures quoted, regardless of its correctness” (further clarification on the meaning of these words and
on the pitfalls of translation can be found in [74]).
The most commonly used statistical indicator is the RMSE [65], which provides a reliable indication
on the altimetric DEM error considering that the sample size N of the reference data (i.e., the number
of GCPs) is big enough [43]:
s
Pn 2
i=1 (ẑi − ziRef )
RMSE = (2)
N
The relationship between the RMSE, the mean, and the standard deviation of the error is
the following:
p
RMSE = µ2 + σ2 (3)

The statistical distribution of the error is often Gaussian [28], although some studies suggest
that this hypothesis is not always valid [75]. To verify the validity of this hypothesis, Q-Q plots
(quantile-quantile plots [63,75]) can be used. This can be helpful to reveal a non-normal distribution
and to suggest more adapted statistical indicators for error analysis, like the median, which is more
robust, i.e., not affected by extreme error values. If we consider that the error follows a normal
distribution N(µ,σ), then it is possible to define the confidence level of the error that corresponds to
the percentage of points whose error value lies in the confidence interval. For instance, the intervals
[µ − σ; µ + σ] and [µ − 1.96σ; µ + 1.96σ] correspond to linear errors with 68% (LE68) and 95% (LE95)
confidence levels, respectively.
The elevation error propagates through elevation derivatives like slope, aspect and curvature,
leading to erroneous drainage network or watershed delineation for instance [76,77]. Although the
computation of the slope or other derivatives may lead to different results according to the selected
algorithms [78–82], the error propagation always generates an uncertainty in the resulting product
(e.g., slope map) that needs to be evaluated due to its wide use in many applications of DEMs
in geoscience [68,82–84]. In general, the elevation derivatives are very sensitive to the spatial
autocorrelation of the error [28]. Indeed, since the elevation derivatives are calculated using a
neighborhood, their quality depends on the error of the points from which they have been extracted
as well as on the autocorrelation of their errors. Then, it is possible to have a low positional quality
but a high quality of the derivatives as in the case of a big systematic error and a small random error,
and vice versa.
Most DEMs are delivered with a simple value of the RMSE or a standard deviation of the elevation
error, and this value is considered as representative of the overall quality of the DEM, where the spatial
or mountainous areas. Thus, to assess the error propagation from the elevation to its derivatives, an
error model has to be built (Figure 5).
This error model includes the elevation error as well as its autocorrelation and it can be
modelled using a semivariogram. The following formula represents an exponential error model [85]:

Remote Sens. 2020, 12, 3522 10 of(4)

36
γ(h) = σ (1 − e )
where γ is the semivariance, h is the horizontal distance, σ is the sill and φ is the range. An error
distribution
model having of the errornugget
a null is completely ignored [71].
(i.e., y-intercept), This is due
a minimal silltoand
the fact that the evaluation
a maximum practical of the
range
spatial autocorrelation of the error requires a dense sample of control points,
provides a high spatial autocorrelation and then a high quality of elevation derivatives. The which in many cases is
not available.
generation of However,
the modelalthough
is basedthe on RMSE is well adapted
the second-order for the hypothesis,
stationary positional quality
whichassessment of a
supposes that
DEM, it may not be informative about the quality of elevation derivatives
the mean and the standard deviation of the error are constant in the DEM and that the since it does not consider
the spatial behaviour
autocorrelation of theof error
the error [27]. This[69].
is isotropic behaviour can reveal
The isotropic systematicwhich
hypothesis, trendsisduenottonecessarily
inaccurate
orbit or sensor
verified model parameters
in the natural relief due toastectonic
well asorlocal error autocorrelation,
hydrographic phenomena, often
mayduealsotobethe influence by
questioned of
landscape characteristics, e.g., higher error in forested or mountainous areas. Thus,
the DEM production process, i.e., the data acquisition geometry (as in the case of side looking radar),to assess the error
propagation
the processing from the elevation
method to its derivatives,
(image matching an error
along lines) or themodel has to be (raster
grid structure built (Figure
effect)5).
[86].

Figure 5. Method for the elaboration of an elevation error model.

Figure 5. Method for the elaboration of an elevation error model.
This error model includes the elevation error as well as its autocorrelation and it can be modelled
The error propagation
using a semivariogram. can be measured
The following either analytically
formula represents an exponential whenerror
possible
modelor[85]:
numerically
through simulation-based (e.g., Monte Carlo) methods  when it is not analytically possible [68]. The
2 h
−on
goal is to evaluate the impact of data uncertainty
γ(h) = σz 1 − e ϕ the subsequent extracted derivatives (4)
[77,85,87,88]. For instance, the analytically measured error propagation in slope (as it is the most
important
where morphological
γ is the index
semivariance, [89])
h is the calculated
horizontal by trigonometry
distance, σ2 is the sillisand
the ϕ
following [90]:An error model
is the range.
z
having a null nugget (i.e., y-intercept), a minimalz −sillz and a maximum practical range provides a high
S= (5)
d
spatial autocorrelation and then a high quality of elevation derivatives. The generation of the model
is based on the second-order stationary hypothesis,
σ which supposes that the mean and the standard
σ =
deviation of the error are constant in the DEM 2(1that
and − rthe )autocorrelation of the error is isotropic [69].
(6)
d
The isotropic hypothesis, which is not necessarily verified in the natural relief due to tectonic or
where S is the slope, d is the horizontal distance separating points 1 and 2, σ is the error of the
hydrographic phenomena, may also be questioned by the DEM production process, i.e., the data
slope and r is the autocorrelation coefficient of the error. Moreover, in the case of finite
acquisition geometry (as in the case of side looking radar), the processing method (image matching
differences, which give a more accurate slope estimation [91], the slope error for a third-order finite
along lines) or the grid structure (raster effect) [86].
The error propagation can be measured either analytically when possible or numerically through
simulation-based (e.g., Monte Carlo) methods when it is not analytically possible [68]. The goal
is to evaluate the impact of data uncertainty on the subsequent extracted derivatives [77,85,87,88].
For instance, the analytically measured error propagation in slope (as it is the most important
morphological index [89]) calculated by trigonometry is the following [90]:

z2 − z1
S= (5)
d
σz
q
σS = 2(1 − rz1 z2 ) (6)
d
Remote Sens. 2020, 12, 3522 11 of 36

where S is the slope, d is the horizontal distance separating points 1 and 2, σS is the error of the slope and
rz1 z2 is the autocorrelation coefficient of the error. Moreover, in the case of finite differences, which give
a more accurate slope estimation [91], the slope error for a third-order finite difference weighted by
reciprocal of squared distance method [79,92–94] is obtained by the following formula [85]:
√  √ 
3σ2z + 4C(w) − 2C(2w) − 4C 5w − C 8w
σ2S = (7)
16w2
h
C(h) = σ2z − γ(h) = σ2z .e− ϕ (8)

where w is the mesh size and C is the spatial autocovariance.

In a numerical approach, a multiple realization of DEMs is produced based on the error model,
where the precise realization is considered to be one of them. For each realization, the targeted
derivative is extracted, which provides a sample for this derivative. Then, a probabilistic method is
used to evaluate the quality of this derivative [25,85].
All the aforementioned descriptors aim at evaluating the absolute vertical quality and the
derivative quality. In the horizontal dimension, a planimetric shift in the input point cloud has an
impact on DEM elevations that behaves as a planimetric error, which can also be evaluated in both
absolute and relative terms. The absolute planimetric quality is obtained by comparing the planimetric
positions to reference data and adapted statistical descriptors are used to quantify the error like for the
altimetric error as shown above. The relative positional quality is measured between points identified
in the DEM. For instance, the Euclidian distance between two points is obtained using the coordinates
of these points; if we consider these two points have the same error, then the measured distance is not
affected by this error, as in the case of a constant planimetric shift of the DEM.

4.1.2. Simulation-Based DEM Production Method Validation

An interesting extension of external validation consists in processing simulated images to derive
a DEM and to compare the output DEM with the input DEM used to simulate the synthetic image
dataset. In this case, the goal is not to evaluate a particular DEM, but a topographic method used for
DEM production.
This approach implies that computational tools must be available to simulate realistic images
based on a landscape model and a sensor model. Such tools are developed to study radiative transfer
processes and to test remote sensing strategies [95,96]. The landscape model includes its geometry
(DEM and objects over the ground: trees, buildings etc.) and all the characteristics that are relevant to
compute the electromagnetic signal received by the sensor over this particular landscape. The sensor
model includes the description of the sensor itself (spectral interval and any technical property needed
to define the geometry of the images and to compute pixel radiometric values) as well as the acquisition
geometry through the platform position and orientation typically.
Figure 6 shows the workflow that can be implemented for simulation-based validation. Comparing
the input and output DEMs allows the estimation of the error produced by the topographic restitution
method as well as its spatial behaviour. It can also help to better understand the effects of error
propagation. This approach has long been implemented to evaluate the expected performances of
image acquisition strategies as well as processing algorithms for different mapping methods based on
remote sensing data, i.e., photogrammetry [97], lidar [98], and radar methods [99,100]. More recently it
has also been applied to terrestrial mobile mapping [101].
Remote Sens. 2020, 12, 3522 12 of 36
Remote Sens. 2020, 12, x FOR PEER REVIEW 12 of 36

Figure 6. General workflow of simulation-based evaluation of topographic restitution methods.

Figure 6. General workflow of simulation-based evaluation of topographic restitution methods.
Although this approach may suffer from a lack of realism that could create biases in the analysis,
Although this approach may suffer from a lack of realism that could create biases in the
it has several advantages such as the possibility to vary the experimental conditions:
analysis, it has several advantages such as the possibility to vary the experimental conditions:
• The topographic restitution method can be tested over a variety of landscapes, leading to more
• The topographic restitution method can be tested over a variety of landscapes, leading to more
comprehensive conclusions than over a particular DEM.
comprehensive conclusions than over a particular DEM.
•
• ItIt can
can also
also be
be tested
tested with
with aa variety
variety of
of sensor
sensor parameters
parameters and
and orbit
orbit configurations,
configurations, leading
leading to
to
recommendations for optimum image acquisition conditions if the images are to be processed
recommendations for optimum image acquisition conditions if the images are to be processed for
DEM
for DEMproduction.
production.
Regarding the lack
Regarding the lack of
of realism,
realism, Figure
Figure 66 suggests some refinements
suggests some refinements to
to overcome
overcome this
this limitation,
limitation,
for instance:
for instance:
•
• The
Theinput
inputDEM
DEMcancan be
bereprocessed,
reprocessed, either
either to to exaggerate
exaggerate the elevation amplitude, or to introduce
topographic
topographic details such
suchas asmicrorelief
microrelieforor buildings
buildings (note
(note thatthat all these
all these changes
changes in theininput
the input
DEM
DEM
can becan be parametrically
parametrically controlled,controlled, allowing analytical
allowing analytical sensitivityFractal
sensitivity studies). studies). Fractal
resampling
resampling can be toimplemented
can be implemented produce a more to realistic
produce a more
input realistic
DEM [102], input DEM [102],
and geomorphology and
provides
geomorphology provides
criteria to verify this realismcriteria to verify
requirement this realism
in accordance requirement
with in accordance
the principles with the
of internal validation
principles
which we of internal
will validation
see further [103]. which we will see further [103].
•
• Sensor
Sensorparameter
parameteruncertainties
uncertaintiescancan be
be introduced
introduced to to consider
consider the
the fact that the DEM production
method
methodalways
alwaysuses
usesanan approximation
approximation of of the
the exact
exact parameters.
parameters.
Since aa dense
Since dense reference
reference dataset
dataset is
is available,
available, the
the principles
principles of
of external
external quality
quality assessment
assessment can
can be
be
fully applied. However, DEM series based on simulated images can also be evaluated
fully applied. However, DEM series based on simulated images can also be evaluated with an internalwith an
internalassessment
quality quality assessment approach,
approach, i.e., with i.e., with no reference
no reference data. data.

4.2. Internal Quality Assessment

Internal quality assessment is implemented with no ground control. It makes use of criteria of
realism based
based on
on aapriori
prioriknowledge
knowledgeofofthe
thegeneral
generalbehaviour
behaviourofof
allall topographic
topographic surfaces
surfaces (intuition
(intuition or
or geomorphological
geomorphological science).
science).
Remote Sens. 2020, 12, 3522 13 of 36
Remote Sens. 2020, 12, x FOR PEER REVIEW 13 of 36

4.2.1. Visual
Visual Control
As recalled above, land surfaces are very familiar to each of us, so that we can easily complain
about
about aa lack
lack of
of realism
realism in
in their
their representation, even in
representation, even in unknown
unknown regions.
regions. This
This makes
makes visual
visual control
control aa
very powerful approach, although often neglected, for DEM quality assessment
very powerful approach, although often neglected, for DEM quality assessment [104]. The expectations [104]. The
expectations
with regards with regards
to relief to reliefhave
modelling modelling have long
long inspired poetsinspired poets and
and painters, painters,
though withthough with a
a noticeable
noticeableinprogress
progress the past in the past For
centuries. centuries. Forthe
instance, instance,
mountainousthe mountainous
backgroundbackground
landscape behindlandscapethe
behind the Mona Lisa portrait by Leonardo da Vinci (Figure 7), though inspired
Mona Lisa portrait by Leonardo da Vinci (Figure 7), though inspired by real Italian landscapes [105], by real Italian
landscapes [105], is clearly
is clearly unrealistic. unrealistic.
At Leonardo’s timeAtthe
Leonardo’s
landscapetime
wasthe landscape
a mere backdrop wastoa fill
mere thebackdrop
background to fill
of
the background
portraits or scenes, of until
portraits or scenes,
the Dutch until
painters of the Dutch
the 17th painters
century of the
began 17thinterest
to find century in began to find
the landscape
interest in the landscape
itself, leading itself, leading
to more realistic shapesto(whether
more realistic shapes
observed or (whether
imagined). observed or imagined).
This natural This
expectation
natural
with expectation
regards to reliefwith
shape regards to relief
modelling shapewhy
explains modelling explains
it is so natural to why it is
rely on so natural
visual controltoforrely
DEMon
visual control for
quality assessment. DEM quality assessment.

7. Unrealistic
Figure 7.
Figure background landscapes
Unrealistic background landscapes in
in Renaissance paintings (example
Renaissance paintings (example of
of Mona
Mona Lisa
Lisa from
from
Leonardo da
Leonardo da Vinci).
Vinci).

Two usual
Two usualmethods
methods areare
based
basedon grey levelslevels
on grey to represent DEMs, DEMs,
to represent namely,namely,
hypsometry and hillshade
hypsometry and
(or shadowing). Hypsometry assigns a grey level to an elevation
hillshade (or shadowing). Hypsometry assigns a grey level to an elevation interval so interval so that a continuous greythat
scalea
is available to
continuous represent
grey scale isthe verticaltoamplitude
available represent from the lowest
the vertical point (black)
amplitude from the to the highest
lowest pointone (white),
(black) to
whereas
the highest hillshade consists
one (white), in illuminating
whereas hillshade the ground
consists surface by a the
in illuminating virtual
ground lightsurface
source,bypreferably
a virtual
located in the Northern side (in the general case where the map
light source, preferably located in the Northern side (in the general case where the map is oriented with North upwards)
is oriented so
that the
with landforms
North upwards) are so
illuminated from the top
that the landforms are to the bottomfrom
illuminated of the
thescreen,
top towhich
the bottomis the of
usual way to
the screen,
see the real land surface.
which is the usual way to see the real land surface.
Since most
Since most DEMDEM artifacts
artifacts have
have significantly
significantly more more impact
impact on on slope than on
slope than on elevation,
elevation, we we can can
expect that hillshade (in which grey level variations result from slope variations)
expect that hillshade (in which grey level variations result from slope variations) is more efficient is more efficient than
hypsometry
than hypsometryfor shape qualityquality
for shape assessment. DEM viewing
assessment. DEM viewingstrategies based on
strategies hillshade
based are moreare
on hillshade efficient
more
for detecting gross errors [106] and resampling artifacts [107]. This is confirmed
efficient for detecting gross errors [106] and resampling artifacts [107]. This is confirmed by Figure by Figure 8: the river
8:
network at the North of the area has almost completely disappeared
the river network at the North of the area has almost completely disappeared in the DEM with in the DEM with Gaussian noise;
step-like artifacts
Gaussian appear in
noise; step-like the 8-bit
artifacts codedinDEM;
appear smallcoded
the 8-bit drainages
DEM;disappear in the DEM
small drainages subsampled
disappear in the
4 times. These effects are very clear in hillshade but not in hypsometry.
DEM subsampled 4 times. These effects are very clear in hillshade but not in hypsometry.
A challenge
A challenge of of DEM
DEM visualization
visualization is is to
to represent
represent aa 2.5D
2.5D surface
surface on on aa 2D
2D screen.
screen. In In the
the examples
examples
of hypsometry and hillshade the grey scale is devoted to representing
of hypsometry and hillshade the grey scale is devoted to representing either the elevation or either the elevation or the
the
slope. However, an integrated relief visualization product can be generated
slope. However, an integrated relief visualization product can be generated by using a color scale for by using a color scale for
hypsometry and the grey scale for hillshade. Such a product is more complete
hypsometry and the grey scale for hillshade. Such a product is more complete than mere shadowing than mere shadowing
thanks totoitsitscolor-induced
thanks color-induced elevation
elevationperception, and itand
perception, can even
it canbe even
transformed into a 3D perspective
be transformed into a 3D
view to offer an increased perception of elevations. However,
perspective view to offer an increased perception of elevations. However, the hillshadethe hillshade effect remains the major
effect
remains the major contribution to visual quality control since it is more sensitive to potential artifacts
or unrealistic shapes than the hypsometric effect.
Remote Sens. 2020, 12, 3522 14 of 36

contribution to visual quality control since it is more sensitive to potential artifacts or unrealistic shapes
than the
Remote Sens.hypsometric
2020, 12, x FOReffect.
PEER REVIEW 14 of 36

Figure 8.
Figure 8. Comparison
Comparison ofofhillshade
hillshade(top)
(top)and
andhypsometry
hypsometry (bottom)
(bottom)forfor
DEMDEMartifact detection
artifact (from
detection left
(from
to right: original DEM, DEM with Gaussian noise, 8-bit coded DEM, DEM subsampled 4
left to right: original DEM, DEM with Gaussian noise, 8-bit coded DEM, DEM subsampled 4 times). times). Source:
DEM produced
Source: by contour-line
DEM produced interpolation
by contour-line in a 1:20,000
interpolation topographic
in a 1:20,000 map of Lebanon.
topographic map of Lebanon.

A visual approach of DEM quality assessment can also be applied to vector information derived
A visual approach of DEM quality assessment can also be applied to vector information derived
from the DEM. This is usually done for vector data quality control in GIS, consisting in verifying
from the DEM. This is usually done for vector data quality control in GIS, consisting in verifying
topological properties such as connectivity [41]. In the case of DEM quality assessment, this approach
topological properties such as connectivity [41]. In the case of DEM quality assessment, this
can typically be applied to the drainage network, which may have geometrical and topological errors,
approach can typically be applied to the drainage network, which may have geometrical and
as illustrated in Figure 9. However, it is not straightforward to decide if the artifacts observed in the
topological errors, as illustrated in Figure 9. However, it is not straightforward to decide if the
extracted drainage network are caused by the DEM or by the drainage extraction algorithm, although it
artifacts observed in the extracted drainage network are caused by the DEM or by the drainage
could be feasible by comparing different hydrographic networks extracted from the same DEM with
extraction algorithm, although it could be feasible by comparing different hydrographic networks
different algorithms.
extracted from the same DEM with different algorithms.
The visual analysis of a raster DEM or a vector product derived from a DEM, which can be helpful
for preliminary control, is an inductive approach based on our repeated experience of landform viewing
in everyday life. It is an intuitive implementation of internal DEM quality assessment, which can also
be implemented with quantitative criteria, although with no ground control.

Figure 9. Detail of the hydrographic network extracted from SRTM (red) and Topodata (green).
Source: [108].
 from the DEM. This is usually done for vector data quality control in GIS, consisting in verifying
topological properties such as connectivity [41]. In the case of DEM quality assessment, this
approach can typically be applied to the drainage network, which may have geometrical and
topological errors, as illustrated in Figure 9. However, it is not straightforward to decide if the
artifacts observed in the extracted drainage network are caused by the DEM or by the drainage
Remote Sens. 2020, 12, 3522 15 of 36
extraction algorithm, although it could be feasible by comparing different hydrographic networks
extracted from the same DEM with different algorithms.

Figure 9. Detail of the hydrographic network extracted from SRTM (red) and Topodata (green).
Figure 9. Detail of the hydrographic network extracted from SRTM (red) and Topodata (green).
Source: [108].
Source: [108].
4.2.2. Quantitative Internal Quality Assessment
Internal quality assessment is based on the hypothesis that all topographic surfaces are supposed
to fulfil some universal rules. This approach is relevant to evaluate the quality of a DEM in terms of
shape rendering, for which external quality assessment is less straightforward. Indeed, the comparison
of a terrain shape as modelled in the evaluated DEM with regards to a reference dataset is limited
by the difficulty of defining a suitable metric, and by the fact that height derivatives (slope, aspect,
curvature) and therefore shapes are not stable with regards to scale, while elevation is stable [109–111],
as discussed below in Section 6. Therefore, the comparison of two DEMs, which may not necessarily
have the same native mesh size, may bring some information on elevation accuracy, but probably not on
slope accuracy or shape realism. For this reason, it is more convenient to evaluate the geomorphological
realism of a DEM through a rule-based approach, i.e., by analyzing its compliance to a number of
general rules that the terrestrial relief is supposed to fulfil.
Two different requirement levels can be considered, which may be called strong and weak
requirements [108]. A strong shape modelling requirement is defined by physical rules and a terrain
that does not fulfil them is impossible. A typical example is the downward streaming of the water.
A practical consequence is that the detection of local sinks along the hydrographic network provides a
criterion to locate and quantify DEM errors [108,112], except in the case of natural sinkholes (dolines)
which only appear in specific geological contexts. Such a quality assessment method is easy to
implement, and it does not need any reference data. Once the hydrographic network has been
automatically extracted from the DEM, the sinks are detected, and their occurrence is analyzed.
Sink density and mean depth are quantitative criteria that are easy to calculate and useful to evaluate
the compliance of this physical rule. Their spatial distribution is also an indicator of error zones.
A weak shape modelling requirement is more difficult to define. It is based on statistical rules
and a terrain that does not fulfil them is improbable, although not impossible. It is a quantitative
implementation of artifact detection, with criteria to identify unrealistic shapes in a DEM [113].
In both cases (i.e., visual or quantitative) the lack of realism is assessed based on geomorphological
skills. The most usual realism criteria are inspired by the principle of the first law of geography,
which states that “everything is related to everything else, but near things are more related than distant
things” [114,115]. Some rules are based on the fractal behaviour of the topographic surface for terrains
modelled by hydric erosion, which is the most general case on the Earth’s surface [116,117].
A typical rule is provided by Horton’s law, also based on the fractal hypothesis, which states that
if the river streams are classified by Strahler orders [118], the total number of streams of a given order
decreases in a geometric progression when the order increases [119]. In other terms, the logarithm of
Remote Sens. 2020, 12, 3522 16 of 36

the total number of streams of order n decreases linearly as a function of n. Figure 10 illustrates an
interesting contribution of Horton’s law in a study of the quality of the Topodata DEM obtained by
oversampling SRTM in Brazil [120]. As expected according to Horton’s law, a linear tendency with
R2 = 0.9836 is observed over 12 orders (Figure 10a), which means that the relief described by the DEM
is nearly fractal. However, it appears that excluding the values obtained for orders 1 and 2, which are
slightly higher than the linear trend, and plotting the results for orders 3 to 12 only, increases the
linearity with R2 = 0.9965 (Figure 10b). This suggests that the Topodata resampling process slightly
increases the number of short streams with regards to a fractal hypothesis, which could be confirmed
Remote Sens. 2020, 12, x FOR PEER REVIEW
in the evaluation of stream extraction among several DEMs (Topodata and SRTM included) by 16 of 36
[121].

Figure 10. Fractal behaviour of the hydrographic network as illustrated by the variation of the logarithm
Figure 10. Fractal behaviour of the hydrographic network as illustrated by the variation of the
of the number of streams for each Strahler order as a function of this order, for orders 1 to 12 (a) and 3
logarithm of the number of streams for each Strahler order as a function of this order, for orders 1 to
to 12 (b). Source: [108].
12 (a) and 3 to 12 (b). Source: [108].
Similarly, if the area is divided in watersheds, the cumulative area of the n largest watersheds
Similarly,
increases linearly if the
witharea is divided
n. All these rules in watersheds,
aim at verifying the that
cumulative
the DEM area
willofprovide
the n largest
reliablewatersheds
indicators
increases linearly with n. All these rules aim at verifying that
for applications based on landforms, which cannot be guaranteed by the absolute positional the DEM will provideaccuracy
reliable
indicators
as calculated forin applications
external DEMbased quality onassessment.
landforms, which cannot be guaranteed by the absolute
positional accuracy as calculated in
Since internal quality assessment consists external DEMinquality assessment.
confronting the data with a rule (Horton law,
Since internal quality assessment consists
rivers always streaming downward, etc.), it is worth noting thatin confronting the thedatadiscrepancies
with a rule (Horton
between law,
therivers
data
and the ideal behaviour can provide quality indicators, such as the number and the mean depthand
always streaming downward, etc.), it is worth noting that the discrepancies between the data of
the sinks
the ideal along
behaviour can provide
the drainages (andquality
this wouldindicators,
even be such as the number
a criterion andsink
to validate the removal
mean depth of the
methods),
sinks
or the along the drainages
R2 coefficient (and this
in the control would
of the even bewith
compliance a criterion
Horton’s to validate
law. Thissinkis why removal methods),
the internal qualityor
the R² coefficient
assessment method, in the controlnot
although of using
the compliance
any referencewithdata,
Horton’s law. This
can really is why theasinternal
be considered quality
a quantitative
assessment method, although not using any reference
approach, with objective criteria to decide if a DEM is acceptable or not. data, can really be considered as a
quantitative approach,
This statistical with objective
approach of internal criteria
qualityto assessment,
decide if a DEM which is acceptable or not. for unrealistic
consists in looking
This statistical approach of internal quality assessment,
shapes in a DEM, has similarities with the use of the famous Benford’s law [122,123]which consists in looking for unrealistic
to detect financial
shapes in a DEM, has similarities with the use of the famous Benford’s
frauds based on the automated detection of unrealistic statistical distributions in the accounts law [122,123] to detect
[124].
financial frauds based on the automated detection of unrealistic statistical
Indeed, this law uses a very widely verified property: in most series of numbers found in many fields distributions in the
accounts [124]. Indeed, this law uses a very widely verified property:
of science, economy, sports etc. the occurrence of n (n = 1 to 9) as the first digit of each number in most series of numbers
found in many
decreases rapidly fields
as aoffunction
science, economy,
of n, following sportsapproximately
etc. the occurrence log (1of +n (n = 1with
1/n), to 9) 1asinthe
30%firstofdigit
the
of each number
numbers, 2 in 18% decreases
and so on.rapidly as a function
Accounts of n,depart
that clearly followingfromapproximately log (1 + 1/n),
this rule are suspected withbeen
to have 1 in
30% of theThe
falsified. numbers,
authors2have
in 18% and so
shown on.Benford’s
that Accountslaw thatcanclearly
revealdepart from this
unrealistic rule are suspected
morphologies in DEMs to
have beenthat
provided falsified. The authors
an appropriate have
metric is shown
used for that
theBenford’s law can reveal
series of numbers. A study unrealistic
carried outmorphologies
on a series
of DEMs shows that elevation is not relevant, while slope and Strahler order respect thecarried
in DEMs provided that an appropriate metric is used for the series of numbers. A study expectedout
on a series of DEMs shows that elevation is not relevant,
behaviour and can therefore reveal artifacts [125], as illustrated in Figure 11.while slope and Strahler order respect the
expected behaviour
Internal qualityand can therefore
assessment can even reveal beartifacts [125], aswhen
more powerful illustrated
the DEMin Figure 11.
production method is
known including the parameters used for image acquisition, elevation calculation and resampling.
Indeed, it is then possible to know what to expect from the quality of the product and this can guide
the search for artifacts and make it easier to distinguish between existing landforms and processing
artifacts. This approach is particularly interesting to reveal resampling effects as illustrated in the
following two examples.

Figure 11. Control of the compliance of a DEM with Benford’s law for two metrics: elevation and
 of each number decreases rapidly as a function of n, following approximately log (1 + 1/n), with 1 in
30% of the numbers, 2 in 18% and so on. Accounts that clearly depart from this rule are suspected to
have been falsified. The authors have shown that Benford’s law can reveal unrealistic morphologies
in DEMs provided that an appropriate metric is used for the series of numbers. A study carried out
Remote
on a Sens. of 12,
2020,
series 3522 shows that elevation is not relevant, while slope and Strahler order respect
DEMs 17 the
of 36
expected behaviour and can therefore reveal artifacts [125], as illustrated in Figure 11.

Remote Sens. 2020, 12, x FOR PEER REVIEW 17 of 36

Figure 11. Control of the compliance of a DEM with Benford’s law for two metrics: elevation and slope
Figure 11. Control of the compliance of a DEM with Benford’s law for two metrics: elevation and
(source: [125]).
slope
artifacts. (source:
This [125]). is particularly interesting to reveal resampling effects as illustrated in the
approach
following
In thetwo firstexamples.
example [126], a DEM obtained by interpolation of contour lines with a mesh of 40 m
InInternal
the quality
first example assessment can even
[126], description
a DEM be moreinterpolation
obtained powerful when the DEM production method is
was evaluated using a fractal of thebyterrestrial relief.ofModelling
contour lines
thewith a mesh
semivariogram of 40led
m
wasknown including the parameters used for image acquisition, elevation calculation and resampling.
to a evaluated
much lower using a fractal
fractal description
dimension of the terrestrial
for distances relief.
of 40 to 200 m Modelling
(D = 2.07) the than semivariogram
for distances of led400to
atoIndeed,
much
1200 m
it is then
lower
(D =fractal
possible to know
dimension
2.25), and forwhat
the horizontal
to expect
distances
distance
from
of between
40 the m
to 200 quality= of
the (D
contour
the than
2.07) product
lines was
and
formost this can
distances
often of
guide
400 to
between
the search for artifacts and make it easier to distinguish between existing landforms and processing
1200
200 andm (D400 = 2.25),
m, whichand the horizontal
confirms distance
that the between
interpolator theacontour
had smoothing lineseffect.
was most often between
In addition, 200
the fractal
and 400 m,over
dimension whichshortconfirms
distances that the interpolator
revealed an anisotropyhadofa the
smoothing effect.
interpolation byInshowing
addition,thatthe
thefractal
DEM
dimension over short distances revealed an anisotropy of the interpolation
was even smoother in the directions NS and EW (directions of the interpolator) than in the diagonal by showing that the DEM
was even smoother
directions. in the directions
This is a fractal NS and
approach which is EW (directions
not so different of
fromthethe
interpolator) than in the
use of information diagonal
entropy as a
directions.
measure ofThis DEMisquality
a fractal[127].
approach which is not so different from the use of information entropy as
a measure
The secondof DEM qualityis[127].
example part of a study previously cited [108]. The directional histograms of the
aspectThe second example
(direction is part ofslope)
of the maximum a studywerepreviously
compared cited [108]. two
between The DEMs,
directional
namelyhistograms
Topodata ofand
the
aspect
SRTM. (direction
The result of in the maximum
Figure 12 showsslope) wereoverall
the same compared between
elliptic shape in twobothDEMs, namelyrelated
histograms, Topodatawithandthe
SRTM. The result in Figure 12 shows the same overall elliptic shape in both
general trend in the regional topography, but the SRTM aspect histogram exhibits higher occurrences histograms, related with
the
in a general
few specifictrenddirections,
in the regional topography,exactly
which correspond but the SRTM
to the aspectdirections
principal histogramofexhibits
the raster higher
grid,
occurrences in a few specific directions, which correspond
while Topodata seems to have redistributed the aspects in a more realistic way. exactly to the principal directions of the
raster grid, while Topodata seems to have redistributed the aspects in a more realistic way.

Figure 12. Aspect directional histograms in Topodata and SRTM and identification of main directions
Figure 12. Aspect directional histograms in Topodata and SRTM and identification of main
(source: [108]).
directions (source: [108]).

Hence when the topographic indices highlight well known characteristics of the sampling method,
Hence when the topographic indices highlight well known characteristics of the sampling
it confirms that it is probably not the terrain that naturally has these characteristics but an artifact in
method, it confirms that it is probably not the terrain that naturally has these characteristics but an
the DEM.
artifact in the DEM.

5. DEM Validation at Different Levels

The different methods reviewed in the previous section for the detection and evaluation of
DEM errors can be considered at several levels. Indeed, several steps follow one another from
preliminary terrain observation to application-driven product generation. The error propagates
from a level to the next one and quality requirements can be expressed at each level. In this section,
we consider quality requirement transfer from point cloud to grid surface model (Section 5.1) and
Remote Sens. 2020, 12, 3522 18 of 36

5. DEM Validation at Different Levels

The different methods reviewed in the previous section for the detection and evaluation of DEM
errors can be considered at several levels. Indeed, several steps follow one another from preliminary
terrain observation to application-driven product generation. The error propagates from a level to the
next one and quality requirements can be expressed at each level. In this section, we consider quality
requirement transfer from point cloud to grid surface model (Section 5.1) and from grid surface model
to derived topographic features (Section 5.2). Moreover, the quality assessment of a global DEM is not
straightforward when based of local data analysis (Section 5.3).

5.1. From Point Cloud to Grid Surface Model

The first result of elevation computation is a point cloud in which the individual quality of each
point is influenced by the selected technique and by the parameters chosen for data acquisition (sensor,
orbit . . . [128–135]) and processing (e.g., matching template size, radar slave image interpolation
method . . . [84,136–140]), as well as by the local slope and the landcover [84,140–145]. There is an
almost linear relationship between the DEM elevation error and the terrain slope [146–149], although the
impact on the landform quality is more severe in low-slope areas [85]. This point cloud can be obtained
directly in the case of laser scanning methods, or through image processing techniques in the case of
photogrammetry or SAR interferometry for instance. The output DEM is a surface model obtained
from the point cloud with a specific grid structure (e.g., raster or TIN), using an interpolation process if
resampling is necessary. The grid surface model inherits the errors of the input point cloud, but the
resampling process contributes too. For example, comparing different base to height ratios (or different
sizes of the matching template) in respective DEMs is not fully relevant if further resampling has
modified the initial DEMs.
In fact, the relative contribution of the input data and the interpolation depends on the criterion
considered to evaluate the DEM:

• In terms of absolute vertical accuracy, the input data are essential: the errors of the input point
cloud are generally autocorrelated (due to orbit, relief . . . ) so that they remain in the resampled
DEM whatever the selected interpolation method.
• In terms of shape realism, the interpolation plays a major role since it can remove or create artifacts
(noise, stripe, pixelation, etc.). The resampling step may filter the noise of the input point cloud
and therefore improve the quality in terms of elevation accuracy. In contrast, the interpolation
implemented for resampling may produce an exaggerated smoothing effect or a raster grid effect,
resulting in a quality degradation in terms of shape rendering. Indeed, the choice of an interpolation
algorithm affects not only the elevation absolute accuracy but also the geomorphometric
indices [150], and this effect depends on the local terrain morphology [151–154]. The effect
of an interpolator on the DEM quality depends on both the DEM production technique and
the application-oriented user requirements [43,155,156]. Hengl and Evans [70] distinguish three
aspects for classifying interpolation methods, namely, the smoothing effect (exact or approximate
interpolation), the proximity effect (local or global interpolation) and the stochastic hypothesis.
For example, an exact interpolation method, such as linear interpolation, is recommended if
the data are very dense and accurate, while a smoothing method should be used if the data are
noisy. Mitas and Mitasova [157] state that the description of the smoothing and tension effects
and the consideration of ridges and streams are the most important evaluation criteria for an
interpolation method.

Another study by [152] also tested the relevance, at different scales and according to the terrain
morphology, of five interpolators, namely, inverse distance weighting (IDW), ordinary (OK) and
universal (UK) kriging, multiquadratic radial basis function (MRBF), and regularized spline with
tension (RST). This study concludes that the impact of the different interpolation methods depends
on the density of the original sample, i.e., the interpolation methods show small differences with a
Remote Sens. 2020, 12, 3522 19 of 36

high sample density, whereas this difference becomes more important when the sample density is low.
This is quite an intuitive result, since with a low density, the elevation calculation depends more on
the interpolator than on the data. On a low density sample and on different scales, the IDW provides
more accurate results in mountainous regions, whereas the OK performs best in regions with naturally
smooth relief, which is consistent with [153]. Finally, the authors conclude that three criteria must
be taken into consideration when selecting the interpolation method: type of relief, sample density,
and applicability to different spatial scales.

5.2. From Grid Surface Model to Derived Topographic Features

Many applications consist in analyzing topographic objects extracted from the DEM [158–162].
Remote Sens.
It is not the2020, 12, x FOR
purpose ofPEER
this REVIEW
article to 19 of to
deal with the exploitation of these objects, which is specific 36

each application, but it is relevant to consider the defects of these extracted objects as indicators of the
This is particularly the case of the hydrographic network, which has universal properties that
defects of the DEM.
the DEM must respect. The errors of the extracted drainage network depend on the one hand on the
This is particularly the case of the hydrographic network, which has universal properties that
errors of the DEM, and on the other hand on the extraction algorithm [163–165]. These two
the DEM must respect. The errors of the extracted drainage network depend on the one hand on the
influences cannot be considered separately because a robust algorithm can compensate for the errors
errors of the DEM, and on the other hand on the extraction algorithm [163–165]. These two influences
of the DEM by “forcing” the runoff despite a noisy profile. Figure 13 shows the main types of errors
cannot be considered separately because a robust algorithm can compensate for the errors of the
in the drainage network extracted from a DEM. The DEM errors have an impact on the network
DEM by “forcing” the runoff despite a noisy profile. Figure 13 shows the main types of errors in the
geometry (a: absolute location error; b: unrealistic shape) or on its topology (c: wrong topology
drainage network extracted from a DEM. The DEM errors have an impact on the network geometry
although possible; d: impossible topology). It should be noted that the error detection requires
(a: absolute location error; b: unrealistic shape) or on its topology (c: wrong topology although possible;
reference data for (a) and (c), while no reference is needed for (d); (b) is an intermediate situation in
d: impossible topology). It should be noted that the error detection requires reference data for (a) and
which unrealistic shapes can be detected in the DEM itself but a reference network or other external
(c), while no reference is needed for (d); (b) is an intermediate situation in which unrealistic shapes can
information such as a remote sensing image can help to confirm the interpretation. Therefore, both
be detected in the DEM itself but a reference network or other external information such as a remote
external and internal quality assessment methods, as defined in Section 4, provide a complete set of
sensing image can help to confirm the interpretation. Therefore, both external and internal quality
criteria to evaluate the quality of a topographic object derived from the DEM, such as the
assessment methods, as defined in Section 4, provide a complete set of criteria to evaluate the quality
hydrographic network.
of a topographic object derived from the DEM, such as the hydrographic network.

13. Main
Figure 13.
Figure Maintypes
typesofof
errors in the
errors in drainage network
the drainage extracted
network from afrom
extracted DEMa(blue
DEM line: realline:
(blue network;
real
red line: network extracted from the DEM).
network; red line: network extracted from the DEM).

Another example of DEM-derived topographic object is the variable layer of sand, snow, or ice
Another example of DEM-derived topographic object is the variable layer of sand, snow, or ice
over the ground surface. Many studies consist in measuring and analyzing the thickness of these
over the ground surface. Many studies consist in measuring and analyzing the thickness of these
layers based on the difference between two DEMs produced at different dates [166,167]. The quality of
layers based on the difference between two DEMs produced at different dates [166,167]. The quality
the result depends on the cumulative quality of both DEMs. Two main influences may be mentioned
of the result depends on the cumulative quality of both DEMs. Two main influences may be
to illustrate the need for a careful DEM quality specification prior to the calculation of a thickness:
mentioned to illustrate the need for a careful DEM quality specification prior to the calculation of a
thickness:
• The accuracy of the thickness, which can be estimated through the RMSE of the DEM difference, is:
• The accuracy of the thickness, which can be estimated through the RMSE of the DEM difference,
q
is:
RMSEthickness = RMSE2DEM1 + RMSE2DEM2 (9)
RMSE = RMSE + RMSE (9)
which in the case of two DEMs having the same quality can be written:
RMSE = √2 RMSE (10)
hence a very strong requirement in terms of DEM accuracy.
• whatever the accuracy of the two DEMs, any horizontal registration error Δx between them will
Remote Sens. 2020, 12, 3522 20 of 36

which in the case of two DEMs having the same quality can be written:
√
RMSEthickness = 2RMSEDEM (10)

hence a very strong requirement in terms of DEM accuracy.

• whatever the accuracy of the two DEMs, any horizontal registration error ∆x between them will
lead to a vertical error ∆z on the thickness, which is a function of the slope θ:

∆z = ∆xtan θ (11)

The relevant requirement to avoid this effect is a high relative positional accuracy of the two
DEMs. Systematic vertical or horizontal errors can be corrected by 3D matching prior to thickness
computation. However, such a correction has to be implemented very carefully since it may absorb the
thickness of the layer which is supposed to be measured.
These examples confirm the need to specify input DEM quality with suitable criteria before
computing derived topographic objects.

5.3. Challenge of Global DEM Quality Assessment

The concept of global DEM has gained increasing interest since DEMs can be produced from
satellite data which cover the whole planet. The goal of a global DEM can be either to provide
global mapping to a single user (e.g., military program), or to provide users around the world with
easy-to-access altimetric information on each area of interest. In this second case it is actually a
multi-user database and it is not straightforward to reconcile multiple user requirements. Those global
DEMs are extensively used in geoscientific studies [168–171].
Juxtaposing national DEMs would lead to a mosaic product with heterogeneous quality, not to
mention restrictions in some countries [172]. The advantage of a global DEM is to offer a homogeneous
quality, even if it is an illusory ideal because no technique can provide a homogeneous quality with
such a variety of climate, relief and landcover.
The launch of satellites with stereo-viewing capabilities (SPOT-1 in 1986, ERS-1 in 1991, Radarsat
in 1994, etc.) and the development of image processing algorithms for photogrammetry and SAR
interferometry enabled since the 1990s an industrial activity for the production of DEM on demand,
and things got closer to the concept of global DEM with SPOT-5 HRS (2002) designed to increase the
accuracy and the production capacity. In parallel, global DEM projects have been envisaged with
dedicated missions (SRTM) or with systematic use of data from an existing mission (GDEM from ASTER,
WorldDEM from TanDEM-X). Planning a global DEM had to face several challenges. Beyond the
technical constraints linked to the enormous quantity of data, it was necessary to develop automatic
methods for production as well as for quality control.
Studies published on the quality of a global DEM, based either on theory or on the study of a
particular test site where reference data are available, often have a limited scope and do not allow easily
to predict the quality on another site, or to a certain extent only [173–177]. The production of the first
DEM after the launch of a satellite [178] should be considered as a demonstration of feasibility, likely to
indicate an order of magnitude, rather than as a prediction of the quality which can be expected
anywhere in the world. Two important reasons can be cited to explain the unpredictability of a global
quality. The first one is that the global DEM can be a mosaic of DEM tiles of heterogeneous theoretical
quality, due to the fact that the acquisition parameters vary from one tile to another, like the base to
height ratio in the case of optical stereo pairs, and even unpredictable, like the baseline in the case of
repeat-pass SAR interferometry. The second reason is related to the influence of relief and landcover:

• The accuracy of elevation and slope depends on slope and landcover [90,140,146].
• The impact of image acquisition parameters on the DEM accuracy (which must be studied to
optimize the values of these parameters) also depends on slope and landcover [140,179].
Remote Sens. 2020, 12, 3522 21 of 36

Landcover can be considered to locally improve the DEM. For instance, the tree height can be
removed in forested areas to model the ground surface, provided that the forest patches have been
identified and located. This removal can be achieved assuming a constant tree height or through
empirical methods based on tree height estimation at forest edges [180] or on the use of ancillary data
such as ICESat [181].
The validation of a global DEM must rely on many test cases with different orbital configurations
and different landscapes, or on simulations (see Section 4.1.2), to meet multiple requirements:

• Allow a user to predict the quality of the DEM on a given site for a given application.
• Guide product improvements by post-processing or by merging data from different sources.
• Consider the variety of geographic conditions as we have just seen, to guarantee that the
conclusions on the global DEM quality do not depend on particular conditions, or to limit the
scope of the conclusions to a particular landscape.
• Consider different quality criteria as suggested in Section 4 (including the artifacts produced by
resampling, which have been shown to also have a significant impact on the geomorphometric
quality of the final DEM), so that a variety of user needs is considered. Indeed, a global DEM is
typically a multi-user database, for which it is difficult to define quality standards that are suitable
for all potential users.

6. Resolution Dependency in DEM Validation

6.1. From Scale to Resolution

In the case of traditional analog maps, usual cartographic conventions used to define map scales
(e.g., 1:20,000 topographic maps or 1:50,000 geological maps) are based on the fact that we do not expect
the same accuracy and the same shape rendering in maps with different scales. The same happens
with digital elevation models, except that the classical concept of scale (defined as the ratio between a
distance on the map and the corresponding distance on the ground) is meaningless for digital products.
Indeed, a distance cannot be defined on a digital map (stored on a hard disk or similar device) as it can
be on a computer screen or on a paper map. Therefore, the classical concept of scale has to be replaced
by an equivalent indicator, typically the grid mesh size in a raster DEM. The compatibility between a
mesh size and a scale has been widely addressed since the very beginning of digital maps, based on the
size of the smallest detail that can be reasonably represented on a map, typically 1 × 1 mm [182,183].
In this context, the NIMA (National Imagery and Mapping Agency, today NGA) defined a standard
for its own Digital Terrain Elevation Data (DTED) [184], with a range of three levels with increasing
resolution: DTED0 with 30” sampling for gross representation of the Earth’s surface, DTED1 with 3”
sampling approximately equivalent to the contour information represented on a 1:250,000 scale map,
and DTED2 (1”, 1:50,000, resp.). More recently, the increasing availability of sensors and methods for
DEM production with higher resolution has led to the definition of new standards such as NGA’s
HRE standard defined for higher resolution DEMs, from 12 m to 12 cm for both mesh size and vertical
accuracy [185].
The term “resolution” is often confusing because it can apply to different aspects of digital
cartography and is commonly used improperly or without clarity. The resolution is the ability of the
DEM to discriminate objects, for instance in a geological landscape with periodic forms. It is linked
to the smallest detectable wavelength and it is therefore an essential characteristic for the ability of
a DEM to describe the shapes [186–188]. The mesh size (e.g., meters per pixel) is different and can
be irrelevant to the actual DEM resolution. Note that the term “pixel size” is also inappropriate for
DEMs, in which the data refer to a point (i.e., intersection of a vertical axis with the terrain surface),
whereas image data refer to a pixel (i.e., surface element over which a radiometric measurement is
integrated), so that DEMs and images have different physical meanings although DEMs can be stored
in a raster format like images.
 (e.g., airborne lidar point clouds have a lower density under forests than on bare ground);
moreover, a cloud (for photogrammetry) or a forest (for repeat-pass SAR interferometry) can
prevent the calculation of elevations and create holes in the DEM so that the resolution is locally
lost.
• Whatever the 12,
Remote Sens. 2020, input
3522 point cloud, the interpolator imposes its form to some extent, mainly
22 of in
36 low
density areas.
• Even if the initial point cloud is dense, this does not guarantee a high resolution because each
A small mesh size is a necessary condition to guarantee a high resolution. Figure 14 shows that
point can be the result of a calculation requiring information over a neighborhood (e.g., case of a
the availability of SRTM 1” (30 m) instead of SRTM 3” (90 m) resulted in an improved resolution.
large template for image matching [84,140]), and this is not an effect of the interpolation.

Figure 14. Visual comparison between hillshaded SRTM 3” (left) and SRTM 1” (right) DEMs displayed
Figure 14. Visual comparison between hillshaded SRTM 3’’ (left) and SRTM 1’’ (right) DEMs
on NASA website to promote the new 1” product.
displayed on NASA website to promote the new 1” product.
However, the resolution also depends on the input data (point cloud before resampling) and
Forthe
on these different so
interpolation, reasons, the elevation
that a small mesh size,ofalthough
each point of theisDEM
necessary, is influenced
not a sufficient by an
condition to area
around this point.
guarantee This
a high confirms
resolution. thatreasons
Three DEM resolution
can be givenistoclearly
explaindifferent
why the from mesh
smallest size,
terrain and user
shapes
may be lostshould
requirements even with
notabesmall grid mesh:
expressed in terms of mesh size if the relevant criterion is actually the
resolution.
• The point cloud generally has a variable density, with low density areas due to physical reasons
This(e.g.,
resolution requirement
airborne lidar point cloudscan
havealso be density
a lower interpreted in terms
under forests than onofbare
signal processing
ground); moreover,[50].
Shannon'sa cloud (for photogrammetry) or a forest (for repeat-pass SAR interferometry) can preventmust
theorem gives a sampling rule and indicates the minimum rate at which a signal the be
sampled,calculation
at the riskofof irretrievably
elevations losing
and create parts
holes of it.
in the DEM Under certain
so that hypotheses
the resolution which
is locally lost.are globally
verified
• for the terrestrial
Whatever the inputrelief,
point a signal
cloud, thecan be decomposed
interpolator imposes into a sum
its form of harmonics,
to some extent, mainlyin potentially
in low
infinite number, each harmonic (k) being characterized by an amplitude ak, a phase φk, and a
density areas.
frequency
• fk: if the initial point cloud is dense, this does not guarantee a high resolution because each
Even
point can be the result of a calculation requiring information over a neighborhood (e.g., case of a
S(t) = [84,140]),
large template for image matching a sin(2πf t +isφnot
and this ) an effect of the interpolation. (12)

The For these

signal different
should reasons, the
therefore elevation of
be sampled ineach
suchpoint
a wayof the
asDEM is influenced
to preserve by an area around
the maximum frequency
this point. This confirms that DEM resolution is clearly different from mesh size, and user
harmonic fmax, which requires a DEM density of at least two points per cycle in each dimension, and requirements
should not be expressed in terms of mesh size if the relevant criterion is actually the resolution.
thus a sampling frequency of at least twice the highest frequency contained in the signal (the
This resolution requirement can also be interpreted in terms of signal processing [50]. Shannon’s
so-called Nyquist condition).
theorem gives a sampling rule and indicates the minimum rate at which a signal must be sampled,
2 f certain hypotheses which are globally verified (13)
at the risk of irretrievably losing partsf of it. Under
for the terrestrial relief, a signal can be decomposed into a sum of harmonics, in potentially infinite
number, each harmonic (k) being characterized by an amplitude ak , a phase ϕk , and a frequency fk :
X
S(t) = ak sin(2πfk t + ϕk ) (12)

The signal should therefore be sampled in such a way as to preserve the maximum frequency
harmonic fmax , which requires a DEM density of at least two points per cycle in each dimension, and thus
a sampling frequency of at least twice the highest frequency contained in the signal (the so-called
Nyquist condition).
fDEM ≥ 2 fmax (13)
 Remote Sens. 2020, 12, x FOR PEER REVIEW 23 of 36
Remote Sens. 2020, 12, 3522 23 of 36
The terrain frequency spectrum has a wide range of frequencies and it is theoretically infinite so
that it is impossible to represent the complete terrain geometry in a finite size DEM, as it is
The terrain
equivalent frequency
to produce spectrum
a map has a1:1,
at scale wide range oftofrequencies
according and it famous
Umberto Eco’s is theoretically
article infinite
“On The so
that it is impossible to represent the complete terrain geometry in a finite size
Impossibility of Drawing a Map of the Empire on a Scale of 1 to 1” [189]. However, a maximum DEM, as it is equivalent
tospatial
produce a map can
frequency at scale 1:1, according
be reasonably to Umberto
defined, Eco’s
based either onfamous article “On
the resolution The Impossibility
limitation of the imagingof
Drawing
sensor used a Map of thethe
to derive Empire
DEM,on or aonScale of 1 torequirement
the user’s 1” [189]. However,
in terms ofa maximum
object size.spatial frequency
This leads to the
can
specification of a minimum grid density. In the case of irregular grid spacing, the sampling densityto
be reasonably defined, based either on the resolution limitation of the imaging sensor used
derive
can bethe DEM, or
reduced on the
over user’s
smooth requirement
terrain in terms
(i.e., with of object
neglectable size. Thisasleads
curvature) donetofor theprogressive
specificationandof a
compositegrid
minimum sampling
density.with a subsampling
In the case of irregular criterion based on
grid spacing, thethe Laplacian,
sampling densityi.e.,can
a curvature
be reduced index
over
[48,190].
smooth terrain (i.e., with neglectable curvature) as done for progressive and composite sampling with
Given thecriterion
a subsampling importance
basedofonDEM resolution i.e.,
the Laplacian, for athe description
curvature indexof[48,190].
shapes, the mesh size that
conditions
Given the theimportance
resolutionofmust
DEMbe considered
resolution in DEM
for the quality
description assessment
of shapes, for two
the mesh size reasons given
that conditions
hereafter.
the resolution must be considered in DEM quality assessment for two reasons given hereafter.

6.2.
6.2.Stability
StabilityofofTopographic
TopographicIndices
Indiceswith
with Regards
Regards to
to Resolution
Resolution
Informationextracted
Information extracted from
from aa DEMDEM (i.e.,
(i.e., the elevation and its derivatives
derivatives as as well
wellas astopographic
topographic
objectssuch
objects suchas asthe
thedrainage
drainage network)
network) can can be compared with the same same information
informationextracted
extractedfrom fromaa
referenceDEM
reference DEMfor forquality
qualityassessment.
assessment. This is the principle
principle of external
external validation
validation discussed
discussedabove above
(Section 4.1). However,
(Section 4.1). However, the DEM the DEM to be evaluated and the
the reference DEM are usually derivedfrom
reference DEM are usually derived from
differentmapping
different mapping processes.
processes. ForFor example,
example, the reference
the reference DEM may DEMbemay be ofscale
of a finer a finer
(higherscale (higher
resolution),
orresolution),
it may be an or official
it may DEMbe anwith
official DEM with
a reputation fora reliability.
reputationIn forother
reliability.
words, In other
one maywords,
have toone may
compare
have to compare indices in DEMs of different scales, i.e., different resolutions.
indices in DEMs of different scales, i.e., different resolutions. It should be considered that some indices It should be
considered
are unstable that
whensome indices
changing are while
scale, unstable when
others changing
remain stablescale, while others
[109–111,191]. Many remain
studiesstable
about[109–
the
111,191].
impact Many
of the mesh studies
size on about the impact of and
geomorphological the mesh size on geomorphological
hydrographic and hydrographic
indices lead to the expected result that
indices lead
increasing thetomesh
the expected
size has aresult that increasing
smoothing effect on thethe mesh
DEM size has a smoothing effect on the DEM
[192–198].
[192–198].
This is of particular concern in DEM applications that use slopes for hydrological modelling or for
mapping Thiserosion,
is of particular
avalanche,concern in DEM hazards.
or landslide applications that useare
Decisions slopes
made forfor
hydrological
the safety ofmodelling
populations or
for mapping erosion, avalanche, or landslide hazards. Decisions
based on results obtained at a given scale, which may over- or underestimate the risk. Indeed, are made for the safety of
populations based on results obtained at a given scale, which may over- or
increasing the mesh size (i.e., degrading the resolution) smooths the terrain and statistically reduces underestimate the risk.
Indeed,
the slope increasing
values [90],the as mesh
shownsize (i.e., degrading
in Figure the resolution)
15: the elevation histogram smooths the terrain
is unchanged and
(i.e., thestatistically
histograms
reduces the slope values [90], as shown in Figure 15: the elevation histogram
corresponding to the different resolutions are strictly superimposed), while the slope histogram is unchanged (i.e., theis
histograms corresponding to the different resolutions are strictly superimposed),
shifted toward lower slopes. Therefore, studies using DEM to exploit slopes should always specify while the slope
the
histogram is shifted toward lower slopes. Therefore, studies using DEM to exploit slopes should
scale (i.e., the mesh size), otherwise their results are meaningless.
always specify the scale (i.e., the mesh size), otherwise their results are meaningless.

Figure
Figure15. 15.Effect of DEM
Effect subsampling
of DEM on elevation
subsampling and slope
on elevation and histograms. The distributions
slope histograms. of elevation
The distributions of
(a) and slope
elevation (a) (b)
andare computed
slope for an input
(b) are computed forDEM (obtained
an input from a SPOT-4
DEM (obtained from astereo
SPOT-4pair overpair
stereo Lebanon
over
with 10 m with
Lebanon ground10 msampling distance) distance)
ground sampling and for subsampled DEMs with
and for subsampled DEMsthe with
increasing ratios ofratios
the increasing 2, 4, 8.
Aofhigher
2, 4, 8.subsampling
A higher subsampling
ratio leadsratio leadsand
to larger to larger and less numerous
less numerous cells[90]).
cells (source: (source: [90]).
Remote Sens. 2020, 12, 3522 24 of 36

By smoothing the surface model, subsampling tends to reduce the curvilinear length of vertical
profiles and contour lines extracted from the DEM. Similarly, it reduces the length of drainage lines
and watershed boundaries. This has an impact on the calculation of some hydrographic indices used
to characterize watersheds. This is the case of the Gravelius index, used to characterize the compacity
in watershed classifications [199]:
P P
K= √ = 0.28 √ (14)
2 πA A
where P is the perimeter and A is the area. The problem is that the area is stable with regard to scale
but the perimeter is not [200], so that the Gravelius index is meaningless if the DEM scale (i.e., mesh
size in practice) is not specified, whatever the DEM quality. Santos et al. [111] proposed a classification
in which stable vs unstable hydrographic indicators are identified.
If we consider the quality criteria defined in Section 4, these examples suggest that mesh size
transformation (like any form of resampling) has more impact on the landform modelling than on
elevation values.

6.3. Relevant Resolution for Landform Modelling

Internal quality assessment, which consists in controlling landform realism by confronting the
DEM to general rules the terrestrial relief is expected to fulfil everywhere, is generally more relevant
at low resolution. This is quite an intuitive truth, since a DEM is then more likely to express pure
geomorphologic properties, while high resolution reveals non-topographic objects like trees and
man-made objects.
By comparing a DTM and DSM in the Amazon rainforest, Polidori and Simonetto [201] show that
using a DSM instead of a DTM for slope calculation leads to an error of 5◦ for a 30 m mesh (e.g., ASTER
GDEM and SRTM 1”) against only 2◦ for a 90 m mesh (e.g., SRTM 3”). Therefore, high resolution
should not be preferred in forested or urban areas for applications based on slopes. This is a trap of
the temptation of high resolution. Moreover, the DEM errors (except highly autocorrelated errors
that increase the RMS error but generate a low impact on shapes) mainly affect the highest spatial
frequencies of the DEM. For instance, the DEM artifacts illustrated in Figure 8 create or remove small
shapes, typically from 1 to a few sampling units. Consequently, the highest frequencies are affected by
both non-topographic objects and DEM artifacts.
The rules that express a fractal behaviour in regions where the relief is structured into watersheds
(i.e., almost everywhere) are particularly well suited to highlight the spatial frequency above which
landforms are altered by DEM artifacts. Thus, in the case of DEMs obtained by interpolating contour
lines, the interpolator produces undesirable effects (smoothing, anisotropy) over periods shorter than
the horizontal interval between the contour lines on the map. This may be revealed by comparing
fractal dimension values under and over this limit [126]. Similarly, Figure 10 shows how a DEM
behaves with regard to Horton’s law, i.e., pure geomorphology for Strahler orders of 3 and higher
(large rivers) while orders of 1 and 2 (small rivers) are probably more affected by DEM artifacts [108].

7. Discussion
As we have seen, DEM quality assessment must be done with precautions, i.e., after specifying
the nominal terrain (Section 2) and the resolution at which the quality makes sense (Section 6), and it
must be based on relevant criteria among the multitude reviewed in Section 4. However, in many
works the nominal terrain and the resolution are not specified by the user, and the used quality criteria
are not suitable for the application. Quality control must be imposed by the user on the basis of
the requirements foreseen for the application, but these requirements are not always easy to express
with criteria useful to the DEM producer, not to mention the case of multi-user databases that have
to reconcile a multitude of quality criteria. In practice, the quality of a DEM is specified by the
producer and the quality indications highlight the product or method rather than informing users of
the limitations of which they should be aware.
Remote Sens. 2020, 12, 3522 25 of 36

This raises the question of who specifies the DEM characteristics. In any situation (i.e., local DEM
produced on-demand or multi-user database), a DEM producer has to answer the following questions:
what to represent (upper surface or ground)? in what format (vector or raster)? and for whom
(for which application)? Many users utilize freely available DEMs in their studies without questioning
the quality standard of these products or their convenience for their studies, which can lead to wrong
results in some cases as we have pointed out [202]. Although it is essential to take the application
into account, a difficulty in letting the user specify the DEM quality is the risk of a disproportionate
requirement, which seeks to get the most out of a method even if it is not necessary. For example,
the disadvantages of high resolution have been mentioned. More generally, over-specification should
be avoided although the temptation exists when a small object must necessarily be represented,
which would require resolution and accuracy performances that most users do not need. For these
reasons, the product specifications adopted for intermediate resolution DEMs like SRTM, TanDEM-X,
ASTER GDEM, with meshes between 10 and 100 m, are generally relevant for a wider user community
than high resolution products like the digital globe AES (Advanced Elevation Suite) DEM, the Airbus
Elevation1 and Elevation4 DEMs, and that of the planned mission CO3D (Constellation Optique 3D),
which seeks to model everything on the surface of the planet in 3D with 0.5 m ground spacing and 1 m
vertical accuracy, at the risk of including ephemeral and uninteresting objects. Indeed, information
on micro-relief and buildings is often available in 2D, and if 3D is really required it could be locally
modelled on demand.
Beyond a posteriori quality control of an existing DEM, the quality expected from a DEM
which does not yet exist may be questioned, depending on the method used and the geographical
characteristics of the area mapped (climate, relief, landcover). The quality of the DEM can be predicted
to a certain extent considering the instrumental and orbital characteristics of the imaging system.
Indeed, theoretical equations make it possible to estimate indicators like the altimetric standard
deviation σz and therefore to predict DEM external quality. However, this approach remains limited
for several reasons:

• σz is a very optimistic indicator and it may give the illusion that the accuracy of a global DEM is the
same all over the world. σz is a very limited indicator as we have shown, excellent for predicting
the vertical accuracy of the DEM but unsuitable for guaranteeing a realistic shape rendering.
• σz also depends on slope and landcover, and this double influence cannot be predicted
quantitatively because the slope is not necessarily known before the DEM is calculated and
the landcover can be described qualitatively at best.
• The interpolation, which is applied to resample the DEM in the required grid structure, also has
an impact on the quality by creating artifacts.

Moreover, some of the parameters in the equation used to predict the DEM quality have
a well-known but unpredictable influence. This is the case of the baseline in repeat-pass SAR
interferometry: its influence on accuracy and phase coherence is well known, but it is impossible to
predict its value before image acquisition, hence very heterogeneous performances between DEMs
made yet with the same sensor and the same processing method.
Therefore, the performance of a DEM production method cannot be predicted from theoretical
equations alone, it also requires a great deal of experience based on the analysis of many DEMs or
on simulation-based studies. The theoretical approach also focuses on external quality criteria which
we have shown to be of limited relevance for many applications, while internal criteria related to the
respect of landforms can be relevant. As shown in Section 4.2, the detection of landform inconsistencies
can first be based on visual analysis, which is a subjective approach of quality assessment, but we have
also shown that objective methods (based on quantitative criteria) can be implemented as automated
procedures to find and quantify inconsistencies. When external data are missing, this internal validation
approach becomes more relevant and the corresponding quality criteria could be programmed in a
dedicated software together with external quality criteria. Those criteria can be generic and applied to
Remote Sens. 2020, 12, 3522 26 of 36

any DEM, but they can also be specific to a given production method or product, like a DEM expected
to model the ground surface in forested areas.
Beyond mere quality assessment, the analysis of DEM errors and the understanding of the
phenomena that degrade the quality lead to DEM improvement possibilities, either during the
elaboration of the product, or a posteriori [203,204]. The external approach (based on reference
data) consists in correcting a bias, for instance removing tree height to transform a DSM into a DTM,
using external data to locate forest patches and an empirical method to estimate tree height. The internal
approach (based on physical or statistical assumptions) consists in removing the artifacts and forcing
the DEM to comply with certain rules. For example, sinks can be filled to force rivers to stream down.
The DEM can be resampled according to a required statistical behaviour, to filter noise or inversely to
create variability by kriging or fractal synthesis; assumptions can also be injected a priori to improve
the DEM calculation, for instance Monti-Guarnieri [205] uses a multifractal model of the Earth’s
surface to model the statistical distribution of slopes, which may be used, among other applications,
to guide phase unwrapping [206]. All these DEM improvement possibilities based on reference data or
geomorphological assumptions make it possible to avoid over-specification, the risk of which has been
mentioned above. This opens new opportunities to reduce costs by accepting intermediate quality
DEMs that can be improved in accordance with the requirements of a given application, in particular
by merging a DEM with other DEMs or additional information. This approach could benefit from
recent advances in deep learning.
Finally, a wide field of research is still opening up after several decades in the field of DEM
quality assessment [72,88] for the benefit of producers and users. In parallel with the development
of new sensors, new image processing algorithms and tests using DEMs under new conditions,
further studies based on different research fields are needed to better understand the error behaviour
and its impact on the expected result for different DEM-based applications. Such studies contribute to
the characterization of complex topographic objects and their interrelations, based on the adaptation of
existing mathematical tools to the complexity of the real world (from signal processing to machine
learning). They open promising research paths and the critical review proposed in this article allows to
identify some of them:
• Mathematical modelling of the error expected for different DEM production techniques as a
function of landscape characteristics and acquisition/processing parameters.
• Definition of invariant properties of the Earth’s surface to support internal quality assessment
methods, based on the advances of geoscience and even comparative planetology.
• Analysis of scale effects, which has become a challenging research issue with the development of
very high-resolution products, leading to increased needs for gigantic data volumes management
and for the characterization of extremely complex phenomena.
Studies on the quality of DEMs require and stimulate fundamental research in both geoscience
and mathematics, but the scientific community must also remain attentive to the expectations of DEM
producers and operational users. Indeed, beyond the standards proposed by DEM producers to specify
their products, there is a need to clarify relevant quality criteria that are understandable to both users
and producers of digital elevation models in a common language.

8. Conclusions
The review presented in this article is based on a comprehensive bibliography and it links the
different aspects of DEM quality assessment that are often addressed separately in the literature.
The main approaches of DEM quality are presented in a methodical synthesis though far from being
exhaustive, in which the quality criteria are interpreted according to user needs. Methods are proposed
to detect and quantify errors and artifacts in DEMs, but also to make choices between different
production techniques and to specify adapted requirements. The main conclusions are as follows:
• The quality of a DEM makes sense for a given application, i.e., depends on user needs.
Remote Sens. 2020, 12, 3522 27 of 36

• The nominal terrain, i.e., the physical surface, which is supposed to be modelled, must be explicitly
defined for the quality to make sense.
• The quality of a DEM can be assessed using different methods, with or without ground control
data, according to quantitative criteria.
• Artifact detection can be carried out with no ground control data, by revealing the non-compliance
of the DEM with physical or statistical characteristics of the Earth’s surface.
• Visual analysis can complement quality control provided that the DEM is visualized appropriately.
• Testing mapping methods on simulated images allows a more in-depth validation of these methods.
• The quality must be considered at a given scale. However, the concept of scale is an abuse of
language that comes from the paper map and must be replaced by equivalent concepts adapted to
the digital world.
• The quality of a global DEM cannot be easily deduced from an analysis of local data nor from a
theoretical approach only: it also requires a great deal of experience based on the analysis of many
DEMs on a variety of landscapes, or on simulation-based studies.

This article should contribute to warn DEM users against preconceived ideas in terms of precision
and resolution, in particular to avoid the temptation of over-specification. It should also be useful
to DEM producers to describe their products with technical characteristics more adapted to users’
expectations. With the multiplication of space-based Earth observation systems and processing
algorithms, this review should enable a more objective dialogue between producers and users,
and provide ideas for defining new research priorities.

Author Contributions: L.P. and M.E.H. conceived the review together based in their previous studies, and they
wrote the manuscript together. Both authors have read and agreed to the published version of the manuscript.
Funding: TOSCA program of the Centre National d’Etudes Spatiales (CNES).
Acknowledgments: This review paper is based on multiple research works carried out by the authors for years.
The review itself has been prepared in the framework of the “Biomass mission valorization” project funded by the
Centre National d’Etudes Spatiales (CNES) through the TOSCA program.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Goldthwaite, R.P.; Matsch, C.L. Glossary of Geology, 4th ed.; American Geological Institute: Alexandria, VA,
USA, 1997.
2. Huggett, R.J. What is Geomorphology? In Fundamentals of Geomorphology, 3rd ed.; Routledge: London, UK,
2011; pp. 3–18.
3. Laussedat, A. Mémoire Sur L’Emploi de la Photographie Dans la Levée Des Plans par M. Laussedat.
Comptes Rendus Séances Académie Sci. 1859, 49, 732–734.
4. Li, W. Digital Photogrammetry: A Practical Course; Springer Science & Business Media: Berlin/Heidelberg,
Germany, 2006; ISBN 978-3-540-29153-4.
5. Kasser, M.; Egels, Y. Digital Photogrammetry; Taylor & Francis: London, UK; New York, NY, USA, 2002.
6. Graham, L. Synthetic Interferometer Radar for Topographic Mapping. Proc. IEEE 1974, 62, 763–768.
[CrossRef]
7. Toutin, T.; Gray, L. State-of-the-Art of Elevation Extraction from Satellite SAR Data. ISPRS J. Photogramm.
Remote Sens. 2000, 55, 13–33. [CrossRef]
8. Leberl, F. Radargrammetric Image Processing; Artech House: London, UK, 1989; ISBN 978-0-89006-273-9.
9. Baltsavias, E. Airborne Laser Scanning: Existing Systems and Firms and Other Resources. ISPRS J.
Photogramm. Remote Sens. 1999, 54, 164–198. [CrossRef]
10. Hodgson, M.E.; Bresnahan, P. Accuracy of Airborne Lidar-Derived Elevation. Photogramm. Eng. Remote Sens.
2004, 70, 331–339. [CrossRef]
11. Moore, I.D.; Grayson, R.B.; Ladson, A.R. Digital Terrain Modelling: A Review of Hydrological,
Geomorphological, and Biological Applications. Hydrol. Process. 1991, 5, 3–30. [CrossRef]
Remote Sens. 2020, 12, 3522 28 of 36

12. Hengl, T.; Reuter, H.I. Geomorphometry: Concepts, Software, Applications; Elsevier Science: Amsterdam,
The Netherlands, 2009; ISBN 978-0-12-374345-9.
13. Park, S.; McSweeney, K.; Lowery, B. Identification of the Spatial Distribution of Soils Using a Process-Based
Terrain Characterization. Geoderma 2001, 103, 249–272. [CrossRef]
14. Zhang, J.; Xu, W.; Qin, L.; Tian, Y. Spatial Distribution Estimates of the Urban Population Using DSM and
DEM Data in China. ISPRS Int. J. Geo-Inf. 2018, 7, 435. [CrossRef]
15. Shimada, M. Ortho-Rectification and Slope Correction of SAR Data Using DEM and Its Accuracy Evaluation.
IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 2010, 3, 657–671. [CrossRef]
16. Reinartz, P.; Müller, R.; Schwind, P.; Suri, S.; Bamler, R. Orthorectification of VHR Optical Satellite Data
Exploiting the Geometric Accuracy of TerraSAR-X data. ISPRS J. Photogramm. Remote Sens. 2011, 66, 124–132.
[CrossRef]
17. Pike, R.J. Geomorphometry-Diversity in Quantitative Surface Analysis. Prog. Phys. Geogr. Earth Environ.
2000, 24, 1–20. [CrossRef]
18. Rasemann, S.; Schmidt, J.; Schrott, L.; Dikau, R. Geomorphometry in Mountain Terrain. In GIS & Mountain
Geomorphology; Bishop, M.P., Shroder, J.F., Eds.; Springer: Berlin/Heidelberg, Germany, 2004; pp. 101–145.
19. Pike, R.; Evans, I.; Hengl, T. Geomorphometry: A Brief Guide. In Developments in Soil Organic Matter;
Hengl, T., Reuter, H.I., Eds.; Elsevier BV: Amsterdam, The Netherlands, 2009; Volume 33, pp. 3–30.
20. Macmillan, R.; Shary, P. Landforms and Landform Elements in Geomorphometry. In Developments in Soil
Science; Hengl, T., Reuter, H.I., Eds.; Elsevier BV: Amsterdam, The Netherlands, 2009; Volume 33, pp. 227–254.
21. Hengl, T.; MacMillan, R.A. Geomorphometry—A Key to Landscape Mapping and Modelling. In Developments
in Soil Science; Hengl, T., Reuter, H.I., Eds.; Elsevier: Amsterdam, The Netherlands, 2009; Volume 33,
pp. 433–460.
22. Olaya, V. Basic Land-Surface Parameters. In Developments Soil Organic Matter; Hengl, T., Reuter, H.I., Eds.;
Elsevier BV: Amsterdam, The Netherlands, 2009; Volume 33, pp. 141–169.
23. Wilson, J.P.; Gallant, J.C. Terrain Analysis: Principles and Applications; John Wiley & Sons: Hoboke, NJ, USA,
2000; ISBN 978-0-471-32188-0.
24. Schneider, B. On the Uncertainty of Local Shape of Lines and Surfaces. Cartogr. Geogr. Inf. Sci. 2001, 28,
237–247. [CrossRef]
25. Holmes, K.; Chadwick, O.; Kyriakidis, P. Error in a USGS 30- Meter Digital Elevation Model and Its Impact
on Terrain Modeling. J. Hydrol. 2000, 233, 154–173. [CrossRef]
26. Shary, P.A.; Sharaya, L.S.; Mitusov, A.V. Fundamental Quantitative Methods of Land Surface Analysis.
Geoderma 2002, 107, 1–32. [CrossRef]
27. Heuvelink, G.B.M. Analysing Uncertainty Propagation in GIS: Why is it not that Simple? In Uncertainty in
Remote Sensing and GIS; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2002; pp. 155–165.
28. Heuvelink, G.B. Error Propagation in Environmental Modelling with GIS; CRC Press: Boca Raton, FL, USA, 1998.
29. Polidori, L.; El Hage, M.; Villard, L.; Koleck, T.; Le Toan, T. Potential of P-Band Sar Topographic Products
Over Forested Areas in Terms of Terrain Morphological Description: A Preliminary Study in the Framework
of the Biomass Mission. In Proceedings of the Simposio Latinoamericano de Percepción Remota y Sistemas
de Información Espacial, La Havana, Cuba, 6–9 November 2018.
30. D’Alessandro, M.M.; Tebaldini, S. Digital Terrain Model Retrieval in Tropical Forests Through P-Band SAR
Tomography. IEEE Trans. Geosci. Remote Sens. 2019, 57, 6774–6781. [CrossRef]
31. Le Toan, T.; Quegan, S.; Davidson, M.; Balzter, H.; Paillou, P.; Papathanassiou, K.; Plummer, S.; Rocca, F.;
Saatchi, S.; Shugart, H.; et al. The BIOMASS Mission: Mapping Global Forest Biomass to Better Understand
the Terrestrial Carbon Cycle. Remote Sens. Environ. 2011, 115, 2850–2860. [CrossRef]
32. Fryer, J.G. A Simple System for Photogrammetric Mapping in Shallow Water. Photogramm. Rec. 2006, 11,
203–208. [CrossRef]
33. Allouis, T.; Bailly, J.-S.; Feurer, D. Assessing Water Surface Effects on Lidar Bathymetry Measurements in
Very Shallow Rivers: A Theoretical Study. In Proceedings of the Second ESA Space for Hydrology Workshop,
Geneva, Switzerland, 12–14 November 2007; pp. 12–14.
34. Gratiot, N.; Gardel, A.; Polidori, L. Remote Sensing Based Bathymetry on the Highly Dynamic Amazonian
Coast. In Proceedings of the 9th International Coastal Symposium, Hornafjörður, Island, 5–8 June 2005;
pp. 5–8.
Remote Sens. 2020, 12, 3522 29 of 36

35. Dai, W.; Li, H.; Zhou, Z.; Cybele, S.; Lu, C.; Zhao, K.; Zhang, X.; Yang, H.; Li, D. UAV Photogrammetry for
Elevation Monitoring of Intertidal Mudflats. J. Coast. Res. 2018, 85, 236–240. [CrossRef]
36. Smith, L.C. Emerging Applications of Interferometric Synthetic Aperture Radar in Geomorphology and
Hydrology. Ann. Assoc. Am. Geogr. 2002, 92, 385–398. [CrossRef]
37. Catani, F.; Farina, P.; Moretti, S.; Nico, G.; Strozzi, T. On the Application of SAR Interferometry to
Geomorphological Studies: Estimation of Landform Attributes and Mass Movements. Geomorphology 2005,
66, 119–131. [CrossRef]
38. Toutin, T. ASTER DEMs for Geomatic and Geoscientific Applications: A Review. Int. J. Remote Sens. 2008, 29,
1855–1875. [CrossRef]
39. Biljecki, F.; LeDoux, H.; Stoter, J.; Zhao, J. Formalisation of the Level of Detail in 3D City Modelling.
Comput. Environ. Urban Syst. 2014, 48, 1–15. [CrossRef]
40. Biljecki, F.; LeDoux, H.; Stoter, J. An Improved LOD Specification for 3D Building Models. Comput. Environ.
Urban Syst. 2016, 59, 25–37. [CrossRef]
41. Longley, P.; Goodchild, M.F. Geographic Information Science and Systems; John Wiley & Sons: Hoboken, NJ,
USA, 2015.
42. Poiker, T.K.; Fowler, R.J.; Little, J.J.; Mark, D.M. The Triangulated Irregular Network. In Proceedings of the
Digital Terrain Models Symposium, Missouri, MO, USA, 9–11 May 1978; pp. 516–540.
43. Li, Z.; Zhu, C.; Gold, C. Accuracy of Digital Terrain Models. In Digital Terrain Modeling Principles and
Methodology; Taylor and Francis: New York, NY, USA, 2005; pp. 158–190.
44. Hutchinson, M.F. Calculation of Hydrologically Sound Digital Elevation Models. In Proceedings of the Third
International Symposium on Spatial Data Handling, Sydney, Australia, 17–19 August 1988; pp. 117–133.
45. Hutchinson, M. A New Procedure for Gridding Elevation and Stream Line Data with Automatic Removal of
Spurious Pits. J. Hydrol. 1989, 106, 211–232. [CrossRef]
46. Makarovic, B. Progressive Sampling for Digital Terrain Models. ITC J. 1973, 3, 397–416.
47. Makarovic, B. Composite Sampling for Digital Terrain Models. ITC J. 1977, 3, 406–433.
48. Charif, M.; Makarovic, B. Optimizing Progressive and Composite Sampling for DTMs. ITC J. 1989, 2, 104–111.
49. Pajarola, R.; Gobbetti, E. Survey of Semi-Regular Multiresolution Models for Interactive Terrain Rendering.
Vis. Comput. 2007, 23, 583–605. [CrossRef]
50. Florinsky, I.V. Errors of Signal Processing in Digital Terrain Modelling. Int. J. Geogr. Inf. Sci. 2002, 16, 475–501.
[CrossRef]
51. Heritage, G.L.; Milan, D.J.; Large, A.R.; Fuller, I. Influence of Survey Strategy and Interpolation Model on
DEM Quality. Geomorphology 2009, 112, 334–344. [CrossRef]
52. Weber, D.; Englund, E. Evaluation and Comparison of Spatial Interpolators. Math. Geol. 1992, 24, 381–391.
[CrossRef]
53. Zheng, X.; Chen, Z.; Han, Q.; Deng, X.; Sun, X.; Yin, Q. Self-similarity Based Multi-layer DEM Image
Up-Sampling. In Advances in Intelligent Systems and Computing; Springer Science and Business Media LLC:
Cham, Switzerland, 2019; pp. 533–545.
54. Polidori, L.; Chorowicz, J. Comparison of Bilinear and Brownian Interpolation for Digital Elevation Models.
ISPRS J. Photogramm. Remote Sens. 1993, 48, 18–23. [CrossRef]
55. Featherstone, W.; Dentith, M.; Kirby, J. Strategies for the Accurate Determination of Orthometric Heights
From Gps. Surv. Rev. 1998, 34, 278–296. [CrossRef]
56. Erol, B.; Erol, S.; Celik, R.N. Height Transformation Using Regional Geoids and GPS/Levelling in Turkey.
Surv. Rev. 2008, 40, 2–18. [CrossRef]
57. Durland, N.H. Defining Mean Sea Level in Military Simulations with DTED. In Proceedings of the 2009
Spring Simulation Multiconference, San Diego, CA, USA, 22–27 March 2009; Society for Computer Simulation
International: San Diego, CA, USA, 2009; pp. 1–3.
58. Paparoditis, N.; Dissard, O. 3D Data Acquisition from Visible Images. In Digital Photogrammetry; Kasser, M.,
Egels, Y., Eds.; Taylor & Francis: London, UK; New York, NY, USA, 2002; Volume 48, pp. 168–220.
59. Rabus, B.; Eineder, M.; Roth, A.; Bamler, R. The Shuttle Radar Topography Mission—A New Class of Digital
Elevation Models Acquired by Spaceborne Radar. ISPRS J. Photogramm. Remote Sens. 2003, 57, 241–262.
[CrossRef]
Remote Sens. 2020, 12, 3522 30 of 36

60. Jarvis, A.; Rubiano, J.; Nelson, A.; Farrow, A.; Mulligan, M. Practical Use of SRTM Data in the Tropics:
Comparisons with Digital Elevation Models Generated from Cartographic Data; International Centre for Tropical
Agriculture: Cali, Columbia, 2004; Volume 198, pp. 1–35.
61. Farr, T.G.; Rosen, P.A.; Caro, E.; Crippen, R.; Duren, R.; Hensley, S.; Kobrick, M.; Paller, M.; Rodriguez, E.;
Roth, L.; et al. The Shuttle Radar Topography Mission. Rev. Geophys. 2007, 45. [CrossRef]
62. Reuter, H.I.; Hengl, T.; Gessler, P.; Soille, P. Preparation of DEMs for Geomorphometric Analysis.
In Developments in Soil Organic Matter; Elsevier BV: Amsterdam, The Netherlands, 2009; Volume 33,
pp. 87–120.
63. Höhle, J.; Potuckova, M. Assessment of the Quality of Digital Terrain Models. Eur. Spat. Data Res. 2011,
60, 91.
64. Cooper, M.A.R. Datums, Coordinates and Differences. In Landform Monitoring, Modelling and Analysis;
Lane, S., Richards, K., Chandler, J., Eds.; Wiley: Chichester, UK, 1998; pp. 21–36.
65. Wise, S. Assessing the Quality for Hydrological Applications of Digital Elevation Models Derived from
Contours. Hydrol. Process. 2000, 14, 1909–1929. [CrossRef]
66. Hu, P.; Liu, X.; Hu, H. Accuracy Assessment of Digital Elevation Models based on Approximation Theory.
Photogramm. Eng. Remote Sens. 2009, 75, 49–56. [CrossRef]
67. Fisher, P.F.; Tate, N.J. Causes and Consequences of Error in Digital Elevation Models. Prog. Phys. Geogr.
Earth Environ. 2006, 30, 467–489. [CrossRef]
68. Hunter, G.J.; Goodchild, M.F. Modeling the Uncertainty of Slope and Aspect Estimates Derived from Spatial
Databases. Geogr. Anal. 2010, 29, 35–49. [CrossRef]
69. Temme, A.; Heuvelink, G.; Schoorl, J.; Claessens, L. Geostatistical Simulation and Error Propagation
in Geomorphometry. In Developments in Soil Organic Matter; Hengl, T., Reuter, H.I., Eds.; Elsevier BV:
Amsterdam, The Netherlands, 2009; Volume 33, pp. 121–140.
70. Hengl, T.; Evans, I. Mathematical and Digital Models of the Land Surface. In Developments in Soil Organic
Matter; Hengl, T., Reuter, H.I., Eds.; Elsevier BV: Amsterdam, The Netherlands, 2009; Volume 33, pp. 31–63.
71. Hebeler, F.; Purves, R.S. The Influence of Elevation Uncertainty on Derivation of Topographic Indices.
Geomorphology 2009, 111, 4–16. [CrossRef]
72. Devillers, R.; Stein, A.; Bédard, Y.; Chrisman, N.; Fisher, P.; Shi, W. Thirty Years of Research on Spatial Data
Quality: Achievements, Failures, and Opportunities. Trans. GIS 2010, 14, 387–400. [CrossRef]
73. Granshaw, S.I. Photogrammetric Terminology: Third Edition. Photogramm. Rec. 2016, 31, 210–252. [CrossRef]
74. Newby, P.R.T. Accuracy, Precision, Extraction, Citation and Valediction. Photogramm. Rec. 2011, 26, 149–153.
[CrossRef]
75. Höhle, J.; Höhle, M. Accuracy Assessment of Digital Elevation Models by Means of Robust Statistical
Methods. ISPRS J. Photogramm. Remote Sens. 2009, 64, 398–406. [CrossRef]
76. Wechsler, S.P. Uncertainties Associated with Digital Elevation Models for Hydrologic Applications: A Review.
Hydrol. Earth Syst. Sci. 2007, 11, 1481–1500. [CrossRef]
77. Hengl, T.; Heuvelink, G.B.M.; Van Loon, E.E. On the Uncertainty of Stream Networks Derived from Elevation
Data: The Error Propagation Approach. Hydrol. Earth Syst. Sci. 2010, 14, 1153–1165. [CrossRef]
78. Snyder, M.W. A Comparison of Four Techniques for the Calculation of Slope and Aspect from Digital Terrain
Matrices. Master’s Thesis, University of Illinois at Urbana-Champaign, Champaign, IL, USA, 1983.
79. Skidmore, A.K. A Comparison of Techniques for Calculating Gradient and Aspect from a Gridded Digital
Elevation Model. Int. J. Geogr. Inf. Syst. 1989, 3, 323–334. [CrossRef]
80. Srinivasan, R.; Engel, B.A. Effect of Slope Prediction Methods on Slope and Erosion Estimates.
Appl. Eng. Agric. 1991, 7, 779–783. [CrossRef]
81. Zhou, Q.; Liu, X. Error Analysis on Grid-Based Slope and Aspect Algorithms. Photogramm. Eng. Remote. Sens.
2004, 70, 957–962. [CrossRef]
82. Zhou, Q.; Liu, X. Analysis of Errors of Derived Slope and Aspect Related to DEM Data Properties.
Comput. Geosci. 2004, 30, 369–378. [CrossRef]
83. Bolstad, P.V.; Stowe, T. An Evaluation of DEM Accuracy: Elevation, Slope, and Aspect. Photogram. Eng.
Remote Sens. 1994, 60, 1327–1332.
84. El Hage, M.; Simonetto, E.; Faour, G.; Polidori, L. Evaluation of Elevation, Slope and Stream Network Quality
of SPOT DEMs. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2012, 2, 63–67. [CrossRef]
Remote Sens. 2020, 12, 3522 31 of 36

85. Oksanen, J.; Sarjakoski, T. Error Propagation of DEM-Based Surface Derivatives. Comput. Geosci. 2005,
31, 1015–1027. [CrossRef]
86. Aguilar, F.J.; Mills, J.P.; Delgado, J.; Aguilar, M.A.; Negreiros, J.; Pérez, J.L. Modelling Vertical Error in
LiDAR-Derived Digital Elevation Models. ISPRS J. Photogramm. Remote Sens. 2010, 65, 103–110. [CrossRef]
87. Florinsky, I.V. Accuracy of Local Topographic Variables Derived from Digital Elevation Models. Int. J. Geogr.
Inf. Sci. 1998, 12, 47–62. [CrossRef]
88. Wilson, J.P. Digital Terrain Modeling. Geomorphology 2012, 137, 107–121. [CrossRef]
89. Evans, I.S. General Geomorphometry, Derivatives of Altitude, and Descriptive Statistics. In Spatial Analysis
in Geomorphology; Chorley, R.J., Ed.; Methuen & Co., Ltd.: London, UK, 1972; pp. 17–90.
90. El Hage, M. Etude de la Qualité Géomorphologique de Modèles Numériques de Terrain Issus de L’Imagerie
Spatiale. Ph.D. Thesis, Conservatoire National des Arts et Metiers, Paris, France, 2012.
91. Warren, S.; Hohmann, M.; Auerswald, K.; Mitasova, H. An Evaluation of Methods to Determine Slope Using
Digital Elevation Data. Catena 2004, 58, 215–233. [CrossRef]
92. Horn, B. Hill Shading and the Reflectance Map. Proc. IEEE 1981, 69, 14–47. [CrossRef]
93. Burrough, P.A.; McDonnell, R.; McDonnell, R.A.; Lloyd, C.D. Principles of Geographical Information Systems;
OUP: Oxford, UK, 2015; ISBN 978-0-19-874284-5.
94. Liu, X.; Bian, L. Accuracy Assessment of DEM Slope Algorithms Related to Spatial Autocorrelation of DEM
Errors. In Advances in Digital Terrain Analysis; Zhou, Q., Lees, B., Tang, G., Eds.; Springer: Berlin/Heidelberg,
Germany, 2008; pp. 307–322. ISBN 978-3-540-77800-4.
95. Gastellu-Etchegorry, J.-P.; Grau, E.; Lauret, N.A. A 3D Model for Remote Sensing Images and Radiative
Budget of Earth Surfaces. In Modeling and Simulation in Engineering; Alexandru, C., Ed.; IntechOpen: London,
UK, 2012.
96. Gastellu-Etchegorry, J.-P.; Yin, T.; Lauret, N.; Cajgfinger, T.; Gregoire, T.; Grau, E.; Feret, J.-B.; Lopes, M.;
Guilleux, J.; Dedieu, G.; et al. Discrete Anisotropic Radiative Transfer for Modeling Airborne and Satellite
Spectroradiometer and LIDAR Acquisitions of Natural and Urban Landscapes. Remote Sens. 2015, 7,
1667–1701. [CrossRef]
97. Paparoditis, N.; Polidori, L.; Savaria, E. Parametric Simulation-Based Evaluation of 3D Mapping Techniques
on Optical Remote Sensing Images. Satell. Remote Sens. II 1995, 2579, 423–435. [CrossRef]
98. Chu, H.-J.; Chen, R.-A.; Tseng, V.S.; Wang, C.-K. Identifying LiDAR Sample Uncertainty on Terrain Features
from DEM Simulation. Geomorphology 2014, 204, 325–333. [CrossRef]
99. Guindon, B. Application of SAR Simulation Techniques to Improve the Understanding of Spaceborne Sar
Scenes of Moderate to Rugged Terrain. Proc. EARSeL I 1991, 9, 100–109.
100. Polidori, L.; Armand, P. On the use of SAR Image Simulation for the Validation of Relief Mapping Techniques.
EARSeL Adv. Remote Sens. 1995, 4, 40–48.
101. Taglioretti, C.; Manzino, A.M. Terrestrial Mobile Mapping: Photogrammetric simulator. ISPRS Int. Arch.
Photogramm. Remote Sens. Spat. Inf. Sci. 2014, 3, 333–339. [CrossRef]
102. Polidori, L. Fractal-Based Evaluation of Relief Mapping Techniques. In Proceedings of the Joint EC-EARSeL
Expert Meeting on Fractals in Geosciences and Remote Sensing, Ispra, Italy, 14–15 April 1994; pp. 277–297.
103. Griffin, M.W. Terrain Synthesis: The Creation, Management, Presentation and Validation of Artificial
Landscapes. Ph.D. Thesis, University of Nottingham, Nottingham, UK, 2001.
104. Podobnikar, T. Methods for Visual Quality Assessment of a Digital Terrain Model. SAPIENS 2009, 2, 1–10.
105. Borchia, R.; Nesci, O.; Codice, P. Atlante Illustrato Del Reale Paesaggio Della Gioconda; Mondadori Electa:
Florence, Italy, 2012; pp. 1–144, ISBN 978-88-370-9422-5.
106. Oksanen, J. Tracing the Gross Errors of DEM-Visualisation Techniques for Preliminary Quality Analysis.
In Proceedings of the 21st International Cartographic Conference, Durban, South Africa, 10–16 August 2003;
pp. 2410–2415.
107. Arrell, K.; Wise, S.; Wood, J.; Donoghue, D.; Donoghue, D.N. Spectral Filtering as a Method of Visualising
and Removing Striped Artefacts in Digital Elevation Data. Earth Surf. Process. Landf. 2008, 33, 943–961.
[CrossRef]
108. Polidori, L.; El Hage, M.; Valeriano, M.D.M. Digital Elevation Model Validation with No Ground Control:
Application to the Topodata Dem in Brazil. Bol. Ciências Geodésicas 2014, 20, 467–479. [CrossRef]
109. Shary, P.A.; Sharaya, L.S.; Mitusov, A.V. The Problem of Scale-Specific and Scale-Free Approaches in
Geomorphometry. Geogr. Fis. Din. Quat. 2005, 28, 81–101.
Remote Sens. 2020, 12, 3522 32 of 36

110. El Hage, M.; Simonetto, E.; Faour, G.; Polidori, L. Impact of DEM Reconstruction Parameters on Topographic
Indices. In Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial
Information Sciences, Paris, France, 1–2 September 2010; Volume 38, pp. 40–44.
111. Santos, V.; El Hage, M.; Polidori, L.; Stevaux, J.C. Effect of Digital Elevation Model Mesh Size on Geomorphic
Indices: A Case Study of the Ivaí River Watershed-State of Paraná, Brazil. Bol. Ciências Geodésicas 2017, 23,
684–699. [CrossRef]
112. Hutchinson, M.; Gallant, J.C. Digital Elevation Eodel and Representation of Terrain Shape. In Terrain Analysis:
Principles and Applications; Wilson, J.P., Gallant, J.C., Eds.; John Wiley & Sons: Hoboke, NJ, USA, 2000;
ISBN 978-0-471-32188-0.
113. Hirt, C. Artefact Detection in Global Digital Elevation Models (DEMs): The Maximum Slope Approach and
Its Application for Complete Screening of the SRTM v4.1 and MERIT DEMs. Remote Sens. Environ. 2018,
207, 27–41. [CrossRef]
114. Tobler, W.R. A Computer Movie Simulating Urban Growth in the Detroit Region. Econ. Geogr. 1970, 46, 234.
[CrossRef]
115. Miller, H.J. Tobler’s First Law and Spatial Analysis. Ann. Assoc. Am. Geogr. 2004, 94, 284–289. [CrossRef]
116. Rodriguez-Iturbe, I.; Rinaldo, A.; Levy, O. Fractal River Basins: Chance and Self-Organization; Cambridge
University Press: Cambridge, UK, 1997.
117. Gaucherel, C.; Salomon, L.; LaBonne, J. Variable Self-Similar Sinuosity Properties Within Simulated River
Networks. Earth Surf. Process. Landf. 2011, 36, 1313–1320. [CrossRef]
118. Strahler, A.N. Quantitative Analysis of Watershed Geomorphology. Trans. Am. Geophys. Union 1957, 38,
913–920. [CrossRef]
119. Gaucherel, C.; Frelat, R.; Salomon, L.; Rouy, B.; Pandey, N.; Cudennec, C. Regional Watershed Characterization
and Classification with River Network Analyses. Earth Surf. Process. Landf. 2017, 42, 2068–2081. [CrossRef]
120. Valeriano, M.D.M.; Rossetti, D.F. Topodata: Brazilian Full Coverage Refinement of SRTM Data. Appl. Geogr.
2012, 32, 300–309. [CrossRef]
121. Fernández, D.C.J. Evaluation of Algorithms and Digital Elevation Models for Drainage Extraction. Master’s
Thesis, Instituto Nacional de Pesquisas Espaciais, Sao Jose dos Campos, Brazil, 2011.
122. Benford, F. The Law of Anomalous Numbers. Proc. Am. Philos. Soc. 1938, 78, 551–572.
123. Berger, A.; Hill, T.P. An Introduction to Benford’s Law; Princeton University Press: Princeton, NJ, USA, 2015.
124. Nigrini, M.J. Benford’s Law: Applications for Forensic Accounting, Auditing, and Fraud Detection; John Wiley &
Sons: Hoboken, NJ, USA, 2012; ISBN 978-1-118-15285-0.
125. Polidori, L.; El Hage, M. Application de la Loi de Benford AU Contrôle de Qualité Des Modèles Numériques
de Terrain. Revue XYZ 2019, 158, 19–22.
126. Polidori, L.; Chorowicz, J.; Guillande, R. Description of Terrain as a Fractal Surface, and Application to
Digital Elevation Model Quality Assessment. Photogramm. Eng. Remote Sens. 1991, 57, 1329–1332.
127. Wise, S. Information Entropy as a Measure of DEM Quality. Comput. Geosci. 2012, 48, 102–110. [CrossRef]
128. Renouard, L. Extraction Automatique de Mnt à Différentes Résolutions. Int. Arch. Photogramm. Remote Sens.
1993, 29, 886.
129. Krupnik, A. Accuracy Assessment of Automatically Derived Digital Elevation Models from SPOT Images.
Photogramm. Eng. Remote Sens. 2000, 66, 1017–1023.
130. Hashemian, M.S.; Abootalebi, A.; Kianifar, F. Accuracy Evaluation of Dem Generated from Spot 5 Hrs
Imageries. In Proceedings of the XXth ISPRS Congress, Commission I, Istanbul, Turkey, 12–23 July 2004;
pp. 12–23.
131. Cuartero, A.; Felicísimo, A.M.; Ariza, F.J. Accuracy of DEM Generation from TERRA-ASTER Stereo Data.
Int. Arch. Photogramm. Remote Sens. 2004, 35, 559–563.
132. Cuartero, A.; Felicísimo, A.M.; Ariza, F. Accuracy, Reliability, and Depuration of SPOT HRV and Terra
ASTER Digital Elevation Models. IEEE Trans. Geosci. Remote Sens. 2005, 43, 404–407. [CrossRef]
133. Poon, J.; Fraser, C.S.; Zhang, C.; Zhang, L.; Gruen, A. Quality Assessment of Digital Surface Models Generated
from IKONOS Imagery. Photogramm. Rec. 2005, 20, 162–171. [CrossRef]
134. Evans, G.; Ramachandran, B.; Zhang, Z.; Bailey, B.; Cheng, P. An Accuracy Assessment of Cartosat-1 Stereo
Image Data-Derived Digital Elevation Models: A Case Study of the Drum Mountains, Utah. Int. Arch.
Photogramm. Remote Sens. Spat. Inf. Sci. 2008, 37, 1161–1164.
Remote Sens. 2020, 12, 3522 33 of 36

135. Bignone, F.; Umakawa, H. Assessment of ALOS PRISM Digital Elevation Model Extraction over Japan.
Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2008, 37, 1135–1138.
136. Gooch, M.J.; Chandler, J.H.; Stojić, M. Accuracy Assessment of Digital Elevation Models Generated Using the
Erdas Imagine Orthomax Digital Photogrammetric System. Photogramm. Rec. 1999, 16, 519–531. [CrossRef]
137. Lane, S.N.; James, T.D.; Crowell, M.D. Application of Digital Photogrammetry to Complex Topography for
Geomorphological Research. Photogramm. Rec. 2000, 16, 793–821. [CrossRef]
138. Hanssen, R.; Bamler, R. Evaluation of interpolation kernels for SAR interferometry. IEEE Trans. Geosci.
Remote Sens. 1999, 37, 318–321. [CrossRef]
139. Slacikova, J.; Potuckova, M. Evaluation of Interpolation Methods in InSAR DEM Derivation from ERS
Tandem Data. In Proceedings of the 31st EARSeL Symposium, Prague, Czech Republic, 30 May–2 June 2011;
pp. 544–551.
140. El Hage, M.; Simonetto, E.; Faour, G.; Polidori, L. Effect of Image-Matching Parameters and Local Morphology
on the Geomorphological Quality of SPOT DEMs. Photogramm. Rec. 2017, 32, 255–275. [CrossRef]
141. Li, Z. Variation of the Accuracy of Digital Terrain Models with Sampling Interval. Photogramm. Rec. 2006, 14,
113–128. [CrossRef]
142. Gao, J. Comparison of Sampling Schemes in Constructing Tdms From Topographic Maps. ITC J. 1995, 1,
18–22.
143. Gao, J. Resolution and Accuracy of Terrain Representation by Grid DEMs at a Micro-Scale. Int. J. Geogr.
Inf. Sci. 1997, 11, 199–212. [CrossRef]
144. Weng, Q. Quantifying Uncertainty of Digital Elevation Models Derived from Topographic Maps. In Proceedings
of the Open Source Approaches in Spatial Data Handling; Springer Science and Business Media LLC:
Berlin/Heidelberg, Germany, 2002; pp. 403–418.
145. Su, J.; Bork, E. Influence of Vegetation, Slope, and Lidar Sampling Angle on DEM Accuracy. Photogramm. Eng.
Remote Sens. 2006, 72, 1265–1274. [CrossRef]
146. Toutin, T. Impact of Terrain Slope and Aspect on Radargrammetric DEM Accuracy. ISPRS J. Photogramm.
Remote Sens. 2002, 57, 228–240. [CrossRef]
147. Toutin, T. Generation of DSMs from SPOT-5 in-Track HRS and across-Track HRG Stereo Data Using
Spatiotriangulation and Autocalibration. ISPRS J. Photogramm. Remote Sens. 2006, 60, 170–181. [CrossRef]
148. Felicísimo, A.M. Digital Terrain Models and their Application to Environmental Sciences. Ph.D. Thesis,
University of Oviedo, Asturias, Spain, 1992.
149. Papasaika, H.; Baltsavias, E. Investigations on the Relation of Geomorphological Parameters to DEM
Accuracy. In Geomorphometry 2009, Proceedings of the Geomorphometry Conference, Zurich, Switzerland,
31 August–2 September 2009; Purves, R., Gruber, S., Straumann, R., Hengl, T., Eds.; University of Zurich:
Zurich, Switzerland, 2009; pp. 162–168.
150. Schwendel, A.C.; Fuller, I.C.; Death, R.G. Assessing DEM Interpolation Methods for Effective Representation
of Upland Stream Morphology for Rapid Appraisal of Bed Stability. River Res. Appl. 2010, 28, 567–584.
[CrossRef]
151. Aguilar, F.J.; Agüera, F.; Aguilar, M.A.; Carvajal, F. Effects of Terrain Morphology, Sampling Density,
and Interpolation Methods on Grid DEM Accuracy. Photogramm. Eng. Remote Sens. 2005, 71, 805–816.
[CrossRef]
152. Chaplot, V.; Darboux, F.; Bourennane, H.; Leguédois, S.; Silvera, N.; Phachomphon, K. Accuracy of
Interpolation Techniques for the Derivation of Digital Elevation Models in Relation to Landform Types and
Data Density. Geomorphology 2006, 77, 126–141. [CrossRef]
153. Binh, T.Q.; Thuy, N.T. Assessment of the Influence of Interpolation Techniques on the Accuracy of Digital
Elevation Model. VNU J. Sci. Earth Sci. 2008, 24, 176–183.
154. Bater, C.W.; Coops, N.C. Evaluating Error Associated with Lidar-Derived Dem Interpolation. Comput. Geosci.
2009, 35, 289–300. [CrossRef]
155. Weibel, R.; Heller, M. Digital terrain modelling. In Geographical Information Systems; Maguire, D.J.,
Goodchild, M.F., Rhind, D.W., Eds.; Longman: London, UK, 1991; Volume 1, pp. 269–297.
156. Pain, C.F. Size Does Matter: Relationships Between Image Pixel Size and Landscape Process Scales.
In International Congress of Modelling and Simulation, Proceedings of the MODSIM, Sydney, Australia,
12–15 December 2005; Modelling and Simulation Society of Australia and New Zealand Inc.: Sydney,
Australia, 2005; pp. 1430–1436.
Remote Sens. 2020, 12, 3522 34 of 36

157. Mitas, L.; Mitasova, H. Spatial interpolation. In Geographical Information Systems: Principles, Techniques,
Management and Applications; Longley, P., Goodchild, M.F., Maguire, D.J., Rhind, D.W., Eds.; Wiley: Hoboken,
NJ, USA, 1999; Volume 1, pp. 481–492.
158. Tarboton, D.G.; Ames, D.P. Advances in the Mapping of Flow Networks from Digital Elevation Data.
Bridg. Gap 2001, 1–10. [CrossRef]
159. Hancock, G.R. The Use of Digital Elevation Models in the Identification and Characterization of Catchments
Over Different Grid Scales. Hydrol. Process. 2005, 19, 1727–1749. [CrossRef]
160. Hancock, G.R.; Martinez, C.; Evans, K.G.; Moliere, D.R. A Comparison of SRTM and High-Resolution Digital
Elevation Models and Their Use in Catchment Geomorphology and Hydrology: Australian Examples.
Earth Surf. Process. Landf. 2006, 31, 1394–1412. [CrossRef]
161. Murphy, P.N.C.; Ogilvie, J.; Meng, F.-R.; Arp, P. Stream Network Modelling Using Lidar and Photogrammetric
Digital Elevation Models: A Comparison and Field Verification. Hydrol. Process. 2008, 22, 1747–1754.
[CrossRef]
162. Trofaier, A.M.; Rees, W.G. The Suitability of Using ASTER GDEM2 for Terrain-Based Extraction of Stream
Channel Networks in a Lowland Arctic Permafrost Catchment. Fenn. Int. J. Geogr. 2015, 193, 66–82.
163. Veregin, H. The Effects of Vertical Error in Digital Elevation Models on the Determination of Flow-path
Direction. Cartogr. Geogr. Inf. Syst. 1997, 24, 67–79. [CrossRef]
164. Raaflaub, L.D.; Collins, M.J. The Effect of Error in Gridded Digital Elevation Models on the Estimation of
Topographic Parameters. Environ. Model. Softw. 2006, 21, 710–732. [CrossRef]
165. Khan, A.; Richards, K.S.; Parker, G.T.; McRobie, A.; Mukhopadhyay, B. How large is the Upper Indus Basin?
The Pitfalls of Auto-Delineation Using DEMs. J. Hydrol. 2014, 509, 442–453. [CrossRef]
166. Chakra, C.A.; Gascoin, S.; Somma, J.; Fanise, P.; Drapeau, L.; Chakra, A. Monitoring the Snowpack Volume
in a Sinkhole on Mount Lebanon using Time Lapse Photogrammetry. Sensors 2019, 19, 3890. [CrossRef]
[PubMed]
167. Shaw, T.E.; Gascoin, S.; Mendoza, P.A.; Pellicciotti, F.; McPhee, J. Snow Depth Patterns in a High Mountain
Andean Catchment from Satellite Optical Tristereoscopic Remote Sensing. Water Resour. Res. 2020, 56.
[CrossRef]
168. Bubenzer, O.; Bolten, A. The Use of New Elevation Data (SRTM/ASTER) for the Detection and Morphometric
Quantification of Pleistocene Megadunes (Draa) in the Eastern Sahara and the Southern Namib. Geomorphology
2008, 102, 221–231. [CrossRef]
169. Pipaud, I.; Loibl, D.; Lehmkuhl, F. Evaluation of TanDEM-X Elevation Data for Geomorphological Mapping
and Interpretation in High Mountain Environments—A Case Study from SE Tibet, China. Geomorphology
2015, 246, 232–254. [CrossRef]
170. Domeneghetti, A. On the Use of SRTM and Altimetry Data for Flood Modeling in Data-Sparse Regions.
Water Resour. Res. 2016, 52, 2901–2918. [CrossRef]
171. Du, X.; Guo, H.; Fan, X.; Zhu, J.; Yan, Z.; Zhan, Q. Vertical Accuracy Assessment of Freely Available Digital
Elevation Models Over Low-Lying Coastal Plains. Int. J. Digit. Earth 2015, 9, 252–271. [CrossRef]
172. Böhme, R. Inventory of World Topographic Mapping; International Cartographic Association by Elsevier Applied
Science Publishers: New York, NY, USA, 1993; ISBN 978-1-85166-357-6.
173. Weydahl, D.J.; Sagstuen, J.; Dick, Ø.B.; Rønning, H. SRTM DEM Accuracy Assessment Over Vegetated Areas
in Norway. Int. J. Remote Sens. 2007, 28, 3513–3527. [CrossRef]
174. Hawker, L.; Neal, J.; Bates, P. Accuracy Assessment of the TanDEM-X 90 Digital Elevation Model for Selected
Floodplain Sites. Remote Sens. Environ. 2019, 232, 111319. [CrossRef]
175. Zhao, S.-M.; Cheng, W.; Zhou, C.; Chen, X.; Zhang, S.; Zhou, Z.; Liu, H.; Chai, H. Accuracy Assessment of
the ASTER GDEM and SRTM3 DEM: An Example in the Loess Plateau and North China Plain of China.
Int. J. Remote Sens. 2011, 32, 8081–8093. [CrossRef]
176. Gómez, M.F.; Lencinas, J.D.; Siebert, A.; Díaz, G.M. Accuracy Assessment of ASTER and SRTM DEMs:
A Case Study in Andean Patagonia. GISci. Remote Sens. 2012, 49, 71–91. [CrossRef]
177. Satgé, F.; Bonnet, M.; Timouk, F.; Calmant, S.; Pillco, R.; Molina, J.; Lavado-Casimiro, W.; Arsen, A.; Crétaux, J.;
Garnier, J. Accuracy Assessment of SRTM v4 and ASTER GDEM v2 over the Altiplano Watershed Using
ICESat/GLAS Data. Int. J. Remote Sens. 2015, 36, 465–488. [CrossRef]
178. Toutin, T.; Cheng, P. DEM Generation with ASTER Stereo Data. Earth Obs. Mag. 2001, 10, 10–13. [CrossRef]
Remote Sens. 2020, 12, 3522 35 of 36

179. Hasegawa, H.; Matsuo, K.; Koarai, M.; Watanabe, N.; Masaharu, H.; Fukushima, Y. DEM Accuracy and the
Base to Height (B/H) Ratio of Stereo Images. Int. Arch. Photogramm. Remote Sens. 2000, 33, 356–359.
180. Gallant, J.C.; Read, A.M.; Dowling, T. Removal of Tree Offsets from SRTM and Other Digital Surface Models.
ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2012, 39, 275–280. [CrossRef]
181. Yamazaki, D.; Ikeshima, D.; Tawatari, R.; Yamaguchi, T.; O’Loughlin, F.; Neal, J.C.; Sampson, C.C.; Kanae, S.;
Bates, P.D. A High-Accuracy Map of Global Terrain Elevations. Geophys. Res. Lett. 2017, 44, 5844–5853.
[CrossRef]
182. EUROSTAT. Manual of Concepts on Land Cover and Land Use Information Systems; European Communities:
Luxembourg, 2001.
183. McBratney, A.; Santos, M.M.; Minasny, B. On Digital Soil Mapping. Geoderma 2003, 117, 3–52. [CrossRef]
184. NGA. Performance Specification Digital Terrain Elevation Data (DTED); Report MILPRF-89020A; NGA:
Springfield, VA, USA, 1996; pp. 1–45.
185. Heady, B.; Kroenung, G.; Rodarmel, C. High Resolution Elevation Data (HRE) Specification Overview.
In Proceedings of the ASPRS/MAPPS 2009 Conference, San Antonio, TX, USA, 16–19 November 2009.
186. Takagi, M.; Asano, H.; Kikuchi, Y. Optimum Spatial Resolution of Digital Elevation Model for Topographical
Analysis. In Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial
Information Sciences, Ottawa, ON, Canada, 1–5 October 2002; Volume 34, pp. 442–446.
187. Hengl, T. Finding the Right Pixel Size. Comput. Geosci. 2006, 32, 1283–1298. [CrossRef]
188. Shary, P.A. Models of Topography. In Advances in Digital Terrain Analysis; Zhou, Q., Lees, B., Tang, G., Eds.;
Springer: Berlin/Heidelberg, Germany, 2008; pp. 29–57, ISBN 978-3-540-77800-4.
189. Eco, U. On the Impossibility of Drawing a Map of the Empire on a Scale of 1 to 1. In How Travel Salmon Other
Essays; Houghton Mifflin: Boston, MA, USA, 1994; pp. 95–106.
190. Charif, M. Optimum Sampling for Digital Terrain Modelling. Int. Arch. Photogramm. Remote Sens. 1992,
29, 77–78.
191. Evans, I.; Hengl, T.; Gorsevski, P.V. Chapter 22 Applications in Geomorphology. In Developments in Soil
Organic Matter; Hengl, T., Reuter, H.I., Eds.; Elsevier BV: Amsterdam, The Netherlands, 2009; Volume 33,
pp. 497–525.
192. Erskine, R.H.; Green, T.; Ramirez, J.A.; Macdonald, L.H. Digital Elevation Accuracy and Grid Cell Size:
Effects on Estimated Terrain Attributes. Soil Sci. Soc. Am. J. 2007, 71, 1371–1380. [CrossRef]
193. Kienzle, S. The Effect of DEM Raster Resolution on First Order, Second Order and Compound Terrain
Derivatives. Trans. GIS 2003, 8, 83–111. [CrossRef]
194. Guoan, T.; Mudan, Z.; Tianwen, L.; Yongmei, L.; Ting, Z. Simulation on Slope Uncertainty Derived from
DEMs at Different Resolution Levels: A Case Study in the Loess Plateau. J. Geogr. Sci. 2003, 13, 387–394.
[CrossRef]
195. Thompson, J.A.; Bell, J.C.; Butler, C.A. Digital Elevation Model Resolution: Effects on Terrain Attribute
Calculation and Quantitative Soil-Landscape Modeling. Geoderma 2001, 100, 67–89. [CrossRef]
196. Vaze, J.; Teng, J. Impact of DEM Resolution on Topographic Indices and Hydrological Modelling Results.
In Proceedings of the MODSIM 2007 International Congress on Modelling and Simulation, Christchurch,
New Zealand, 10–13 December 2007.
197. Vaze, J.; Teng, J.; Spencer, G. Impact of DEM Accuracy and Resolution on Topographic Indices.
Environ. Model. Softw. 2010, 25, 1086–1098. [CrossRef]
198. Wu, S.; Li, J.; Huang, G. Modeling the Effects of Elevation Data Resolution on the Performance of
Topography-Based Watershed Runoff Simulation. Environ. Model. Softw. 2007, 22, 1250–1260. [CrossRef]
199. Gravelius, H. Grundrifi der Gesamten Gewcisserkunde. Compend. Hydrol. 1914, I, 265–278.
200. Mandelbrot, B.B. The Fractal Geometry of Nature, Revised ed.; W.H. Freeman: San Francisco, CA, USA, 1982.
201. Polidori, L.; Simonetto, E. Effect of Scale on the Correlation Between Topography and Canopy Elevations in
an Airborne InSAR Product Over Amazonia. Procedia Technol. 2014, 16, 180–185. [CrossRef]
202. Wechsler, S.P. Perceptions of Digital Elevation Model Uncertainty by DEM Users. URISA J. 2003, 15, 57–64.
203. Lopez, C. Improving the Elevation Accuracy of Digital Elevation Models: A Comparison of Some Error
Detection Procedures. Trans. GIS 2000, 4, 43–64. [CrossRef]
204. Milledge, D.; Lane, S.N.; Warburton, J. The Potential of Digital Filtering of Generic Topographic Data for
Geomorphological Research. Earth Surf. Process. Landf. 2009, 34, 63–74. [CrossRef]
Remote Sens. 2020, 12, 3522 36 of 36

205. Monti-Guarnieri, A. Topographic Statistics for Phase Unwrapping. In Proceedings of the EUSAR, Cologne,
Germany, 4–6 June 2002; pp. 1–4.
206. Guarnieri, A.M. Using Topography Statistics to Help Phase Unwrapping. Sonar Navig. IEE Proc. Radar 2003,
150, 144. [CrossRef]

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional
affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).