Comparison of Filtering Algorithms
Comparison of Filtering Algorithms
net/publication/228579450
CITATIONS READS
70 2,040
2 authors:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by George Vosselman on 16 May 2014.
ABSTRACT
To determine the performance of filtering algorithms a study was conducted in which eight groups filtered data supplied to them. The
study aimed to determine the general performance of filters, the influence of point resolution on filtering and future research
directions. To meet the objectives the filtered data was compared against reference data that was generated manually. In general the
filters performed well in landscapes of low complexity. However, complex landscapes as can be found in city areas and
discontinuities in the bare earth still pose challenges. Comparison of filtering at lower resolutions confirms that amongst other factors
the method of filtering also has an impact on the success of filtering and hence on the choice of scanning resolution. It is suggested
that future research be directed at heuristic classification of point-clouds (based on external data), quality reporting, and improving
the efficiency of filter strategies.
1 INTRODUCTION In line with these aims a web site was set up in which twelve
sets of data were provided for testing. Individuals and groups
While commercial Airborne Laser Scanning (ALS) systems wishing to participate in the study were kindly requested to
have come a long way, the choice of appropriate data processing process all twelve data sets if possible. A total of 8 data sets
techniques for particular applications is still being researched. (results) were received. The algorithms used by participants
Data processing, here, is understood as being either come from a cross-section of the most common strategies (or
semiautomatic or automatic, and includes such tasks as variants) for extracting the bare-earth from ALS point-clouds.
“modelling of systematic errors”, “filtering”, “feature The report is broken into three main parts. Section 2 discusses
extraction” and “thinning”. Of these tasks, manual classification common characteristics of filtering algorithms. Sections 3 and 4
(filtering) and quality control pose the greatest challenges, describe the data used, and discuss the results of the
consuming an estimated 60 to 80% of processing time (Flood, comparisons. In section 5 the results from the study is discussed
2001), and thus underlining the necessity for research in this and conclusions are drawn in respect to the objectives setout.
area. Algorithms have been developed for semi
automatically/automatically extracting the bare-earth from 2 FILTER CHARACTERISTICS
point-clouds obtained by airborne laser scanning and InSAR.
While the mechanics of some of these algorithms have been Filters are built from combinations of different elements.
published, those of others are not known because of proprietary Therefore, to understand or predict the behavior and output of a
restrictions. Some comparison of known filtering algorithms and filter the way in which elements are combined has to be
difficulties have been mentioned in Huising and Gomes Pereira understood. Seven elements have been identified:
(1998), Haugerud and Harding (2001), Tao and Hu (2001).
However, an experimental comparison was not available. 2.1 Data Structure
Because of this it was felt that an evaluation of filters was
required to assess the strengths and weaknesses of the different The output of an ALS is a cloud of irregularly spaced 3D points.
approaches based on available control data. In line with the Some filters work with the raw point-cloud. However, to take
framework of ISPRS Commission III, the Working Group III/3 advantage of image processing toolkits some filtering algorithms
"3D Reconstruction from Airborne Laser Scanner and InSAR resample the ALS produced point-cloud into an image grid,
Data" initiated a study to compare the performance of various before filtering.
automatic filters developed to date, with the aim of:
2.2 Test neighborhood and the number of points filtered
at a time
1. Determining the comparative performance of existing
filters.
Filters always operate on a local neighborhood. In the
2. Determining the performance of filtering algorithms under classification operation (bare earth or object) two or more points
varying point densities. are classified at a time. In regard to the neighborhood this
classification can be done in three possible ways.
3. Determining problems in the filtering of point-clouds that
still require further attention
Point-to- Point - In these algorithms two points are compared at information is gathered about the neighborhood of a point and
a time. The discriminant function is based on the positions of the thus a much more reliable classification can be obtained.
two points. If the output of the discriminant function is above a
certain threshold then one of the points is assumed to belong to
an object. Only one point is classified at a time.
Slope based - In these algorithms the slope or height difference 2.7 Using first pulse and reflectance data
between two points is measured. If the slope exceeds a certain
threshold then the highest point is assumed to belong to an Some scanners record multiple pulse returns. This feature is
object. advantageous in forested areas, where the first pulse is usually
off vegetation and subsequent pulses are from surfaces below
Block-minimum - Here the discriminant function is a horizontal the vegetation canopy. Additional to multiple pulse
plane with a corresponding buffer zone above it. The plane measurements the intensity of the returned pulses is also
locates the buffer zone, and the buffer zone defines a region in measured. Different surfaces in the landscape will absorb/reflect
3D space where bare earth points are expected to reside. pulses differently and therefore it may be possible to use this
information in classifying points.
Surface base - In this case the discriminant function is a
3 TEST DATA AND ALGORITHMS
parametric surface with a corresponding buffer zone above it.
The surface locates the buffer zone, and as before the buffer
As part of the second phase of the OEEPE project on laser
zone defines a region in 3D space where ground points are
scanning (OEEPE 2000) companies were invited to fly over the
expected to reside.
Vaihingen/Enz test field and Stuttgart city center. These areas
were chosen because of their diverse feature content. Eight sites
Clustering/ Segmentation - The rational behind such
were selected for the comparison of filtering algorithms. The
algorithms is that any points that cluster must belong to an
landscape was scanned with an Optech ALTM scanner, and the
object if their cluster is above its neighborhood. It is important
data was produced by FOTONOR AS. Both first and last pulse
to note that for such a concept to work the clusters/ segments
data were recorded. Eight test sites (four urban and four rural)
must delineate objects and not facets of objects.
were chosen. The urban sites were at a resolution of 1-1.5m. The
2.5 Single step vs. iterative rural sites were at a resolution of 2-3.5m. This data was offered
to participants for processing. Some characteristics of the test-
Some filter algorithms classify points in a single pass while sites are listed below:
others iterate, and classify points in multiple passes. The
advantage of a single step algorithm is computational speed. (i) Steep slopes, (ii) Mixture of vegetation and
However, computational speed is traded for accuracy by buildings on hillside, (iii) Large buildings, (iv)
iterating the solution, with the rational that in each pass more Irregularly shaped buildings, (v) Densely packed
buildings with vegetation between them, (vi) Building
with eccentric roofs, (vii) Open spaces with mixtures 4 RESULTS/ COMPARISONS
of low and high features, (viii) Railway station with
trains (low density of terrain points), (ix) Bridge, (x) 4.1 Qualitative assessment
High bridge, (xi) Underpass, (xii) Ramps, (xiii) Road
with embankments, (xiv) Road with bridge and small The fifteen samples were extracted with a view to examining
tunnel, (xv) Quarry (break-lines), (xvi) Vegetation on and comparing how the different filters behave and to identify
river bank, (xvii) Data gaps difficulties in filtering. Based on an examination of the eight
data sets and the fifteen sample sets, each of the filters was
Two sites were also provided at three different resolutions (1- assessed for difficulties.
1.5m, 2-3.5m, 4-6m for the urban and 2-3.5m, 4-5.5m, 7-10m
4.1.1 Filtering difficulties
for the rural) to test filter performance at three different point-
cloud resolutions. To obtain the first reduced resolution the scan
The filtering difficulties identified from the qualitative
lines in the original point-clouds were detected and every second
comparison relate to Outliers, Object Complexity, Attached
point in a scan line was dropped. Similarly, the second reduced
Objects, Vegetation, and Discontinuities in the Bare Earth. Each
point-cloud was produced from the first reduced point-cloud.
is briefly discussed below.
3.1 Reference data sets
4.1.1.1 Outliers
The reference data was generated, by manually filtering the High outliers - These are points that normally do not belong to
eight data sets. In the manual filtering, knowledge of the the landscape. They originate from hits off objects like birds,
landscape and some aerial imagery where available. All points low flying aircraft, etc. Most filters handle such features easily,
in the reference data sets were labelled “Bare Earth” or because they are so far elevated above neighboring points.
“Object”. From the eight data sets fifteen samples were Therefore it is included here for completeness only.
abstracted. These fifteen samples were rechecked and it is these
samples that are used in the quantitative analysis. The fifteen Low outliers - These are points that also normally do not belong
samples are representative of different environments (but are to the landscape. They originate from multi-path errors and
more focused in respect to the expected difficulties). errors in the laser range finder. Most filters work on the
assumption that the lowest points in a point-cloud must belong
3.2 Filtered data sets to the terrain. These points are naturally an exception to the rule.
Many algorithms also work on the assumption that points
Corresponding samples to the fifteen reference samples were neighboring a lower point must belong to an object. In practice
also extracted from the filtered data provided by the participants. this assumption usually holds. However, in cases where the
The filtered data sets contain only the bare earth points. lowest point is an outlier, the assumption fails completely,
resulting in an erosion of points in the neighborhood of the low
3.3 Participants outlier.
Eight individuals/groups submitted results for the test. A brief 4.1.1.2 Object complexity
breakdown of the participants and a description of their filters
are given in Table 3.1. Very large objects - Because many of the filtering algorithms
are localized, large objects may not be filtered if the size of
Table 3.1 Participants objects exceeds that of the test neighborhood.
Elmqvist
Axelsson
Roggero
Brovelli
4.1.1.3 Attached objects
Sithole
Pfeifer
Wack
Sohn
Building on slopes - Such objects have roofs that are elevated
above the bare earth on some sides and minimally or not at all Outliers
on other sides. Because of this it becomes difficult to distinguish High points G G G G G G G G
between such objects and the bare earth. High points G G G G G G G G
influence
Low points G F F G G F F G
Bridges - Artificial structures spanning the gap (road, river, Low points G G G G G P P G
etc.,) between bare earth surfaces. influence
Object complexity
Objects G G G G G G G G
Ramps - Natural/Artificial structures spanning the gaps between Large objects G G G G G G G G
bare earth surfaces; where one is lower than the other. Small objects F F G F F G F G
Complex objs. F F F F F F F F
4.1.1.4 Vegetation Low objects P P G G G F F F
Disconnected F F F F F F F F
Vegetation on slopes - Vegetation points can be filtered based terrain
on the premise that they are significantly higher than their Detached objects
Building on G F F F F G F G
neighborhoods. This assumption naturally falls away in steep slopes
terrain where terrain points may lie at the same height as Bridges F
vegetation points. G/ G/ G/ G/ G/ G/ G/
R R R R R R R
Ramps P P P P P F P P
Low vegetation - Similar to the problem of low objects, except Vegetation
this time complicated by steep slopes. Vegetation G G G G G G G G
Veg. on slopes G G F F F F F G
4.1.1.5 Discontinuity Low veg. G F F F G F F G
Discontinuity
Preservation (Steep slopes) - Generally objects are filtered Preservation P P P P F F P F
because they are discontinuous to the terrain. Occasionally it Sharp ridges P P P P F P P P
also happens that the bare earth is piecewise continuous. At G, Good; F, Fair; P, Poor; R, Removed
discontinuities some filters will operate as they would on
objects. Therefore, discontinuities in the Bare Earth are lost. 4.2 Quantitative assessment
Sharp ridges - The preservation of ridges is a similar but more The quantitative assessment was done by generating cross-
drastic problem of retaining convex slopes as described by matrices and generating visual representations of the cross-
Huising and Pereira (Huising et. al. 1998). matrices (figure 4.1). The cross-matrices were then used to
evaluate Type I and Type II errors, and visual representation
4.1.2 Assessment were then used to determine the relationship of Type I and Type
II errors to features in the landscape. Furthermore the size of the
The qualitative assessment was based on a visual examination error between the reference and filtered DEMs was computed
and comparison of the filtered data sets. The Qualitative and analyzed. The purpose of this was to determine the potential
assessment of filters is summarized in Tables 4.1, 4.2. The main influence of the filtering algorithms on the resulting DEM,
problems faced by the filter algorithms are in the reliable based on the predominant features in the data sets. It must be
filtering of complex scenes, filtering of buildings on slopes, stressed that what is presented here covers the difficulties in
filtering of disconnected terrain (courtyards), and discontinuity filtering as observed in the data and in general all the filters
preservation. worked quite well for most landscapes.
Table 4.1 Meaning of Good, Fair and Poor (used in Table 4.2) 4.2.1 Type I vs Type II
Rating Item filter rating Influence rating All the filtering algorithms examined make a separation between
Good Item filtered most of No influence Object and Bare Earth based on the assumption that certain
the time (> 90%) structures are associated with the former and others with the
Fair Item not filtered a few Small influence on filtering of
times neighboring points latter. This assumption while often valid does sometimes fail.
Poor Item not filtered most Large influence on filtering of This failure is caused by the fact that filters are blind to the
of the time (< 50%) neighboring points context of structures in relation to their neighborhoods. Because
of this a trade off is involved between making Type I (reject
Bare Earth points) and Type II errors (accept Object points).
Axelsson Unused 121 number of Type I errors. However, when the height difference at
Filtered discontinuities increases the performance of the slope-based
Ref BE Obj filters remains the same. This is not the case with some of the
BE 21880 602 22482 68.99% other filters, where a discontinuity can also influence filtering in
Obj 588 9515 10103 31.01%
the neighborhood of the discontinuity. Another interesting
22468 10117 32585
68.95% 31.05% aspect of filtering at discontinuities is where the Type I errors
ratio BE-Obj/ Obj-BE 1.02 occur. Some filters only cause Type I errors at the Top edge of
Typical numerical results (including cross-matrix) discontinuities, whereas others cause errors at both the top and
bottom edges. The potential for the latter case happening is
relatively higher for surface based filters.
4.2.4 Bridges
The Axelsson filter generated the least total error (total number
of points misclassified) on steep slopes. One explanation for this
could lie in the Axelsson filter’s (or parameterizations) bias
towards Type II errors. In general there are fewer Object points
then there are Bare Earth points, and if bias is on making Type
II errors then it also means that the Type II misclassifications
will be fewer than Type I misclassifications. Nonetheless,
filtering in steep terrain still remains a problem especially at
reduced resolutions.
4.2.3 Discontinuities
Figure 4.2 Complex scene
The two slope based filters have the most difficulty with
discontinuities in the Bare Earth. This is borne by the large
4.2.6 Outliers Site 1: Type I
40%
The number of outliers (both high and low) are relatively small 35%
25%
small. However, their influence on filtering in their
% error
20%
neighborhoods can be considerable. The filters by Axelsson and 15%
Sithole produce such Type I errors. While single outliers cause 10%
depending on the concept base of the filter. Sohn Axelsson Pfeifer Brovelli Roggero Wack Sithole
Site 1: Type II
% error
10%
errors in the underlying slope, and in the case of the Elmqvist 8%
30%
4.3 Effect of resolution 25%
% error
20%
As the resolution of the data is lowered, the bare earth and 15%
0%
earth and objects breakdown. To test this the filtered data the 1 2 3
10%
site 1 and 8 it can be seen that there are variations and
8%
exceptions. Four possible reasons are offered. 6%
4%
are much larger than those for site 8. This is due to (i) more 0%
1 2 3
The effect of lowered resolutions on the performance of filters OEEPE: 2000 Working Group on laser data acquisition. ISPRS
was also tested. Comparison of the results at lower resolutions Congress 2000. http://www.geomatics.kth.se/~fotogram/
OEEPE/ISPRS_Amsterdam_OEEPE_presentation.pdf
confirms that amongst other factors the method of filtering also
has an impact on the success of filtering and hence on the choice Pfeifer N.; Kostli A., Kraus K., 1998: "Interpolation and
of scanning resolution. However, more tests are required to form filtering of laser scanner data - implementation and first results."
International archives of photogrammetry and remote sensing,
a clear impression of which filter characteristics have a
Vol XXXII, Columbus, part 3/1 Columbus, pp.153 - 159.
significant impact on filtering at lower resolutions.
Roggero M., 2001: “Airborne Laser Scanning: Clustering in raw
The filtering of complex urban landscapes still poses the greatest data”. IAPRS, Vol XXXIV –3/W4 Annapolis, MD, 22-24 Oct,
2001. pp. 227-232.
challenges. As has been suggested elsewhere, filtering using
segmentation, and understanding of the context of the landscape Sithole G., 2001: “Filtering of laser altimetry data using a slope
being filtered and data fusion might be one ways in which this adaptive filter”. IAPRS, Vol. XXXIV –3/W4 Annapolis, MD,
challenge could be overcome. 22-24 October 2001. pp. 203-210.
Sohn G., Dowman I., 2002: “Terrain Surface Reconstruction by
The full report of the test can be found at the following URL: the Use Of Tetrahedron Model With the MDL Criterion”.
http://www.geo.tudelft.nl/frs/isprs/filtertest/ IAPRS, Vol XXXIV Part 3A. ISPRS Commission III,
Symposium. September 9 - 13, 2002, Graz, Austria. pp. 336-
344.
7 ACKNOWLEDGEMENTS
Tao C. V., Hu Y., 2001: “A review of post-processing
This study would not have been possible without the help and algorithms for airborne LIDAR Data”. Proceedings ASPRS
co-operation of participants who took time from their schedules conference April 23-27, 2001. St. Louis Missouri. CD-ROM, 14
pages.
to filter the twelve data sets. The authors wish to extend their
gratitude to P. Axelsson, C. Briese, M. Brovelli, M. Elmqvist, N. Vosselman G., 2000: “Slope based filtering of laser altimetry
Pfeifer, M. Roggero, G. Sohn, R. Wack and A. Wimmer. data”. IAPRS, Vol XXXIII, Part B3, Amsterdam, The
Netherlands. pp. 935-942.
Wack R., Wimmer A., 2002: “Digital Terrain Models From
REFERENCES Airborne Laser Scanner Data – A Grid Based Approach”.
Axelsson P., 1999: "Processing of laser scanner data - IAPRS, Vol XXXIV Part 3B. ISPRS Commission III,
algorithms and applications". ISPRS Journal of Photogrammetry Symposium. September 9 - 13, 2002, Graz, Austria. pp. 293-
& Remote Sensing, 54 (1999). pp. 138 -147. 296.