Download the Full Ebook and Access More Features - ebooknice.

com

(Ebook) Object and Pattern Recognition in Remote

Sensing: Modelling and Monitoring Environmental
and Anthropogenic Objects and Change Processes by
Stefan Hinz; Andreas Braun; Professor Martin
Weinmann ISBN 9781849951289, 1849951284
https://ebooknice.com/product/object-and-pattern-
recognition-in-remote-sensing-modelling-and-monitoring-
environmental-and-anthropogenic-objects-and-change-
processes-51882070

OR CLICK HERE

DOWLOAD EBOOK

Download more ebook instantly today at https://ebooknice.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Start reading on any device today!

(Ebook) Piano adventures Performance 3b by Nancy and

Randall Faber

https://ebooknice.com/product/piano-adventures-performance-3b-52393612

ebooknice.com

(Ebook) Advances in Environmental Remote Sensing: Sensors,

Algorithms, and Applications (Remote Sensing Applications
Series) by Qihao Weng ISBN 9781420091755, 1420091751
https://ebooknice.com/product/advances-in-environmental-remote-
sensing-sensors-algorithms-and-applications-remote-sensing-
applications-series-2390016
ebooknice.com

(Ebook) Advanced Environmental Monitoring with Remote

Sensing Time Series Data and R by Alexandra Gemitzi,
Nikolaos Koutsias, Venkat Lakshmi ISBN 9780367205270,
0367205270
https://ebooknice.com/product/advanced-environmental-monitoring-with-
remote-sensing-time-series-data-and-r-11198072

ebooknice.com

(Ebook) Remote Sensing and Global Environmental Change by

Sam J. Purkis, Victor V. Klemas ISBN 9781405182256,
9781444339352, 1405182253, 1444339354
https://ebooknice.com/product/remote-sensing-and-global-environmental-
change-2388674

ebooknice.com
(Ebook) Practical Machine Learning and Image Processing:
For Facial Recognition, Object Detection, and Pattern
Recognition Using Python by Himanshu Singh ISBN
9781484241486, 9781484241493, 1484241487, 1484241495
https://ebooknice.com/product/practical-machine-learning-and-image-
processing-for-facial-recognition-object-detection-and-pattern-
recognition-using-python-9961714
ebooknice.com

(Ebook) Urban Remote Sensing : Monitoring, Synthesis and

Modeling in the Urban Environment, 2nd Edition by Xiaojun
Yang ISBN 9781119625841, 111962584X
https://ebooknice.com/product/urban-remote-sensing-monitoring-
synthesis-and-modeling-in-the-urban-environment-2nd-edition-35173678

ebooknice.com

(Ebook) Inspection and Monitoring Technologies of

Transmission Lines with Remote Sensing by Yi Hu and Kai
Liu (Auth.) ISBN 9780128126455
https://ebooknice.com/product/inspection-and-monitoring-technologies-
of-transmission-lines-with-remote-sensing-6614680

ebooknice.com

(Ebook) Remote Sensing and GIS for Ecologists: Using Open

Source Software (Data in the Wild) by Martin Wegmann
(editor), Benjamin Leutner (editor), Stefan Dech (editor)
ISBN 9781784270223, 1784270229
https://ebooknice.com/product/remote-sensing-and-gis-for-ecologists-
using-open-source-software-data-in-the-wild-11120736

ebooknice.com

(Ebook) Understanding Forest Disturbance and Spatial

Pattern: Remote Sensing and GIS Approaches by Michael A.
Wulder, Steven E. Franklin ISBN 9780849334252,
9781420005189
https://ebooknice.com/product/understanding-forest-disturbance-and-
spatial-pattern-remote-sensing-and-gis-approaches-2137710

ebooknice.com
FM 2021/4/24 page i

Object and Pattern Recognition in

Remote Sensing
Modelling and Monitoring Enviromental and
Anthropogenic Objects and Change Processes

Edited by
S. Hinz, A.C. Braun, M. Weinmann
FM 2021/4/24 page ii

Published by
Whittles Publishing,
Dunbeath,
Caithness KW6 6EG,
Scotland, UK

www.whittlespublishing.com

© 2021 S. Hinz, A.C. Braun, M. Weinmann

ISBN 978-184995-128-9

All rights reserved.

No part of this publication may be reproduced,
stored in a retrieval system, or transmitted,
in any form or by any means, electronic,
mechanical, recording or otherwise
without prior permission of the publishers.

The publisher and authors have used their best efforts in preparing this book, but assume no
responsibility for any injury and/or damage to persons or property from the use or
implementation of any methods, instructions, ideas or materials contained within this book.
All operations should be undertaken in accordance with existing legislation, recognized codes
and standards and trade practice. Whilst the information and advice in this book is believed
to be true and accurate at the time of going to press, the authors and publisher accept no legal
responsibility or liability for errors or omissions that may have been made.

Printed by
FM 2021/4/24 page iii

Contents

Foreword vii
Editors and Authors xi

Part I: Methodology 1

1 Introduction 1
1.1 Sensors and Imaging 3
1.2 From Image Information to Semantics of Objects 4
1.3 Conclusion 6
References 7

2 Object, Data and Sensor Modelling 9

2.1 Sensors and Sensor Models 9
2.2 Data Content and Complexity 20
2.3 Object Modelling 22
2.4 Extraction Strategy 29
2.5 Conclusions 32
References 33

3 Feature Extraction from Images and Point Clouds:

Fundamentals, Advances and Trends 35
3.1 Feature Extraction from 2D Imagery 36
3.2 Feature extraction from 3D point clouds 52
3.3 Spatio-temporal Features 63
3.4 Conclusions 65
References 65

4 A Short Survey on Supervised Classification in Remote Sensing 73

4.1 Supervised and Unsupervised Classification 73
4.2 Rule-based and Exemplar-based Learning 74
4.3 Recent Exemplar-based Supervised Classifiers 75
4.4 One-class Versus Multi-class Classification 92
4.5 Conclusions 94
References 95

5 Context-based Classification 101

5.1 Graphical Models 102
5.2 Markov Random Fields (MRF) 105
5.3 Conditional Random Fields (CRF) 109
5.4 Inference and Training 113
FM 2021/4/24 page iv

iv Object and Pattern Recognition

5.5 Concluding Remarks 116

References 117

6 Toward a Framework for Quality Assessment in

Remote Sensing Applications 121
6.1 Introduction 121
6.2 Related Work 122
6.3 Basics of Quantitative Quality Assessment 123
6.4 Framework for Quantitative Quality Assessment 137
6.5 Example for Quantitative Quality Assessment 140
6.6 Conclusions 142
References 143

Part II: Application 149

7 From Raw 3D Point Clouds to Semantic Objects 151

7.1 Point Cloud Acquisition 151
7.2 Basic Point Cloud Processing Techniques 156
7.3 Fusion of 2D and 3D Information 158
7.4 Scene Analysis 168
7.5 Conclusions 173
References 174

8 Traffic Extraction and Characterisation from

Optical Remote Sensing Data 179
8.1 Motivation 179
8.2 Traffic Model 181
8.3 Traffic Extraction and Characterisation 183
8.4 Evaluation and Discussion 191
8.5 Conclusions and Outlook 193
References 194

9 Object Extraction in Image Sequences 199

9.1 Object Tracking 199
9.2 Persons in Aerial Image Sequences 204
9.3 Detection of Persons 205
9.4 Tracking of Persons 209
9.5 Experiments 216
9.6 Conclusion 224
References 225
FM 2021/4/24 page v

Contents v

10 A Process-based Model Approach to Predict Future Land-use

Changes and Link Biodiversity with Soil Erosion in Chile 227
10.1 Introduction 227
10.2 Approach of this Study 228
10.3 Process-based Land-use Assessment 229
10.4 Prediction of Future Land-Use Changes 231
10.5 Estimation of Potential Erosion Risks 235
10.6 Discussion and Conclusion 239
References 241

11 Interferometric SAR Image Analysis for 3D Building

Reconstruction 247
11.1 SAR and InSAR Principle 247
11.2 Building Signature 251
11.3 Overview of Building Reconstruction Strategies 252
11.4 Approach of Building Reconstruction 255
11.5 Results 262
11.6 Summary and Outlook 263
References 264

12 Detection and Classification of Collapsed Buildings after

a Strong Earthquake by means of Laser Scanning
and Image Analysis 267
12.1 Motivation 267
12.2 Related Work 268
12.3 Damage Scales and Damage Features 270
12.4 Applied Models 273
12.5 Strategy for Damage Analysis 275
12.6 Results 283
12.7 Outlook 285
References 287

13 A Settlement Process Analysis in Coastal Benin:

Confronting Scarce Data Availability in Developing Countries 289
13.1 Introduction 289
13.2 Objectives of this chapter 290
13.3 Study Area and Data Availability 292
13.4 Methods for Settlement Process Studies based on Pattern
Analysis 294
13.5 Results of the Settlement Process Analysis 299
13.6 Discussion 302
13.7 Conclusion and Outlook 305
References 307
FM 2021/4/24 page vi

vi Object and Pattern Recognition

Part III: Conclusion 311

14 Benchmarking: A Basic Requirement for Effective

Performance Evaluation 313
14.1 Introduction 313
14.2 3D Reconstruction, Image Orientation and Navigation 313
14.3 Object Detection and Tracking 314
14.4 Object Classification and Semantic Segmentation 314
References 319

15 Remote Sensing and Computer Vision Image Analysis:

Summary and Recent Trends 323
15.1 Summary 323
15.2 The Complexity of Ground Truth Acquisition for
Model Validation and Accuracy Assessment 327
15.3 Recent Trends from our Perspective 330
15.4 Conclusions 339
References 340

Index 347
FM 2021/4/24 page vii

Foreword

An image says more than a thousand words – this well-known proverb, which
has its origin in the advertisement industry, describes much of the fascination of
photogrammetry and remote sensing, and in particular of its images. In sharp
contrast to machines – and the statement remains true also in today’s era of
deep learning and artificial intelligence – humans are capable of interpreting
images and videos, and of extracting the information contained therein without
much effort and in real-time. However, we do not know just how we accomplish
this task, which perhaps explains some of the fascination of aerial and satellite
imagery.
Major characteristics of photogrammetry and remote sensing are:

• contact-free data acquisition,

• short image acquisition time which allows the capture of dynamic pro-
cesses,
• derivation of 3D information from stereoscopic images,
• complete iconic documentation of the whole scene,
• the possibility to measure and monitor objects of arbitrary size; methods in
use vary in scale from microscopy all the way to planetary remote sensing.

Traditionally, remote sensing deals with earth observation from space – mon-
itoring of processes on the Earth’s surface, as well as in the atmosphere are of
prime interest – while the focus of photogrammetry is the 3D geometric and the-
matic exploitation of images of all kind. Technical innovations, however, have
led to more and more overlap between the two disciplines. Photogrammetry
was of course established long before satellites became available; nevertheless
it is today regarded as part of the wider field of remote sensing. Moreover,
photogrammetry forms one of the foundations of modern computer vision.
In recent years, photogrammetry and remote sensing have witnessed great
changes in virtually every stage of image acquisition and processing. These
developments have had a profound impact on the theory, development and
operational use of photogrammetry and remote sensing. Examples include:

• new and better sensors and platforms such as 3D cameras, flash lidars,
unmanned aerial vehicles and mobile mapping vehicles,
• the deployment of satellite constellations with the aim of imaging the earth
in high resolution every single day,
• a shift from single sensors to integrated sensor systems, e. g. for assisted
and autonomous driving,
FM 2021/4/24 page viii

viii Object and Pattern Recognition

• an increasing use of, and integration with, methods developed in computer

science and computer vision, both for geometric and sematic tasks,
• a shift from mapping to monitoring and to dynamic geospatial services,
e.g. for sustainable development.

Innovation in our field has been strongly influenced by progress in infor-

mation and communication technology, and terms like ubiquitous computing,
geosensor networks, digital earth, big data, deep learning, cloud computing, the
semantic web, the internet of things and crowd sourcing have become part of our
discipline. Besides a large variety of new applications, all the way from robotics
and driver-assistance systems to animal behaviour studies and environmental
monitoring, new methodologies for automatic, reliable information extraction
and cooperative decision making employing concepts from data mining and
machine learning, are now commonplace in our field.
This observation leads directly to the title of this book – Object and pattern
recognition in remote sensing. The book reports on research and development
projects which were carried out in approximately the last 10 years at the Institute
of Photogrammetry and Remote Sensing of the Karlsruhe Institute of Technol-
ogy (KIT) under the leadership of Professor Stefan Hinz, and it gives an excellent
overview of the current state-of-the-art in photogrammetry and remote sens-
ing. For decades KIT has been one of the prime centres in this area, and the
work documented in this book proves that this remains the case. While the
book is not designed as a text book, it is of high relevance to students and other
people wanting to learn about photogrammetry and remote sensing. The indi-
vidual chapters discuss a number of important topics and, at the same time,
show the clear direction of the current activities, at KIT and elsewhere, in
photogrammetry and remote sensing research.
I believe such success has basically two ingredients: (a) individual excellence,
which the KIT group greatly possesses, and (b) an international network, to be
able to share and learn from latest trends in their own discipline and in neigh-
bouring fields. Together with his group, Stefan Hinz is very active in the network
set up and maintained within the International Society of Photogrammetry and
Remote Sensing (ISPRS).
ISPRS, www.isprs.org, is an international non-governmental organization
that promotes international cooperation between the worldwide organizations
with interests in the photogrammetry, remote sensing and spatial information
sciences. Established in 1910, ISPRS is the oldest international umbrella organi-
zation in its field, which may be summarized as addressing ‘‘information from
imagery’’. The ISPRS scientific and technical programmes are organized by five
Commissions. Each Commission is sponsored by an ISPRS Member organi-
zation for the four-year period between Congresses. The Commissions have
established more than 60 Working Groups which are responsible for particular
topics within the Commissions’ areas of responsibility.
FM 2021/4/24 page ix

Foreword ix

KIT has been active in ISPRS for a very long time. As early as 1964 Kurt
Schwidefsky, one of the predecessors of Stefan Hinz and later an ISPRS Hon-
orary Member, became President of the ISPRS Commission II, then entitled
‘‘Theory, methods, instruments of restitution’’. From 2016 – 2020, Stefan Hinz
is President of Commission I, ‘‘Sensor systems’’; in the preceding period Boris
Jutzi held the position of Vice President of the same Commission, continuing
in this position for the present period. In addition, scientists from KIT chaired
ISPRS working groups, organised ISPRS events and presented numerous papers
at the ISPRS Congresses, symposia and workshops. As mentioned, such engage-
ment is an essential ingredient in modern science and development, which in
the days of open science, open source, open data and open access is heavily
influenced by international and interdisciplinary cooperation.
I congratulate the authors, and in particular the editors under the leadership
of Stefan Hinz, for this nice piece of work and hope that the readers will find
the material as interesting as I did.

Christian Heipke
ISPRS President 2016–2020
Institut für Photogrammetrie und GeoInformation (IPI)
Leibniz Universität Hannover
FM 2021/4/24 page x
FM 2021/4/24 page xi

Editors and Authors

Stefan Hinz studied geodesy and geoinformatics at the TU Muenchen (MSc, 1998) and
received the doctorate in 2003 and the venia legendi (Habilitation) in 2008. His research
work concentrated on image analysis for automatic mapping from aerial and satellite images
by methods of artificial intelligence. In the course of the habilitation, this research extended
towards thermal and radar data with application to short and long-term monitoring of moving
objects, land use and urban infrastructure. With the appointment in 2008 as full professor
for Remote Sensing and Image Processing at KIT, Professor Hinz became director of the
Institute for Photogrammetry and Remote Sensing (IPF) and has since then been leading a
group of scientists of about 30–35 people. He is head of the Graduate School for Climate
and Environment (GRACE) of approximately 100 PhD students and served as dean of the
Faculty for Civil Engineering, Geo- und Environmental Sciences for the period 2012–2018.
Since 2016, he has been President of Technical Commission 1 Sensors and Platforms of the
International Society for Photogrammetry and Remote Sensing (ISPRS) and member of the
ISPRS Journal editorial advisory board.

Andreas Braun received the Diploma in Geoecology from Technical University of Karlsruhe
in 2009. After continuing his research in remote sensing at Karlsruhe Institute of Technology
(Institute of Photogrammetry and Remote Sensing), he received his PhD in 2013 and spent
a year as a postdoctoral researcher at the university of Freiburg (Chair of Remote Sensing
and Landscape Information Systems). From 2011 he additionally started his studies of soci-
ology, economy and philosophy at the FernUniversität in Hagen (Germany) and received his
MA in sociology in 2016. In 2015 he was assigned a junior professorship for risk oriented
regional development at the Institute of Regional Science (KIT), which he currently holds,
investigating human vulnerability in developing countries (mainly in Latin America).

Martin Weinmann received the Diploma in Electrical Engineering and Information Tech-
nology from the Technical University of Karlsruhe in 2009 and the PhD degree from the
Karlsruhe Institute of Technology (KIT) in 2015. Currently, he is postdoctoral researcher at
the Institute of Photogrammetry and Remote Sensing at KIT. His main research interests are
in the fields of computer vision, pattern recognition, active 3D vision, image and point cloud
processing, and scene analysis. He served several years as reviewer for the International
Society for Photogrammetry and Remote Sensing (ISPRS), the Institute of Electrical and
Electronics Engineers (IEEE) and the German Society for Photogrammetry, Remote Sensing
and Geoinformation (DGPF).

Callum Banfield received his Diploma in Geoecology from the Karlsruhe Institute of Tech-
nology in 2013. After having spent one year travelling, he started pursuing a PhD in soil science
at the Department of Temperate Ecosystems of the Georg-August University of Goettingen.
During his PhD studies he is focusing on subsoil carbon dynamics of arable fields, combining
biomarker approaches with stable and radioactive tracers.

Clémence Dubois received her Double-Diploma in Topography, Geodesy and Geomat-

ics from the National Institute of Applied Sciences (INSA) of Strasbourg, France, and the
Karlsruhe Institute of Technology (KIT), Germany, in 2011, and her PhD degree from the
Karlsruhe Institute of Technology (KIT) in 2015. From 2012 to mid-2016, she has been with
the Institute of Photogrammetry and Remote Sensing (IPF) of the Karlsruhe Institute of
Technology (KIT), and also a member of the GRAduate School for Climate and Environment
(GRACE) of KIT. Since 2016, she has been with the department "Geoscientific Information,
FM 2021/4/24 page xii

xii Object and Pattern Recognition

International Cooperation" of the Federal Institute for Geosciences and Natural Resources,
Germany. Her research activity mainly covers SAR, InSAR and stereo-SAR image exploita-
tion, especially object detection and reconstruction, change detection, and lately DInSAR
and PS processing for diverse geo-hazard and remote sensing applications.

Franz Rottensteiner received a Dipl. Ing. Degree in surveying and PhD and venia docendi
in photogrammetry, all from Vienna University of Technology, Vienna, Austria. He has been
a Postdoctoral Researcher with the Vienna University of Technology, the University of New
South Wales (Kensington, NSW, Australia), and the University of Melbourne (Parkville, Vic-
toria, Australia). He is currently with the Institute of Photogrammetry and GeoInformation,
Leibniz Universität Hannover (Hannover, Germany), where he is an Associate (apl.) Pro-
fessor, leading the Photogrammetric Image Analysis Group. His research interests include
photogrammetry, automated extraction of topographic objects, processing of lidar data, and
sensor orientation.

Miriam Hommel received her Diploma in Geodesy and Geoinformatics from Universität
Karlsruhe (TH) in 2006 and her PhD degree from Karlsruhe Institute of Technology (KIT)
in 2010. From 2006 to 2010 she was staff member of the Institute of Photogrammetry and
Remote Sensing (IPF) at KIT and from 2010 to 2011 of the Fraunhofer Institute for Manu-
facturing Engineering and Automation (IPA). Since 2012 she has been an employee at Aalen
University of Applied Sciences. Her main research interests are object detection and recon-
struction from lidar and multi-spectral data, automatic change and damage detection, as well
as classification and statistical data analysis.

Boris Jutzi received a Diploma in Electrical Engineering at University of Kaiserslautern,

Germany, obtained his PhD degree at Technische Universität München (TUM), Germany,
and the Venia Legendi at Karlsruhe Institute of Technology (KIT), Germany. Currently he
is Adjunct Professor and head of the ‘‘Active Sensors & Computer Vision’’ Department at
the Institute of Photogrammetry and Remote Sensing at KIT. His main research interests are
computer vision, lidar remote sensing, automatic image analysis, and optical active sensing
with full-waveform lidar and time-of-flight imaging. He served several years for the Inter-
national Society for Photogrammetry and Remote Sensing (ISPRS), e.g. 2012–2016 as Vice
President of the ISPRS Technical Commission I “Sensors and Platforms for Remote Sensing”
and 2016–2020 as Secretary of the ISPRS Technical Commission I “Sensor Systems”.

Sina Keller holds a degree in mathematics and geography from the University of Karlsruhe in
2010. She received her PhD degree from the Karlsruhe Institute of Technology (KIT) in 2015.
During her PhD she worked on a model assessing natural hazards on critical infrastructure.
Currently, she is a postdoctoral researcher at the Institute of Photogrammetry and Remote
Sensing at KIT. Since 2012, she has been working as a lecturer for mathematics at the Baden-
Wuerttemberg Cooperative State University Karlsruhe. In 2015, she also took over leadership
of the research division at the company ci-Tec GmbH. Her main research interests address
the modelling of hydrological processes with hyperspectral and thermal data as well as the
modelling and investigation of road network vulnerability.

Jens Leitloff received his Diploma in Geodetic Engineering from the Technische Universität
Berlin in 2003 and his PhD degree from Technische Universität München in 2011. From
2008 to 2011 he was with the Remote Sensing Technology Institute at the German Aerospace
Center (DLR) in Oberpfaffenhofen. In 2011 he joined the Institute of Photogrammetry and
Remote Sensing at Karlsruhe Institute of Technology (KIT). His main research interests are
computer vision, image processing, image and point cloud classification, camera systems and
camera calibration.
FM 2021/4/24 page xiii

Editors and Authors xiii

Florian Schmidt received his Diploma in Geodesy and Geoinformatics from the Techni-
cal University of Karlsruhe in 2009 and his PhD degree from the Karlsruhe Institute of
Technology (KIT) in 2012. From 2009 to 2012, he was staff member of the Institute of Pho-
togrammetry and Remote Sensing (IPF) at KIT. His main research interests include object
detection and tracking, machine learning and analysis of aerial image sequences.

Ulrike Sturm-Hentschel (nee Ulrike Sturm) received her diploma in geoecology in 2004 after
studying at Technical University in Karlsruhe (today: Karlsruhe Institute of Technology) and
the Universidade Federal do Parana in Brazil. From 2004 to 2006 she worked as a scientific
volunteer at Staatliches Museum für Naturkunde (Federal Museum of Natural History) in
Karlsruhe. From 2006 to 2011 she has been working as a scientific assistant at the Institute
of Photogrammetry and Remote Sensing of Karlsruhe Insitute of Technology. In 2012, she
received her PhD for her study on the settlement processes in the coastal area of Benin, in
which she applied several remote sensing techniques for a spatially explicit assessment of
settlement growth. These studies were backed up by field research, and the two types of data
sets were related to each other. From 2012 to 2013 she has been working as a freelancer in
the field of sustainable development. Since 2013, she has been a scientific assistant at the
German Museum in Munich, responsible for the Department of Ground Transportation.

Antje Thiele received her Diploma in Geodesy from the Technical University Dresden in
2004 and her PhD degree from Leibniz Universität Hannover (LUH) in 2013. Since 2005,
she has been with the Fraunhofer Institute of Optronics, System Technologies and Image
Exploitation (IOSB) and since 2009 also with the Institute of Photogrammetry and Remote
Sensing (IPF) at the Karlsruhe Institute of Technology (KIT). In 2014, she took over lead-
ership of the group ‘‘SAR Image Exploitation’’ at Fraunhofer IOSB. Her research activity
mainly concerns SAR and InSAR image exploitation by focusing on object detection and
reconstruction, classification, change detection and simulation.

Uwe Weidner received his Diploma in Geodesy in 1992 and his PhD degree in 1997, both
from the Rheinische Friedrich-Wilhelms University, Bonn. From 1997 to 2000 he worked for
a company in the field of aerial photogrammetry and from 2000 to 2002 for a company in
the field of industrial image processing and robotics. Since February 2003 he has been with
the Institute of Photogrammetry and Remote Sensing, KIT, Karlsruhe, as senior scientist
in the field of remote sensing. His research interests include various aspects of automatic
object extraction using multi-/hyperspectral remote sensing data and evaluation of meth-
ods in remote sensing. He served several years as reviewer for the International Society
for Photogrammetry and Remote Sensing (ISPRS), the Institute of Electrical and Electron-
ics Engineers (IEEE) and the German Society for Photogrammetry, Remote Sensing and
Geoinformation (DGPF).

Thomas Vögtle received his Diploma in Geodesy in 1982 and his PhD degree in 1989, both
from the University of Karlsruhe. Since 1984, he has been a member of the Institute of
Photogrammetry and Remote Sensing, working on several scientific projects. Since 1991 he
has been Senior Scientist and Lecturer at the same institute. His research activities concern
automation in image analysis and point cloud processing as well as 3D object extraction
from different data sets. He has served as reviewer for several international journals, has
published two lecture books as editor/co-author, and has received two awards (Dambach
award, Hansa-Luftbild award).

Sven Wursthorn received his Diploma in Geodesy from the Technical University of Karlsruhe
in 2002 and his PhD degree from the Karlsruhe Institute of Technology (KIT) in 2014.
Since 2002, he has been with the Institute of Photogrammetry and Remote Sensing (IPF)
at the Karlsruhe Institute of Technology (KIT). His main research interests are geographic
information systems, augmented reality and photogrammetry.
FM 2021/4/24 page xiv
Chap-01 2021/2/20 page 1

Part I
Methodology
Chap-01 2021/2/20 page 2
Chap-01 2021/2/20 page 3

Introduction CHAPTER

Stefan Hinz
1

There have been many attempts to grasp the meaning of the main topic of
this book – image and scene analysis. Summarising the perceptions of many
review books and articles, image and scene analysis focuses on the automated
characterisation of a certain part of the real world – the scene – recorded by a
specific sensor. In the context of remote sensing (as a discipline), the majority
of these devices are imaging sensors, hence relating scene analysis closely to the
general tasks of image processing. Indeed, it is often used synonymously with
terms like image understanding or even scene interpretation1 . A nice historical
review of the origins of digital image processing, which can be traced back to
the early 1960s, can be found in Gonzalez and Woods (2002). Condensing the
various aspects of image and scene analysis into one sentence, Rosenfeld and
Kak (1982) stated that image analysis is defined as ‘‘the automatic derivation
of a meaningful description of physical objects in the real world from images’’.
The aim of image analysis is thus to generate an explicit description of objects
from the information implicitly contained in the data. This task can be com-
pared to the human skill of photo interpretation. The primary aim is to achieve
the same results of interpretation as a human would achieve, although how
a human actually interprets images plays a minor role here. Needless to say,
the human approach to image interpretation can nevertheless give useful hints
when designing remote sensing image analysis frameworks.

1.1 Sensors and Imaging

When transferring the opportunities and potentials of image analysis to dis-
ciplines such as remote sensing, computer vision or photogrammetry, the
term image adopts a much broader notion than its original sense of a digital
frame image (i.e. a regularly sampled array of pixels) taken in the visible spec-
trum. Modern remote sensing and computer vision not only involve spectra far
beyond the visible domain, but also make use of active sensors such as radar
and laser scanning sensors (also known as lidar sensors). These kinds of sensor
emit specifically coded signals and record the echo(es) of exactly these signals,
thereby allowing analysis of a scene and the objects therein with a specific focus
on the properties of the emitted signals. A more in-depth discussion of the
different sensors and sensor concepts will be given in the next chapter.
Having been underestimated in the very early years of its development, the
tasks related to image analysis are known to be very hard to solve, depending on
Chap-01 2021/2/20 page 4

4 Object and Pattern Recognition

the complexity of the data and the scene to be analysed, in particular when deal-
ing with outdoor environments. In addition to the challenges of scene analysis
arising from the complex structure of the world, there are further challenges
in recording and storing the results from the process of image formation. The
decisive factor in this is the sensor, as well as the characteristics of the informa-
tion carrier. Usually, electromagnetic radiation (EMR) serves as the information
carrier, but other types of waves, in particular acoustic waves, are sometimes
used. The information carrier triggers, which characteristics of the objects in
the real world can be recorded by the sensor – in the case of EMR, the radiation
and reflection properties of the objects. The sensor, meanwhile, determines
how this information is effectively recorded and stored. Here, a series of geo-
metric and physical transformations come into play, which change the original
information fundamentally. These may include a geometrical transformation
from 3D to 2D, the spatial discretisation of the image by the CCD2 or CMOS3
sensor elements, physical conversion of the EMR into a measurable physical
entity (e.g. charge), and finally its quantisation into bits or integer numbers.

1.2 From Image Information to Semantics of Objects

If we transfer this concept to typical remote sensing data4 , which are most
relevant in the context of this book, the following characteristics (regarding the
information contained in the image) arise:

• Radiometric information: The intensity values are approximately propor-

tional to the reflection properties of the objects, but limited by the discrete
radiometric and spectral resolution of the sensor.
• Geometric information: The original geometry of the objects now exists
only in a discrete representation. In the case of common frame images
and assuming a pinhole camera, the dimension is reduced by the cen-
tral perspective projection from 3D to 2D. This may lead to a change in
the topology of adjacent scene elements in certain image areas. In the
case of occlusions, for example, two adjacent pixels do not correspond to
two adjacent surface elements in the real world. This holds similarly for
other imaging geometries in remote sensing, and also for 3D sensors like
laserscanners or time-of-flight cameras.
• Semantic information: Objects or object classes and their semantics are
defined only by human beings; that is, semantics are not attached to the
objects in a physical sense, and the meaning of objects is only assigned
by humans interpreting the radiometric and geometric information of the
images. The recorded data only implicitly contain information about the
semantics of the imaged scene. Transforming this implicitly contained
information into explicit information is thus the key issue of image and
scene analysis.
Chap-01 2021/2/20 page 5

Introduction 5

Figure 1.1 Example showing (a) radiometric information represented as digital num-
bers; (b) the same information represented as greyvalues; (c) assigned semantics in the
form of an explicit vehicle model.

Figure 1.1 illustrates the aspects mentioned above. The radiometric information
transformed into 8-bit digital numbers ranging from 0 (black) to 255 (white) of
a small part of a panchromatic aerial image is shown in Figure 1.1a. Figure 1.1b
visualises the same implicit information, but displayed in greyvalues. In con-
trast, Figure 1.1c shows the result after applying an image analysis approach
designed for detecting vehicles (Hinz, 2004). Now, as a vehicle model repre-
senting the vehicle as well as its substructures (such as windows, hood and
rear) has been fitted to the greyvalues, the semantics of the corresponding
greyvalue patches are available; thus, the implicit information has been made
explicit.
Determining the semantics of the objects is only possible through the
integration of additional knowledge. Usually, one can divide this into two
components:

• The model, comprising the radiometric, geometric and semantic aspects

of the object(s). Knowledge is represented by a model that describes a
specification of the general imaging process with respect to the objects to
be extracted. The knowledge contained in the model forms the basis for
the identification of objects from the raw intensity values.
• The extraction strategy, which builds the conceptual basis of the processing
scheme to automatically extract the objects from the image, thereby mak-
ing use of the model. The extraction strategy is the formalism that defines
individual image processing operations and the systematics of their appli-
cation. As the strategy determines the use of the model knowledge, it also
has the same relevance as the model itself.
Chap-01 2021/2/20 page 6

6 Object and Pattern Recognition

The rationale behind the extraction strategy is very similar to model inversion or
parameter estimation schemes (e.g. inversion or search algorithms). It aims to
invert the process of image generation and also to determine the object seman-
tics and object relations and their specific properties in the scene from the
recorded raw image data. To this end, the utilisation of 3D information turns
out to be extremely helpful. Sensors and methods like laser scanning, inter-
ferometric synthetic aperture radar (SAR) and time-of-flight imaging, among
others, deliver 3D data directly. However, with 2D image data, by using multiple
images providing different views 3D information can be reconstructed for static
scenes. If the orientation of the images is known – at least theoretically – each
object point, that is included in two or more images can be defined in 3D by
means of well-known photogrammetric methods (e.g. the advanced approach
of Hirschmueller, 2008). This task, in general, is not of a semantic nature and
can thus be automated quite reliably.

1.3 Conclusion
Automatic object extraction, in general, thus rests on two fundamentals: an
appropriate computer-oriented formalisation of knowledge (modelling) and the
use of knowledge for extraction (strategy). For both, a variety of different and
complementary approaches exist. An object model, for instance, may rest on
a specific, parametric (CAD-like) 3D description or on a view-based descrip-
tion automatically learned from many example images of a particular object
(or even learned from simulated images). Approaches for extraction strategies
range from more or less monolithic image processing pipelines designed for the
extraction of a specific object class to very generic concepts that involve auto-
matic feature learning, embedding of spatial context and object classification
(e.g., approaches based on Artificial Neural Networks (ANN) and more specific
deep learning concepts like Convolutional Neural Networks (CNN)).
The remainder of this book attempts to give an insight into the most impor-
tant methodological components and concepts of these different approaches
to automatic object extraction and scene analysis (Part I), while outlining the
particular advantages and deficiencies of each concept. Based on this, selected
approaches on object extraction, detection, classification and object tracking in
remote sensing data are presented (Part II). Science develops rapidly, and none
of these approaches will alone represent the current state-of-the-art. However,
the approaches are selected in such a way that they capture the diversity of dif-
ferent attempts and views on the most important aspects of object extraction
and scene analysis. The final part of the book (Part III) summarises and reviews
these aspects and focuses on recent scientific trends and developments in the
fields of computer vision and remote sensing, so that the reader will be able to
obtain an insight into the basic facts, the range of opportunities, the potential
Chap-01 2021/2/20 page 7

Introduction 7

challenges and also the limitations of this highly dynamic and complex field of
computer and engineering science.
As the ultimate scope of this book is knowledge-based understanding of
images, this book focuses more on semantic approaches to image interpretation
rather than on learning-based ‘‘black-box’’ approaches. Especially under the
light of the success of advanced deep learning schemes it seems necessary to put
semantic concepts to image analysis back into focus more than ever. Because
only by them, it will be possible to make the mechanisms involved in deep
learning systems transparent and explainable (Roscher et al, 2019).

References
Gonzalez, R. and Woods, R. (2002). Digital image processing (2nd edition). Prentice
Hall, Upper Saddle River, USA.
Hinz, S. (2004). Detection of vehicles and vehicle queues in high resolution aerial images.
Photogrammetrie Fernerkundung Geoinformation, 3(4):201–214.
Hirschmüller, H. (2008). Stereo processing by semiglobal matching and mutual informa-
tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2):328–341.
Rosenfeld, A. and Kak, A. (1982). Digital picture processing (2nd edition). Academic
Press, Orlando, USA.
Roscher, R., Bohn, B., Duarte, M. F. and Garcke, J. (2019). Explainable Machine Learn-
ing for Scientific Insights and Discoveries, available at: https://arxiv.org/abs/1905.
088832019 (last access Aug. 25, 2019)

Notes
1. In the literature, the terms scene analysis, image analysis, image understanding,
computer vision, machine vision and robot vision are often used synonymously.
2. Charged coupled device.
3. Complementary metal oxide semiconductor.
4. Here, we make no difference between digitally recorded images and scanned
photographs.
Chap-01 2021/2/20 page 8
Chap-02 2021/4/28 page 9

Object, Data and Sensor CHAPTER

Modelling 2
Stefan Hinz

2.1 Sensors and Sensor Models

In the following, we sketch the characteristics of the most important remote
sensing sensors on a very general and simplified basis in order to provide an
overview of the basic components of sensor models. If needed for the under-
standing of particular image and scene analysis approaches, the application-
oriented chapters of Part II of this book provide more details of specific sensors
and sensor models.

2.1.1 Cameras and Image-based 3D Reconstruction

Apart from satellite remote sensing where so-called line scanners play a major
role, frame cameras are the most important sensor for acquiring images
from airborne, terrestrial or UAV1 platforms. High-quality panchromatic or
multi-spectral aerial images, captured during photogrammetric aerial flights,
often serve as a primary data source. However, in recent years, rapid devel-
opments in the consumer camera market have allowed the design of much
cheaper (multi-)camera systems that are still suitable for applications with high
geometric and radiometric requirements.
Aerial or UAV images are often taken with high along-track overlap, ranging
from 60% up to more than 90% overlap of consecutive images, as this model
parameter depends primarily on the maximum frame rate and storage capacity
onboard (Figure 2.1). Across-track overlap influences the flight duration and is
thus more sensitive to the budget of a flight campaign than along-track overlap.
Hence, across-track overlap typically ranges from 20% to 60% in photogram-
metry, and occasionally more for specific applications. Due to automated flight
navigation and, especially in photogrammetric campaigns, stabilised camera
platforms, nearly the entire image bundle of a campaign can be captured in
predefined viewing directions. While remote sensing and photogrammetric
applications typically prefer nadir views, close-range and UAV images are often
taken in oblique viewing directions, too. The primary sensor model parameters
for determining the detailedness (i.e. the scale or geometric resolution) of an
image are the exterior orientation parameters (position and attitude of the cam-
era) and the interior orientation parameters of the camera, most prominently
focal length, pixel size and lens characteristics such as radial/tangential distor-
tion, spherical/chromatic aberration and vignetting. The procedure of camera
Chap-02 2021/4/28 page 10

10 Object and Pattern Recognition

Figure 2.1 Tpyical set-up of a photogrammetric flight (from Albertz, 2009) with hg
being the hight above ground, b the basis, s the side length of an image projected on
ground and c’ the camera focal length.

calibration usually subsumes the determination and correction of these param-

eters (Hartley and Zisserman, 2008; Kraus, 2007; Steger et al., 2007), resulting
in a geometric resolution of the calibrated images being able to capture object
details in the range of a few centimetres or decimetres for UAV and aerial images
and roughly 0.5 m to several metres (or even more) for satellite images.
The radiometric resolution of digital images is often 8 bit per channel,
although the original radiometric resolution of the sensor is often higher. Reduc-
ing the radiometric resolution to 8 bit per channel is sometimes done, though, in
order to apply more efficient image processing algorithms. A better radiometric
resolution gives rise to a higher dynamic range of the recorded intensities, which
eases the analysis of dark or shadowed areas in particular. During the develop-
ment of photogrammetric digital cameras in the late 1990s, it was shown, that
a higher dynamic range leads to significantly better accuracy of the resulting
digital surface model (DSM)2 in shadowed areas. Typical parameters further
influencing the quality of the radiometric resolution are quantum efficiency,
dark current, blooming, photon noise and charge transfer efficiency, which
relate to the linearity and stochastic behaviour of the detector elements and the
measuring units of the charge coupled device (CCD) and complementary metal
oxide semiconductor (CMOS) sensors that transform the incoming electromag-
netic radiation (EMR) into digital numbers. Luhmann (2010), for instance, gives
further insight into this topic.
Most of today’s cameras feature at least one panchromatic channel and
three multi-spectral channels (red, green and blue), often with one or even
more infrared channels covering the near and occasionally mid and far/thermal
infrared bands. This so-called spectral resolution is expanded in the special case
Chap-02 2021/4/28 page 11

Object, Data and Sensor Modelling 11

Figure 2.2 Example of hyperspectral imaging (according to Albertz, 2009) with λ being
the wavelength of the electromagnetic radiance and ρ being the reflection.

of hyperspectral images, where up to 200–300 channels cover the spectral sig-

natures of objects almost continuously (Figure 2.2). However, a higher spectral
resolution involves proportionally more CCD or CMOS detector elements, and
thus usually entails a loss in geometric resolution since the number of detector
elements that can be placed in a camera is still limited. Although for many appli-
cations that focus on a geometric analysis of the images one single panchromatic
channel is sufficient, further colour and/or infrared channels can deliver addi-
tional information about the reflection properties of the imaged objects, which
provides significant support to the classification of different vegetation types
and materials, for example. On the other hand, many objects (e.g. buildings
and vegetation) occur as prominent structures in a DSM, which can be derived
– as a consequence of the high overlap and multiple views of the objects of
a scene – by well-known 3D reconstruction techniques (Hirschmüller, 2008;
Kraus, 2007). Two major issues when using a DSM for image and scene analysis
are its accuracy and its consistency with the content of the images. To ensure the
latter, either a simultaneous recording of image and elevation data is necessary
(e.g. by an integrated system for image and laser data acquisition) or the height
information is derived from the images themselves. This indirect derivation
of height values is traditionally seen as cheaper and is thus more available. It
may achieve a similar or even better accuracy of the reconstructed 3D points
than laser scanning technology. This is restricted, however, to there being suf-
ficient (and non-repetitive) texture in the underlying images. In texture-less
regions, direct 3D measuring with laser scanning systems is superior. Also, for
inferring the volumetric radiometric and geometric characteristics of partially
Chap-02 2021/4/28 page 12

12 Object and Pattern Recognition

Figure 2.3 Characteristics of the data base from a the ‘‘city relief’’. Left: image
information (partial occlusions). Right: height information (geometric inaccuracies).

penetrable objects like trees and bushes, laser scanning is the technology of
choice. Figure 2.3 visualises the elevation model of an inner city area and the
corresponding stereo image pair. It clearly indicates that heights are mapped
realistic yet blurred when purely using stereo images as in this example (see also
the exemplary height profile).
Chap-02 2021/4/28 page 13

Object, Data and Sensor Modelling 13

Figure 2.4 Example of a multiple line camera (HRSC camera) (see (Albertz, 2009)).

Once the DSM has been reconstructed, each pixel of the original image
can theoretically be related to a specific height value, for example, by using
well-known collinearity equations (Kraus, 2007). The height accuracy is strongly
linked to the number of views from which the height value has been calculated.
Furthermore, it should be noted that the reconstructed height value (Z) is mod-
elled as a function of the lateral coordinates X, Y (i.e. Z = f (X, Y )), and thus
no full 3D description of the object space is possible. Instead, the term ‘‘2.5D’’
is frequently used for such a height representation. Further photogrammetric
products of images and DSMs include ortho-images or ortho-mosaics. For the
tasks of image analysis, the use of a DSM also allows projection from image
to object space, the integration of multiple views in object space, as well as
the extraction of further context information such as potential occlusion and
shadow areas.
A special case of cameras is the concept of line scanners. These cameras are
usually equipped with multiple one-dimensional CCD/CMOS arrays arranged
perpendicular to the platform’s heading direction and pointing in nadir, forward
and backward views (Figure 2.4). Through the platform’s motion, continuous
image stripes are acquired.
The configuration of lines pointing forward and backward allows the delin-
eation of 3D information ‘‘on-the-fly’’. Yet, in contrast to frame cameras, the
complete trajectory of the platform needs to be estimated, because each image
line has different position and attitude parameters, meaning that only an accu-
rate calculation of the exterior orientation of each image line at each acquisition
delivers geometrically consistent image stripes. In addition, to ensure complete
coverage of the scene, the so-called pushbroom constraint must be fulfilled; that
is, the (quasi-) ‘‘frame rate’’ of each line (projected into object space) must be
adapted to the velocity of the platform so that each part of the scene is imaged
at least once.
Chap-02 2021/4/28 page 14

14 Object and Pattern Recognition

Figure 2.5 Signal profile with multiple reflections.

2.1.2 Laser scanning

In contrast to the sensors mentioned above, active laserscanner systems allow a
direct and illumination-independent measurement of the distance, often called
the ‘‘range’’. By measuring the direction of the laser beam as well as the position
and attitude of the platform, one can ultimately obtain the 3D coordinate of each
recorded reflection (i.e. echo) of the pulse (Figure 2.5). Hence, laser scanning
systems have become very popular for remote sensing and scene analysis tasks,
especially when the geometric properties of objects play a major role. The
operative wavelength of lasers is located between 0.1 µm and 3 mm, but for
eye safety reasons wavelengths outside the visible spectrum are typically used.
One usually distinguishes between so-called pulsed laserscanner systems
and continuous-wave (cw) laserscanner systems. These rely on different mea-
surement principles, have complementary advantages and disadvantages, and
involve different sensor parameters.
Using an amplitude-modulated (AM) cw laser system, the range is measured
by exploiting the phase of a sinusoidal modulated signal. The phase difference
between the emitted and received signals indicates the distance between the
laser and the reflecting object. Due to the periodicity of the emitted signal,
the distance measurement is only unambiguous in the interval [0, 2π]. Increas-
ing the wavelength expands the maximum unambiguous range, at the cost,
however, of decreasing range resolution (i.e. measurement accuracy). The prob-
lem of expanding the unambiguous measurement range can be solved by using
multiple simultaneous offset sinusoidal modulation frequencies (multiple-tone
sinusoidal modulation). In this case, the maximum modulation wavelength
defines the unambiguity and the minimum modulation wavelength defines the
range resolution.
This technique makes cw laserscanners very attractive for terrestrial mea-
surements, because they achieve a better measurement accuracy of single points
(few millimetres) than pulsed laser systems, thereby keeping the footprint very
small (few millimetres at a distance of 10–30 m). Because of the small footprint
Chap-02 2021/4/28 page 15

Object, Data and Sensor Modelling 15

and the unique identification of the reflection point, it is sufficient to record only
one echo per emitted signal. On the other hand, cw laser systems require more
energy to operate over long ranges so that they are less attractive for airborne
platforms.
Conventional pulsed laserscanner systems for topographic mapping are
based on time-of-flight ranging techniques to determine the range to the reflect-
ing object. The time of flight is measured from the elapsed time between
the emitted and backscattered laser pulses, with each pulse being only a few
nanoseconds long. For objects such as vegetation showing more than one sin-
gle scatterer inside the laser beam, more than one backscattered pulse may be
detected per emitted pulse. Hence, for such objects, some sort of volume scat-
tering can be reconstructed. Most airborne laserscanner (ALS) systems are able
to capture, at a minimum, the range for the first and last detected backscattered
pulses. Some systems acquire ranges for multiple backscattered pulses, up to the
quasi continuous waveform of the echo per emitted pulse. Hence, these systems
are also known as full-waveform laserscanners. In contrast to earlier large foot-
print systems like the spaceborne Geoscience Laser Altimeter System (GLAS)
mounted on the Ice, Cloud and land Elevation Satellite (ICESat), small-footprint
ALS systems illuminate only a small patch of the scene within the footprint (e.g.
20 cm diameter), yielding waveforms with distinct return pulses corresponding
to specific surface characteristics.
This technique is better suited for long-distance measurements (more than
1 km), because less energy as for cw laser scanning systems is needed. At ranges
of 300 m to 1 km, pulsed systems reach an accuracy of single points measuring
5–15 cm with a footprint diameter of about 20 cm. Pulsed laserscanners are as
attractive remote-sensing measurement systems, because – depending on the
system and platform – they can reach a high point density, thus mitigating the
measurement noise by simple averaging, and, as stated above, they are able to
record the full waveform of the received signal.
An example of a signal profile applied to multiple pulses is depicted in Fig-
ure 2.5. The waveform parameters for each detected pulse of this signal profile
are estimated by a Gaussian decomposition method (the extracted features are
given in the table), and the estimated waveform is shown below the original
waveform. By comparing the range values in the table of Fig. 2.5, we see that
the distance between the first and second pulse is about 10 m and between the
third and fourth pulse about 2.5 m. The third pulse shows the highest ampli-
tude, and the pulse widths of the first and second pulses are slightly lower than
the pulse widths of the third and fourth pulses. It is not possible to classify the
type of surface illuminated within a single beam corridor using the amplitude,
pulse width and range properties alone. To assign each return pulse to a specific
surface type, additional information is required, which can result from the 3D
geometrical relationships of the returns within a point cloud.
Figure 2.6 visualises an example of a full-waveform data set of a test scene
captured by scanning along the azimuth and elevation and recording the return
Chap-02 2021/4/28 page 16

16 Object and Pattern Recognition

Figure 2.6 Vertical image slices with ground, vegetation and building structures.

intensity sampled over time t. When neglecting angular variations of the scan,
the measured intensities as a function of time t sampled over the azimuth and
elevation can be interpreted as a 3D data set forming a cuboid with Cartesian
coordinates x, y and t. Sampling along the time axis can be recalculated into
corresponding range values z. From this cuboid, various parameters of the echo
pulse are achieved by combining echo properties and the spatial relationship of
the returns. Figure 2.6 shows a set of image slices (yt planes). The second slice
from the left (x = 4) shows vegetation in the centre (near range) and building
structures on the right side (far range). Greyvalues correspond to the intensity
of the signal. Note that, although this way of displaying the data suggests that a
full 3D representation of the scene has been obtained, this is in fact not the case.
Just like point clouds measured by conventional laserscanners, the data cube
represents only 2.5D information. This is because of occlusion effects that are
dependent on the target size in relation to the beam footprint size. It is possible
that the laser pulse is mostly intercepted by and backscattered from the first
illuminated surface along the propagation direction and that following surfaces
along the laser vector are hidden, giving weak or no reflections. For instance,
a tree with dense foliage may return only a single reflection response per laser
pulse even though multiple surfaces are present after the first detected return
along the path of the laser vector.
Much more detailed approaches towards exploiting the data and sensor char-
acteristics of full-waveform laserscanners will be addressed in later chapters of
this book.

2.1.3 Synthetic Aperture Radar

Like laser scanning systems, a synthetic aperture radar (SAR) system is an active
sensor. With the success of the first environmental SAR satellites, ERS-1 and
ERS-2, SAR technology has become very popular in the remote sensing commu-
nity. In particular, because SAR is an active system and largely independent of
Chap-02 2021/4/28 page 17

Object, Data and Sensor Modelling 17

weather conditions, it is an attractive imaging technology for rapidly acquiring

area-wide information for remote regions prone to harsh weather conditions.
The first SAR satellites were only able to map areas with a geometric reso-
lution of 20–30 m, but, since 2007, a new class of space-borne SAR sensors
(e.g. TerraSAR-X, TanDEM-X, SAR-Lupe or Cosmo-SkyMed) provide images
with 1–3 m spatial resolution (or even below) in special spotlight modes. One
major focus of these missions is the improvement of mapping of rural and
urban areas. The development of methods to automatically derive detailed
cartographic information has undergone much development in recent years.
Furthermore, due to the fact that high-precision repeat-pass tracks and the
TanDEM-X mission provide high-resolution interferometric SAR data, meth-
ods for automatic 3D information extraction have come into play. These will be
addressed in Chapter 11 of this book.
In the following we give a short description of the image formation process
of SAR, as it involves the most important sensor characteristics mandatory to
consider for further image analysis tasks.
The radar emits frequency-modulated signals – so-called chirps – with a pre-
defined pulse repetition rate (PRF) in a side-looking, oblique imaging geometry,
and records the echoes scattered at the objects on the ground; see Figure 2.7
(upper left) for an illustration of the radar imaging geometry. The echoed chirps
are correlated with the sent (reference) chirp, eventually yielding a compressed
pulse-shaped signal whose width is mainly determined by the chirp’s bandwidth
(Figure 2.7, upper right). The travelling time of the signals is proportional to
the distance to the objects and defines the image dimension perpendicular to
the flight direction, the so-called range or across-track coordinates. The sec-
ond dimension, the azimuth or along-track, is simply aligned with the flight
direction.
While the resolution in the range direction is determined by the chirp band
width and is typically in the (sub-)metre area, the resolution in the azimuth
direction of the raw data depends on the antenna’s real aperture characteris-
tics (antenna length, carrier wavelength and range) and is impractically coarse
for geospatial applications (Fig. 2.7, lower left). Hence, to enhance the azimuth
resolution, the well-known SAR principle is applied; that is, the motion of the
real antenna is used to construct a very long synthetic antenna by exploiting
each point scatterer’s range history recorded during a point’s entire observation
period (Fig. 2.7, lower right), also forming a chirp-like signal (i.e. the so-called
azimuth chirp). An optimally focused image is thus obtained by complex-valued
correlation of the reference azimuth chirp and the recorded azimuth chirp.
To construct the reference azimuth chirp correctly, the actual range or phase
history of each target in the image must be known, which can be inferred
from sensor and scatterer position. Usually, the time dependence of the scat-
terer position is ignored. From an algorithmic point of view, this concept of
azimuth focusing of the SAR image is performed using sophisticated matched
Chap-02 2021/4/28 page 18

18 Object and Pattern Recognition

Figure 2.7 Radar imaging geometry (upper left) and compression of the chirp signal
sent in line of sight direction (also called slant range) into a pulse (upper right). Forming a
high resolution synthetic apertur in azimuth direction based on capturing low resolution
echos sent from the moving real aperture (lower left) and processing them in such a way
as if one long antenna had been used (lower right).

filter concepts (Bamler and Schättler, 1993; Cumming and Wong, 2005), also
commonly referred to as the stationary-world matched filter (SWMF) concept.
Since the length of the synthetic aperture increases proportionally with flying
height, the resolution in the azimuth direction purely depends on the length of
the physical antenna given a sufficiently large PRF to avoid aliasing. Figure 2.7
compares a real aperture and formation of a synthetic aperture for a typical
spaceborne SAR configuration, and Figure 2.8 shows examples of focused SAR
images.
The image formation process of SAR involves both the amplitudes and phases
of the recorded signal, so SAR belongs to the category of coherent imaging tech-
niques (like, e.g., tomography). Amplitude variations of SAR images thus follow
Chap-02 2021/4/28 page 19

Object, Data and Sensor Modelling 19

Figure 2.8 High-resolution SAR scene with 1 m geometric resolution (top) and
mountainous area of medium-resolution SAR (SRTM, bottom).

a Rayleigh distribution, and intensity values (i.e. the square of amplitude) follow
an exponential distribution, leading to the so-called fully developed (multiplica-
tive) speckle noise (Bamler and Schättler, 1993; Cumming and Wong, 2005). Yet
these relations hold only under the condition of an almost infinitely high num-
ber of scatterers in each resolution cell, which can be assumed for medium- and
low-resolution SAR but is rarely fulfilled for high-resolution SAR images.
It is important to note, that SAR is a 2D imaging sensor, although distances
to objects are measured. Yet, in contrast to laserscanners, it is not possible to
measure the pointing direction – here called elevation angle – of the radar beam
with sufficient resolution. Only by using multiple SAR images can the elevation
angle be determined with reasonable resolution. This leads to the typical SAR
characteristics that all scattered signals with the same distance to the sensor are
superimposed in the same resolution cell, also called ‘‘layover’’, leading to the
impression that steep or vertical objects lean towards the sensor (Figure 2.8,
bottom). The opposite effect, the radar shadow, is comparable to occlusions in
optical images and appears as scatterless resolution cells.
The above description of sensors and sensor models used in remote sensing
only touches on the most important measuring techniques and their typical
characteristics. In fact, many more remote sensing technologies exist to capture
Chap-02 2021/4/28 page 20

20 Object and Pattern Recognition

2D, 3D or nD information about objects. Further popular sensors include struc-

tured light sensors (such as Kinect) or time-of-flight video cameras. Depending
on the requirements of certain image analysis tasks, some issues concerning
these sensors will be tackled in later chapters.
Besides the sensor models, the complexity of the recorded scenes and of the
data to be treated is of essential importance when designing an image analysis
system. To enable successful and efficient object extraction, the model and
strategy must be adapted to this complexity. In the following, we will outline
these principles.

2.2 Data Content and Complexity

As far as images are concerned, the term complexity is closely related to the visual
impression of the individual viewer. Figure 2.9 shows some example subsets of
aerial images. Viewers naturally feel that the subsets of rural areas can be more
easily interpreted and are therefore less complex than those of urban areas. The
subjective impression of complexity makes its quantification – ideally with a
single measure – extremely difficult (Shufelt, 1999). However, in order to allow
at least a qualitative categorisation, a number of factors can be mentioned that
contribute to the complexity to a greater or lesser extent. First of all, the content
of the scene in the real world and the observability of the content, defined by
the sensor, are to be distinguished (Mayer, 1998).
Important factors that contribute to the complex content of a scene from the
real world (based on unit area) are as follows:

• Number of objects and object classes: The more individual objects from dif-
ferent classes (such as buildings, roads and water bodies) that are present
in a scene, the more comprehensive and variable is the scene.
• Intra-class variability (i.e. the variability within an object particular class):
This is basically defined by largely varying characteristics of objects
belonging to the same class. The class ‘‘vegetation’’, for example, has
a much higher intra-class variability than the class ‘‘road marking’’.
• Distinctiveness of object classes: Different object classes may be charac-
terised by pronounced differences in their characteristics (in terms of
different means, standard deviations, different texture features, etc.).
However, distributions between object classes may also be similar (e.g.
roads and flat roofs made of concrete), thus reducing inter-class variability
and complicating their separation.
• Relations between objects: The more different relations between objects
that occur, the more variable is the appearance of the objects.

Besides these, complexity is also affected by sensor-related factors. In the follow-

ing, specific characteristics are summarised (effects due to long time intervals
between image acquisitions are neglected):
Chap-02 2021/4/28 page 21

Object, Data and Sensor Modelling 21

Figure 2.9 Complexity of images depending on their resolution. Rows (left to right):
increasing complexity of the content of the scene. Columns (top to bottom): decreasing
observability (resolution: high, medium, low).

• Imaging geometry: This determines which part of the scene is recorded

under which kind of geometric projection. Aerial frame cameras pro-
duce images with central perspective projection in all viewing directions.
However, the images of CCD line sensors are characterised by a central
perspective projection only perpendicular to the flight direction, and show
a parallel projection along the flight direction.
• Image scale/ground sampling distance (GSD): The smaller the GSD, the
smaller the objects and details that are observable and the higher the
complexity. For images of man-made objects, a GSD larger than 1 m is
commonly defined as low, between 0.25 m and 1 m as medium, and less
than 0.25 m as high-resolution data (Mayer, 1998). The image scale is given
by the ratio of the focal length of the camera and the flying height, and
it may differ from pixel to pixel depending on the viewing direction and
terrain relief.
• Spectral resolution: The more channels are available, the more information
about the spectral properties of the objects is included in the image. One
typically divides images into pan-chromatic images (one band), multi-
spectral images (roughly up to a dozen of bands) and hyperspectral images
(up to several hundreds of bands).
Chap-02 2021/4/28 page 22

22 Object and Pattern Recognition

• Radiometric resolution: This specifies the dynamics of a sensor (i.e. how

sensitively the energy of the EMR can be recorded). While 8 bit (256 grey-
values) per channel is typically the usual quantisation, the radiometric
resolution of many newer sensors in remote sensing can already capture
EMR dynamics with 12 bit and more.
• Temporal resolution: Remote sensing data sets are acquired with different
repetition cycles. Although short videos can be also taken from space
meanwhile, a second image of the same point on Earth may be available
only after a couple of hours or days after the first, or may be acquired
years after the first3 . In both cases, temporal resolution may complicate
the analysis, either because the availability of numerous images inflates
the data volume, or because the lack of images may lead to there being
insufficient information (e.g. summer conditions in vegetation analyses).
On the other side, if the temporal behaviour of an object class can be
modelled – such as the phenomenology of vegetation – multi-temporal
data may significantly facilitate object extraction.

Primarily, a higher complexity of data complicates the process of image analysis.

The variety and variability of different objects and object classes, as well as the
relations among them, can easily lead to missing extractions or misinterpreta-
tions. A high degree of complexity, however, is not necessarily an obstacle to
object extraction. Often it is the details of an object and its relations to other
objects that make it identifiable. A typical classroom example for this is shown
in Figure 2.10. The sub-object ‘‘chimney’’, for example, gives a clear indica-
tion of a house roof or a building, and the relation ‘‘vehicle driving on a road’’
implies a hidden lane and thus a road. These examples prove that a powerful
image analysis system is characterised by an appropriate adaption of model and
strategy to the complexity of the scene and data.

2.3 Object Modelling

2.3.1 Introduction
Any object detection is based on a model of the object to be extracted. As
stated above, only by means of a model can the implicitly contained information
of an image or other remote sensing data be related to the semantics of the
corresponding subset of the scene in the real world. Therefore, the aim of
modelling must be to structure, formalise and represent the available knowledge
about the object in order to successfully reconstruct the semantics and the
properties of an object on the basis of the raw data.
In the remainder of this book we do not make a sharp distinction between
objects and object classes. A certain object such as a specific car in an image is
always understood as a typical representative of the object class ‘‘car’’. If truely
a specific instance of an object class is meant, we indicate this accordingly.
Chap-02 2021/4/28 page 23

Object, Data and Sensor Modelling 23

Figure 2.10 The context information of the left image allows the identification of
objects in the two image patches on the right.

There are different approaches to the components of modelling, which will

be described in more detail in the following subsections. Even if not every com-
ponent of a model is used for automatically detecting and characterising objects
in the data or even if a model is not fixed a priori but learned from sample data,
it is nonetheless helpful to (conceptually) set-up an object model on the level
of the real world and, by incorporating a sensor model, to transfer this into
an image model of the object (or more generally, a data model of the object).
Doing so has the advantage that the appearance of the object and its properties
in the image can be explicitly connected to symbolic descriptions and hierar-
chical structuring of objects – an approach that supports the understanding
of the performance and deficiencies of the implemented image analysis algo-
rithms. Moreover, especially when dealing with complex scenes, it is necessary
to integrate knowledge about the surrounding influences on the object and its
appearance in the model. This is realised through the context model.
The aim of modelling is to provide a description of an object, which is
structured, preferably consistent and formally representable in the computer.
One usually distinguishes an explicit and implicit representation of the model
knowledge. In the following sections, we describe the basic principles of these
modelling approaches starting with explicit modelling.
Chap-02 2021/4/28 page 24

24 Object and Pattern Recognition

2.3.2 Explicit Modelling

An explicit description of the model knowledge about objects and their relations
is advantageous because the results generated by the algorithm can be anal-
ysed more easily, and can therefore be directly used for predicting the system’s
behaviour for other data sets. It is, however, limited to those cases for which
the particular object classes, data and sensor characteristics can be modelled
properly.
The knowledge contained in the model can be structured, based on different
aspects that complement one another, as discussed in the following.

2.3.2.1 Structuring in Levels

A first categorisation is often done in the form of different levels. As has been
outlined before, structuring model knowledge can be achieved by dividing the
model into an object model, which corresponds to a description using concepts
of the real world, and an image model, which describes the appearance of the
object in image space. Transfer of the object model into image space is done
by the sensor model, which captures the geometric and physical transforma-
tions during the sensing process. The two main levels are thus the ‘‘real-world’’
level and the ‘‘image’’ or ‘‘data’’ level. At each level an object can be hierarchi-
cally subdivided into sub-objects by specialisation and part-of relations based
on specific criteria. At the image level, these criteria are used to relate image
features that can be extracted from the data to the sub-objects; that is, this level
is closely related to the capabilities and efficiency of image-processing algo-
rithms. In contrast, the function of the objects plays an important role in the
‘‘real world’’ level. For example, existing highways must have multiple lanes to
allow high-speed and safe overtaking, and must correspond in curvature and
slope to given construction guidelines. Such functional properties can be very
favourably used for structuring the object into its parts and for setting plausible
parameter values for geometrical and physical object properties.

2.3.2.2 Structuring According to Semantic Abstraction and Resolution

A model can be divided into different levels of resolution. In the image space,
this means that the image model is available at different scales (i.e. in discrete
realisations of the continuous scale–space). The most common scale-space is the
linear or Gaussian scale-space, where lower-resolution levels of an nD original
signal are produced by convolution with the Gaussian smoothing operator:
1 −x
x
g(x, t) = · e 2t (2.1)
(2π t)n/2
in which x is 2D (n = 2) for conventional images and the smoothing or
scale-space parameter t represents variance σ 2 . After smoothing, resampling
according to the Nyquist theorem is usually carried out, so the lower resolution
of the images is expressed by a larger GSD.
Chap-02 2021/4/28 page 25

Object, Data and Sensor Modelling 25

Figure 2.11 Levels of abstraction formulated as a semantic network (left) and the
corresponding scale-space (right). With increasing smoothing σ of the image, details
such as markings and cross-walks, and later larger structures such as vehicles and the
constructional separation of the lanes, are eliminated. The results of these events can be
assigned to different levels of abstraction in the real world.

As shown in Figure 2.11, a scale-dependent modelling at the level of the real

world corresponds to a subdivision of the object model into several levels of
abstraction. This kind of structuring emphasises the generalisation of knowl-
edge at higher levels of abstraction and underlines the different characteristics
of the object at different scales. The advantages of this kind of modelling were
particularly pointed out in classical works on semantic object extraction such
as Mayer (1998) and Mayer and Steger (1998).
Generally speaking, this indicates that an extraction solely using images of the
best resolution does not necessarily lead to the best results. Each object property
has a certain scale range in which it is particularly observable. Thus scale-
based modelling not only contributes to a better structuring and efficient use of
knowledge, but is also an important basis for successful extraction, especially
when the results from different scales are combined into a scale-dependent
extraction framework.
Chap-02 2021/4/28 page 26

26 Object and Pattern Recognition

The combination of certain abstraction levels with their corresponding scales

in the image domain is usually done in advance by fixing a certain discrete num-
ber of scales. Some research activities are also directed towards an automated
analysis of so-called scale-space events, be it numerically or analytically (Mayer
and Steger, 1998). In the case of complex topographic objects, automatic scale
estimation, which is necessary in this context, only succeeds exemplarily. How-
ever, for the extraction of simple image features such as lines and edges it can
be formulated more generally (Lindeberg, 1998).

2.3.2.3 Structuring by Context

In natural environments, an object is often influenced by certain neighbouring
objects or its entire surroundings. It is therefore not sufficient to model the
object as an isolated unit. Roads, for example, often appear in aerial images
as a bright grey surface. In the case of shadows cast by adjacent buildings or
trees, the bright surface becomes dark and therefore it no longer corresponds
to the assumptions of the model. Knowledge about such influences can be
formulated by a so-called context model. That means that the object model
is extended by relations to other objects, or, in the case of a more compre-
hensive modelling, it is embedded in a model for the complete scene. Taking
into account the original concept of Strat (1992), context in the concept of
explicit modelling describes knowledge about the environment of the object to
be extracted and the resulting impact on its shape and appearance. The inter-
action of object and environment is particularly evident when due to impacts
from the immediate neighborhood. However, because dependencies on the
environment occur also when taking into account large-scale object properties
and object relations, analogous to Mayer (1998) and Baumgartner (2003) a sub-
division into local and global context is made. For detailed context modelling
it must be pointed out that interactions of object relations could also appear.
For example, a car passing through the shadow of a building practically does
not cast an additional shadow on the road. The shadow relation between the
road and the vehicle is thus erased by the shadow relation between the road
and the building.
The objective for model global context knowledge is related to the obser-
vation that, depending on the particular large-scale environment, specific
properties and local contexts of the model are particularly striking and oth-
ers become less important. In open and rural areas, for example, there is, from
an economic point of view, usually only one optimal connection between two
points on the road network, while in urban areas, due to dense building devel-
opment, several nearly equivalent alternatives often exist (van Cleynenbreugel
et al., 1990). Model properties, arising from the connecting function of roads,
therefore have a much higher relevance in open and rural areas than in urban
environments. The same applies to the importance of various local contexts.
By including such global context areas – often ‘‘settlement’’, ‘‘open landscape’’
Chap-02 2021/4/28 page 27

Object, Data and Sensor Modelling 27

and ‘‘forest’’ – into the model, certain components of a complex model can
be extracted more effectively and certain model parameters can be adapted
to the particular context area (van Cleynenbreugel et al., 1990). This leads
to a self-adaptive extraction system with a specially adapted model and spe-
cially developed strategy for each global context area, so that even very large
scenes can be handled by effective models and efficient strategies. Summaris-
ing, local context describes the interactions of an object with objects in the
immediate neighborhood. In contrast, the global context describes the large-
scale specification of object classes, and therefore enables focusing on the typical
characteristics, appearances and local contexts within the particular global con-
text. This kind of context modelling profoundly supports the flexibility of the
extraction strategy as, depending on different local and global contexts, the
extraction strategy can provide several alternatives and thus allows dynamic
control (regulation) of the extraction process.

2.3.2.4 Representation and Formalisation of Model Knowledge

An essential point for explicit object modelling in its classical sense is the rep-
resentation and formalisation of the knowledge contained in the model. The
form of knowledge representation should be chosen in order to fairly repre-
sent the model knowledge on the computer without any restrictions in terms of
size and structure. At the same time, however, it should ensure a high degree
of clarity for a detailed examination and testing of the image analysis system.
In Brachman (1979) five levels of knowledge are presented: linguistic, concep-
tual, epistemological, logic and implementation levels. The linguistic level is
based on natural language representation and is considered inappropriate for
object extraction due to the absence of a clear formalisation of knowledge rep-
resentation. The logic and implementation levels, however, are subject to very
strong constraints. They can easily lead to complex and sometimes contra-
dictory (intermediate) interpretations in a more comprehensive image analysis
system, as shown, for example in McKeown et al. (1989). The epistemologi-
cal level allows an application-independent representation of knowledge and
therefore is suitable for a comparative analysis of different approaches, as in
Mayer (1998), while the conceptual level – relating to object extraction – has
the advantage that it allows an object-centred and thus a problem-oriented rep-
resentation, but at the same time enforces the need for a clear formalisation of
the knowledge representation. The conceptual level is often used for explicit
modelling in early and pioneering object extraction systems; the specific repre-
sentation formalism, which is particularly suitable and frequently incorporated
in these systems, is a so-called semantic network (Brachman, 1977, 1979; Liedke
et al., 2001; Sagerer, 1982; Woods, 1975). In addition, semantic networks show
the advantage that they can be easily formulated as Bayesian networks – or
more generally as so-called graphical models – so the relations between parts
of objects and their appearances in the image can be attached with conditional
probabilities among themselves. In this way, unavoidable uncertainties during
Chap-02 2021/4/28 page 28

28 Object and Pattern Recognition

the extraction process (in assigning image features to specific object classes)
can be modelled and handled adequately. Besides classical Bayesian theory, a
variety of mathematical and stochastic frameworks could be applied, such as
Dempster–Shafer, evidence or fuzzy theory (Shafer and Pearl, 1990; Zadeh,
1989). However, if using these theories within the conceptual framework of an
extraction system, it is particularly important to ensure that both modelling and
extraction strategy do not violate the underlying assumptions of these theories.

2.3.3 Implicit Modelling

For many applications it is sufficient, easier or even better not to perform an
explicit transition from the object model into the image space, but to only
utilise an image model. For such approaches, implicit or appearance-based
modelling comes into play. Like in the case of explicit modelling, the utilisation
of different resolution and context plays an important role. The main difference
from the concepts described above concerns the direct use of image patches
and derived features and parameters therefrom as a model of the objects and
relations.
The easiest case of implicit modelling uses a template image of the object and
matches this template to the search image with any kind of similarity metric
(e.g. greyvalue differences, cross-correlation, etc.). These techniques assume
that one single instance of an object represents the whole object class properly.
Partial occlusions, small deformations, clutter, illumination differences between
template and search image, and so on are supposed to be accommodated by the
application of sophisticated matching techniques being invariant against these
influences (e.g. rotation and scale-invariant matching, feature-based matching,
implicit shape matching, deformable matching, etc.).
While such matching techniques test the similarity of the template and search
image by applying one or more thresholds of the technique’s specific met-
ric(s), stochastic appearance-based approaches rely on a statistical model of
the appearance of an object (and occasionally the object’s relations). As in any
kind of supervised classification scheme, the model is learned from extracted
features of sample data and represented in a multi-dimensional feature space.
An unknown image patch is eventually tested against the model by evaluating
some sort of distance – in feature space – between the model and the fea-
tures extracted from the unknown patch. The model may be parameterised
in feature space with analytical probability density functions (e.g. for Gaussian
normal distributions) or be represented by discrete data as in the case of support
vector machines (SVMs). A further distinction between different classification
techniques can be made on the type of modelling. Generative classification
approaches learn models of object classes and test whether an unknown image
patch belongs to a certain class (based on a given distance metric), whereas dis-
criminative classification approaches model only boundaries between different
classes, thus not taking the shape or distribution of an object class in feature
Chap-02 2021/4/28 page 29

Object, Data and Sensor Modelling 29

space into account, and test on which side of the boundary the features of
an unknown image patch eventually lie. The following chapters in Part I of this
book give a more detailed introduction into feature extraction and classification
techniques.
Even more sophisticated appearance-based approaches include the stochastic
modelling of local contexts (e.g. Markov random fields (MRFs) and conditional
random fields (CRFs)), the distribution of relative distances and orientations
between objects (e.g. component-based matching) and the learning, selection
and fusion of the best suitable features (e.g. CNNs). It needs to be noted,
however, that a higher complexity of the model to be learned usually calls for
significantly more training data to learn the model and the involved coefficients
and parameters to be sufficiently precise. Thus, a trade-off between the avail-
ability of adequate sample data and the mandatory model complexity must be
observed in any case when designing an appearance-based extraction system.
Later chapters of this book give an insight into this issue.

2.4 Extraction Strategy

In the context of object extraction, the extraction strategy is defined as ‘‘a plan
for the recognition of objects in images by matching data to a model using var-
ious methods’’ (Mayer, 1998). This statement reveals that the strategy consists
of two basic, interconnected elements: on the one hand, the development of the
extraction process, and on the other hand, the choice of the particular image
and data processing methods for its realisation. Mostly, the choice of meth-
ods highly depends on the application, so the methods relevant for a particular
object extraction system will be described in the application-oriented chapters.
In particular, the planning of the extraction process controls at what stage which
kind of knowledge about the object is used for extraction and how the results of
intermediate extractions are integrated in the further procedure. In this context,
one usually distinguishes data-driven, model-driven and hybrid/mixed meth-
ods. In Figure 2.12, the basic steps of data-driven and model-driven approaches
are compared. It has to be noted that this is an idealised scheme, as is the
following description. For practical applications, only the use of the hybrid
approach is realistic. Differences exist, however, in the weighting of data-driven
and model-driven parts.

2.4.1 Data-driven Approach

The data-driven approach starts with purely iconic pre-processing steps, where
the input and output of processing are of the same data type (image). In the next
step, features from the images are extracted and combined to form superior
structures (cf. Chapter 3). This process is continued in 2D and 3D until struc-
tures have been built that sufficiently meet the requirements of the model, and
thus enable an unambiguous assignment to the model. To apply this procedure
successfully it is mandatory that, at each step, new information is gathered,
Chap-02 2021/4/28 page 30

30 Object and Pattern Recognition

Figure 2.12 Procedures of image analysis in an idealised form: data-driven (left) and
model-driven (right).

thereby constructing the complete object successively from the extracted parts.
An advantage of this procedure is that a generic object model is sufficient; that
is, there is no need for a pre-defined parametric model because the model com-
ponents are directly obtained from the data by feature extraction and feature
grouping methods.

2.4.2 Model-driven Approach

Within a purely model-driven approach, a (3D) object model is projected into
the image or data domain via the sensor model, basically at every location and for
each particular view onto the object. At each projection step, a hypothesis for the
image-like appearance of the object is generated. The fitting of a hypothesis for a
particular view and a particular location in the image is evaluated by specialised
similarity metrics – usually after decomposing the object into individual object
parts. A necessary precondition for this procedure is that the model and its
appearance in the image can be described very specifically, so that a hypothesis
in the image space corresponds to the actual appearance of the object. The
advantage of the specific description is that, on the one hand, the search space
for assignments can be strongly limited, especially when external knowledge
about the scene is already available (e.g. a geospatial database) and, on the other
hand, effects like occlusions, small distortions, background clutter or partially
lacking data have less impact compared to the data-driven approach.

2.4.3 Hybrid Approach

A hybrid processing combines data-driven and model-driven aspects. In the
case of interpretation of complex remote sensing data this is the only feasible
approach. Decisive for the efficiency of a hybrid extraction system is the choice
Chap-02 2021/4/28 page 31

Object, Data and Sensor Modelling 31

Figure 2.13 Efficient methods for different levels of complexity of data and models
(according to Suetens et al., 1992; Mayer, 1998).

of ‘‘turning points’’ – when to move from data-driven processing to model-

driven processing and vice versa. A typical approach is to start with data-driven
feature extraction in order to obtain reliable hypotheses for object parts that
can be easily and unambiguously extracted. These are evaluated for plausibility
in a model-driven manner and used to generate new hypotheses, which are
then verified or falsified by assigning newly extracted features. For example, the
boundary polygon of a flat roof is relatively easy to extract from aerial images.
Reliable roof hypotheses can thus be generated by (more or less) purely data-
driven processing. Then, with the help of roof hypotheses, the search area
of other object parts such as facades and shadow edges can be predicted and
restricted by model-driven hypotheses generation. Hence, on the one hand,
extracted object parts verify the roof hypothesis; on the other hand, they also
allow a more detailed reconstruction of the buildings (Kim et al., 2001). A major
advantage of the hybrid processing lies in the fact that additional knowledge
can be implemented by the appropriate choice of turning points, that is, the
(engineering type of ) knowledge about typical capabilities and limitations of
image processing methods concerning a certain task.

2.4.4 Suitability of the Extraction Strategy

The suitability of an extraction strategy can be categorised depending on the
complexity of the model and the used data, as follows (Figure 2.13). For simple
data and simple models, segmentation or classification approaches suffice for
the extraction of objects. This can be seen in the widely used and successfully
applied pixel-based multi-spectral classification methods, for instance, to clas-
sify satellite images of rural scenes. The assignment of models to radiometry –
thus a strong model-driven strategy – can be applied if the data are characterised
by high complexity, whereas the model itself can be described easily and specif-
ically or the search space can be strongly limited. This aspect plays an essential
role for the extraction of objects that can be easily parameterised in a CAD-like
manner, such as vehicles, for instance. In the case of lower data complexity
but higher model complexity, a strategy assigning geometric and radiometric
features to the model is suitable. As the content of the underlying data is quite
Random documents with unrelated
content Scribd suggests to you:
“Fräulein Harlberg,” he said with a certain suavity of manner, “I just
did myself the honour to call at the farm and found your father a
little concerned at your long absence. Knowing the danger of losing
one’s way in the forest I offered to go in search of you.”
“It was very good of you, Count,” the girl replied almost indifferently.
“But I was hardly in danger of being quite lost.”
The Count now turned his attention to Von Tressen, looking at him
with a peculiar wolfish smile, which was at the same time no smile
at all, but just the mask of one. “I see, Fräulein, you have already
found an efficient escort. You have been shooting in the forest, mein
Herr?”
“Unfortunately, yes.”
“Unfortunately?” The Count took up the word quickly, with a snap,
as it were, and glanced with a smile of protest at the girl.
“Most unfortunately,” the Lieutenant repeated. “I have unhappily hit
the Fräulein.”
Again Zarka echoed his words, drawing back his lips into an
expression of incredulity.
“It is nothing,” the girl said a little impatiently.
But it had occurred to Von Tressen that it would be just as well to
mention the accident. Zarka looked to him the man inevitably to find
it out; besides which it seemed due to the girl that their chance
acquaintanceship should be accounted for.
“The Fräulein is good enough to make light of it,” he said. “But it is
desirable that a doctor should see her hand without delay.
Therefore, perhaps, the Herr Graf will pardon me if I suggest that
we move on.”
The Herr Graf did not look exactly in a pardoning mood, although
the suave smile was still on his face. He wheeled his horse. “I will do
myself the pleasure of bearing you company to the Meierhof,” he
said in a tone which had in it less of a suggestion than a
determination. “Perhaps then I may be allowed to ride into
Kulhausen for a doctor. It will be quickest.”
They had moved on together, the Count walking his horse abreast of
them and in his insinuating way trying to draw out a circumstantial
account of the accident. At a turn in the forest road Von Tressen
said, “It is properly I, the culprit, who should go for the doctor. I
cannot allow you, Count, to take the trouble. I have a horse at my
camp and——”
As he spoke he felt a pressure on his arm. The girl had given him a
warning touch. Zarka signified by an indifferent bow that he
accepted Von Tressen’s suggestion. But his face grew a shade darker
as Fräulein Harlberg said:
“There is really no hurry. We can easily send from the farm. My
father will naturally think it right, Herr Lieutenant, that you should
come and make his acquaintance.”
The Count gave a tolerant smile, which probably served to mask
some darker expression, and the three went on together a short
half-mile to the house, Zarka chatting volubly and Von Tressen
wondering why the girl had so manifestly objected to his leaving
them.
 CHAPTER III
A MOMENTOUS MEETING

Gorla’s Meierhof, or Grange, was a picturesque house which had

been converted into a kind of shooting-box from a farmhouse,
which, in turn, had been adapted from the ruins of an ancient
building left centuries before by the Turks. It was a rough and
primitive abode, but one which in that wild country would be
considered comfortable enough and a not undesirable summer
mountain residence, situated as it was on the fringe of the vast hill
forests and commanding a view along the great sweep of the valley.
As the three approached the house they saw a man sitting before it
smoking and reading a newspaper. At the sound of their voices he
turned his head, then rose and sauntered to meet them. He was
small but well set-up, somewhat dandified even in the loose
lounging suit he wore; there was a good deal of the town man, Von
Tressen thought, in his appearance and manner, and, what struck
him forcibly, a decided military air in his carriage. This rather
surprised him, for had the other been a soldier he would surely at
his age have borne a high military title, whereas the Count had
distinctly alluded to him more than once as plain Herr Harlberg. But
that he had seen enough soldiering to have acquired a manifest
military bearing was to the Lieutenant’s mind a certainty.
“At last!” Harlberg exclaimed, a little peevishly Von Tressen thought.
“My dear Philippa, where have you been wandering?”
“Not so far, father,” she answered, with a laugh, and she introduced
Von Tressen, who had been the object of his rather suspicious
scrutiny.
The accident was related and the Lieutenant’s apologies accepted
not ungraciously; the Count, who had dismounted and led his horse
up the ascent to the house, standing in silence with his lips drawn
back in the inevitable smile. At length he spoke, and it was to the
purpose.
“The Herr Lieutenant has most kindly offered to ride into Kulhausen
for a doctor to see Fräulein Philippa’s hand. Dare one suggest that
the sooner it is professionally examined the better it will be?”
“There is no hurry; it hardly pains at all,” the girl protested.
For an instant the expression on the Count’s sharp face was not a
pretty one. But he replied merely by a shrug of mingled protest and
annoyance.
“Certainly. I am going at once,” Von Tressen said, watching the girl’s
face involuntarily for a sign. “I only came so far, sir,” he added to
Harlberg, “at the desire of the Fräulein, who was good enough to
express a wish to present me to you.”
“But how will you get to Kulhausen?” Harlberg asked, with what
seemed to the young man a rather too suggestive glance at the
Count’s horse.
Anyhow Zarka accepted it with some alacrity. “If the Lieutenant will
honour me by making use of my horse, it will be the quickest way,
and I shall be only too charmed.”
As he turned to the animal to bring him over, Von Tressen
instinctively glanced at the girl. She was biting her lip, and as their
eyes met she gave a little, almost imperceptible, shake of the head.
“The Herr Lieutenant,” she said, “tells us he also has a horse close
by. If he is kind enough to ride over to Kulhausen it would be
perhaps a pity to deprive the Count of his means of getting home.”
The Count, however, did not seem to look at the proposed
arrangement in that light. “I should be only too content and pleased
to wait,” he protested. “There, Herr Lieutenant——”
He brought the horse round for Von Tressen to mount. But the hint
had not been lost.
“I could not think of inconveniencing the Count,” he objected
resolutely. “And it is absurd when my own horse is so near.” He
made as though to move off. Zarka for a moment forgot his
somewhat oppressive politeness.
“It is waste of time, man!” he hissed rather than spoke. “Take the
horse; he will carry you well.”
But the other was resolved he would not be forced. He could not
quite guess the reason of the girl’s anxiety, but he did not like the
Count, and could understand that he might not be singular in his
antipathy.
“No, no! Not for the world!” he cried, backing off. “Herr Harlberg,
Fräulein, I have the honour. Auf Wiedersehen!” He turned and ran
off, divided between amusement at the Count’s furious disgust and
pleasure at the look of thanks in Fräulein Philippa’s eyes.
Zarka smoothed the strong muscles of his expressive face.
“An obstinate young Bursche,” he observed spitefully. “I hope the
dangers of our forest are not to be increased by these mad
marksmen.”
“It was entirely my own fault that he fired and hit me,” the girl said
emphatically, as though annoyed at his tone. “You, Count, or any
other sportsman, would have fired under the same provocation.”
The Count could smile again; he had evidently quite recovered his
equanimity. “Then I can only congratulate myself that I was not in a
position to inflict harm on you,” he returned. “You are not going,
Fräulein?” for, with a slight bow as disdaining further argument, she
had turned towards the house.
“Yes. I am tired with my long walk. I bid you good-evening, Count.”
And she left them.
The two men did not speak till she was out of earshot. Then
Harlberg remarked:
“It might have been an awkward contretemps, Count. As it is, I used
to know this young fellow’s father. He was a cavalryman.”
Zarka gave a shrug. “It is nothing. The Lieutenant is of no account
and an unsuspecting”—soldier, he was going to say, but substituted
—“young swaggerer. I shall keep my eye on him. I gather that he is
camping in the forest alone.”
“I hope he likes it,” Harlberg said wearily. “I find it dull enough.”
The Count laughed unsympathetically. “You miss the Königstrasse,
my friend. Patience! It will not be for long. The grass will soon have
grown over this excitement.”
“You have heard no news?”
“None. Except that the search is active. Naturally. A prince is a
prince even though he be a fool, and cannot be allowed to disappear
like a rag-picker. Well, good-evening, General——”
“Hush!” Harlberg held up a silencing hand.
“Oh, it is all safe here,” Zarka laughed in his masterful fashion.
“There is no one to overhear us. You may trust me not to make a
slip at the wrong time. I shall see you to-morrow, and, I hope,
Fräulein Philippa.”
So with a sweeping glance at the house he mounted and rode off.
Harlberg went in and, lighting a fresh cigar, took up a novel and
proceeded to make himself as comfortable as the place permitted.
He had scarcely settled himself in the easiest chair the room
afforded when his step-daughter came in.
“The Count has gone?” she asked.
“Yes,” he answered casually, glancing up from the book. “He talks of
coming again to-morrow. He is an agreeable fellow and will enliven
our exile. By the way, my dear girl,” he went on in a voice of languid
expostulation, “you must take care of yourself in the forest. How
foolish of you to play the water-rat. Lucky the fellow was a bad shot
and only hit your hand.”
“It was hardly a question of his being a bad shot,” the girl replied
indifferently. “He could see nothing to aim at except the movement
of my hand, and he hit that.”
“It is unfortunate.”
“No; the wound is absurdly slight.”
“I meant,” he said a little querulously, “the fact of the Lieutenant’s
breaking in upon our privacy.”
“I do not see,” she returned, “that we have anything to fear. I
thought you would be glad of company beyond our own.”
“Quite so. But under the circumstances, perhaps the fewer
acquaintances we make the better. We have always the Count.”
“Yes,” she repeated, “we have always the Count. Father,” she added
suddenly with a change of tone, “I do not care for Count Zarka’s
attentions.”
His look of surprise was rather obviously unreal. “Have they been
very marked?” he asked.
“No,” she answered drily, “because I have not given him the chance.
Only I think it well you should know I do not care to see very much
of Count Zarka.”
He threw out his hands deprecatingly. “Of course you know best, my
dear. Only,” he added, changing from a resigned to a persuasive
tone, “I should have imagined you would not have slighted the
chance of an alliance with a man of the Count’s wealth and position.”
“And character?” The sharp question made him feel uncomfortable.
“Do you know anything against his character?” he inquired blandly.
“Nothing definite,” she answered quietly. “But I am not a fool, and
Count Zarka’s personality does not seem to me to belie a certain evil
reputation which I believe he enjoys.”
“Philippa——” he began, but she cut him short.
“Apart from this, father, I do not like Count Zarka, and I think he
knows it. Anyhow, I have told you now so that there may be no
misunderstanding or cross purposes between us on the subject.”
Philippa spoke quietly, but with a slight tremor in her voice which
betrayed the feelings she repressed. She knew well how little
affection her step-father really had for her. A handsome, vivacious
girl, much admired wherever she appeared, her companionship was
far less irksome to her sole guardian than might have been the case
had she been plain and uninteresting. She knew all this, and
although she accepted it as the inevitable logic of her step-father’s
character, which was to have a real affection for no one outside his
own skin, yet she rebelled at the idea of being disposed of to suit his
convenience.
Harlberg spread out his hands in a gesture of protestation. “I have
nothing to do with it, my dear,” he said, almost petulantly. “You are
quite old enough to choose for yourself; and if our friend Count
Zarka wishes to marry you, why, he has a tongue in his head, and a
pretty glib one too.”
“I only wish you not to encourage him in that idea,” Philippa said.
“You may be sure I shall not,” he replied, taking up his novel again
with a suggestion that argument was fatiguing, and he did not feel
just then in the humour for it. The girl was far from sure, but,
realizing the uselessness of further discussion, she said no more.

Meanwhile Lieutenant Von Tressen had saddled his horse and ridden
post haste in search of the doctor. Having found the only practitioner
of which the little place boasted, and arranged for him to come out
to Gorla’s Farm without delay, he was starting back again, when he
saw on the other side of the street a face which he recognized.
“Galabin!” he shouted. “So it is, by Jupiter. Why, Horaz, my friend,
what on earth brings you here?”
The other man, on hearing his name called out, had glanced up
quickly with a look of mingled suspicion and annoyance. But on
recognizing Von Tressen his expression changed to a smile; he went
across and shook hands.
“What on earth are you doing in these outlandish parts?” the
Lieutenant repeated.
“Is it only in the military service that men take holidays?” Galabin
retorted.
“A holiday?”
“Why not, my friend? Do we spend our leave in town?”
“But here? Why, Horaz, you are never married?”
“And on my honeymoon? No, thank you. I have come for the
mountains and a little sport in the forest.”
“So? That is good to hear. I, too, am staying in the forest under
canvas for sport. You must join me. The deer-stalking will begin in a
few days. It will be glorious. You know Molvar of my regiment? He
has deserted me. We arranged the expedition together, and at the
last moment he cried off. Ah, well, he could not help it. If you are in
earnest you shall take his place. I can promise you fine sport.”
Galabin’s face had become thoughtful, almost business-like. “You are
camping in the forest?” he asked. “Anywhere near the Schloss
Rozsnyo?”
“At present I am within half an hour’s walk of it. By the way, do you
know Count Zarka?”
“Not I. Perhaps you do?”
“I met him just now for the first time.”
“An agreeable fellow, eh?”
“H’m! Yes, doubtless. Now, my dear Horaz, will you join me?”
“To-morrow? Yes, I shall be delighted.”
“Very well. I will come in the morning and fetch you and your traps.”
 CHAPTER IV
THE UNACCOUNTABLE

Next morning, as in duty bound, Von Tressen stopped on his way to

Kulhausen to inquire after Fräulein Harlberg’s injury. The surgeon
had pronounced it to be trifling, had extracted a shot and answered
for a speedy healing.
“So you see,” Philippa said to the Lieutenant, “you have nothing to
reproach yourself with.”
She had come out of the house to greet him, her father not being
visible.
“I have indeed,” he returned, “when I think how awful the result
might have been.”
“It was a curiously informal introduction,” she said laughingly.
“That is to me the only pleasant aspect of the affair. I feel inclined
never to fire a gun again.”
“You must not say that. You should have good sport to-day if my
wishes were of any avail.”
“I don’t deserve,” he said self-reproachfully, “that you should be so
forgiving.”
“A woman,” she replied—and as she spoke her eyes rested on him
with a sort of wistful trust—“can afford to overlook in a man slight
failings in consideration of qualities she respects.”
He coloured a little at the implied compliment.
“You are good to say so,” he murmured.
“Oh,” she said lightly, “it is nothing. You are a soldier; I am sure you
are brave and true and loyal, that you have a sense of duty. What is
a moment’s carelessness to set against that? There! Perhaps I have
said too much for the proprieties, but I can’t bear to see you
weighed down by unnecessary self-reproach. Now you must go and
shoot away with a clear conscience.”
Respecting her motive for frankness, he only gave her a grateful
bow.
“I am not going to shoot this morning,” he informed her. “I have
been lucky enough to find a companion.”
“Ah!” She turned quickly to him with a look of something more than
curiosity. “Here in these wilds?”
“Not exactly here,” he laughed. “But in Kulhausen last evening. An
old friend of mine. I am going now to fetch him over to my gipsy
camp.”
“A brother officer?”
“No, a civilian. His name is Horaz Galabin. He is one of the
secretaries in the Chancellor’s Bureau.”
He spoke quite carelessly, as though his friend’s identity were
scarcely a matter to interest his companion, and he was surprised to
notice a rather anxious look on her face.
“What in the world,” she asked—and he could not help thinking her
voice rather betrayed an unsuccessful attempt at indifference—“is a
secretary of the Chancellerie doing out in these uncivilized parts?”
“He comes for sport, he tells me.”
“Ah! And you both by lucky chance find a companion.” She had
regained her self-possession now. “Come! Here is another reason
why you need not regret that mistaken shot. If you had not ridden
into town for the doctor you would not have met your friend.”
A chance which he had been hoping for had presented itself, and he
seized it.
“If I had taken the Count’s horse,” he said with a reminiscent laugh,
“I should have got to Kulhausen sooner, and thereby should have
probably missed Galabin.”
“No doubt,” she agreed. “I am glad you did not take it.”
“Not for that reason alone, Fräulein?”
For a moment her eyes rested on him searchingly as though to
determine whether she might trust him. Evidently the result of the
scrutiny was favourable, for she answered:
“No; I did not want you to take the Count’s horse.”
“I gathered that,” he said with a smile; “and I have been puzzled for
a reason, which perhaps I have no right to seek.”
“The explanation is quite simple,” she replied, smiling now in her
turn. “I did not wish the Count detained here till you could bring his
horse back.”
“The Count,” he said, “did not seem to share your idea that it would
be inconvenient to him.”
“To him? Did I say so?”
“Ah, then to you. You do not like Count Zarka?”
Without looking at him she gave a little impatient shake of the head.
“Not very much.”
“I thought so yesterday.” The girl was silent. “Perhaps,” he added, “I
can guess why.”
“It is scarcely worth speculating about,” she said with a touch of
pride. “Ah, here is my father.”
Herr Harlberg had sauntered from the house, and now came
towards them with a not particularly gracious look on his
discontented face. Von Tressen paid his respects, explained the
object of his call, and expressed his relief at the doctor’s favourable
report. Then, accepting a hint from Harlberg’s manner that he had
stayed long enough, he took his leave.
“Tell me about this Count Zarka,” Galabin asked Von Tressen as they
drove out to the forest together. “A wonderful fellow, is he not?”
“My dear man, I know little more than you. He has at least a
wonderful house. Why are you so curious about the fellow?”
“Oh, I have heard of him in town,” the other answered carelessly.
“He is reported to have become rich in an astonishingly short time,
and no one can tell how.”
“He has the reputation of being a great gambler, and, what seldom
follows, a successful one.”
“So I have heard.”
“And do you not believe it?”
Galabin gave a shrug. “I have no grounds on which to form an
opinion. Yet I confess a man may well be sceptical. The gambler’s
trade hardly pays so prodigiously—at least when he plays fair.”
They soon reached Von Tressen’s encampment, and after luncheon
took their guns and strolled out.
“If it is all the same to you,” Galabin suggested, “suppose we shape
our course in the direction of the Schloss Rozsnyo. I am rather
curious to see the place.”
“I think, my friend,” Von Tressen returned slily, “you are very anxious
to see it, and are much interested in Count Zarka. Why, I do not
know. To me he is not an attractive person.”
Galabin gave a shrug. “I am a student of human nature, my dear
Osbert. This man is a curiosity. At least you will allow that. Most men
are negative characters. I love a positive, whether it be good or evil.”
“The positive characters in general are evil, are they not?”
“True. And I imagine our friend over there in particular. Still he will
be a study.”
“Mark!” Von Tressen’s gun rang out, and a black-cock fell twenty
yards in front of them. “But, my good Horaz,” he said as he
reloaded, “you did not come out here to study character. You came
for sport, did you not?”
Galabin pointed to the fallen bird. “There are more kinds of sport
than that, my friend,” he returned.
Von Tressen looked at him sharply, and, as the eyes of the two men
met, the light of a mutual understanding seemed to fill them. “Now,
Horaz,” he said with a laugh, “is it worth while wasting time by
playing at cross purposes? We are old comrades; you can trust me.”
“Yes,” he replied after a moment’s hesitation, “I can trust you. We
both serve the same master, and it is on his business that I am here.
Besides, you should be able to help me; there is no reason why we
should not work together.”
“State service? My dear Horaz, you may be sure I shall only be too
ready.”
“We diplomats,” Galabin observed cautiously, “have to be more than
ordinarily careful. Gersdorff would say it is a mistake to trust one’s
dearest friend. The very essence of our work is secrecy. Still,
confidence here is less dangerous than the risk of our playing at
cross purposes; for there, my friend, you might easily and quite
unwittingly spoil my game.”
“No doubt,” Von Tressen agreed. “Nevertheless, you must not take
me into your confidence against your better judgment.”
“I would not. Still I feel sure you can help me here if you will. It is a
business where pluck and nerve are likely to be needed. You will
give me your word of honour to be secret?”
“Certainly. You have my word of honour. Beyond that, if, as I
understand, it is a state affair, and no private business, I am already
bound by my oath of service.”
“Of course, my dear Osbert, I know you would be loyal to the
death.”
“Now what is this mysterious undertaking?”
“Principally to find out all about your friend at Rozsnyo.”
Von Tressen laughed. “As I had already guessed. He is suspect?”
Galabin nodded. “There is an idea that he is in the pay of Russia.
And, incidentally that he knows as much, or rather more, than
anybody else about the disappearance of Prince Roel of Rapsberg.”
“And your mission is to convert the conjecture into a certainty.”
“Precisely. Now, tell me what you know.”
He threw himself down on the natural bank which rose towards the
trunk of a great pine, and Von Tressen followed his example.
“Very little,” the Lieutenant replied. “And about Count Zarka, except
that I have made the man’s acquaintance, probably much less than
you. My meeting him was the result of a rather curious adventure
yesterday.”
“Tell me.”
Von Tressen thereupon related the story of his unlucky shot, and his
meeting with Fräulein Harlberg, telling everything with perfect
frankness. When he had come to an end Galabin remarked:
“So you were not favourably impressed with the Herr Graf, and the
young lady shares your repugnance. Harlberg? H’m! It is curious that
an elderly man should drag his daughter out to a lonely farmhouse
in these wilds for sport. You have come across him in your shooting
excursions?”
“No. But I have been here only a few days.”
“You have no corroborative evidence that he does shoot?”
Von Tressen laughed. “No. Nor can I say that he looks a keen
sportsman. One thing I did notice about him, though.”
“That he had the air of a military man?”
The Lieutenant stared. “Why, Horaz, how did you guess that?”
“You shall know before long. I have an idea. These people are
intimate with the Count, and mein Herr comes here for sport. The
Count, their friend, has a great house quite near. Why does he not
invite them to stay at Rozsnyo instead of allowing them to undergo
the discomforts of an old farmhouse?”
He seemed to be arguing the matter with himself rather than putting
the question to his companion.
Von Tressen shook his head. “It is very singular.”
“Not so very strange,” Galabin returned with a laugh. “If you are an
admirer of Fräulein Harlberg, my dear friend, I dare say there is no
reason why you should not continue your admiration.”
Von Tressen was silent for a few moments.
“Why are they here?” he asked.
“Ah! That is a question which perhaps no one could answer so
satisfactorily as Count Zarka. But if we have patience we may find it
out for ourselves. At present I can only hazard the merest guess.”
“And that is——?”
“That the Count may share your and Prince Roel’s admiration for the
young lady. Is it too improbable?”
Von Tressen shook his head resentfully. “No; I fear it is quite likely.”
“At least it supplies a motive. I fancy Rozsnyo is the central point of
a very pretty series of conspiracies, public and private, one within
the other, which it will be my task to unravel.”
“I shall only be too glad to help you,” Von Tressen declared heartily.
“Very well, then. We will make a reconnaissance of Rozsnyo to-
night.”
 CHAPTER V
THE MYSTERY OF ROZSNYO

At about nine o’clock that evening the two friends set out through
the forest for the Schloss Rozsnyo. They had been careful to utter no
hint of their intention which might be overheard by Bela, Von
Tressen’s soldier servant, and their nocturnal expedition was
ostensibly to the feeding grounds of the deer, which in those regions
are stalked at night.
A cloudy sky with occasional bright intervals suited their purpose
well, and the forest paths had become sufficiently familiar to Von
Tressen to enable him to guide his companion without difficulty
across the wooded valley to the elevation on which Rozsnyo stood.
As they drew near the castle the moon shone out brilliantly for a few
minutes, affording them, from their dark covert, a magnificent view
of the romantic building perched high above them. A curious edifice
blending, as it did, antique and modern styles of architecture, the
rough solidity of the ancient fortified dwelling with the fantastic
pretentiousness of the Gothic of yesterday. But the whole effect was
picturesque enough, especially as seen by moonlight.
“A fine lair for a beast of prey,” was Galabin’s comment. “What if the
inside should be as foul as the outside is fair?”
“Not an easy place to reconnoitre,” Von Tressen observed, having
taken in with a professional scrutiny the situation of the castle and
its points of approach.
“No,” Galabin replied; “the undertaking is by no means easy or safe,
as the Chancellor gave me clearly to understand. Still, I mean to go
through with it, although there is no reason why one need act with
precipitation.”
Keeping within the dark fringe of the wood, they began the ascent of
the hill and soon had reached the small plateau on which Rozsnyo
stood. As the castle came into view from the side approach the two
men could get a very good idea of its real size and form. It was a
large rambling structure, covering far more space than apparently its
real size and capacity would warrant. Indeed, the idea in its
construction seemed to have been distance; for one part, that is, to
lie as far from the other extremity as possible. From their point of
observation in the valley below, the two men had noticed a light in
one of the windows; here on the inner side all was dark: there
seemed no sign of life about the place.
The principal approach to the castle was by a great bridge of wood
and iron thrown over a moat drained of its water and planted with
flower-beds in curiously modern contrast to the grey massive walls
which rose from it.
All was still and silent save for the rustling of the pines as an
occasional gust swept through them. The two men emerged boldly
into the open which divided the surrounding wood from the castle
precincts.
“We are doing no harm in strolling round the place,” Galabin said.
“Let us make a tour of inspection and keep our eyes open.”
Keeping on the outer edge of what had been the moat, they
followed the circuit of the building until they came round again on
the farther side of the declivity of rock which dropped almost sheer
down to the valley. Nothing but a general idea of the castle
rewarded their scrutiny. All was dark and silent.
“Not a very promising place to investigate,” Von Tressen laughed.
“We had better go back again as we came. To try and get down into
the valley from this side looks like breaking our necks.”
“Yes,” Galabin assented grimly. “The vulture has well chosen his
eyrie. But for a great house there seems to be a singular absence of
life about the place.”
A heavy bank of clouds had now drifted over the moon, and the
darkness, intensified by the wall of pine woods, was so thick that the
two men had some difficulty in finding their way round the moat
again, at least without stumbling down the grass-grown bank.
Suddenly Von Tressen, who was in front, stopped, so abruptly that
Galabin cannoned against him.
“Look!”
A ray of light had shot out across the moat at a point some twenty
yards in front of them. It came from the castle, and was rendered
more vivid by the intense darkness elsewhere. The two diverged
from the path now until they came opposite to the spot whence the
light proceeded. It streamed from a window at some distance from
the ground in an otherwise blank wall which connected two Gothic
towers. There was nothing, of course, mysterious or even
remarkable in the appearance of the light, which was, indeed, rather
to be expected than the utter darkness in which all the back part of
the castle had been shrouded, yet somehow both men felt that there
was an element of mystery about it. For one thing, the room whence
it came was situated at about the most distant point from that other
lighted window they had seen from below, while all the intervening
block of buildings seemed dark and silent as a ruin.
“I should like,” said Galabin, “to take the liberty of looking inside that
room.”
“Impossible, so far as one can see,” Von Tressen replied. “There is
nothing but sheer wall. Still, we might creep across and examine it.”
Keeping well outside the band of light which stretched slantwise
across the moat, they descended into its hollow and crept up the
other side. So much of it, that is, as they found practicable. For the
angle made by the inner bank only rose about eight or ten feet from
the bottom and then continued in sheer, straight stone wall. The two
men were now directly underneath the window from which the light
passed high over their heads; but, although they listened intently
they could detect no sound from the room above them. To think of
climbing the wall was absurd. At that part, at any rate, the stones
were smoothly laid and faced, no hold for hands or feet was
possible. After a thorough examination both men agreed that it was
not feasible.
“If only some of yonder trees grew this side of the moat,” Von
Tressen observed.
“Ah!” An idea occurred to his friend. “Suppose we climb one as it is
and try what we can see. We shall at least get on a level with the
window.”
Von Tressen nodded, and they stealthily recrossed the dip and
regained the shelter of the wood.
“Let me go up and take an observation,” Von Tressen proposed. “I
used to be a good climber.”
Selecting the foremost tree opposite the window, the Lieutenant
with some little difficulty swarmed up the bare trunk. Galabin stood
below eagerly watching his progress, which became easier as he got
higher and the stem grew narrower. At length he calculated that Von
Tressen must be on a level with the window, and drew back to
measure the distance with his eye. As he did so, suddenly and
silently the light vanished. He turned quickly, only to see nothing but
a mass of black wall rising indistinct in the darkness. It was some
minutes before he heard his companion descending; perhaps he had
lingered in the hope that the light might re-appear, for assuredly
nothing was to be seen in its absence.
“Just too late,” Von Tressen exclaimed regretfully, as he reached the
ground. “It was provoking; but, never mind, we may have better
luck another night.”
“Yes; we will come to-morrow,” Galabin said. “Although, after all,
there may be nothing worth troubling to see. But I must confess
——”
He stopped as Von Tressen knocked his arm sharply. An
extraordinary thing had happened. For the moon coming out in its
full brightness showed the wall blank now, without a trace of the
window whence the light had shone.
At first neither man could realize it nor believe his eyes. Then, as
they became certain of the strange disappearance they looked at
each other in amazement.
“Where has the window gone to?” Von Tressen exclaimed, with a
short laugh.
Galabin shook his head. “Rather mysterious, is it not? Let us cross
over again and make sure that it is not an optical delusion.”
In a few seconds they were under the wall, looking in vain for a sign
of the window.
“It was directly above us here,” Von Tressen said. “I would swear to
that. I marked it by the triangular flower-bed. But where is it now?”
The dark wall above them presented an unbroken surface of stone.
The moon shone out for a moment and proved it clearly. There was
no window.
“Well,” observed Galabin, as they turned away from their puzzled
search, “at least we have found something mysteriously interesting
to follow up. Count Zarka is evidently a man worth the trouble of
watching. We will come again to-morrow night, and may have better
luck.”
So without seeing anything further to excite their curiosity they
returned through the forest to their camping-ground.
 CHAPTER VI
ZARKA PLAYS TERRORIST

Next day, as the two friends were preparing for a morning’s sport,
they were surprised by a visit from no less a personage than Count
Zarka himself. He came in, all smiles, to invite them to Rozsnyo.
“I really cannot allow you to live here like gipsies,” he said, “when I
have a great, almost empty house, within a stone’s throw. You must
be my guests while you stay in the forest.”
His manner was polished and civil to a degree which with some
people would have seemed charming, yet somehow to the two men
it was in the matter of sincerity absolutely unconvincing. The
invitation was declined as gracefully as possible, but without
hesitation. They liked the free life of the forest, Von Tressen said, the
novel change in their mode of existence. Moreover, they were not
prepared for visiting, and would feel uncomfortable in a big house.
Zarka forebore to press the invitation beyond the slight combating of
their excuses which the appearance of sincerity demanded.
“At least,” he said with his somewhat sinister smile, “you must let me
make you free of that part of the forest which is my preserve. I can
promise you good sport there.”
They thanked him and could not well refuse.
“Now,” he continued, “if you will not stay at my somewhat
formidable house, you will at least not refuse to come and see it.
You have doubtless an hour to spare this afternoon. I have some
curiosities which may interest you, and the view from my Belvidere
is magnificent. I may expect you? Yes?”
Von Tressen glanced at Galabin, who, without hesitation, accepted
the invitation. Whereupon with a parting volley of polite remarks and
small-talk the Count wheeled his horse and with a flourish rode off.
“An interesting specimen of character,” Galabin observed as they
stood watching him down the forest road. “I wonder why he wanted
us to stay at Rozsnyo; that is, if he did want us, which I doubt.”
At that moment the Count turned in his saddle and, looking back,
saw they were watching him. He waved his riding whip. It was a
mere flourish to cover his action of curiosity, and as such the two
men recognized it. Then he put his horse to a trot and was quickly
out of sight. The two looked at each other and laughed.
“I am glad, anyhow, he asked us up there,” Galabin said. “I want to
take every opportunity I can get of examining the place. And I have
a curiosity to see what our mysterious window looks like by
daylight.”
Count Zarka rode on to Gorla’s Farm and announced himself with,
for a ceremonious person, scant ceremony to Philippa Harlberg,
whom he found in the house. Perhaps he had an idea that a more
formal entry might result in his not seeing her.
“My father is smoking his cigar outside,” she said, as they shook
hands.
He returned a protesting smile.
“I did not come particularly to see the General. I came to see you.”
Her reception of the announcement was hardly encouraging, yet she
had to submit to the visit with as good a grace as possible.
“I have had news to-day from town,” he observed; then stopped,
watching her.
“Ah, yes?” There was repressed apprehension in her tone which he
was too clever to fail to notice.
“Prince Roel has not yet been found—dead or alive.”
“Poor fellow!” Her pity was genuine enough, yet there was
something behind it.
“The search,” Zarka continued, still eyeing her keenly, “is being
energetically carried on by his family as well as by the Government.
It is just as well that you did not stay in the city.”
“Yes.” She responded mechanically without conviction.
“A great friend of Prince Roel’s is reported to have set out for Paris.”
“Ah!” She looked at him enquiringly, yet unwilling to show how great
her curiosity was.
“Yes,” he proceeded with his evil smile. “Perhaps after all you may
have been very wise in changing your intention of going to Paris.”
“Perhaps,” she agreed in the same preoccupied tone. Then with a
flash her manner changed. “No. I was wrong to leave town. You
should know perfectly well, Count, that I was neither directly nor
indirectly the cause of Prince Roel’s disappearance.”
She spoke vehemently, as though lashed by the man’s insinuations
into taking a stand against him. He merely smiled, more inscrutably
than ever.
“Of course if you will tell me so I am bound to believe it,” he replied.
“Only, other people might not be so easily convinced.”
“And why not, pray?” she demanded, with a touch of haughtiness.
“The Prince was well known to be rather more than an admirer of
yours, Fräulein.”
“Absurd!” she burst out. “An admirer, perhaps, but nothing more,
and you have no warrant for supposing such a thing. Do men make
away with themselves for unreturned admiration? I am not to be at
the mercy of such a suggestion, Count.”
Behind the tolerant smile of a strong-willed man who holds, or thinks
he holds, a winning card, there was a look of intense, hardly
disguised admiration in Zarka’s eyes. The girl had at last roused
herself to face him; instead of mere avoidance she had sounded now
a bold note of open defiance. He realized that, perhaps he had
expected it, anyhow he was prepared to meet it.
He replied quietly, veiling the sentiment her outburst had called up—
“It is most unfair,” he said insinuatingly, “that you should be the
victim of an unfortunate suspicion; particularly hard that the crime of
which you stand accused is simply that of exciting in this man an
admiration which you were unable to return. My dear Fräulein, it
must often have been your fate—and will be—to commit that
offence, if it be one.” As she was not looking at him, he saved
himself the trouble of pointing his compliment with one of his
characteristic smiles. “But in this case,” he went on suavely—“you
will, I am sure, forgive my hinting at it—have not Prince Roel’s
friends perhaps something more to go upon than a mere
suggestion?”
She turned upon him sharply, and met the insinuating smile she so
detested.
“What do you mean, Count?”
He spread out his hands deprecatingly.
“I mean,” he continued in the same quiet voice, subdued because
the words themselves carried sting and point sufficient, “have they
not evidence of a pre-determination on the Prince’s part not to
survive your cruelty?”
“My cruelty!” she cried, and her face went white. “What evidence?”
“The evidence,” he answered quietly and unhesitatingly, “of the
roses.”
She was at a loss, that was plain. And the idea of a false underhand
accusation struck more fear to her than the certainty of her visitor’s
determined persecution.
“The roses?” she repeated.
“The red and the white,” he answered, with an almost mocking
seriousness. “The white signifying life, and the red, death. It was
perhaps a cruel choice to force upon you.”
“Choice?” she exclaimed in blank amazement. “I know of no choice.”
“Surely!” he insisted blandly. “And you wore the red roses at the
Margravine von Reuspach’s ball.”
“I wore——” she replied. “Yes, I remember wearing red roses which
Prince Roel sent me. I hesitated whether I should put them in my
dress, and only did so because I thought it would be ungracious to
refuse.”
Zarka raised his eyebrows in affected astonishment.
“Ungracious, Fräulein? To refuse to send your lover to his death?”
“Prince Roel was not my lover,” she retorted indignantly. “And how
could wearing his roses send him to his death?”
“The red ones, Fräulein,” he answered with suave insistence. “The
red were for death, the white for life. And you chose to wear the
red.”
The girl looked at him half in doubt, half in consternation.
“I know nothing of any white roses,” she replied steadily, although
her heart began to be full of a sickening fear of treachery, “nor of
any particular significance attached to my wearing red ones.”
The Count looked incredulous.
“Indeed! But Prince Roel is known to have sent roses of both kinds,
with a note intimating the significance to turn of which colour you
should choose to wear that night.”
She shook her head.
“I know nothing of this. All I received was a bunch of red roses.”
“And no note?”
“And no note. Perhaps, Count,” she went on, with a touch of scorn,
“as you know so much more of the affair than I, you will tell me the
words of the note.”
Zarka gave a slight bow of acceptance.
“A copy of the note addressed to you is in the hands of the
Chancellor, as having been found among Prince Roel’s papers,” he
replied. “I have, as you know, had to interest myself in the matter
from political reasons. So far as my memory serves me, the words
were these:
“‘I send you herewith two bunches of roses, white and red; the
white signify love and life; the red, hate and death. Those which you
will wear to-night must decide my fate. R.’ Those,” he added with a
smile, which seemed to deprecate further denial on her part, “were
the words. And you wore the red roses.”
She met his look and replied, unfalteringly—
“Count, I can only repeat I never got the note you speak of, nor the
white roses. You, who seem to know so much, should at least know
that.”
Her manner was one of defiance rather than defence or explanation,
and Zarka felt that intimidation here would hardly serve his purpose.
Accordingly he changed his tone.
“I am very glad to hear it,” he said sympathetically, “and more
especially for your own sake. But it seems to me that some hideous
mistake has been made, possibly by an enemy of the Prince’s, a
mistake which is likely to have cost him his life. It strikes, as I have
hinted, a particularly cruel blow at you, Fräulein. For the world will
hardly believe that you wore the red roses by accident, not design.
And—I do not wish to alarm you, but it is necessary to realize and
face the situation—the effect on the Prince’s family and friends must
be bitter enough to lead to danger to yourself.”
“Danger!” the girl echoed scornfully. “I am not afraid, knowing that I
have done nothing to deserve their ill-feeling. What do you mean by
danger, Count? It is, as you say, best to know how one stands.”
Zarka affected to hesitate, as shrinking from a truth which might
alarm her.
“These Eastern Huns,” he replied slowly and with an assumed
deliberation, “are a peculiar race, given to fits of ungovernable
passion, and actuated by a blind spirit of revenge for a wrong,
fancied or real. They are dangerous people to cross, hot-headed and
unreasoning, and there is no knowing to what length their
vindictiveness may carry them.”
“I understand your suggestion,” the girl said almost coldly. “Do these
people wreak their vengeance on women? I always thought they
were chivalrous.”
Zarka’s eyes were fixed on her like those of a snake, ever ready to
dart in the direction his prey might try to escape.
“You have never heard,” he replied, almost softly, “of the Blutrache,
the blood vengeance?”
“A kind of vendetta,” she replied, in a tone approaching indifference.
“Yes. But you will hardly expect me to stand in terror of that.”
“Ah!” he returned. “Then you know little of it.”
“If I knew everything I should not fear it.”
“Indeed, Fräulein?” His exclamation was an incredulous protest.
“No. For two reasons,” she went on. “In the first place, I am entirely
innocent of Prince Roel’s death, and in the second, even did it lie at
my door, I can hardly suppose that the most blood-thirsty of his
avengers would seek retribution against a woman.”
Zarka gave a shrug of doubt.
“Perhaps not. Although I have never heard that these people
allowed the sex of their wronger to stand in the way of their
vengeance. What I wish to say, Fräulein,” he continued with a
change of tone, “is, that I hope I shall be permitted to stand
between you and any danger which may exist. Let me assure you of
my devotion both to your safety and happiness as to yourself.”
He spoke earnestly, with a touch of repressed passion in his voice.
Before she could reply, to her great relief her father came in, and no
more on the subject could be said.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebooknice.com