Autonomous Mobile Robots

"Position"
Localization Cognition
Global Map

Environment Model Path

Local Map

Perception Real World Motion Control

Environment

Perception
Sensors
Vision
Uncertainties, Fusion
Features

Zürich Autonomous Systems Lab

 4a - Perception - Sensors
4a
2 Perception – Sensors
 4a - Sensors
 case studies
 classification, performance
 technology overview
 4b - Vision
 4c – Uncertainties, Fusion
 4d - Features
 Edges
 Histograms
 Fingerprints
 Lines
 Planes

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
3 Sensors for Mobile Robots

 Why should a robotics engineer know about sensors?

 They are the key components for perceiving the environment
 Understanding the physical principles enables appropriate use

 Understanding the physical principle behind sensors enables us:

 To properly select the sensors for a given application
 To properly model the sensor system, e.g. resolution, bandwidth,
uncertainties

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
4 Example of “Simple” Real World Situations
 A typical play in robocup

Curtesy of Manuela Veloso <[email protected] >

Carnegie Mellon University, [email protected]
 What the robot sees

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
5 Dealing with Real World Situations
 Reasoning about a situation

 Cognitive systems have to interpret situations based on uncertain and only

partially available information
 They need ways to learn functional and contextual information (semantics /
understanding)

Probabilistic Reasoning
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
6 Perception for Mobile Robots

Places / Situations
A specific room, a meeting situation, …
•Functional / Contextual
Servicing / Reasoning Relationships of Objects
• imposed
Objects • learned
Compressing Information

• spatial / temporal/semantic
Doors, Humans, Coke bottle, car , …

Interaction •Models / Semantics

• imposed
• learned
Features
Lines, Contours, Colors, Phonemes, …

Navigation •Models
• imposed
Raw Data • learned
Vision, Laser, Sound, Smell, …
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
8 The Challenge

 Perception and models are strongly linked

What is the difference

in brightness?

Courtesy E. Adelson
http://web.mit.edu/persci/people/adelson/checkershadow_downloads.html
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
4a
9

*
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
11
Case Studies

 Lets look at a couple of case studies before we begin

 What sensors are commonly employed on robots

 How has the choice of sensors evolved over time

• Shakey
• Tourguide robots late 90ies
• Willow Garage PR2
• SmartTer – the autonomous car

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
13 Shakey the Robot (1966-1972), SRI International
 Operating environment
 Indoors
 Engineered

 Sensors
 Wheel encoders
 Bumb detector
 Range finder
 Camera

C SRI International

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
14
Rhino Tourguide Robot (1995-1998), University of Bonn
 Operating environment
 Indoors (Museum: unstructured and dynamic)

 Sensors
 Wheel encoders
 Ring of sonar sensors
 Pan-tilt camera

C University of Bonn
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
16 PR2 (2010-),
 Operating environment
 Indoors and outdoors
 Onroad only

 Sensors
 Wheel encoders
 Bumper
 IR sensors
 Laser range finder
 3D nodding laser range finder
 Inertial measurement unit
 Pan-tilt stereo camera with
texture projector (active)
 Pressure sensor and
C Willow Garage

accelerometer inside hands

 ...

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

4a
18 The SmartTer Platform

 Three navigation SICK laser scanners

 Obstacle avoidance and local navigation
 Two rotating laser scanners (3D SICK)
 3D mapping of the environment
 Scene interpretation
 Omnidirectional camera
 Texture information for
the 3D terrain maps Motion Estimation / Localization
 Scene interpretation  Differential GPS system
 Monocular camera (Omnistar 8300HP)
 Scene interpretation  Inertial measurement unit
(Crossbow NAV420)
 Optical Gyro
 Odometry
(wheel speed, steering angel)
 Motion estimation
 Localization
Internal car state sensors
 Vehicle state flags (engine, door,
etc.)
 Engine data, gas pedal value
Camera for life video streaming
 Transmission range up to 2 km

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

4a
19 Autonomous Navigation and 3D Mapping

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 AMAR - Demonstration Scenario
 Interactive service tasks in a kitchen environment

Courtesy of Dillmann & Asfour,

Karlsruhe Institiute of Technology, Germany

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

Multimodal detection and tracking

Pedestrians Cars

Rich info
High prec
Inexpensive
Light indep
Noise
Low info
No distance
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
Video

Detection and tracking What the robot sees:

displayed on camera laser projected on
data image

Detection and tracking

displayed on laser data

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
23 Classification of Sensors
 What:
 Proprioceptive sensors
• measure values internally to the system (robot),
• e.g. motor speed, wheel load, heading of the robot, battery status
 Exteroceptive sensors
• information from the robots environment
• distances to objects, intensity of the ambient light, unique features.
 How:
 Passive sensors
• Measure energy coming from the environment
 Active sensors
• emit their proper energy and measure the reaction
• better performance, but some influence on environment

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
24 General Classification (1)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
25 General Classification (2)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
27 Characterizing Sensor Performance (2)
 Basic sensor response ratings (cont.)

 Range
• upper limit - lower limit

 Resolution
• minimum difference between two values
• usually: lower limit of dynamic range = resolution
• for digital sensors it is usually the A/D resolution.
• e.g. 5V / 255 (8 bit)

 Linearity
• variation of output signal as function of the input signal
• linearity is less important when signal is treated with a computer
x  f ( x)   x    y  f (  x    y)    f ( x)    f ( y)
y  f ( y)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
28 Characterizing Sensor Performance (3)
 Basic sensor response ratings (cont.)

 Bandwidth or Frequency
• the speed with which a sensor can provide a stream of readings
• usually there is an upper limit depending on the sensor and the sampling rate
• lower limit is also possible, e.g. acceleration sensor
• one has also to consider phase (delay) of the signal

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
29 In Situ Sensor Performance (1)
Characteristics that are especially relevant for real world environments

 Sensitivity
 ratio of output change to input change
dy
dx
 however, in real world environment, the sensor has very often high sensitivity to
other environmental changes, e.g. illumination

 Cross-sensitivity (and cross-talk)

 sensitivity to other environmental parameters (e.g. temperature, magnetic field)
 influence of other active sensors

 Error / Accuracy
 difference between the sensor’s output and the true value
error
m = measured value
v = true value

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
30 In Situ Sensor Performance (2)
Characteristics that are especially relevant for real world environments

 Systematic error -> deterministic errors

 caused by factors that can (in theory) be modeled -> prediction
 e.g. calibration of a laser sensor or of the distortion caused by the optics of a
camera

 Random error -> non-deterministic

 no prediction possible with given sensors
 however, they can be described probabilistically

 Precision
 reproducibility of sensor results:

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
31
Sensors: outline
 Optical encoders
 Heading sensors
 Compass
 Gyroscopes
 Accelerometer
 IMU
 GPS
 Range sensors
 Sonar
 Laser
 Structured light
 Vision (next lecture)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
32
Encoders
 Definition:
 electro-mechanical device that converts linear or angular position of a
shaft to an analog or digital signal, making it an linear/anglular transducer

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
33 Wheel / Motor Encoders
 Use cases
 measure position or speed of the wheels or steering
 integrate wheel movements to get an estimate of the position -> odometry
 optical encoders are proprioceptive sensors
 typical resolutions: 64 - 2048 increments per revolution.
 for high resolution: interpolation

 Working principle of optical encoders

 regular: counts the number of transitions but cannot tell the direction of motion
 quadrature: uses two sensors in quadrature-phase shift. The ordering of which wave
produces a rising edge first tells the direction of motion. Additionally, resolution is 4 times
bigger
 a single slot in the outer track generates a reference pulse per revolution

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
34 Wheel / Motor Encoders (2)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
35 Wheel / Motor Encoders (3)

scanning scale
reticle fields slits

scanning
reticle fields
scale
slits

2. Main Characteristics
• The four fields on the scanning
Notice what happens when the direction changes: reticle are shifted in phase relative
to each other by one quarter of the
grating period, which equals
360°/(number of lines)
• This configuration allows the
detection of a change in direction
• Easy to interface with a micro-
controller
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
36 Heading Sensors
 Definition:
 Heading sensors are sensors that determine the robot’s orientation and
inclination.

 Heading sensors can be proprioceptive (gyroscope, accelerometer) or

exteroceptive (compass, inclinometer).
 Allows, together with an appropriate velocity information, to integrate the
movement to a position estimate.
 This procedure is called deduced reckoning (ship navigation)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
37 Compass
 Used since before 2000 B.C.
 when Chinese suspended a piece of natural magnetite from a silk thread and used it to
guide a chariot over land.

 Magnetic field on earth

 absolute measure for orientation (even birds use it for migrations (2001 discovery))

 Large variety of solutions to measure magnetic/true north

 mechanical magnetic compass
 Gyrocompass
 direct measure of the magnetic field (Hall-effect, magneto-resistive sensors)

 Major drawback of magnetic solutions

 weakness of the earth field (30 μTesla)
 easily disturbed by magnetic objects or other sources
 bandwidth limitations (0.5 Hz) and susceptible to vibrations
 not suitable for indoor environments for absolute orientation
 useful indoor (only locally)
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
38 Gyroscope
 Definition:
 Heading sensors that preserve their orientation in relation to a fixed reference
frame
 They provide an absolute measure for the heading of a mobile system.
 Two categories, the mechanical and the optical gyroscopes
 Mechanical Gyroscopes
• Standard gyro (angle)
• Rate gyro (speed)
 Optical Gyroscopes
• Rate gyro (speed)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
39 Mechanical Gyroscopes
 Concept:
 Inertial properties of a fast spinning rotor
 Angular momentum associated with a spinning wheel keeps the axis of the gyroscope
inertially stable.
 No torque can be transmitted from the outer pivot to the wheel axis
 spinning axis will therefore be space-stable
 however friction in the axes bearings will introduce torque and so drift ->precession
 Quality: 0.1° in 6 hours (a high quality mech. gyro costs up to 100,000 $)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
40 Rate gyros
 Same basic arrangement shown as regular mechanical gyros

 But: gimbals are restrained by torsional springs

 enables to measure angular speeds instead of the orientation.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
41 Optical Gyroscopes
 Optical gyroscopes
 angular speed (heading) sensors
using two monochromic light (or laser) beams
from the same source.
 One is traveling in a fiber clockwise,
the other counterclockwise around a cylinder
 Laser beam traveling in direction opposite to
the rotation
 slightly shorter path
 phase shift of the two beams is proportional to
the angular velocity  of the cylinder
 In order to measure the phase shift, coil
consists of as much as 5Km optical fiber
 New solid-state optical gyroscopes based on
the same principle are build using
microfabrication technology.

Single axisSiegwart
© R. optical gyro 3-axis
& D. Scaramuzza, ETHoptical
Zurichgyro
- ASL
 4a - Perception - Sensors
4a
42 Mechanical Accelerometer
 Accelerometers measure all external
forces acting upon them, including
gravity
 accelerometer acts like a spring–
mass–damper system

Where m is the proof mass, c the

damping coefficient, k the spring
constant
 at steady-state:

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
43 Mechanical Accelerometer
 On the Earth's surface, the
accelerometer always indicates 1g
along the vertical axis
 To obtain the inertial acceleration (due
to motion alone), the gravity must be
subtracted. Conversely, the device's
output will be zero during free fall
 Bandwidth up to 50 KHz
 An accelerometer measures
acceleration only along a single axis.
By mounting three accelerometers
orthogonally to one another, a three-
axis accelerometer can be obtained

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
44 Other Accelerometers
 Modern accelerometers use Micro Electro-
Mechanical Systems (MEMS) consisting of
a spring-like structure with a proof mass.
Damping results from the residual gas
sealed in the device.

 In capacitive accelerometers the

capacitance between a fixed structure and
the proof mass is measured

 Piezoelectric accelerometers are based

on the property exhibited by certain crystals
to generate a voltage when a mechanical
stress is applied to them

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
45 Factsheet: MEMS Accelerometer (1)

seismic mass
1. Operational Principle
The primary transducer is a vibrating mass that
relates acceleration to displacement. The
secondary transducer (a capacitive divider)
converts the displacement of the seismic mass into
an electric signal.

capacitive 2. Main Characteristics

divider • Can be multi-directional
• Various sensing ranges from 1 to 50 g

3. Applications
• Dynamic acceleration
• Static acceleration (inclinometer)
• Airbag sensors (+- 35 g)
• Control of video games (Wii)

<http://www.mems.sandia.gov/>

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
46 Factsheet: MEMS Accelerometer (2)
capacitive
divider

spring
M a M a M

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
48 Factsheet: Piezoelectric Accelerometer
box
1. Operational Principle
Primary transducer is typically a single-degree-of-freedom
spring-mass system that relates acceleration to
spring displacement. Secondary transducer (piezoelectric discs)
converts displacement of the seismic mass into an
electrical signal (voltage).

mass M
2. Main Characteristics
• Piezoelectric elements cannot produce a signal under
constant acceleration (i.e., static) conditions
piezoelectric • 2-D and 3-D accelerometers can be created by
discs combining 2 or 3 1-D modules
u

3. Applications
• Vibration analysis
• Machine diagnostics
• Active vehicle suspension
• Autonomously guided vehicles
• Earthquake sensors
• Modal analysis
<http://www.pcb.com/>
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
49
Inertial Measurement Unit (IMU)
 Definition
 An inertial measurement unit (IMU) is a device that uses measurement systems such
as gyroscopes and accelerometers to estimate the relative position (x, y, z), orientation
(roll, pitch, yaw), velocity, and acceleration of a moving vehicle.

 In order to estimate motion, the gravity vector must be subtracted. Furthermore,

initial velocity has to be known.

 IMUs are extremely sensitive to measurement errors in gyroscopes and

accelerometers: drift in the gyroscope unavoidably undermines the estimation of
the vehicle orientation relative to gravity, which results in incorrect cancellation of
the gravity vector. Additionally observe that, because the accelerometer data is
integrated twice to obtain the position, any residual gravity vector results in a
quadratic error in position.

 After long period of operation, all IMUs drift. To cancel it, some external reference
like GPS or cameras has to be used.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
50 Ground-Based Active and Passive Beacons
 “Elegant” way to solve the localization problem in mobile robotics
 Beacons are signaling guiding devices with a precisely known position
 Beacon base navigation is used since the humans started to travel
 Natural beacons (landmarks) like stars, mountains or the sun
 Artificial beacons like lighthouses
 The recently introduced Global Positioning System (GPS) revolutionized modern
navigation technology
 Already one of the key sensors for outdoor mobile robotics
 For indoor robots GPS is not applicable,
 Major drawback with the use of beacons in indoor:
 Beacons require changes in the environment -> costly.
 Limit flexibility and adaptability to changing
environments.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
51 Global Positioning System (GPS) (1)
 Facts
 Recently it became accessible for commercial applications (1995)
 24+ satellites orbiting the earth every 12 hours at a
height of 20.190 km.
 4 satellites are located in each of 6 orbits with
60 degrees orientation between each other.

 Working Principle
 Location of any GPS receiver is determined through a time of
flight measurement (satellites send orbital location (ephemeris)
plus time; the receiver computes its location through
trilateration and time correction)

 Technical challenges:
 Time synchronization between the individual satellites and the GPS
receiver
 Real time update of the exact location of the satellites
 Precise measurement of the time of flight
 Interferences with other signals
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
52 Global Positioning System (GPS) (2)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
53 Global Positioning System (GPS) (3)
 Time synchronization:
 atomic clocks on each satellite
 monitoring them from different ground stations.
 Ultra-precision time synchronization is extremely important
 electromagnetic radiation propagates at light speed
 Light travels roughly 0.3 m per nanosecond
 position accuracy proportional to precision of time measurement
 Real time update of the exact location of the satellites:
 monitoring the satellites from a number of widely distributed ground stations
 master station analyses all the measurements and transmits the actual position to each of the
satellites
 Exact measurement of the time of flight
 the receiver correlates a pseudocode with the same code coming from the satellite
 The delay time for best correlation represents the time of flight.
 quartz clock on the GPS receivers are not very precise
 the range measurement with four satellite allows to identify the three values (x, y, z) for the
position and the clock correction ∆T
 Recent commercial GPS receiver devices allows position accuracies down to a couple
meters.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
55 Differential Global Positioning System (dGPS) (4)
 DGPS requires that a GPS receiver, known as the base station, be set up on a precisely
known location. The base station receiver calculates its position based on satellite signals
and compares this location to the known location. The difference is applied to the GPS data
recorded by the roving GPS receiver
 position accuracies in sub-meter to cm range

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
56
Range sensors
 Sonar

 Laser range finder

 Time of Flight Camera

 Structured light

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
57 Range Sensors (time of flight) (1)
 Large range distance measurement  thus called range sensors
 Range information:
 key element for localization and environment modeling
 Ultrasonic sensors as well as laser range sensors make use of
propagation speed of sound or electromagnetic waves respectively.
 The traveled distance of a sound or electromagnetic wave is given by

 d = distance traveled (usually round-trip)

 c = speed of wave propagation
 t = time of flight.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
58 Range Sensors (time of flight) (2)
 It is important to point out
 Propagation speed v of sound: 0.3 m/ms
 Propagation speed v of of electromagnetic signals: 0.3 m/ns,
 Electromagnetic signals travel one million times faster.
 3 meters
• Equivalent to 10 ms for an ultrasonic system
• Equivalent to only 10 ns for a laser range sensor
• Measuring time of flight with electromagnetic signals is not an easy task
• laser range sensors expensive and delicate

 The quality of time of flight range sensors mainly depends on:

 Inaccuracies in the time of fight measurement (laser range sensors)
 Opening angle of transmitted beam (especially ultrasonic range sensors)
 Interaction with the target (surface, specular reflections)
 Variation of propagation speed (sound)
 Speed of mobile robot and target (if not at stand still)

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
5
59 Factsheet: Ultrasonic Range Sensor

emitter
1. Operational Principle
An ultrasonic pulse is generated by a piezo-
electric emitter, reflected by an object in its path,
receiver and sensed by a piezo-electric receiver. Based
on the speed of sound in air and the elapsed time
from emission to reception, the distance between
v  t the sensor and the object is easily calculated.
d
2

2. Main Characteristics
• Precision influenced by angle to object (as
illustrated on the next slide)
• Useful in ranges from several cm to several
meters
• Typically relatively inexpensive

3. Applications
• Distance measurement (also for transparent
surfaces)
<http://www.robot-electronics.co.uk/
• Collision detection
shop/Ultrasonic_Rangers1999.htm>

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
60 Ultrasonic Sensor (time of flight, sound) (1)
 transmit a packet of (ultrasonic) pressure waves
 distance d of the echoing object can be calculated based on the
propagation speed of sound c and the time of flight t.

c t
d
2
 The speed of sound c (340 m/s) in air is given by

Where c    R T
: adiabatic index ( isentropic expansion factor) - ratio of specific heats of a gas
R: gas constant
T: temperature in degree Kelvin

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
62 Ultrasonic Sensor (time of flight, sound) (2)
 typical frequency: 40kHz - 180 kHz
 Lower frequencies correspond to longer maximal sensor range
 generation of sound wave via piezo transducer
 transmitter and receiver can be separated or not separated
 Range between 12 cm up to 5 m
 Resolution of ~ 2 cm
 Accuracy 98%  relative error 2%
 sound beam propagates in a cone (approx.)
 opening angles around 20 to 40 degrees 0° measurement cone
 regions of constant depth -30° 30°
 segments of an arc (sphere for 3D)

-60° 60°

Amplitude [dB]

Typical intensity distribution of a ultrasonic sensor

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
63 Ultrasonic Sensor (time of flight, sound) (3)
 Other problems for ultrasonic sensors
 soft surfaces that absorb most of the
sound energy
 surfaces that are fare from being
perpendicular to the direction of
the sound  specular reflections

a) 360° scan b) results from different geometric primitives

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
64
Ultrasonic Sensor (time of flight, sound) (4)
 Bandwidth
 measuring the distance to an object that is 3 m away will take such a
sensor 20 ms, limiting its operating speed to 50 Hz. But if the robot has a
ring of 20 ultrasonic sensors, each firing sequentially and measuring to
minimize interference between the sensors, then the ring’s cycle time becomes
0.4 seconds => frequency of each one sensor = 2.5 Hz.
 This update rate can have a measurable impact on the maximum speed
possible while still sensing and avoiding obstacles safely.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
65 Laser Range Sensor (time of flight, electromagnetic) (1)
 Laser range finder are also known as Lidar (LIght Detection And Ranging)

SICK

Alaska-IBEO

Hokuyo

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
66 Laser Range Sensor (time of flight, electromagnetic) (1)

D
Transmitter
P

L Target

Phase Transmitted Beam

Measurement Reflected Beam

 Transmitted and received beams coaxial

 Transmitter illuminates a target with a collimated laser beam
 Receiver detects the time needed for round-trip
 A mechanical mechanism with a mirror sweeps
 2D or 3D measurement

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
67 Laser Range Sensor (time of flight, electromagnetic) (2)

 Operating Principles:
 Pulsed laser (today the standard)
• measurement of elapsed time directly
• resolving picoseconds
 Phase shift measurement to produce range estimation
• technically easier than the above method

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
68 Laser Range Sensor (time of flight, electromagnetic) (3)
 Phase-Shift Measurement
D
Transmitter
P

L Target

Phase Transmitted Beam

Measurement Reflected Beam

 c
D  L  2D  L   
2 f
Where:
c: is the speed of light; f the modulating frequency; D’ the distance covered by the
emitted light is.
 for f = 5 MHz (as in the A.T&T. sensor),  = 60 meters

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
69 Laser Range Sensor (time of flight, electromagnetic) (4)

 Distance D, between the beam splitter and the target


D 
4
 where
 : phase difference between transmitted and reflected beam
 Theoretically ambiguous range estimates
 since for example if  = 60 meters, a target at a range of 5 meters = target at 35
meters
Amplitude [V]

lambda

q Phase Transmitted Beam

Reflected Beam

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
70 Laser Range Sensor (time of flight, electromagnetic) (5)
 Uncertainty of the range (phase/time estimate) is inversely proportional to
the square of the received signal amplitude.
 Hence dark, distant objects will not produce such good range estimated as
closer brighter objects …

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
71 Laser Range Sensor (time of flight, electromagnetic)

 Typical range image of a 2D laser range sensor with a rotating mirror. The length of
the lines through the measurement points indicate the uncertainties.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
72
The SICK LMS 200 Laser Scanner
 Angular resolution 0.25 deg
 Depth resolution ranges between 10 and 15 mm and the typical accuracy
is 35 mm, over a range from 5 cm up to 20 m or more (up to 80 m),
depending on the reflectivity of the object being ranged.
 This device performs seventy five 180-degrees scans per second

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
75
3D Laser Range Finder (2)
 The Velodyne HDL-64E uses 64 laser emitters.
 Turn-rate up to 15 Hz
 The field of view is 360° in azimuth and 26.8° in elevation
 Angular resolution is 0.09° and 0.4° respectively
 Delivers over 1.3 million data points per second
 The distance accuracy is better than 2 cm and can measure depth up to 50 m
 This sensor was the primary means of terrain map construction and obstacle detection for all the top
DARPA 2007 Urban Challenge teams. However, the Velodyne iscurrently still much more expensive
than Sick laser range finders (SICK ~ 5000 Euros, Velodyne ~50,000 Euros!)

C Carnegie Mellon University

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
76
Europa Robot: Obstacle and Terrain Detection

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
77 3D Range Sensor (4): Time Of Flight (TOF) camera
 A Time-of-Flight camera (TOF camera, figure ) works similarly to a lidar with the advantage that the whole
3D scene is captured at the same time and that there are no moving parts. This device uses an
infrared lighting source to determine the distance for each pixel of a Photonic Mixer Device (PMD) sensor.

Swiss Ranger 3000

(produced by MESA)
© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL
 4a - Perception - Sensors
4a
78 Incremental Object Part Detection
 Range Camera
 3D information with high data rate (100 Hz)
 Compact and easy to manage
 High, non-uniform measurement noise
 High outlier rate at jump edges
 However very low resolution (174x144 pixels)
Range Camera SR-3000

C MESA Imaging AG

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
79 Triangulation Ranging

 Use of geometrical properties of the image to establish a distance measurement

 If a well defined light pattern (e.g. point, line) is projected onto the environment.
 reflected light is then captured by a photo-sensitive line or matrix (camera) sensor device
 simple triangulation allows to establish a distance.
 If size of a captured object is precisely known
 triangulation without light projecting

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
80 Laser Triangulation (1D)
D

Laser / Collimated beam

P

L Target

L
D f
x x Transmitted Beam
Lens Reflected Beam
f

Position-Sensitive Device (PSD)

or Linear Camera

L
 Principle of 1D laser triangulation: D  f
x

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
81 Structured Light (vision, 2D or 3D): Structured Light

b
a b
u

 Eliminate the correspondence problem by projecting structured light on the scene.

 Slits of light or emit collimated light (possibly laser) by means of a rotating mirror.
 Light perceived by camera
 Range to an illuminated point can then be determined from simple geometry.

© R. Siegwart & D. Scaramuzza, ETH Zurich - ASL

 4a - Perception - Sensors
4a
82 Structured Light (vision, 2 or 3D)
 Baseline length L:
 the smaller L is the more compact the sensor can be.
 the larger L is the better the range resolution is.
Note: for large L, the chance that an illuminated point is not visible to the
receiver increases.
 Focal length f:
 larger focal length f can provide
• either a larger field of view
• or an improved range resolution
 however, large focal length means a larger sensor head
D

Laser / Collimated beam

P
L Target
L
D f
x x Transmitted Beam
Lens Reflected Beam
f

Position-Sensitive Device (PSD)

or Linear Camera © R. Siegwart & D. Scaramuzza, ETH Zurich - ASL