Intelligent Vehicles - 2021

7.5 credits

Course Responsibility: Björn Åstrand

Laboratory exercises: Björn Åstrand

Contact: [email protected]

Subject: DT8020_VT20
 Course Evaluations and course ambassadors

• Digital course evaluation at the end of the

course
• Course ambassadors
– aims to strengthen student participation during the
course and in the course evaluation work.
– two to four students
– 3 meetings in total 2 hours + discussion with other
student’s
– Previous year's course evaluation
 Course Objectives
• Knowledge and understanding
– describe and compare different navigation systems for indoor
and outdoor navigation
– explain how basic GPS-based systems work
• Skills and abilities
– apply, appraise and explain fundamental models for dead-
reckoning and kinematic models and methods for combining
information from several sensors
– select and present a scientific article in the area of sensor fusion
• Judgement and approach
– assess and together with others discuss scientific articles in the
area of intelligent vehicles
 Course Examination
• 4 Exercises – Matlab exercises – Written report
(groups of two students). Hand in latest one
week after exercise!
– Self-Assign to an Exercise group in Blackboard!
• Oral presentation of selected paper. The paper
should have a focus on sensor fusion
techniques.
• Participate in ”paper discussion lectures”
- Participate in 5 of 7 occasions.
• Written exam – In the end of the period
 Course information
• Lectures
– course literature: Introduction to Autonomous Mobile
Robots
• Exercises (start w.5, next week)
– will be available at Blackboard
– 1-2 days preparation before exercise begins!
• Student presentation
• Group discussions (start this week, Wednesday)
– Papers and questions will be available at Blackboard.
Prepare answers before lecture start.
 Course information - Schedule
Activity/Book/Material Lecture / Discussion Exercise
w. 3 Course description Monday, 10-12
Overview
(Chapter 1 – 2)

Basic statistics I and II Wednesday, 10-12

(Compendium + Chapter 4.2)

GPS (two papers) Wednesday, 13-15

(Chapter 4.1.5.1 + Compendium + papers:
Lechner and Baumann 2000,
Adamchuk 2001)
w. 4 Kinematics and dead reckoning Monday, 09-12 Exercise #1 Supervision, GPS,
(Chapter 3, 5.1-5.2 + paper: Wang 1988) Group 1, Wednesday, 08-10
Group 2, Wednesday, 10-12
w. 5 Perception I Monday, 09-12
(Chapter 4 + paper: Klang and Baerveldt
1997)

w. 6 Perception II Monday, 09-12 Exercise #2 Supervision, Kinematics, dead reckoning, error

(Chapter 4 + paper: Cox 1991) propagation,
Group 1, Wednesday, 08-10
Group 2, Wednesday, 10-12
w.7 Localization and sensor fusion - Kalman Monday, 09-12
filter
(Chapter 5 + paper: Maybeck.)
w.8 Sensor fusion Monday, 09-12 Exercise #3 Supervision, Perception, [Cox 1991],
(Chapter 5 + paper: Arras et al.) Group 1, Wednesday, 08-10
Group 2, Wednesday, 10-12
w. 9 Planning, navigation and obstacle Monday, 09-12 Exercise #4 Supervision, Sensor fusion,
Avoidance (two papers) Kalman filter,
(Chapter 6 + papers: Borestein and Koren Group 1, Wednesday, 08-10
1991, Shoval et al. 2003) – Note: two papers this time! Group 2, Wednesday, 10-12
w. 10 Applications Group 1, Monday, 08-12 Exercise # Extra time
(Student presentations) Group 2, Tuesday, 08-12 Group 1, Wednesday, 08-10
Group 2, Wednesday, 10-12
 Exercise reports*
• Written as a short paper!
– Abstract
– Introduction
– Theory / Method
– Experiments / results
– Conclusions and further improvements
– References
• Note!
– All reports are written in groups of two
students! *For details se course description
 Course information
• Textbook:
“Introduction to
Autonomous Mobile
Robots”
Introduction
Introduction

Introduction of Mobility
Today’s trends
Locomotion Concepts: Principles Found in Nature

© R. Siewart, I. Nourbakhsh
 A typical mobile system
Laser (Bearing)

Laser (Bearing + Range)

Laser (Bearing + Range)
Micro controllers
Kinematic model
Control systems
Emergency stop

v
Yr

a
Wheels

Xr
H(r)
Control panels Yw

Encoder
Motors

Power (battery packs)

Xw

∆d (t ) = v(t ) cos(α (t ))T

v(t )sin (α (t ))T
∆θ (t ) =
L
© O. Bengtsson
 Sensors – How to use them
Encoder 1

Gyro 1
Compass
Gyro 2
GPS Micro controller - Reaction / Decision
How to use all these
Laser
sensor values

IR
Ultra sound
Encoder 2
Sensor fusion
© O. Bengtsson
 Mobile Int. Systems – Basic
questions
1. Where am I? (Self localization)
2. Where am I going? (Navigation)
3. How do I get there? (Decision making /
path planning)

4. How do I interact with the environment?

(Obstacle avoidance)
5. How do I interact with the user? (Human –
Machine interface)
© O. Bengtsson
The dream of fully autonomous
robots

R2D2

?
C3PO

MARVIN

© O. Bengtsson
Autonomous robots of today
• Stationary robots
- Assembly
- Sorting
- Packing
- Collaborative

• Mobile robots
- Service robots
- Guarding
- Cleaning
Huskvarna, Automover
- Hazardous env.

- Humanoid robots

Trilobite ZA1, Image from: Xiaomi S50

© O. Bengtsson http://www.electrolux.se
History of Mobile Robots

Not a complete but……

Automatic Guided Vehicles
Newest generation of
Automatic Guided
Vehicle of VOLVO used
to transport motor
blocks from on
assembly station to an
other. It is guided by an
electrical wire installed
in the floor but it is also
able to leave the wire to
avoid obstacles. There
are over 4000 AGV only
at VOLVO’s plants.

© R. Siewart, I. Nourbakhsh
Helpmate

• HELPMATE is a mobile robot used in hospitals

for transportation tasks. It has various on board
sensors for autonomous navigation in the
corridors. The main sensor for localization is a
camera looking to the ceiling. It can detect the
lamps on the ceiling as reference (landmark).
http://www.ntplx.net/~helpmate/
© R. Siewart, I. Nourbakhsh
BR700 Cleaning Robot
• BR 700 cleaning
robot developed and
sold by Kärcher Inc.,
Germany. Its
navigation system is
based on a very
sophisticated sonar
system and a gyro.
http://www.kaercher.
de

• link

© R. Siewart, I. Nourbakhsh
ROV Tiburon Underwater Robot

• Picture of robot ROV Tiburon for

underwater archaeology
(teleoperated)- used by MBARI for
deep-sea research, this UAV
provides autonomous hovering
capabilities for the human
operator.
© R. Siewart, I. Nourbakhsh
 The Pioneer

• Picture of Pioneer, the teleoperated

robot that is supposed to explore the
Sarcophagus at Chernobyl
© R. Siewart, I. Nourbakhsh
 Robots for Tube Inspection

• HÄCHER robots for sewage

tube inspection and reparation.
These systems are still fully
teleoperated.
http://www.haechler.ch

• EPFL / SEDIREP: Ventilation

inspection robot © R. Siewart, I. Nourbakhsh
Sojourner, First • The mobile robot
Sojourner was

Robot on Mars
used during the
Pathfinder mission
to explore the
mars in summer
1997. It was nearly
fully teleoperated
from earth.
However, some on
board sensors
allowed for
obstacle detection.
http://ranier.oact.h
q.nasa.gov/telerob
otics_page/telerob
otics.shtm

© R. Siewart, I. Nourbakhsh
Last contact 10 June 2018
 The Pioneer

• PIONEER 1 is a modular mobile robot offering various options like

a gripper or an on board camera. It is equipped with a
sophisticated navigation library developed at Stanford Research
Institute (SRI). http://www.activmedia.com/robots
© R. Siewart, I. Nourbakhsh
The B21 Robot

• B21 of Real World

Interface is a
sophisticated
mobile robot with up
to three Intel
Pentium processors
on board. It has all
different kinds of on
board sensors for
high performance
navigation tasks.
http://www.rwii.com

© R. Siewart, I. Nourbakhsh
 The Khepera Robot

• KHEPERA is a small mobile robot for research and education. It sizes only
about 60 mm in diameter. Additional modules with cameras, grippers and
much more are available. More then 700 units have already been sold (end
of 1998). http://diwww.epfl.ch/lami/robots/K-family/ K-Team.html
© R. Siewart, I. Nourbakhsh
Baxter

Mobile plattform
Research @ Halmstad
university
Agricultural robots
for plant scale operations

Mechanical
weed-control in
sugar beets
 Agricultural robots
for plant scale operations

Vision-based perception system for

– Guidance (row following).
– Discrimination between crops and
weeds (plant scale operations)
– Navigation
– Odometry

Björn Åstrand
AGV:s – Today!
Flexibility, Intelligent, Safety

Landmark
AGV:s – Today!
Flexibility, Intelligent, Safety

Standard EN1525 - Safety of industrial trucks

Robot de Neuro

© O. Bengtsson
Rocla

© O. Bengtsson
 MALTA
Multiple Autonomous forklifts for Loading and Transportation Applications
 Results
Identify and localize paper reels and load them into a container
 Results
Identify and localize paper reels and load them into a container
 Result
AGV decide by itself where to put the cargo

Replanning

Navigation without infrastructure

 Result
Navigation (localization and map building) without infrastructure
 Results
Obstacle avoidance and replanning
 Result
People tracking for safety
 Truck safety
A proposal for the safety standard of driverless trucks
The aim of this work is to propose an extension to existing
standard with new objects that are better suited for
validation and verification of non-contact (3D) safety
sensors.

Björn Åstrand
 AIMS
Automatic Inventory and Mapping of Stock
An intelligent warehouse
environment that
autonomously builds a map of
articles in a warehouse and
relates article identity from the
warehouse database with the
article’s position (metric) and
visual appearance in the
warehouse.
 AIMS
Planar mapping of infrastructure
 AIMS
Planar mapping of infrastructure
 AIMS
Environment Perception: Semantic Mapping,
Knowledge Transfer

Knowledge transfer

Semantic mapping of Hand-crafted features vs

infrastructure robot representation.
When to introduce
Map alignment
semantics?
SAS2 – Situation Aware Safety Systems

Research on the design of systems that, as autonomously

as possible, can construct knowledge from real life data
created through the interaction between a system and its
environment. This data necessarily includes streaming
data. Such systems should be able to handle events that
are unknown at the time of design.
SAS2 – Situation Aware Safety Systems
 Towards social acceptance of autonomous vehicles using
3D video data and road user behavior modeling

• Motivation: Autonomous vehicle (AV)

technology is rapidly growing and policy
makers need to take an action for
towards social acceptance of AVs.
• Goal: To analyze the social acceptance
of autonomous vehicles from the
perspective of safety and accessibility
using 2D video streams.
• Partners: Viscando, RISE-Viktoria,
Halmstad Municipality, HH
LiDAR-only Semantic Perception

mean
std
density
reflectivity
max
200x400
min

Region of Interest
Raw Velodyne
LiDAR-only Semantic Perception

mean
std
density
reflectivity
max
min
Detected Road
Region of Interest
Raw Velodyne
KITTI Dataset: Results I

SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving
Aksoy E.E., Baci S., Cavdar S.
IEEE Intelligent Vehicles Symposium, 2020
Legged Mobile Robots
 The beast

Legged robots

Soccer competition Quad-robot

Forester Robot

• Pulstech developed
the first ‘industrial like’
walking robot. It is
designed moving
wood out of the forest.
The leg coordination is
automated, but
navigation is still done
by the human operator
on the robot.
http://www.plustech.fi/

© R. Siewart, I. Nourbakhsh
 Sony: walking

Humanoid robots
Sony: pushing
Honda: P3

Sony: dancing

NAO
 Boston Dynamics
• Link Wildcat, Altas
 Locomotion

Wheeled Mobile Robots

 Locomotion Concepts
• Concepts found in nature
– difficult to imitate technically

• Most technical systems use wheels or

caterpillars

• Rolling is most efficient, but not found in nature

– Nature never invented the wheel !

• However, the movement of a walking biped

is close to rolling!
© R. Siewart, I. Nourbakhsh
Walking of a Biped

• Biped walking mechanism

– not to fare from real rolling.

– rolling of a polygon with side length

equal to the length of the step.

– the smaller the step gets, the more

the polygon tends to a circle (wheel).

• However, fully rotating joint was not

developed in nature.

© R. Siewart, I. Nourbakhsh
 Mobile Robots with Wheels
• Wheels are the most appropriate solution for most
applications

• Three wheels are sufficient and to guarantee stability

• With more than three wheels a flexible suspension is

required

• Selection of wheels depends on the application

© R. Siewart, I. Nourbakhsh
 The Four Basic Wheels Types
a) b)

• a) Standard wheel: Two

degrees of freedom; rotation
around the (motorized) wheel
axle and the contact point

• b) Castor wheel: Three

degrees of freedom; rotation
around the wheel axle, the
contact point and the castor
axle

© R. Siewart, I. Nourbakhsh
 The Four Basic Wheels Types
c) d)
• c) Swedish wheel: Three
degrees of freedom; rotation
around the (motorized) wheel
axle, around the rollers and swedish 90°
around the contact point

• d) Ball or spherical wheel:

Suspension technically not
solved s wedis h 45°

© R. Siewart, I. Nourbakhsh
 Characteristics of Wheeled Robots and
Vehicles
• Stability of a vehicle is be guaranteed with 3 wheels
– center of gravity is within the triangle with is formed by the ground
contact point of the wheels.
• Stability is improved by 4 and more wheel
– however, these arrangements are hyperstatic and require a flexible
suspension system.
• Bigger wheels allow to overcome higher obstacles
– but they require higher torque or reductions in the gear box.
• Most arrangements are non-holonomic (see chapter 3)
– require high control effort
• Combining actuation and steering on one wheel makes the design
complex and adds additional errors for odometry.

© R. Siewart, I. Nourbakhsh
 Different Arrangements of Wheels I
• Two wheels

• Three wheels

Synchro Drive
Omnidirectional Drive © R. Siewart, I. Nourbakhsh
Different Arrangements of Wheels II
• Four wheels

• Six wheels

© R. Siewart, I. Nourbakhsh
 Cye, a Two Wheel Differential Drive
Robot

• Cye, a commercially available domestic robot that can

vacuum and make deliveries in the home, is built by
Probotics, Inc. © R. Siewart, I. Nourbakhsh
 Synchro Drive
• All wheels are actuated
synchronously by one motor
– defines the speed of the vehicle
• All wheels steered
synchronously by a second
motor
– sets the heading of the vehicle

• The orientation in space of

the robot frame will always
remain the same
– It is therefore not possible to
control the orientation of the robot
frame.
© R. Siewart, I. Nourbakhsh
Tribolo, Omnidirectional Drive with 3
Spheric Wheels

© R. Siewart, I. Nourbakhsh
 Uranus, CMU: Omnidirectional
Drive with 4 Wheels
• Movement in the plane has 3
DOF
– thus only three wheels can be
independently controlled
– It might be better to arrange three
swedish wheels in a triangle

© R. Siewart, I. Nourbakhsh
 Stepping / Walking with Wheels
• SpaceCat, and
micro-rover for
Mars, developed by
Mecanex Sa and
EPFL for the
European Space
Agency (ESA)

© R. Siewart, I. Nourbakhsh
 SHRIMP, a Mobile Robot with
Excellent Climbing Abilities
• Objective
– Passive locomotion
concept for
rough terrain
• Results: The Shrimp
– 6 wheels
• one fixed wheel in the rear
• two boogies on each side
• one front wheel with spring suspension
– robot sizing around 60 cm in length and 20 cm in height
– highly stable in rough terrain
– overcomes obstacles up to 2 times its wheel diameter

© R. Siewart, I. Nourbakhsh