Full download test bank at ebook textbookfull.

com

Robotics Vision and Control

Fundamental Algorithms In MATLAB

CLICK LINK TO DOWLOAD

https://textbookfull.com/product/robotics-
vision-and-control-fundamental-algorithms-in-
matlab-peter-corke/

textbookfull
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Automatic Control, Robotics, and Information Processing

Piotr Kulczycki

https://textbookfull.com/product/automatic-control-robotics-and-
information-processing-piotr-kulczycki/

Modern Robotics Mechanics Planning and Control Frank C.

Park

https://textbookfull.com/product/modern-robotics-mechanics-
planning-and-control-frank-c-park/

MATLAB Computer Vision Toolbox User s Guide The

Mathworks

https://textbookfull.com/product/matlab-computer-vision-toolbox-
user-s-guide-the-mathworks/

Machine Vision Algorithms in Java Techniques and

Implementation Whelan Paul F Molloy Derek

https://textbookfull.com/product/machine-vision-algorithms-in-
java-techniques-and-implementation-whelan-paul-f-molloy-derek/
Computer vision: theory, algorithms, practicalities
Fifth Edition Davies

https://textbookfull.com/product/computer-vision-theory-
algorithms-practicalities-fifth-edition-davies/

Numerical Methods of Exploration Seismology With

Algorithms in Matlab r Gary F. Margrave

https://textbookfull.com/product/numerical-methods-of-
exploration-seismology-with-algorithms-in-matlab-r-gary-f-
margrave/

Nonlinear control systems using MATLAB First Edition

Boufadene

https://textbookfull.com/product/nonlinear-control-systems-using-
matlab-first-edition-boufadene/

Control Theory in Biomedical Engineering: Applications

in Physiology and Medical Robotics 1st Edition Olfa
Boubaker (Editor)

https://textbookfull.com/product/control-theory-in-biomedical-
engineering-applications-in-physiology-and-medical-robotics-1st-
edition-olfa-boubaker-editor/

Wind Energy Design First Edition Corke

https://textbookfull.com/product/wind-energy-design-first-
edition-corke/
 Peter Corke

Robotics,
Vision
and
Second Edition
Control
FUNDAMENTAL
ALGORITHMS
IN MATLAB®

123
Springer Tracts in Advanced Robotics
Volume 118
Editors: Bruno Siciliano · Oussama Khatib
Peter Corke

Robotics,
Vision and Control
Fundamental Algorithms in MATLAB®

Second, completely revised, extended and updated edition

With 492 Images

Additional material is provided at www.petercorke.com/RVC

Professor Bruno Siciliano
Dipartimento di Ingegneria Elettrica e Tecnologie dell’Informazione,
Università di Napoli Federico II, Via Claudio 21, 80125 Napoli, Italy,
e-mail: [email protected]

Professor Oussama Khatib

Artificial Intelligence Laboratory, Department of Computer Science,
Stanford University, Stanford, CA 94305-9010, USA,
e-mail: [email protected]

Author
Peter Corke
School of Electrical Engineering and Computer Science
Queensland University of Technology (QUT), Brisbane QLD 4000, Australia
e-mail: [email protected]

ISSN 1610-7438 ISSN 1610-742X (electronic)

Springer Tracts in Advanced Robotics

ISBN 978-3-319-54412-0 ISBN 978-3-319-54413-7 (eBook)

DOI 10.1007/978-3-319-54413-7

Library of Congress Control Number: 2017934638

1st ed. 2011 © Springer-Verlag Berlin Heidelberg 2011

© Springer International Publishing AG 2017

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way,
and transmission or information storage and retrieval, electronic adaptation, computer software,
or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are
exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in
this book are believed to be true and accurate at the date of publication. Neither the publisher
nor the authors or the editors give a warranty, express or implied, with respect to the material
contained herein or for any errors or omissions that may have been made. The publisher remains
neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Production: Armin Stasch and Scientific Publishing Services Pvt. Ltd. Chennai, India
Typesetting and layout: Stasch · Bayreuth ([email protected])

Printed on acid-free paper

This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Editorial Advisory Board

Nancy Amato, Texas A & M, USA

Oliver Brock, TU Berlin, Germany
Herman Bruyninckx, KU Leuven, Belgium
Wolfram Burgard, Univ. Freiburg, Germany
Raja Chatila, ISIR – UPMC & CNRS, France
Francois Chaumette, INRIA Rennes – Bretagne Atlantique, France
Wan Kyun Chung, POSTECH, Korea
Peter Corke, Queensland Univ. Technology, Australia
Paolo Dario, Scuola S. Anna Pisa, Italy
Alessandro De Luca, Sapienza Univ. Roma, Italy
Rüdiger Dillmann, Univ. Karlsruhe, Germany
Ken Goldberg, UC Berkeley, USA
John Hollerbach, Univ. Utah, USA
Lydia Kavraki, Rice Univ., USA
Vijay Kumar, Univ. Pennsylvania, USA
Bradley Nelson, ETH Zürich, Switzerland
Frank Park, Seoul National Univ., Korea
Tim Salcudean, Univ. British Columbia, Canada
Roland Siegwart, ETH Zurich, Switzerland
Gaurav Sukhatme, Univ. Southern California, USA

More information about this series at http://www.springer.com/series/5208

To my family Phillipa, Lucy and Madeline for their indulgence and support;
my parents Margaret and David for kindling my curiosity;
and to Lou Paul who planted the seed that became this book.
Foreword

Once upon a time, a very thick document of a dissertation from a faraway land came
to me for evaluation. Visual robot control was the thesis theme and Peter Corke was
its author. Here, I am reminded of an excerpt of my comments, which reads, this is a
masterful document, a quality of thesis one would like all of one’s students to strive for,
knowing very few could attain – very well considered and executed.
The connection between robotics and vision has been, for over two decades, the
central thread of Peter Corke’s productive investigations and successful developments
and implementations. This rare experience is bearing fruit in this second edition of his
book on Robotics, Vision, and Control. In its melding of theory and application, this
second edition has considerably benefited from the author’s unique mix of academic
and real-world application influences through his many years of work in robotic min-
ing, flying, underwater, and field robotics.
There have been numerous textbooks in robotics and vision, but few have reached
the level of integration, analysis, dissection, and practical illustrations evidenced in
this book. The discussion is thorough, the narrative is remarkably informative and
accessible, and the overall impression is of a significant contribution for researchers
and future investigators in our field. Most every element that could be considered as
relevant to the task seems to have been analyzed and incorporated, and the effective
use of Toolbox software echoes this thoroughness.
The reader is taken on a realistic walkthrough the fundamentals of mobile robots,
navigation, localization, manipulator-arm kinematics, dynamics, and joint-level con-
trol, as well as camera modeling, image processing, feature extraction, and multi-view
geometry. These areas are finally brought together through extensive discussion of
visual servo system. In the process, the author provides insights into how complex
problems can be decomposed and solved using powerful numerical tools and effec-
tive software.
The Springer Tracts in Advanced Robotics (STAR) is devoted to bringing to the
research community the latest advances in the robotics field on the basis of their sig-
nificance and quality. Through a wide and timely dissemination of critical research
developments in robotics, our objective with this series is to promote more exchanges
and collaborations among the researchers in the community and contribute to further
advancements in this rapidly growing field.
Peter Corke brings a great addition to our STAR series with an authoritative book,
reaching across fields, thoughtfully conceived and brilliantly accomplished.

Oussama Khatib
Stanford, California
October 2016
 Preface
Tell me and I will forget.
Show me and I will remember.
Involve me and I will understand.
Chinese proverb

Simple things should be simple,

complex things should be possible.
Alan Kay

These are exciting times for robotics. Since the first edition of this book was published
we have seen much progress: the rise of the self-driving car, the Mars science labora-
tory rover making profound discoveries on Mars, the Philae comet landing attempt,
and the DARPA Robotics Challenge. We have witnessed the drone revolution – flying
machines that were once the domain of the aerospace giants can now be bought for just
tens of dollars. All this has been powered by the continuous and relentless improve-
ment in computer power and tremendous advances in low-cost inertial sensors and
cameras – driven largely by consumer demand for better mobile phones and gaming
experiences. It’s getting easier for individuals to create robots – 3D printing is now
very affordable, the Robot Operating System (ROS) is both capable and widely used,
and powerful hobby technologies such as the Arduino, Raspberry Pi, Dynamixel servo
motors and Lego’s EV3 brick are available at low cost. This in turn has contributed to
the rapid growth of the global maker community – ordinary people creating at home
what would once have been done by a major corporation. We have also witnessed an
explosion of commercial interest in robotics and computer vision – many startups
and a lot of acquisitions by big players in the field. Robotics even featured on the front
cover of the Economist magazine in 2014!
So how does a robot work? Robots are data-driven machines. They acquire data,
process it and take action based on it. The data comes from sensors measuring the ve-
locity of a wheel, the angle of a robot arm’s joint or the intensities of millions of pixels
that comprise an image of the world that the robot is observing. For many robotic ap-
plications the amount of data that needs to be processed, in real-time, is massive. For
a vision sensor it can be of the order of tens to hundreds of megabytes per second.
Progress in robots and machine vision has been, and continues to be, driven by
more effective ways to process data. This is achieved through new and more efficient
algorithms, and the dramatic increase in computational power that follows Moore’s
“Computers in the future may weigh no law.
When I started in robotics and vision in the mid 1980s, see Fig. 0.1, the IBM PC
more than 1.5 tons.” Popular Mechanics, had been recently released – it had a 4.77 MHz 16-bit microprocessor and 16 kbytes
forecasting the relentless march of sci-
(expandable to 256 k) of memory. Over the intervening 30 years computing power has
ence, 1949
perhaps doubled 20 times which is an increase by a factor of one million.
Over the fairly recent history of robotics and machine vision a very large body of
algorithms has been developed to efficiently solve large-scale problems in perception,
planning, control and localization – a significant, tangible, and collective achievement
of the research community. However its sheer size and complexity presents a very real
barrier to somebody new entering the field. Given so many algorithms from which to
choose, a real and important question is:

What is the right algorithm for this particular problem?

One strategy would be to try a few different algorithms and see which works best
for the problem at hand, but this is not trivial and leads to the next question:

How can I evaluate algorithm X on my own data without spending days coding and
debugging it from the original research papers?
xii Preface

Fig. 0.1.
Once upon a time a lot of equip-
ment was needed to do vision-
based robot control. The author
with a large rack full of real-time
image processing and robot
control equipment (1992)

Two developments come to our aid. The first is the availability of general purpose
mathematical software which it makes it easy to prototype algorithms. There are
commercial packages such as MATLAB®, Mathematica®, Maple® and MathCad®, as Respectively the trademarks of The Math-
well as open source projects include SciLab, Octave, and PyLab. All these tools deal Works Inc., Wolfram Research, MapleSoft
and PTC.
naturally and effortlessly with vectors and matrices, can create complex and beauti-
ful graphics, and can be used interactively or as a programming environment. The
second is the open-source movement. Many algorithms developed by researchers are
available in open-source form. They might be coded in one of the general purpose
mathematical languages just mentioned, or written in a mainstream language like C,
C++, Java or Python.
For more than twenty years I have been part of the open-source community and
maintained two open-source MATLAB Toolboxes: one for robotics and one for machine
vision. They date back to my own Ph.D. work and have evolved since then, growing The term machine vision is uncommon
features and tracking changes to the MATLAB language. The Robotics Toolbox has today, but it implied the use of real-time
computer vision techniques in an indus-
also been translated into a number of different languages such as Python, SciLab and
trial setting for some monitoring or con-
LabView. More recently some of its functionality is finding its way into the MATLAB trol purpose. For robotics the real-time
Robotics System Toolbox™ published by The MathWorks. aspect is critical but today the interest-
These Toolboxes have some important virtues. Firstly, they have been around for ing challenges are in nonindustrial ap-
a long time and used by many people for many different problems so the code can be plications such as outdoor robotics. The
term robotic vision is gaining currency
accorded some level of trust. New algorithms, or even the same algorithms coded in
and is perhaps a modern take on ma-
new languages or executing in new environments, can be compared against imple- chine vision.
mentations in the Toolbox.

» allow the user to work with real problems,

not just trivial examples
Secondly, they allow the user to work with real problems, not just trivial examples.
For real robots, those with more than two links, or real images with millions of pixels
the computation required is beyond unaided human ability. Thirdly, they allow us to
gain insight which can otherwise get lost in the complexity. We can rapidly and easily
experiment, play what if games, and depict the results graphically using the power-
ful 2D and 3D graphical display tools of MATLAB. Fourthly, the Toolbox code makes
many common algorithms tangible and accessible. You can read the code, you can
apply it to your own problems, and you can extend it or rewrite it. It gives you a “leg
up” as you begin your journey into robotics.

» a narrative that covers robotics and computer vision

– both separately and together
 Preface xiii

The genesis of the book lies in the tutorials and reference material that originally
shipped with the Toolboxes from the early 1990s, and a conference paper describing
the Robotics Toolbox that was published in 1995. After a false start in 2004, the first
edition of this book was written in 2009–2010. The book takes a conversational ap-
proach, weaving text, mathematics and examples into a narrative that covers robotics
and computer vision – both separately and together. I wanted to show how complex
problems can be decomposed and solved using just a few simple lines of code. More
formally this is an inductive learning approach, going from specific and concrete ex-
amples to the more general.

» show how complex problems can be decomposed

and solved
The topics covered in this book are based on my own interests but also guided by
real problems that I observed over many years as a practitioner of both robotics and
computer vision. I want to give the reader a flavor of what robotics and vision is about
and what it can do – consider it a grand tasting menu. I hope that by the end of this
book you will share my enthusiasm for these topics.

» consider it a grand tasting menu

I was particularly motivated to present a solid introduction to computer vision
for roboticists. The treatment of vision in robotics textbooks tends to concentrate
on simple binary vision techniques. In this book we will cover a broad range of top-
ics including color vision, advanced segmentation techniques, image warping, stereo
vision, motion estimation, bundle adjustment, visual odometry and image retrieval.
We also cover nonperspective imaging using fisheye lenses, catadioptric optics and
the emerging area of light-field cameras. These topics are growing in importance for
robotics but are not commonly covered. Vision is a powerful sensor, and roboticists
should have a solid grounding in modern fundamentals. The last part of the book
shows how vision can be used as the primary sensor for robot control.
This book is unlike other text books, and deliberately so. Firstly, there are already
a number of excellent text books that cover robotics and computer vision separately
and in depth, but few that cover both in an integrated fashion. Achieving such inte-
gration is a principal goal of the book.

» software is a first-class citizen in this book

Secondly, software is a first-class citizen in this book. Software is a tangible instan-
tiation of the algorithms described – it can be read and it can be pulled apart, modified
and put back together again. There are a number of classic books that use software in
an illustrative fashion and have influenced my approach, for example LaTeX: A docu-
ment preparation system (Lamport 1994), Numerical Recipes in C (Press et al. 2007),
The Little Lisper (Friedman et al. 1987) and Structure and Interpretation of Classical
Mechanics (Sussman et al. 2001). Over 1 000 examples in this book illustrate how the
Toolbox software can be used and generally provide instant gratification in just a
couple of lines of MATLAB code.

» instant gratification in just a couple of lines

of MATLAB code
Thirdly, building the book around MATLAB and the Toolboxes means that we are
able to tackle more realistic and more complex problems than other books.

» this book provides a complementary approach

xiv Preface

The emphasis on software and examples does not mean that rigor and theory are
unimportant – they are very important, but this book provides a complementary ap-
proach. It is best read in conjunction with standard texts which do offer rigor and
theoretical nourishment. The end of each chapter has a section on further reading
and provides pointers to relevant textbooks and key papers. I try hard to use the least
amount of mathematical notation required, if you seek deep mathematical rigor this
may not be the book for you.
Writing this book provided the impetus to revise and extend the Toolboxes and
to include some great open-source software. I am grateful to the following for code
that has been either incorporated into the Toolboxes or which has been wrapped into
the Toolboxes. Robotics Toolbox contributions include: mobile robot localization
and mapping by Paul Newman; a quadrotor simulator by Paul Pounds; a Symbolic
Manipulator Toolbox by Jörn Malzahn; pose-graph SLAM code by Giorgio Grisetti
and 3D robot models from the ARTE Robotics Toolbox by Arturo Gil. Machine Vision
Toolbox contributions include: RANSAC code by Peter Kovesi; pose estimation by
Francesco Moreno-Noguer, Vincent Lepetit, and Pascal Fua; color space conversions
by Pascal Getreuer; numerical routines for geometric vision by various members of
the Visual Geometry Group at Oxford (from the web site of the Hartley and Zisserman
book; Hartley and Zisserman 2003); k-means, SIFT and MSER algorithms from the
wonderful VLFeat suite (vlfeat.org); graph-based image segmentation software by
Pedro Felzenszwalb; and the OpenSURF feature detector by Dirk-Jan Kroon. The Camera
Calibration Toolbox by Jean-Yves Bouguet is used unmodified.
Along the way I became fascinated by the mathematicians, scientists and engineers
whose work, hundreds of years ago, underpins the science of robotic and computer
vision today. Some of their names have become adjectives like Coriolis, Gaussian,
Laplacian or Cartesian; nouns like Jacobian, or units like Newton and Coulomb. They
are interesting characters from a distant era when science was a hobby and their day
jobs were as doctors, alchemists, gamblers, astrologers, philosophers or mercenaries.
In order to know whose shoulders we are standing on I have included small vignettes
about the lives of some of these people – a smattering of history as a backstory.
In my own career I have had the good fortune to work with many wonderful peo-
ple who have inspired and guided me. Long ago at the University of Melbourne John
Anderson fired my interest in control and Graham Holmes tried with mixed suc-
cess to have me “think before I code”. Early on I spent a life-direction-changing ten
months working with Richard (Lou) Paul in the GRASP laboratory at the University
of Pennsylvania in the period 1988–1989. The genesis of the Toolboxes was my Ph.D.
research (1991–1994) and my advisors Malcolm Good (University of Melbourne) and
Paul Dunn (CSIRO) asked me good questions and guided my research. Laszlo Nemes
(CSIRO) provided great wisdom about life and the ways of organizations, and encour-
aged me to publish and to open-source my software. Much of my career was spent at
CSIRO where I had the privilege and opportunity to work on a diverse range of real
robotics projects and to work with a truly talented set of colleagues and friends. Part
way through writing the first edition I joined the Queensland University of Technology
which made time available to complete that work, and in 2015 sabbatical leave to com-
plete the second.
Many people have helped me in my endeavor and I thank them. I was generously
hosted for periods of productive writing at Oxford (both editions) by Paul Newman,
and at MIT (first edition) by Daniela Rus. Daniela, Paul and Cédric Pradalier made
constructive suggestions and comments on early drafts of that edition. For the second
edition I was helped by comments on draft chapters by: Tim Barfoot, Dmitry Bratanov,
Duncan Campbell, Donald Dansereau, Tom Drummond, Malcolm Good, Peter Kujala,
Obadiah Lam, Jörn Malzahn, Felipe Nascimento Martins, Ajay Pandey, Cédric Pradalier,
Dan Richards, Daniela Rus, Sareh Shirazi, Surya Singh, Ryan Smith, Ben Talbot, Dorian
Tsai and Ben Upcroft; and assisted with wisdom and content by: François Chaumette,
Donald Dansereau, Kevin Lynch, Robert Mahony and Frank Park.
 Preface xv

I have tried my hardest to eliminate errors but inevitably some will remain. Please
email bug reports to me at [email protected] as well as suggestions for improve-
ments and extensions.
Writing the second edition was financially supported by EPSRC Platform Grant EP/
M019918/1, QUT Science & Engineering Faculty sabbatical grant, QUT Vice Chancellor’s
Excellence Award, QUT Robotics and Autonomous Systems discipline and the ARC
Centre of Excellence for Robotic Vision (grant CE140100016).
Over both editions I have enjoyed wonderful support from MathWorks, through
their author program, and from Springer. My editor Thomas Ditzinger has been a great
supporter of this project and Armin Stasch, with enormous patience and dedication in
layout and typesetting, has transformed my untidy ideas into a thing of beauty.
Finally, my deepest thanks are to Phillipa who has supported me and “the book”
with grace and patience for a very long time and in many different places – without
her this book could never have been written.

Peter Corke
Brisbane,
Queensland
October 2016

Note on the Second Edition

It seems only yesterday that I turned in the manuscript for the first edition of this book,
but it was in fact December 2010, the end of 20 months of writing. So the oldest parts
of the book are over 6 years old – it’s time for an update!
The revision principle was to keep the good (narrative style, code as a first-class citi-
zen, soft plastic cover) and eliminate the bad (errors and missing topics). I started with
the collected errata for the first edition and pencilled markup from a battered copy of the
first edition that I’ve carried around for years. There were more errors than I would have
liked and I thank everybody who submitted errata and suggested improvements.
The first edition was written before I taught in the university classroom or created
the MOOCs, which is the inverse of the way books are normally developed. Preparing
for teaching gave me insights into better ways to present some topics, particularly
around pose representation, robot kinematics and dynamics so the presentation has
been adjusted accordingly.
New content includes matrix exponential notation; the basics of screw theory and
Lie algebra; inertial navigation; differential steer and omnidirectional mobile robots; a
deeper treatment of SLAM systems including scan matching and pose graphs; greater
use of MATLAB computer algebra; operational space control; deeper treatment of ma-
nipulator dynamics and control; visual SLAM and visual odometry; structured light;
bundle adjustment; and light-field cameras.
In the first edition I shied away from Lie algebra, matrix exponentials and twists
but I think it’s important to cover them. The topic is deeply mathematical and I’ve
tried to steer a middle ground between hardcore algebraic topology and the homog-
enous transformation only approach of most other texts, while also staying true to the
overall approach of this book.
All MATLAB generated figures have been regenerated to reflect recent improve-
ments to MATLAB graphics and all code examples have been updated as required and
tested, and are available as MATLAB Live Scripts.
The second edition of the book is matched by new major releases of my Toolboxes:
Robotics Toolbox (release 10) and the Machine Vision Toolbox (release 4). These newer
versions of the toolboxes have some minor incompatibilities with previous releases of the
toolboxes, and therefore also with the code examples in the first edition of the book.
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Robots, Jobs and Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 About the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2.1 MATLAB Software and the Toolboxes . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.2 Notation, Conventions and Organization . . . . . . . . . . . . . . . . . . . . . 10
1.2.3 Audience and Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.4 Learning with the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.5 Teaching with the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.6 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Part I Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Representing Position and Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1 Working in Two Dimensions (2D) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.1.1 Orientation in 2-Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1.2 Pose in 2-Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2 Working in Three Dimensions (3D) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2.1 Orientation in 3-Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.2 Pose in 3-Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.1 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.2 Understanding the Exponential Mapping . . . . . . . . . . . . . . . . . . . . . 50
2.3.3 More About Twists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.3.4 Dual Quaternions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.3.5 Configuration Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.4 Using the Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3 Time and Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.1 Time-Varying Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1.1 Derivative of Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1.2 Transforming Spatial Velocities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.1.3 Incremental Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.1.4 Incremental Rigid-Body Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Accelerating Bodies and Reference Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2.1 Dynamics of Moving Bodies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2.2 Transforming Forces and Torques . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2.3 Inertial Reference Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.3 Creating Time-Varying Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.3.1 Smooth One-Dimensional Trajectories . . . . . . . . . . . . . . . . . . . . . . . 70
xviii Contents

3.3.2 Multi-Dimensional Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.3.3 Multi-Segment Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.4 Interpolation of Orientation in 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.3.5 Cartesian Motion in 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.4 Application: Inertial Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.4.1 Gyroscopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.4.2 Accelerometers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.4.3 Magnetometers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.4.4 Sensor Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Part II Mobile Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

4 Mobile Robot Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.1 Wheeled Mobile Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.1.1 Car-Like Mobile Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.1.2 Differentially-Steered Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.1.3 Omnidirectional Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.2 Flying Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.3 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.3.1 Nonholonomic
and Under-Actuated Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.4 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Toolbox and MATLAB Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

5 Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.1 Reactive Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.1.1 Braitenberg Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.1.2 Simple Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.2 Map-Based Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.2.1 Distance Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.2.2 D* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.2.3 Introduction to Roadmap Methods . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.2.4 Probabilistic Roadmap Method (PRM) . . . . . . . . . . . . . . . . . . . . . . 137
5.2.5 Lattice Planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.2.6 Rapidly-Exploring Random Tree (RRT) . . . . . . . . . . . . . . . . . . . . . 144
5.3 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
MATLAB Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

6 Localization . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

6.1 Dead Reckoning . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.1.1 Modeling the Vehicle .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.1.2 Estimating Pose . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
6.2 Localizing with a Map . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
6.3 Creating a Map . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
6.4 Localization and Mapping . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6.5 Rao-Blackwellized SLAM . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.6 Pose Graph SLAM . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
 Contents xix

6.7 Sequential Monte-Carlo Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

6.8 Application: Scanning Laser Rangefinder . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Laser Odometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Laser-Based Map Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Laser-Based Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
6.9 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Toolbox and MATLAB Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Part III Arm-Type Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

7 Robot Arm Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.1 Forward Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.1.1 2-Dimensional (Planar) Robotic Arms . . . . . . . . . . . . . . . . . . . . . . . 194
7.1.2 3-Dimensional Robotic Arms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
7.2 Inverse Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.2.1 2-Dimensional (Planar) Robotic Arms . . . . . . . . . . . . . . . . . . . . . . . 205
7.2.2 3-Dimensional Robotic Arms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
7.3 Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.3.1 Joint-Space Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.3.2 Cartesian Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
7.3.3 Kinematics in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
7.3.4 Motion through a Singularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
7.3.5 Configuration Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
7.4 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.4.1 Joint Angle Offsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.4.2 Determining Denavit-Hartenberg Parameters . . . . . . . . . . . . . . . . 217
7.4.3 Modified Denavit-Hartenberg Parameters . . . . . . . . . . . . . . . . . . . 218
7.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
7.5.1 Writing on a Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
7.5.2 A Simple Walking Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.6 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
MATLAB and Toolbox Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

8 Manipulator Velocity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

8.1 Manipulator Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
8.1.1 Jacobian in the World Coordinate Frame . . . . . . . . . . . . . . . . . . . . 229
8.1.2 Jacobian in the End-Effector Coordinate Frame . . . . . . . . . . . . . . . 232
8.1.3 Analytical Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
8.2 Jacobian Condition and Manipulability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
8.2.1 Jacobian Singularities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
8.2.2 Manipulability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
8.3 Resolved-Rate Motion Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
8.3.1 Jacobian Singularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8.4 Under- and Over-Actuated Manipulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8.4.1 Jacobian for Under-Actuated Robot . . . . . . . . . . . . . . . . . . . . . . . . . 241
8.4.2 Jacobian for Over-Actuated Robot . . . . . . . . . . . . . . . . . . . . . . . . . . 242
8.5 Force Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
8.5.1 Transforming Wrenches to Joint Space . . . . . . . . . . . . . . . . . . . . . . 244
8.5.2 Force Ellipsoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
8.6 Inverse Kinematics: a General Numerical Approach . . . . . . . . . . . . . . . . . . . 245
8.6.1 Numerical Inverse Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
xx Contents

8.7 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 247

8.7.1 Computing the Manipulator Jacobian Using Twists . . . . . . . . . . . 247
8.8 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 247
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 248
MATLAB and Toolbox Notes . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 248
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 248

9 Dynamics and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

9.1 Independent Joint Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
9.1.1 Actuators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
9.1.2 Friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
9.1.3 Effect of the Link Mass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
9.1.4 Gearbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
9.1.5 Modeling the Robot Joint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
9.1.6 Velocity Control Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
9.1.7 Position Control Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
9.1.8 Independent Joint Control Summary . . . . . . . . . . . . . . . . . . . . . . . . 262
9.2 Rigid-Body Equations of Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
9.2.1 Gravity Term . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
9.2.2 Inertia Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
9.2.3 Coriolis Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
9.2.4 Friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
9.2.5 Effect of Payload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
9.2.6 Base Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
9.2.7 Dynamic Manipulability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
9.3 Forward Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
9.4 Rigid-Body Dynamics Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
9.4.1 Feedforward Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
9.4.2 Computed Torque Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
9.4.3 Operational Space Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
9.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
9.5.1 Series-Elastic Actuator (SEA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
9.6 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

Part IV Computer Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

10 Light and Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
10.1 Spectral Representation of Light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
10.1.1 Absorption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
10.1.2 Reflectance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
10.1.3 Luminance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
10.2 Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
10.2.1 The Human Eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
10.2.2 Measuring Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
10.2.3 Reproducing Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
10.2.4 Chromaticity Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10.2.5 Color Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
10.2.6 Other Color and Chromaticity Spaces . . . . . . . . . . . . . . . . . . . . . . . 301
10.2.7 Transforming between Different Primaries . . . . . . . . . . . . . . . . . . . 304
10.2.8 What Is White? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
10.3 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
10.3.1 Color Temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
10.3.2 Color Constancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
 Contents xxi

10.3.3 White Balancing . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 308

10.3.4 Color Change Due to Absorption . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
10.3.5 Dichromatic Reflectance . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 310
10.3.6 Gamma . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 310
10.4 Application: Color Image . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 312
10.4.1 Comparing Color Spaces . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 312
10.4.2 Shadow Removal . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 313
10.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Further Reading . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 316
Data Sources . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 316
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 317

11 Image Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

11.1 Perspective Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
11.1.1 Perspective Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
11.1.2 Modeling a Perspective Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
11.1.3 Discrete Image Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
11.1.4 Camera Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
11.1.5 Projecting Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
11.1.6 Lens Distortion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
11.2 Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
11.2.1 Homogeneous Transformation Approach . . . . . . . . . . . . . . . . . . . . 331
11.2.2 Decomposing the Camera
Calibration Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
11.2.3 Pose Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
11.2.4 Camera Calibration Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
11.3 Wide Field-of-View Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
11.3.1 Fisheye Lens Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
11.3.2 Catadioptric Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
11.3.3 Spherical Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
11.4 Unified Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
11.4.1 Mapping Wide-Angle Images to the Sphere . . . . . . . . . . . . . . . . . . 345
11.4.2 Mapping from the Sphere to a Perspective Image . . . . . . . . . . . . . 347
11.5 Novel Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
11.5.1 Multi-Camera Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
11.5.2 Light-Field Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
11.6 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
11.6.1 Projecting 3D Lines and Quadrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
11.6.2 Nonperspective Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
11.7 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
Further Reading and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
Toolbox Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

12 Images and Image Processing . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 359

12.1 Obtaining an Image . . . . . . . . . . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 359
12.1.1 Images from Files . . . . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 359
12.1.2 Images from an Attached Camera ... . . . . . . . . . . . . . . . . . . . . . . . 363
12.1.3 Images from a Movie File . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 365
12.1.4 Images from the Web . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 366
12.1.5 Images from Maps . . . . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 367
12.1.6 Images from Code . . . . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 367
12.2 Image Histograms . . . . . . . . . . . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 369
12.3 Monadic Operations . . . . . . . . . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . 370
xxii Contents

12.4 Diadic Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

12.5 Spatial Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
12.5.1 Linear Spatial Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
12.5.2 Template Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
12.5.3 Nonlinear Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
12.6 Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
12.6.1 Noise Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
12.6.2 Boundary Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
12.6.3 Hit or Miss Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
12.6.4 Distance Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
12.7 Shape Changing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
12.7.1 Cropping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
12.7.2 Image Resizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402
12.7.3 Image Pyramids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
12.7.4 Image Warping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
12.8 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
Sources of Image Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
MATLAB Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
General Software Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410

13 Image Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413

13.1 Region Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
13.1.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
13.1.2 Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
13.1.3 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
13.1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
13.2 Line Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
13.2.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
13.3 Point Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
13.3.1 Classical Corner Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
13.3.2 Scale-Space Corner Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
13.4 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
MATLAB Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457

14 Using Multiple Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

14.1 Feature Correspondence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460
14.2 Geometry of Multiple Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
14.2.1 The Fundamental Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
14.2.2 The Essential Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
14.2.3 Estimating the Fundamental Matrix
from Real Image Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
14.2.4 Planar Homography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
14.3 Stereo Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
14.3.1 Sparse Stereo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
14.3.2 Dense Stereo Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
14.3.3 Peak Refinement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
14.3.4 Cleaning up and Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
14.3.5 3D Texture Mapped Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
14.3.6 Anaglyphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
14.3.7 Image Rectification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
14.4 Bundle Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
 Contents xxiii

14.5 Point Clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

14.5.1 Fitting a Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503
14.5.2 Matching Two Sets of Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
14.6 Structured Light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507
14.7 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
14.7.1 Perspective Correction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
14.7.2 Mosaicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512
14.7.3 Image Matching and Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
14.7.4 Visual Odometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
14.8 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
MATLAB and Toolbox Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529

Part V Robotics, Vision and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533

15 Vision-Based Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
15.1 Position-Based Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
15.2 Image-Based Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
15.2.1 Camera and Image Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542
15.2.2 Controlling Feature Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
15.2.3 Estimating Feature Depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551
15.2.4 Performance Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554
15.3 Using Other Image Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
15.3.1 Line Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
15.3.2 Circle Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557
15.3.3 Photometric Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559
15.4 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562

16 Advanced Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565

16.1 XY/Z-Partitioned IBVS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565
16.2 IBVS Using Polar Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
16.3 IBVS for a Spherical Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570
16.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572
16.4.1 Arm-Type Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572
16.4.2 Mobile Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573
16.4.3 Aerial Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576
16.5 Wrapping Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578
Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578
Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579

Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
A Installing the Toolboxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583
B Linear Algebra Refresher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587
C Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
D Lie Groups and Algebras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611
E Linearization, Jacobians and Hessians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617
F Solving Systems of Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621
G Gaussian Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
H Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635
I Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641
J Peak Finding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645
xxiv Contents

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 649

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663
Index of People . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663
Index of Functions, Classes and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664
General Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
Nomenclature

The notation used in robotics and computer vision varies considerably across books and
research papers. The symbols used in this book, and their units where appropriate, are
listed below. Some symbols have multiple meanings and their context must be used to
disambiguate them.
xxvi Nomenclature
Nomenclature xxvii
xxviii Nomenclature
 Nomenclature xxix

MATLAB® Toolbox Conventions

A Cartesian coordinate, a point, is expressed as a column vector.

A set of points is expressed as a matrix with columns representing the coordinates
of individual points.

A rectangular region by two opposite corners [xmin xmax; ymin ymax].

A robot configuration, a set of joint angles, is expressed as a row vector.

Time series data is expressed as a matrix with rows representing time steps.

A MATLAB matrix has subscripts (i, j) which represent row and column respec-
tively. Image coordinates are written (u, v) so an image represented by a matrix I
is indexed as I(v, u).

Matrices with three or more dimensions are frequently used:
– A color image has 3 dimensions: row, column, color plane.
– A greyscale image sequence has 3 dimensions: row, column, index.
– A color image sequence has 4 dimensions: row, column, color plane, index.

Common Abbreviations
2D 2-dimensional
3D 3-dimensional
DOF Degrees of freedom
n-tuple A group of n numbers, it can represent a point of a vector
Chapter

1 Introduction

The term robot means different things to different people. Science fiction books and
movies have strongly influenced what many people expect a robot to be or what it can
do. Sadly the practice of robotics is far behind this popular conception. One thing is
certain though – robotics will be an important technology in this century. Products
such as vacuum cleaning robots have already been with us for over a decade and self-
driving cars are coming. These are the vanguard of a wave of smart machines that will
appear in our homes and workplaces in the near to medium future.
In the eighteenth century the people of Europe were fascinated by automata such as
Vaucanson’s duck shown in Fig. 1.1a. These machines, complex by the standards of the
day, demonstrated what then seemed life-like behavior. The duck used a cam mecha-
nism to sequence its movements and Vaucanson went on to explore mechanization of
silk weaving. Jacquard extended these ideas and developed a loom, shown in Fig. 1.1b,
that was essentially a programmable weaving machine. The pattern to be woven was
encoded as a series of holes on punched cards. This machine has many hallmarks of a
modern robot: it performed a physical task and was reprogrammable.
The term robot first appeared in a 1920 Czech science fiction play “Rossum’s Universal
Robots” by Karel .apek (pronounced Chapek). The term was coined by his brother
Josef, and in the Czech language means serf labor but colloquially means hardwork
or drudgery. The robots in the play were artificial people or androids and as in so
many robot stories that follow this one, the robots rebel and it ends badly for human-
ity. Isaac Asimov’s robot series, comprising many books and short stories written be-
tween 1950 and 1985, explored issues of human and robot interaction and morality.
The robots in these stories are equipped with “positronic brains” in which the “Three
laws of robotics” are encoded. These stories have influenced subsequent books and
movies which in turn have shaped the public perception of what robots are. The mid
twentieth century also saw the advent of the field of cybernetics – an uncommon term
today but then an exciting science at the frontiers of understanding life and creating
intelligent machines.
The first patent for what we would now consider a robot was filed in 1954 by
George C. Devol and issued in 1961. The device comprised a mechanical arm with

Fig. 1.1.
Early programmable machines.
a Vaucanson’s duck (1739) was
an automaton that could flap its
wings, eat grain and defecate. It
was driven by a clockwork mech-
anism and executed a single
program; b The Jacquard loom
(1801) was a reprogrammable
machine and the program was
held on punched cards (photo-
graph by George P. Landow
from www.victorianweb.org)
2 Chapter 1 · Introduction

a gripper that was mounted on a track and the sequence of motions was encod-
ed as magnetic patterns stored on a rotating drum. The first robotics company,
Unimation, was founded by Devol and Joseph Engelberger in 1956 and their first
industrial robot shown in Fig. 1.2 was installed in 1961. The original vision of Devol
and Engelberger for robotic automation has become a reality and many millions of
arm-type robots such as shown in Fig. 1.3 have been built and put to work at tasks
such as welding, painting, machine loading and unloading, electronic assembly,
packaging and palletizing. The use of robots has led to increased productivity and
improved product quality. Today many products we buy have been assembled or
handled by a robot.

Fig. 1.2.
Universal automation. a A plan
view of the machine from Devol’s
patent; b the first Unimation
robot working at a General
Motors factory (photo courtesy
of George C. Devol)

Unimation Inc. (1956–1982). Devol sought financing to develop his unimation technology and at
a cocktail party in 1954 he met Joseph Engelberger who was then an engineer with Manning,
Maxwell and Moore. In 1956 they jointly established Unimation, the first robotics company, in
Danbury Connecticut. The company was acquired by Consolidated Diesel Corp. (Condec) and
became Unimate Inc. a division of Condec. Their first robot went to work in 1961 at a General
Motors die-casting plant in New Jersey. In 1968 they licensed technology to Kawasaki Heavy
Industries which produced the first Japanese industrial robot. Engelberger served as chief execu-
tive until it was acquired by Westinghouse in 1982. People and technologies from this company
have gone on to be very influential on the whole field of robotics.

George C. Devol, Jr. (1912–2011) was a prolific American inventor. He was born in Louisville,
Kentucky, and in 1932 founded United Cinephone Corp. which manufactured phonograph
arms and amplifiers, registration controls for printing presses and packaging machines. In
1954, he applied for US patent 2,988,237 for Programmed Article Transfer which introduced
the concept of Universal Automation or “Unimation”. Specifically it described a track-mounted
polar-coordinate arm mechanism with a gripper and a programmable controller – the precur-
sor of all modern robots.
In 2011 he was inducted into the National Inventors Hall of Fame. (Photo on the right: cour-
tesy of George C. Devol)

Joseph F. Engelberger (1925–2015) was an American engineer and entrepreneur who is often referred
to as the “Father of Robotics”. He received his B.S. and M.S. degrees in physics from Columbia
University, in 1946 and 1949, respectively. Engelberger has been a tireless promoter of robotics.
In 1966, he appeared on The Tonight Show Starring Johnny Carson with a Unimate robot which
poured a beer, putted a golf ball, and directed the band. He promoted robotics heavily in Japan,
which led to strong investment and development of robotic technology in that country.
Engelberger served as chief executive of Unimation until 1982, and in 1984 founded Transitions
Research Corporation which became HelpMate Robotics Inc., an early entrant in the hospital ser-
vice robot sector. He was elected to the National Academy of Engineering, received the Beckman
Award and the Japan Prize, and has written two books: Robotics in Practice (1980) and Robotics
in Service (1989). Each year the Robotics Industries Association presents an award in his honor
to “persons who have contributed outstandingly to the furtherance of the science and practice
of robotics.”
 Chapter 1 · Introduction 3

These first generation robots are fixed in place and cannot move about the fac-
tory – they are not mobile. By contrast mobile robots as shown in Figs. 1.4 and
1.5 can move through the world using various forms of mobility. They can loco-
mote over the ground using wheels or legs, fly through the air using fixed wings or
multiple rotors, move through the water or sail over it. An alternative taxonomy
is based on the function that the robot performs. Manufacturing robots operate
in factories and are the technological descendents of the first generation robots.
Service robots supply services to people such as cleaning, personal care, medical
rehabilitation or fetching and carrying as shown in Fig. 1.5b. Field robots, such as
those shown in Fig. 1.4, work outdoors on tasks such as environmental monitor-
ing, agriculture, mining, construction and forestry. Humanoid robots such as shown
in Fig. 1.6 have the physical form of a human being – they are both mobile robots
In practice the categorization of robots and service robots.

is not very consistently applied.

Fig. 1.3.
Manufacturing robots, tech-
nological descendants of the
Unimate shown in Fig. 1.2.
a A modern six-axis robot de-
signed for high accuracy and
throughput (image courtesy
ABB robotics); b Baxter two-
armed robot with built in vision
capability and programmable
by demonstration, designed for
moderate throughput piece work
(image courtesy Rethink Robotics)

Rossum’s Universal Robots (RUR). In the introductory scene Helena Glory is visiting Harry Domin
the director general of Rossum’s Universal Robots and his robotic secretary Sulla.
Domin Sulla, let Miss Glory have a look at you.
Helena (stands and offers her hand) Pleased to meet you. It must be very hard for you
out here, cut off from the rest of the world [the factory is on an island]
Sulla I do not know the rest of the world Miss Glory. Please sit down.
Helena (sits) Where are you from?
Sulla From here, the factory
Helena Oh, you were born here.
Sulla Yes I was made here.
Helena (startled) What?
Domin (laughing) Sulla isn’t a person, Miss Glory, she’s a robot.
Helena Oh, please forgive me …
The full play can be found at http://ebooks.adelaide.edu.au/c/capek/karel/rur. (Image on the
left: Library of Congress item 96524672)
4 Chapter 1 · Introduction

A manufacturing robot is typically an arm-type manipulator on a fixed base such

as Fig. 1.3a that performs repetitive tasks within a local work cell. Parts are presented
to the robot in an orderly fashion which maximizes the advantage of the robot’s high
speed and precision. High-speed robots are hazardous and safety is achieved by ex-
cluding people from robotic work places, typically placing the robot inside a cage. In
contrast the Baxter robot shown in Fig. 1.3b is human safe, it operates at low speed
and stops moving if it encounters an obstruction.
Field and service robots face specific and significant challenges. The first chal-
lenge is that the robot must operate and move in a complex, cluttered and chang-
ing environment. A delivery robot in a hospital must operate despite crowds of
people and a time-varying configuration of parked carts and trolleys. A Mars rover
as shown in Fig. 1.5a must navigate rocks and small craters despite not having an
accurate local map in advance of its travel. Robotic, or self-driving cars, such as
shown in Fig. 1.5c, must follow roads, avoid obstacles and obey traffic signals and Fig. 1.4. Non-land-based mobile
the rules of the road. The second challenge for these types of robots is that they robots. a Small autonomous un-
derwater vehicle (Todd Walsh
must operate safely in the presence of people. The hospital delivery robot operates © 2013 MBARI); b Global Hawk
among people, the robotic car contains people and a robotic surgical device oper- unmanned aerial vehicle (UAV)
ates inside people. (photo courtesy of NASA)

Cybernetics, artificial intelligence and robotics. Cybernetics flour- (Wiener 1965). A characteristic of a cybernetic system is the use
ished as a research field from the 1930s until the 1960s and was of feedback which is common in engineering and biological sys-
fueled by a heady mix of new ideas and results from neurology, tems. The ideas were later applied to evolutionary biology, psy-
control theory and information theory. Research in neurology chology and economics.
had shown that the brain was an electrical network of neurons. In 1956 a watershed conference was hosted by John McCarthy
Harold Black, Henrik Bode and Harry Nyquist at Bell Labs were at Dartmouth College and attended by Minsky, Shannon, Herbert
researching negative feedback and the stability of electrical net- Simon, Allen Newell and others. This meeting defined the term
works, Claude Shannon’s information theory described digital artificial intelligence (AI) as we know it today with an em-
signals, and Alan Turing was exploring the fundamentals of phasis on digital computers and symbolic manipulation and
computation. Walter Pitts and Warren McCulloch proposed led to new research in robotics, vision, natural language, se-
an artificial neuron in 1943 and showed how it might perform mantics and reasoning. McCarthy and Minsky formed the AI
simple logical functions. In 1951 Marvin Minsky built SNARC group at MIT, and McCarthy left in 1962 to form the Stanford
(from a B24 autopilot and comprising 3 000 vacuum tubes) AI Laboratory. Minsky focused on artificially simple “blocks
which was perhaps the first neural-network-based learning world”. Simon, and his student Newell, were influential in AI
machine as his graduate project. William Grey Walter’s ro- research at Carnegie-Mellon University from which the Robotics
botic tortoises showed life-like behavior. Maybe an electronic Institute was spawned in 1979. These AI groups were to be very
brain could be built! influential in the development of robotics and computer vision
An important early book was Norbert Wiener’s Cybernetics in the USA. Societies and publications focusing on cybernetics
or Control and Communication in the Animal and the Machine are still active today.
Another random document with
no related content on Scribd:
the disturbed shires, or wherever conventicles had been kept, and to
take effectual measures for reducing them to due obedience “to Us
and Our laws,” by taking free quarter from those that were
disaffected, disarming such as they should suspect, and seizing and
securing all horses above such a value as they should think fit; at the
same time, causing heritors and liferenters to give bond for their
tenants and all who resided on their lands; and the tenants and
fathers of families to do the like for those who resided with them, that
they should keep no conventicles, but live orderly, attend the parish
churches regularly, and not harbour or converse with any
intercommuned person.
For the more completely carrying these orders into execution, they
were not only to punish the disobedient, but whomsoever “they might
judge disaffected,” by fining, confining, imprisonment, or banishment.
They were also to plant garrisons wherever they thought it
necessary; and if the forces now ordered were not sufficient, they
were empowered to call to their assistance the troops stationed in
the north of England and Ireland. A report was at this time very
generally spread and believed, that the Duke of York had said there
would be no peace in the country till the west were turned into a
hunting-forest; and the conduct of those in power appeared as if they
had heard and approved of the sentiment.
 BOOK XII.

A.D. 1678-1679.

Privy council forbids emigration—Mitchell’s trial and execution—Highland host—

Committee of the council arrive at Glasgow—Deputation from Ayr sent to the
Commissioner—Bond refused—Committee proceed to Ayr—Earl of Cassilis—
Law-burrows—Case of Lord Cochrane—Ravages of “the Highland Host”—
Their return home—Earl of Cassilis goes to court—Duke of Hamilton follows—
Complaints dismissed—State of the country—Murder of Sharpe.

Capricious as cruel in their tyranny, the council would neither allow

the Presbyterians to live peaceably at home, nor permit them to seek
liberty abroad, especially if they were persons of rank, whom they
wished to make participators of their tyranny as they could not
induce them to be willing associates in their crimes. Having learned
that several noblemen and others of high station, disgusted with their
proceedings, were preparing to leave the country, they issued a
proclamation, January 3, forbidding any person, lord or commoner,
to remove forth thereof upon any pretext whatsoever, without a
special license from them, under the highest penalties; and to make
assurance doubly sure, they ordered the principal among them,
whom they considered as their political rivals, or who were more
moderate in their principles, to attend “a committee of the privy
council,” appointed to accompany the forces in the west country, to
receive their orders and obey their commands.[91]
 91. Numbers of the persecuted in England had left that country for America, and
were founding the states of New England, New Jersey, Massachusetts, &c.,
which formed asylums for their brethren during these perilous times. Many of
the Scots who to Holland, also sought refuge in the New World. But it would
appear the Scottish prelatists being, as all turncoats are, more violent than
the English persecutors, wished to retain their more conscientious
countrymen at home, that they might have the pleasure of tormenting them,
and enjoying the yet higher gratification of revelling on their fines. The
prohibition in Scotland was intended, besides, to answer another purpose, to
prevent any of the nobility proceeding to court without leave; for Lauderdale
knew well the advantage of engrossing the royal ear.

Before proceeding to detail the transactions of this savage horde

and their directors, I shall advert to a transaction still more
disgraceful to the council, as setting at defiance all moral decency,
and bursting asunder every tie that gives security to society, which
can only exist well where the obligations both of the rulers and the
ruled are held sacred. I mean the trial and execution of James
Mitchell.
Cowards are proverbially cruel, and the renegade primate was not
remarkable for courage. He seems to have been constantly haunted
with the terrors of assassination. Fearing his own treacherous “law-
skreened murders” might provoke some other resolute arm to
retaliate, he could not be at rest while Mitchell lived, and appears to
have imagined that the destruction of that poor man was necessary
to secure his own safety. He therefore resolved, by making an
example of him, to show that the sacred person of a priest was not to
be threatened with impunity. Accordingly, Mitchell was brought from
the Bass to Edinburgh in the end of last year, and received an
indictment to stand trial for his attempt. On January 7, 1678, he was
brought to the bar of the Justiciary, where Primrose, justice-general,
sat as one of his judges, and Sir George Mackenzie acted as
accuser, both of whom were perfectly acquainted with the promise of
pardon which had been made. Primrose had been summoned as a
witness, but was dispensed with; and, had he possessed the
smallest particle of common feeling, or of common honesty, he
would never have consented to sit as a judge—much less would
Mackenzie, who had acted as his advocate on the former trial, have
now come forward as his prosecutor; yet so it was. Primrose,
however, transmitted privately to Mitchell’s advocates, a copy of the
act of council in which the assurance was contained. Lauderdale had
been previously warned of its existence by Kincardine. The
pleadings, before entering upon the evidence, were long and
ingenious. His advocates, Sir George Lockhart and Mr John Ellis,
contended that the libel was not relevant,—as a mere attempt, when
unsuccessful, could never constitute the crime of murder; that by the
laws of this kingdom, by the civil law, and the common opinion of
civilians, it was not a capital offence, except in cases of parricide or
treason; and, besides, the act charged was assassination or murder
committed for hire—a term and a crime unknown in Scottish law; nor
is it charged against the prisoner that he was hired by any person to
commit the deed. As to the confession, if such a thing existed, which
the panel refused to acknowledge, it was extrajudicial, not being
made in presence of the assize, who are judges of the whole proof,
and therefore could not be admitted, unless taken together with the
promise of pardon by which it was elicited. But they especially
insisted upon the promise of pardon, as rendering any charge
founded upon such confession totally irrevelant.
The Lord Advocate replied, that, by act 16, parl. James VI. nudus
conatus, attempting and invading, though nothing followed, is found
relevant to infer the pain of death; and by the common law, an
attempt is capital, where the panel has been guilty of the proximate
act, and done all that it was in his power to do:—adding, most
unfairly and untruly, that Mitchell belonged to a sect that hated and
execrated the hierarchy, who deemed it lawful to kill persons of a
prelatical character; and he could prove that Mr James himself held
such opinions, which he endeavoured to defend by wrested places
of Scripture, and acknowledged that the reason why he shot at the
archbishop was, because he thought him a persecutor of the
nefarious and execrable rebels who appeared on the Pentland Hills.
As to assassination not being known in Scots law, the term might not
be there, but the nation would be worse than the Tartars, if lying in
wait with a design to kill clandestinely, where a person, after mature
deliberation, ripens his villany and watches his opportunity, if this
should not be held in greater detestation, and punished more
severely than ordinary murder. As to being hired, if taking money
constitute the criminality of assassination, how much greater is it
when committed to earn a higher reward. He that takes money to kill,
will stab only in the dark, and where he may escape; but the sun,
and the cross, and the confluence of all the world, cannot secure
against the stroke of the murderer who expects heaven as his
reward, and thinks that the deed deserves it. Respecting the promise
of pardon, the promise of life from a judge, who has not the power to
grant it, is of no avail unless the panel can prove that he expressly
pactioned that his confession should not operate against him; and a
confession emitted without any such regular bargain, is of no avail,
even though the judge should promise life; for this would be to make
a judge a king.[92] As to the confession being extrajudicial, so far from
this being the case, it was taken by the authority of the privy council,
the highest judicatory of the nation, uniting in itself the powers both
of the Court of Session and the Court of Justiciary; and if
confessions emitted before the lords of session are a sole, final, and
plenary probation before the Court of Justiciary, it were absurd to
suppose that a confession emitted before the Privy Council should
not be deemed valid.
92. The advocate, as a proof that civilians were on his side, quoted Ægidius
Bossius, who, Titulo de Examine Reorum 15 and 16, says—“Judex qui
induxit reum ad confitendum sub promissione veniæ non tenetur servare
promissum in foro contensioso.” The judge who induces a panel to confess,
by a promise of pardon, is not bound to keep his promise in a contested trial,
which seems, says Lord Fountainhall in his notes, “to be ane disingenous
opinion.”
The court decided that the crime, as libelled, was relevant, i. e.
sufficient, to infer the pains of law; but, at the same time, found that
the defence if proven was relevant to secure the panel of his life and
limb. There were no witnesses to establish the fact; his confession
was the only evidence adduced; to substantiate which, Rothes was
first examined, who deponed that he saw the panel sign the
confession. Being asked, whether or not his lordship did offer to the
panel, upon his confession, to secure his life, in these words, upon
his lordship’s life, honour, and reputation? he swore that he did not at
all give any assurance to the panel for his life, and that the panel
never sought any such assurance from him, nor did he remember
receiving any warrant from the council for that purpose. Upon this,
Mitchell entreated the Chancellor to remember the honour of the
family of Rothes, and mind that he took him by the hand, and said
—“Jacobe, man, confess; and, as I am Chancellor of Scotland, ye
shall be safe in liffe and limb;” to which all the answer returned by
the Chancellor was, “that he hoped his reputation was not yet so low
as that what the panel said, either there or elsewhere, would be
credited, since he had sworn.” The panel, however, still averred the
contrary.
Lord Hatton, Lauderdale, and Sharpe swore to the same effect.
When Sharpe had done, Nicol Somerville, agent, brother-in-law to
the panel, boldly contradicted him, and bid him remember certain
times and expressions. The archbishop, who did not much relish
getting his memory so refreshed, “fell in a mighty chaff and passion,
exceedingly unbecoming his station and the circumstances he was
then stated in, and fell a scolding before thousands of onlookers.
Nicol yielded in nothing; and after the bishop had sworne, he cryed
out that upon his salvation what he had affirmed was true.” “And the
misfortune was, that few there but they believed Nicol better than the
archbishop.”[93] Sir John Nisbet, who was Lord Advocate at the time,
and one of the committee who examined Mitchell, summoned as a
witness for the crown—probably to prevent him from being adduced
for Mitchell—was not called, Sir George Mackenzie, it is likely, being
afraid to trust him.
93. Fountainhall’s notes.
After the public prosecutor had declared his proof closed, the
panel’s advocates produced the copy of the act of council, and
craved that the books of council, which were lying in the next room,
might be produced, or the clerks ordered to give extracts, which they
had formerly refused. At this Lauderdale, who had no right to speak,
“stormed mightily,” and told the court “the books of council contained
the king’s secrets, and he would not permit them to be examined; he
came there to depone as a witness, not to be staged for perjury”—an
unguarded remark, which must have been understood by the judges
as a plain confession that he knew he had sworn falsely; yet, with a
mean servility, they would not assert their own dignity, nor do justice
to the panel. They refused to grant warrant for producing the
registers, because not applied for before, which Fountainhall
observes “choaked both criminal law and equity, for it is never too
late to urge any thing in favour of a panel until the assize be closed.”
Sir George Lockhart defended him with admirable strength of
reasoning; and the trial, which is characterized as the most solemn
which had taken place in Scotland for a hundred years, lasted four
days. The jury returned a verdict, finding him guilty upon his own
confession; but the promise of pardon they found not proven. He
was condemned to be hanged on Friday the 18th of January.
On leaving court, the four “noble” witnesses proceeded to the
Council-chamber and inspected the books, where they saw the
indelible record of their own guilt and infamy, which still remains,
and, like convicted rogues, began each to vindicate himself. After a
vain attempt to fix it upon the late Lord Advocate, Nisbet, had failed,
Lauderdale, who seems to have had some compunctious visitations,
proposed to grant a reprieve, and refer the matter to the king. But the
primate insisted that if favour were shown to this assassin, it would
be exposing his person to the next murderer who should attempt it.
“Then,” said Lauderdale, “let Mitchell glorify God in the
Grassmarket.”[94] He was accordingly executed, pursuant to his
sentence. Sharpe, whose vanity and ambition were unbounded,
aping an equality with royalty, had obtained an order from court, that
Mitchell’s head should be affixed on some public place of the city, as
if his crime had been high treason! But it was told him what was
pronounced for doom could not be altered; so he missed this
gratification. Nor did the fate of Mitchell tend to avert his own.
Mitchell’s misguided act was forgotten in the deeper and more
deliberate revenge of the archbishop, and in the atrocious breach of
public faith by the council. His dying declaration, widely circulated
through the country, exhibited such a view of the treachery and
almost unexampled perjury of the first ministers in the church and in
the state, as excited universal horror and execration.[95]
94. The usual place of execution at that time.
 95. The question then much agitated—“The extraordinary execution of
judgement by private men”—was supported by an apophthegm borrowed
from Tertullian—“Every man is a soldier enrolled to bear arms against all
traitors and public enemies;” and by the authority of Dr Ames, who, in his
treatise on Conscience, published 1674, says—“Sometimes it is lawful to kill,
no public precognition proceeding, when the cause evidently requires it
should be done, and public authority cannot be got: For in that case a private
man is publickly constitute the minister of justice, as well by the permission of
God as the consent of all men.” Mitchell, when questioned by the Chancellor,
thus defended his attack upon Sharpe—and it is easy to conceive that such
reasoning would appear irrefragable to a mind excited as his was—“I looked
upon him to be the main instigator of all the oppression and bloodshed
thereupon, and the continual pursuing after my own; and, my lord, it was
creditably reported to us (the truth of which your lordship knows better than
we) that he kept up his majesty’s letter inhibiting any more blood upon that
account, until the last six were executed; and I being a soldier, not having
laid down my arms, but being upon my own defence; and in prosecution of
the ends of the same covenant [which he also had sworn] which was the
overthrow of prelates and prelacy; and I being a declared enemy to him on
that account, and he to me in like manner: as he was always to take his
advantage of me, as it appeareth, so I of him, to take any opportunity offered.
Moreover, we being in no terms of capitulation, but on the contrary, I, by his
instigation being excluded from all grace and favour, thought it my duty to
pursue him at all events.”

Upon the 24th of January, the threatened army, better known by

the name of “the Highland Host,”[96] assembled at Stirling. The Earls
were their colonels, who received a handsome pay; but the active
officers were a set of thievish lairds; and their retainers, wild
savages, unacquainted with any other law than the will of their
chiefs, whose mandates they obeyed without inquiry upon every
occasion—only in the division of the spoil, they sometimes helped
themselves without waiting the directions of their superiors. They
amounted in all, including about two thousand regulars and two
thousand militia, to about ten thousand men, with four field-pieces,
and with a great quantity of spades, shovels, and mattocks, as if they
were marching to besiege fortified cities; their daggers were formed
to fasten on the muzzles of the muskets, as a kind of rude bayonets,
to attack cavalry; yet were they accompanied with other instruments
that betokened any thing but going to meet a regular force—iron
shackles and thumb-screws!
96. Because Highlanders formed the majority; the regulars or king’s guards were
the worst; the militia, although not good, seem to have been the best, if any
could be called best among them, unless it were that in the act of plundering,
they were not quite so fierce as the others. “The debauched clargie thought it
no shame to call thes dragoons the ruling elders of the church.” Wodrow,
MS. Advocates’ Lib. xl. art. 47, quoted by Dr M’Crie. Mem. of Geo. Brysson,
p. 275.
The approach of such an array amazed the peaceable inhabitants
of the west, nor were the military gentlemen themselves less
astonished when they passed through a country represented as in a
state of rebellion, but where they saw every thing perfectly loyal and
tranquil. Nevertheless, the mountaineers in their march, and during
the time they remained in the west, gratified the expectations of their
employers to the full. Behaving with the unbridled insolence of
victorious mercenaries in a conquered country, they made free with
whatever they wanted without ceremony, seizing every serviceable
horse for the transport of their baggage, even those at the ploughs in
the midst of the tillage, extorting money and beating and wounding
whoever resisted, without distinction. Nor were the few heritors who
took the bond exempted. They found, when too late, that they had
violated their consciences, or at least their consistency, in vain; and
some of them afterwards deeply lamented their compliance,
regretting that they had not rather, like the majority of their
neighbours, taken quietly the spoiling of their goods.
Their head-quarters were first at Glasgow, but the tumultuous
bands soon spread through Clydesdale, Renfrew, Cunninghame,
Kyle, and Carrick. Previously to their arrival, the ministers had held a
day of fasting, humiliation, and prayer. This the prelates represented
as preparatory to a communion, after which there was to be a
general insurrection. The report was soon discovered to be false, but
it had quickened the advance of the host, and was either believed, or
pretended to be believed, by Lauderdale; for, when a deputation
from the nobility and gentry of Ayrshire came to Edinburgh to
represent to the council the tranquillity and unimpeachable loyalty of
the whole district, he would not so much as give them audience; and
when some of them offered to engage for the peace of the shire, the
proposal was peremptorily refused, and they were informed that no
compromise could be entered into, nothing less would be accepted
than that the whole of them present should instantly put their
signatures to the bond, and pledge themselves for all the other
heritors doing the same. The deputation could not promise for
others, and they returned to witness the authorized enormous
disorders they had employed every legal method in their power to
prevent.
On the 28th of the month, the committee of council, armed with
Justiciary power, met at Glasgow to consider their instructions and
proceed to action.[97] They were directed to order the sheriffs of the
different counties to convene all the heritors, and require them to
subscribe a bond, obliging themselves, wives, bairns, and servants,
as also their haill tenants, and cottars, with their wives, bairns, and
servants, to abstain from conventicles, and not to receive, assist, or
speak to any forfeited persons, intercommuned ministers, or vagrant
preachers, but to use their utmost endeavours to apprehend all such,
promising, if any of their families or dependents should contravene,
to present them to the judge-ordinary that they might be fined or
imprisoned for their delinquencies. All who took the bond were to
receive a protection that their lands would not be quartered upon.
They were also to cause the leaders of the horsemen of the militia
troops to deliver to them the haill militia arms, and to disarm heritors
and all other persons, except privy councillors and military men; but
noblemen and gentlemen of quality were to be allowed to wear their
swords. The arms were to be lodged in the Castle of Dumbarton.
97. The committee consisted of the Marquis of Atholl, the Earls of Marr, Murray,
Glencairn, Wigton, Strathmore, Linlithgow, Airly, Caithness, Perth, and Lord
Ross, all of whom were commanders in the army, except Glencairn and
Wigton.

Framed as this bond was, it required no ghost to tell that it would

not generally be taken; and its refusal was looked forward to by the
government with joyful anticipation, as what would justify their
pressing it with a rigour that would produce the grand, much longed-
for consummation—a desperate resistance. But this was for the
present postponed—the disarming of the people, although not
complete, prevented any immediate outbreaking, while the example
of the Duke of Hamilton, Lords Loudon, Cochrane, and especially
the constancy of Lord Cassilis, encouraged the great body of the
gentry to continue steadfast in opposition to a bond which the council
had exceeded their powers in enacting, and could not legally oblige
the lieges to subscribe without the authority of parliament. Besides
its illegality, these patriots considered it cruel and degrading—cruel,
in forbidding them to give relief to Christian ministers, and others in
distress, even though their own relatives, and to shut up their bowels
of compassion from them, merely on account of difference of opinion
about church government—degrading, in desiring them to act as
beadles or common messengers at arms, without their own consent.
From Glasgow the committee proceeded to Ayr; and among their
first proceedings, they ordered the Earl of Cassilis to demolish all the
meeting-houses in Carrick. The Earl represented the probability of
opposition, and having been disarmed, requested that a party of
soldiers, or at least some of the neighbouring gentlemen, might be
ordered to protect him; even this reasonable request could not be
granted; and while he hesitated, a friendly councillor hinted that there
was but an hair’s-breadth between him and imprisonment. Such,
however, was the esteem in which this young nobleman was held,
that the people themselves demolished the offensive places of
worship, rather than that he should be troubled about them.[98] But
the council, not willing that he should get so quietly rid of the job,
ordered him to bring back the doors and all the timber of these
meeting-houses, and burn it on the spot where they had stood. his
lordship complied with this useless but tyrannically-teasing order.
98. The meeting-houses were not in common very costly fabricks. Like the
temple at Jerusalem, no mason’s iron was heard in their building, being
generally framed of rough unhewn stones, covered with turf; and the people
were thankful when the government did not interfere with their cheap church-
extension scheme. Stately cathedrals they asked not, they cheerfully left
them to the Romanists and the renegade prelatical conformists, their
brethren. Consecrated walls were words unknown in their vocabulary, all
they asked was shelter from the weather and very humble accomodation for
 their wearied limbs. Nor did they always ask even these; for their ministers,
following the example of Him whose servants they professed to be, oftener
had the mountain for their pulpit and the heavens for their sounding-board,
than the crimson-covered desk with velvet cushion and gilded canopy; while
they themselves were satisfied, if they could hear the gospel faithfully
preached, although on the mountain side, or in “divot theikit beilds.”

Notwithstanding the stubborn opposition it met with, the council

appeared determined to urge the bond, and issued a fresh
proclamation, February 11, forbidding any person to be received as a
tenant or servant without a certificate from the landlord or master
they last left, or from the minister of the parish, that they had lived
orderly and attended the parish church, and had not heard any of the
vagrant preachers, who without license impiously assumed the holy
orders of the church. To this was annexed a new bond of similar
import with the former, and as an encouragement, all the members
of council signed it, and appointed the lords of session to do the
same when they met. Every inducement proved ineffectual; and the
reports of the commissioners appointed to see their orders carried
into execution, were by no means satisfactory to the council. The
arms were only partially delivered up, and the bond would not at all
go down; and what was the most vexatious part of the business, it
was decidedly rejected by some eminent lawyers in the capital, and
several of the chief nobility in Fife, Stirlingshire, and Teviotdale. The
report from Lanark, too, was provoking beyond measure; of two
thousand nine hundred heritors, only nineteen of the smallest
complied.
Perceiving at length that the opposition was too extensive, and
based upon principles which could not be sneered at as fanatical,
Lauderdale is said at one of their meetings to have bared his arm in
fury, and sworn by Jehovah that he would force them to take the
bond. But it was to be tendered in another shape, under the guise of
a legal quibble—probably the new Lord Advocate might have had
the merit of suggesting it; for certainly the scheme was more like the
device of a pettifogging attorney than the counsel of a sound
statesman. When a deadly feud had arisen between two neighbours,
as the ancient Scots were an ardent irascible race, it generally
terminated fatally, and not infrequently involved the whole relations in
a species of domestic warfare, which lasted for generations till one
party was worn out or gave in. To prevent these consequences, it
had been enacted in the reign of James II. and confirmed in the 7th
parliament of James VI., that an individual who feared bodily harm
from another, by an application upon oath to a magistrate, might
obtain a “writ of law-burrows” to oblige the person of whose violence
he was apprehensive, to give security that he should keep the
peace, nor “skaith or damage” the applicant. This legal pledge, a
wise and necessary precaution to insure personal safety in turbulent
times, such as the frequent minorities of the Jameses had produced,
the council contrived to convert into a more oppressive obligation
than even the bond itself. Assuming an absurd legal fiction, that his
majesty and his government were put in bodily fear by the persons
who refused to take the bond, they issued writs of law-burrows, not
only against individuals, but against a county.
By additional instructions, the committee were directed to pursue
all heritors who had not taken the bond for all conventicles kept on
their lands since the 24th of March 1674—the fine to be exacted for
each conventicle being fifty pounds. They were also to summon
them and their tenants, &c. to answer for building, or being present
at the building, of any preaching-house—the fine imposed to be
arbitrary. No nobleman or gentleman who refused the bond was to
be allowed to wear his sword, and whoever delayed beyond six days
to appear at the council-bar, after they were summoned, were to be
amerced in two years’ valued rent, and were likewise liable for the
delinquencies of their tenants and servants.
Immediately after this, a number of gentlemen in Ayrshire were
summoned before the committee, upon a charge of law-burrows; but
while they made the strongest professions of loyalty, they steadily
resisted putting their hands to a deed which they deemed illegal,
irreconcilable with their profession as Presbyterians, and
impracticable with respect to all their retainers and dependents. One
of them, unfortunately his name is not preserved, who had
indignantly refused, on being told by the president that, if he
continued obstinate, the Highlanders, who had been quartered upon
a neighbouring gentleman’s property till he had complied, would be
transferred to his till he became convinced of the propriety of
obedience, replied, “he had no answer to that argument; but before
he would comply with the law-burrows, he would rather go to prison.”
Lord Cochrane was next before them. He had been served with an
indictment, charging him with encouraging field and house
conventicles, and conversing with intercommuned ministers; in a
word, he or his wife, or some of his family or tenants, had rebelled
against the king by attending upon the preaching of the gospel,
impiously proclaimed by men who owned no bishop, and who wore
no surplice; and was called to answer to the charge within twenty-
four hours. His lordship objected to the shortness of the time allowed
to answer, and contended that, as his indictment contained a capital
charge, it was necessary the “diet” or meeting should be prolonged,
that he might have time to consult with his advocates; and, when
called to answer upon oath, declined to do so, “no man being bound
by any law to give his oath, where the punishment may be in any
way—corporis afflictiva, quia nemo est dominus membrorum
suorum”—destructive to the body, because nobody is lord of his own
body. The committee told him their diets were peremptory, i. e. their
meetings were fixed for certain times, and therefore the accused
were bound to answer upon the instant; but, at the same time,
passed an interlocutor, restricting the libel to an arbitrary
punishment, i. e. declaring that whatever his lordship might depone
should never infer a capital infliction.
His lordship next pleaded that, by an act of council so late as the
5th of October last year, all libels against conventicles were
restricted to a month backwards, consequently he was free. He was
asked if he had brought an extract of the act? He replied he had not,
but it was well enough known, and referred to their lordships
themselves and the public prosecutor. They all declared they knew
nothing about it. He then begged that the clerk might be questioned;
but they would not allow their clerk to give evidence in that matter;
and he was again called upon to swear, otherwise he would be held
as confessed. Seeing at last that nothing else, no not even their own
acts, would avail, he made oath “that he was free of all conventicles,
as were all his servants, to the best of his knowledge.” Some new
queries were now put to him by the Lord Advocate. He refused to
answer to any matters not alleged against him in his indictment, and
appealed to their lordships. They gave it in his favour! finding “that
he was not obliged to depone to any thing not contained in his
indictment;” and the court adjourned.
When they met in the afternoon of the same day—21st February
1678—Lord Cathcart, Sir John Cochrane, with some others, among
whom was the Laird of Kilbirnie, refused the bond upon the same
grounds—the act of council, October the 5th. The lords again denied
their knowledge of such an act; but when Kilbirnie, prepared for this,
offered to produce a copy, they would not receive it, saying, if there
was such an act it was superseded by posterior acts. He then
offered to protest against their proceeding without allowing him to
produce it. This the Earl of Caithness opposed, by representing the
danger he incurred in so doing; but when he persisted, his lordship
suddenly adjourned the sederunt, and thus prevented him from
getting it formally entered on the record.
While the committee were denying the provisions of their own acts
which had the least semblance of moderation, “the Highland Host”
were ravaging the devoted west without mercy.[99] Free quarters
were every where exacted by the militia and king’s forces, although
they received regular pay. But the Highlanders, not content with free
quarters, would march in large bands to gentlemens’ and heritors’
houses, as well as their tenants, and take up their lodgings, and
force the proprietors to furnish them with whatever they chose to
demand, or they would take whatever struck their fancy; and, when
some of their own officers interposed, would present their daggers to
their breasts, and dare them to touch their plunder. They infested the
high-roads in a most ferocious manner, not only robbing the
passengers of their money or baggage, but even stripping them of
their clothes, and sending them to travel naked for miles ere they
could reach home. From the country-folks’ and cottars’ houses, they
carried off pots, pans, wearing-apparel, bedclothes, or whatever was
portable; and, notwithstanding the government had taken care to
order provisions, both officers and men carried off or wantonly killed
the cattle, under pretence that they wanted food, unless they were
bribed by money; yet that did not always avail, the plunderers often
both pocketing the coin and driving the cattle. In some places they
proceeded the horrible length of scorching the people before large
fires, in order to extort a confession, if they suspected they had any
hidden valuables; and to these rapacious, needy hordes, the lowest
necessary utensils of civilized life were precious.
99. Garrisons were ordered to be “planted, 100 foot and 20 horse, in the house
of Blairquhan, Carrick; 50 foot and 10 horse in Barskimming, and the same in
Cessnock. The commissioners of the shire to provide 126 beds, 24 pots, as
many pans, 240 spoons, 60 timber dishes, as many timber cups, and 40
timber stoups; to be distribute to the said garrisons conform to the number of
men; also to provide coal and candle for the garrisons respective.” Act of the
committee of council, Ayr, March 4th, 1678. By an act of the 9th, the
commissioners were ordered to furnish the garrisons with necessary
provisions, such as meat and drink, and to say at what prices they would
agree to do so; but having failed, the committee took the business into their
own hands, and ordered the prices to be as follow:—Hay, per stone, 2s.
Scots; straw, per threave, 4s.; oats, per boll, 50s. in Carrick—55s. in Kyle;
meal, per boll, 5 merks; malt, per boll, £5; cheese, per stone, £1, 10s.; pork,
per stone, £1, 16s.; French grey salt, per peck, 10s.—Scots ditto, 5s.; butter,
per stone, £2, 8s.; each dozen of eggs, 1s. 4d.; milk, per pint, 1s.; each hen,
4s.; each mutton bulk, £2. These prices, reduced to our currency, at 1s.
Scots—1d. sterling; £1 Scots—1s. 8d. sterling,—will show the scarcity of
cash in these days, and its relative value to the present prices.

In other villages, the meanest soldiers exacted sixpence sterling a-

day, and the guards a shilling or merk Scots; their captains and
superior officers, half-crowns and crowns at their discretion, or as
they thought the poor people could procure it, threatening to burn
their houses about their ears if they did not produce sufficient to
answer their demands; besides money, the industrious, sober,
religious peasantry were constrained to furnish brandy and tobacco;
and, what was scarcely less painful, were obliged to witness their
filthy brutal excesses. Then, again, some of the ruffians would levy
contributions in order, as they pretended, to secure the payers from
plunder; yet, after they had filched them of their money, at their
departure rifled them of all they could find the means of transporting.
Their insolences to the females, our historians have drawn a veil
over, as too abominable to admit of description.
 An instance or two of their wanton waste are narrated, from which
the extent of the damage occasioned by their visitations, may, in
some measure, be guessed at, especially as the perpetrators were
not the most savage of the crew, but men from whom better things
might have been expected. The Angus-shire troop of gentlemen
heritors, or yeomanry cavalry, as they would now be called,
commanded by the Laird of Dun, was quartered upon the lands of
Cunningham of Cunninghamhead, then a boy at school, who, even
by the strictest interpretation of the strictest acts of this detestable
period, could not be held liable to such an infliction. Pretending that
the country houses, upon which they were billetted, were not
sufficiently comfortable for persons of their situation, these genteel
troopers obliged the tenants to pay to each five pounds sterling for
“dry quarters;” but, after they had received the money, they either
remained themselves, or sent three or more footmen of the wildest
Highlanders to supply their place. A cornet in this troop, Dunbar of
Grange, nephew to the commander, perceiving that the entrance to
the old tower of Cunninghamhead was strongly secured by an iron
grating before a massy wainscoat door—most likely expecting to find
some treasure secreted within—called for the keys that he might
open and examine the interior. The keeper being absent, he was told
that there was nothing of any consequence in the place, for the
second story was used as a granary, and the rest of the building was
unoccupied. At this, in great wrath, after abusing the people, he set
fire to the door, and blew up the grating with gunpowder. Having
forced his way in to the foot of a staircase, after ascending, he found
himself opposed by a second stout door upon the girnel, also grated;
but full of the hope of plunder, he was not thus to be disappointed,
and removed this obstruction in the same summary manner. Sure of
the prize, he rushed in with his attendants, all equally eager with
himself for a share of the spoil, but they saw nothing except oatmeal,
as they had been told, which, in their rage at finding themselves
“begunkit,” they either “fyled” with their boots and shoes, all clay from
the open field, or scattered about and destroyed, under pretence of
searching for arms. The loss to the minor, as the greater part of the
rents then were paid in produce, has not been mentioned in money;
but as the deed happened in the month of February, the pecuniary
value, although then high, might not be equal to the detriment its
destruction must have occasioned.
William Dickie, a merchant in Kilmarnock, had nine Highlanders
quartered upon him six weeks, during which he was obliged to
furnish them with meat and drink, and, not having sufficient
accommodation for them in his own house, was forced to pay for
“dry quarters,” i. e. good beds, in some other, as were numbers
besides. When they went off, they carried away with them several
sacks full of household stuff, and goods, and a hose full of silver
money; and, before leaving, broke two of the honest man’s ribs—
stabbed his wife in the side, who was big with child at the time, and
otherwise so terrified her, that she died in consequence.
These were the apostles of Episcopacy! and their employers have
even found apologists in our own day; but if they who, by preaching,
and prayer, and suffering, attempted to diffuse the knowledge of the
gospel in their country, were or are called fanatics, by what epithet
shall honest indignation designate the miscreants who could
endeavour by such means to obtrude an illiterate, ignorant,
dissolute, and shameless priesthood, upon an unoffending, and
comparatively uncorrupted, part of the population? It is, however,
pleasant to notice that, among the Highland leaders, there were
several exceptions. The Marquis of Atholl, and the Earl of Perth, are
particularly mentioned as having pled the cause of justice and
humanity at the council-board of the committee, though, unhappily,
their pleadings were overborne by numbers—and their men
comported themselves no better than their comrades.
The whole may be summed up in the words of a contemporary
writer, an eyewitness, quoted by Wodrow:—“It is evidently apparent
that the proceedings of these few months by-past, are a formed
contrivance (if God in mercy prevent not) to subvert all religion, and
to ruine and depopulate the country—they are open and evident
oppression, public violence, and robbery, and invasion of the person
and goods of a free and loyal people—a violation of the ancient
rights and privileges of the lieges—and a treacherous raising of
hatred and discord ’twixt the king and his subjects—and
consequently, manifest treason against the commonwealth and the
king’s majesty. In a word, when considered in its full extent, and in all
its heinous circumstances, it is a complication of the most atrocious
crimes that almost ever could have been conceived or perpetrated.”
The losses sustained by the county of Ayr alone, were estimated, in
an account intended to be laid before the king if he would have
received it, at one hundred and thirty-seven thousand, four hundred
and ninety-nine pounds, six shillings, Scots—or eleven thousand,
four hundred and fifty pounds sterling; and this being only what could
be ascertained and proved, was not supposed to be one-half of the
real amount of damage inflicted.
While the Highlanders were plundering openly, the committee
were equally busy in their vocation—fining or imprisoning all who
came before them, whether upon charges of attending conventicles,
or not signing the bond. On the 22d February, the Earl of Cassilis
appeared, and resolutely refusing to subscribe what he considered
as illegal in itself, and impossible for him to perform, was ordered to
answer next day to an indictment accusing him of high crimes and
misdemeanours, in frequenting conventicles, or allowing them upon
his grounds. His lordship did accordingly appear, and denied upon
oath, the truth of the averments in the libel, only, if there had been
any conventicles upon his ground, or if his tenants had been at them,
he knew nothing but by hearsay, he himself having never seen any
such meetings, nor any of his tenants present at them. Immediately
upon his refusal to subscribe, although he had cleared himself by
oath of all the crimes laid to his charge, the lords appointed a
messenger to charge him with letters of law-burrows, to pledge
himself in the books of the privy council, that his wife, children, men,
tenants, cottars, and servants, should not be present at conventicles,
or any other disorderly meetings, under a penalty of double his
valued yearly rent; and, in case of failure, he was to be denounced a
rebel within six days. Hereupon he wrote to their lordships entreating
a week’s delay, but they refused to grant him even this small favour,
on which he immediately repaired to Edinburgh to offer the council
every satisfaction that could be legally required. But upon his coming
thither, a proclamation was issued, commanding all noblemen,
heritors, and others of the west country, to depart from the capital,
and repair to their own houses within three days, before which time,