A Five-axis Robotic Motion Controller for Designers

ABSTRACT
Andrew Payne

T h i s p a p e r p r o p o s e s t h e u s e o f a n e w s e t o f s o f t w a r e t o o l s , c a l l e d F i r e f l y, p a i r e d

Harvard University

w i t h a l o w - c o s t f i v e - a x i s r o b o t i c m o t i o n c o n t r o l l e r.

This serves as a new means

for customized tool path creation, realtime evaluation of parametric designs using
forward kinematic robotic simulations, and direct output of the programming
language (RAPID code) used to control ABB industrial robots.

Firefly bridges

t h e g a p b e t w e e n G r a s s h o p p e r, a v i s u a l p r o g r a m m i n g e d i t o r t h a t r u n s w i t h i n t h e
Rhinoceros 3D CAD application, and physical programmable microcontrollers like
the Arduino; enabling realtime data flow between the digital and physical worlds.
The custom-made robotic motion controller is a portable digitizing arm designed
to have the same joint and axis configuration as the ABB-IRB 140 industrial robot,
enabling direct conversion of the digitized information into robotic movements.
Using this tangible controller and the underlying parametric interface, this
paper presents an improved workflow which directly addresses the shortfalls of
multifunctional robots and enables wider adoption of the tools by architects and
designers.
K e y w o r d s : r o b o t i c s , C A D / C A M , f i r e f l y, d i r e c t f a b r i c a t i o n , d i g i t i z i n g a r m .

162
acadia 2011

_proceedings

integration through computation

1 I n trodu c tion
There are literally thousands of different applications currently performed by industrial
robots. W ith more than one million multifunctional robots in use worldwide, they have
become a standard in automation (Gramazio 2008). The reason for their widespread
use lies in their versatility; they have not been optimized for one single task, but can
perform a multiplicity of functions. Unlike other computer numerically controlled (CNC)
machines - which are task-specific - robots can execute both subtractive and additive
routines. Among other operations, they can load, unload, place, cut, bend, stack, spray,
weld, and mill.
However, industrial robots were not designed to be user friendly their size, support
infrastructure, and programming demands make them ill-suited for untrained operators.
They can also be dangerous. Industrial robots are often kept in isolated areas in order
to protect human workers.

In fact, many robots are painted security orange (RAL

color reference 2003 ) to remind us that this complex piece of heavy machiner y is not
particularly obser vant of misplaced hands or feet and requires caution when operating.
(Edgar 2008).

Lastly, the design-to-fabrication workflow for industrial robots has

traditionally been a slow and cumbersome process (see Section 3). Machine tooling,
kinematic simulations, and robotic movement programming often require intimate
knowledge of scripting and manufacturing processes, all of which limit the utilization of
such tools by the architect/designer.
Despite considerable advances in digital software used to control robots, there often
remains a detachment between the designer and the final output.

Bringing physical

input and output closer together through purpose-built tools for fabrication can enable
many new creative opportunities for designers. Working from obser vations about the
way architects design, this paper presents a prototype for a 3D drawing tool that takes
realtime input and translates that into machine code for robotic fabrication. The purpose
of this paper is not to suggest that the proposed workflow is a ready-made solution
to replace existing fabrication process; rather the work should be seen as a proof
of concept that could enable wider use of digital fabrication tools by architects and
designers. Finally, the paper concludes with a look at the results and limitations of the
proposed system and outlines a number of considerations for future development.
2

R e la te d Wor k

The fields of digital fabrication and Tangible User Interface (TUI) design have seen a
dramatic increase in activity in recent years. However, despite advancements made in
both fields, there have only been a handful of projects which bridge the gap to embody
direct physical input and output. John Frazer s Flexible Intelligent Modeling System is
a notable early example which was developed in the early 1980s as a response to the
existing CAD systems which at that time were clunky and cumbersome (Shaer 2010).
Frazer and his team developed a 3D modeling system where users built objects by
stacking up sensor embedded cubes or blocks into various configurations. The computer
then deduced location, orientation, and type of each component in the system and
output a 2D representation of the pattern to a plotter.
More recently, there have been a number of projects which have explored various gestural
interfaces as a form generator for digital fabrication. One notable example is the Sketch
Fur niture system developed by Front Design which uses a professional motion-capture
system to create a new way to materialize free hand sketches (Front 2006). Designers
can create 3D digital geometr y by drawing lines in physical space.

This data is then

processed in the computer and converted into a mesh which can be fabricated at full
scale using a Selective Laser Sintering (SLS) process. In another example, the Spatial
Sketch application uses a stereo-vision 3D input system to capture gestural movements
which can be translated into a series of 2D profile cur ves to be output to a laser cutter
for final fabrication (W illis 2010). Finally, the Shaper prototype developed at Carnegie
Mellon University in 2010 uses touch-screen technology to interactively control a threeaxis CNC machine which deposits expanding polyurethane foam material according to
the user s finger placement (W illis 2011). While these works offer unprecedented fluidity
between the design interface and final output, they often require a significant investment

163
integrative tools and techniques

Fig. 1

in time and equipment in order to be used effectively. Additionally, none of the projects
mentioned above have been designed specifically for use with multi-functional robots.
Making an inexpensive and intuitive gestural interface for robotic control became the
primar y driver for many of the design decisions made during the development of this
project.
3

Ex i sti ng Desi g n-to -Ro bo ti c F abri cati o n Wo rk fl o w

Despite significant progress made in CAM software over the last decade, the existing
design-to-fabrication workflow can be difficult to traverse for architects and designers.
In the traditional sense, design conception typically occurs as a 2D sketch which is
then converted into a 3D CAD model following one of two trajectories. The architect or
designer can build a 3D CAD model directly from the sketches; refining certain design
parameters as the idea becomes more fully developed. Or, an alternative is to create
a physical model from which a 3D model can be captured through some form of 3D
scanning equipment such as a digitizing arm.

A digitizing arm is a device which has

several degrees of freedom and uses sensors to measure the angle of each joint. From
this data, the location (or coordinate) of the tool tip can be calculated. A designer can
trace physical objects by moving the tool tip along the surface of an object to return a
string of point coordinates which can be processed by a computer.
Once the digital model has been made, the architect or industrial designer must choose
a fabrication methodology from which they will execute their design (Figure 1). For
working with ABB robots, this means exporting the CAD model into a format (.iges, .stl,
.3ds) that can be understood by the various ABB supported software applications such
as MasterCAM with the Robot Master plug-in or the stand alone program Robot Studio.
While these applications offer sophisticated algorithms for tool path creation, collision
detection, and singularity analysis; they are often only employed by trained operators
as they require a level of manufacturing education and scripting knowledge that
architects and designers typically do not have. This type of workflow introduces a level
of detachment in the design process. The CAM operator, an intermediar y between the
designed input and final physical output, must make decisions based on prior experience
in order to make the design a reality.

However, if errors are found in the fabrication

methodology either from inaccuracies discovered in the CAD model or from simulation
feedback the design must be modified and the entire process begun anew. This can
significantly increase production times and consequently the overall costs of the project.
4

The Tang i bl e C o ntro l l er

Thi s p ap er pre se n ts a prototype for a tan gible c on trolle r an d a n e w parame tric in te rfa ce
Figure 1. Existing design-to-fabrication workflow
for ABB industrial robots

s p eci fi cal l y de sign e d to stre amlin e th e robotic fabric ation proc e ss for arc h ite c ts a nd
d es i g ners . T h e low-c ost c u stom-made digitizin g arm was c on stru c te d u sin g th e sa m e

164
acadia 2011

_proceedings

integration through computation

Fig. 2

j o i n t a n d axi s co nfi g urat i o n as t he ABB -IR B 140 in du strial robot; e n ablin g dire c t
c o n v e r s i o n o f t he d i g i t i zed i nfo rmat i o n i nt o ro botic move me n ts (F i g ure 2).
A d e g r e e o f freed o m (DOF) i s a g eo met ri c d e fin ition of fre e dom of move me n t e ith e r
a l o n g a n a x i s i n s p ace o r ro t at i o n ab o ut an axi s in spac e (Pottman 2007). T h e digitizin g
a r m h a s f i v e j o i nt s o r p i vo t p o i nt s each corre spon din g to on e DOF on th e ABB
r o b o t . E a ch j o i nt o n t he d i g i t i zi ng arm us es a h igh pre c ision pote n tiome te r to me asu re
t h e a n g u l a r ro t at i o n ab o ut each axi s .

Ro t ar y pote n tiome te rs work by c h an gin g th e ir

r e si s t a n c e as t he cent ral s haft i s t urned t o t he le ft or righ t. T h is c h an ge s th e voltage

w h i c h i s r ead b y t he anal o g -t o -d i g i t al co nverte r on th e mic roc on trolle r re tu rn in g a
s e n s o r v a l u e b et ween 0 and 1 0 2 3 . Thes e p art i c u lar pote n tiome te rs we re able to rotate

Fig. 3

u p t o 3 4 0 , s o t he ang ul ar p o s i t i o n (i n rad i ans) was de te rmin e d u sin g th e followin g

equation:

Angle = (PotVal * (1023/340)) * (/180)

I n a d d i t i o n t o t he fi ve anal o g s ens o rs (p o t ent i ome te rs), th e digitizin g arm is e qu ippe d

w i t h a t o o l t i p ci rcui t b o ard wi t h t wo p us h b utton c on trols.

T h e se allow th e u se r to

e a si l y r e c o rd o r res et t he d i g i t i zed i nfo rmat i o n on th e fly (F i g ure 3). All of th e se in pu ts

a r e c o n n e c t ed t o a cus t o m d es i g ned ci rcui t board wh ic h proc e sse s th e in formation
a n d se n d s a fo rmat t ed s t ri ng o f i nfo rmat i o n o ver th e se rial port to th e virtu al in te rfac e .
5

Th e Pa r a me tr ic I n te r fa c e

T h e d e v e l op ment o f a new p aramet ri c i nt erface was vital to e n able wide r adoption of

r o b o t i c f a b ri cat i o n t echni q ues b y archi t ect s a n d de sign e rs. T h e followin g se c tion s
h i g h l i g h t so me o f t he feat ures o f t he p ro p o s ed syste m.
5.1

GRASSHOPPER AND FIREFLY

O v e r t h e l as t 1 5 years , t here has b een a mi gration in de sign prac tic e s toward th e

u t i l i z a t i o n o f p aramet ri c mo d el i ng . The t erm re fe rs to a me th od of digitally mode lin g a
s e r i e s o f d e s i g n vari ant s who s e rel at i o ns hi p s t o e ac h oth e r are de fin e d th rou gh on e or
s e v e r a l p a ramet ers whi ch t hen fo rm a p arametric spac e wh ic h may c omprise doze n s
o r t h o u sa n d s o f rel at ed b ut d i s t i nct fo rms (Lagios 2010).

Eve n th ou gh parame tric

m o d e l i n g beg an as a means t o d evel o p new an imation te c h n iqu e s in th e gamin g/film

i n d u st r i e s i n t he mi d 1 9 9 0 s , t here has b een a c on flu e n c e of disc iplin e s in re c e n t ye ars
w h o h a v e emb raced t hi s t yp e o f d es i g n met h odology primarily du e to its ability to
c r e a t e d e s i g ns t hat can q ui ckl y b e ad ap t ed o r modifie d.

Figure 2. The digitizing arm was designed to

have the same joint and axis configuration as the
ABB-IRB 140 robot
Figure 3. Two custom circuit boards were
designed to stream the sensor information
from the digitizing arm into the 3D parametric
environment

165
integrative tools and techniques

Fig. 4

One recent example is Grasshopper - a visual programming language developed by

David Rutten at Robert McNeel & Associates which provides an intuitive parametric
interface to the Rhino CAD application. In Grasshopper, programs are created by dragging
components onto the editor (called the canvas). The outputs to these components are
then connected to the inputs of subsequent components creating an acyclic graph
of information flow (Figure 4).

Grasshopper enables a way for designers to look at

design problems as a set of sophisticated relationships and to map those relationships

graphically and programmatically into a system that allows them to interactively play with
alternatives (Day 2009).
As a generative modeling tool Grasshopper offers unprecedented capabilities, but by
default, it lacks the ability to communicate with hardware devices such as programmable
microcontrollers or other haptic interfaces. Firefly is a set of tools which bridge the gap
between digital and physical worlds by enabling direct serial communication between
hardware devices and the Grasshopper plug-in. Firefly allows realtime access to each
of the sensors mounted on the digitizing arm; forming the parametric basis for the
kinematic robotic simulation.
5.2

ROBOTIC SIMULATION

Ki nemat i c simu lation of in du strial robots h as be c ome an importan t me an s for t he

i ncreas ed effic ac y of robotic fabric ation . Kin e matic s pe rtain s to th e motion of bodies
i n a ro b o t i c me c h an ism with ou t re gard to th e forc e s/torqu e s th at c au se th at mot ion
(S i ci l i ano 2 008). Sin c e most CAM simu lation tools start with th e in pu t of on e or more
t arg et p o i nts (or goal obje c ts), th e y ofte n e mploy an i n ve rse ki n e mati c solu tion or
t he p ro ces s of de te rmin in g all of th e join t an gle s an d c on figu ration s in orde r to re a ch
a d es i red position . T h e re ve rse proc e du re is c alle d forward ki n e mati c s. If give n all of
t he rel at i ve an gle s of e ac h join t an d th e le n gth s of e ac h le g; th e tool tip (also kn own a s
t he end effec tor) c an be fou n d by pe rformin g a se rie s of matrix tran sformation s on e a ch
b o d y i n t he robotic me c h an ism.

A c u stom-made VB.N ET sc ript in side Grassh opp er

us es each of th e pote n tiome te r valu e s from th e digitizin g arm to c re ate a forwa rd

ki nemat i c s imu lation of th e ABB robot. As th e u se r move s th e digitizin g spac e trac ing
p hys i cal o bje c ts or c re atin g th e ir own c u stom tool path s they can see an immediate
simulation of the robot performing the same actions (Figure 5). On e of th e pu sh bu tto ns
l o cat ed o n th e tip of th e digitizin g arm allows u se rs to re c ord th e tool path in formation
whi ch i s d i re c tly tran slate d in to RAPID c ode th e programmin g lan gu age u se d by a ll
AB B ro b o t s . T h e oth e r pu sh bu tton re se ts or c le ars th e re c orde d data.
5 .3
Figure 4. A portion of the Grasshopper definition:
1) Read sensor values via Firefly 2) Tool definition
3) Custom VB.NET component 4) Final RAPID
code output

ROBOTIC PROGRAMMING

A cri t i cal a spe c t of th e propose d workflow was to be able to ou tpu t th e robot ic

p ro g rammi n g c ode s in orde r to qu ic kly te st th e re c orde d move me n t patte rn s on the
larger industrial robot.

RAPID is the high-level programming language used to

166
acadia 2011

_proceedings

integration through computation

control all ABB robots and entire books have been written about how to generate
your own custom programs. This section does not attempt to replicate the existing
literature; rather serves to outline a few of the features which were implemented in
this project.
Like other computer controlled machines, robots require a set of instructions
i n o r d e r t o m a k e t h e m r u n p r o p e r l y.

While there are many different data types,

operators, and functions that can be included in a set of codes; there exists two
main declarations to every program: 1) a statement that defines one or more target
points (called robtargets) to accurately locate the end effector in space and 2) a set
of movement commands that tell the robot how to move to those positions. Each
composite data type requires their own set of parameters, or arguments - as seen
in Figures 6 and 7. All of these definitions were created on the fly as the digitizing
a r m w a s c o n t r o l l e d b y t h e u s e r.
A robtarget is a composite data type composed from a predefined number of values.
Each robtarget is defined by its name, absolute position as XYZ coordinates,
rotation or orientation of the robot as four quaternion values, joint configurations for
axis# 1, 4, 6, and cfx, and finally any extra external axis configurations (Figure 6).

Fig. 5

Each robtarget is given a unique identification each time the solution is recomputed.
The position information can be calculated by performing the matrix transformations
on all of the joints to determine the location of the tool tip in space. Quaternions
provide a convenient mathematical notation for representing the orientation and
rotation of objects in three dimensions. Space would not permit the full derivation
of the quaternion math used in this project, but if given the roll = , pitch = , and
yaw = of the tool tip; then the four quaternion values can be found using the

Fig. 6

following equations.
q0 = cos(/2)*cos(/2)*cos(/2)+sin(/2)*sin(/2)*sin(/2)
q 1 = s i n ( / 2 ) *c o s ( / 2 ) *c o s ( / 2 ) - c o s ( / 2 ) * s i n ( / 2 ) * s i n ( / 2 )
q2 = cos(/2)*sin(/2)*cos(/2)+sin(/2)*cos(/2)*sin(/2)
Fig. 7

q3 = cos(/2)*cos(/2)*sin(/2)-sin(/2)*sin(/2)*cos(/2)
The configuration data is used to identify the current quadrant of the robot axis for
joints 1, 4, and 6.

The last configuration number is used to select one of eight

predefined robot configurations.

L a s t l y, a n y e x t e r n a l a x i s c o n f i g u r a t i o n s c a n b e

supplied to locate the position of any external axis.

The value 9E9 is defined for

axes which are not connected.

Movement instructions tell the robot how to move from its current location to a
s p e c i f i e d r o b t a r g e t ( F i g u r e 7 ) . P r o v i d e d w i t h s i x o r m o r e D O F, t h e r o b o t c a n m o v e
in any number of ways and the first argument in the move command specifies the
type of desired movement. In this example, the robot will attempt to move linearly
to the next robtarget. The velocity of the end effector is measured in mm/s and can
be determined by taking the distance from the current tool location to its previous
position and dividing it by the time interval between recordings.

The zone data

d e s c r i b e s t h e p o s i t i o n a c c u r a c y, w h i l e t h e t o o l a n d w o r k o b j e c t s t o b e u s e d a r e
defined elsewhere in the program.
6 R e su lts
As of this writing, there have only been a limited number of tests that have been
c o n d u c t e d u s i n g t h e f i v e - a x i s r o b o t i c m o t i o n c o n t r o l l e r. H o w e v e r, t h e i n i t i a l r e s u l t s
suggest that the proposed direct-to-fabrication process could prove to be a viable
alternative to existing robotic workflows. Because the digitizing arm was designed
to have the same proportions and axis configuration as the larger industrial robot,
it provides an intuitive interface for working with the robot. Designers immediately

Figure 5. The interface provides a fast and intuitive

way to create custom tool paths or to simulate
robotic manufacturing processes

understand that the movements they record using the digitizing arm will be

Figure 6. Definition of a robot target

immediately translated into robotic movements, while the realtime visual feedback

Figure 7. Definition of a movement command

167
integrative tools and techniques

from the simulation further reinforces this fact. Given that robots can support any
number of off-the-shelf or custom-made end effectors, the potential applications
are limitless (Figure 8).
There were a number of factors in the design of the controller which contributed
to the fidelity of the final robotic output and the results from these tests helped to
identify areas of improvement. The decision to use high precision potentiometers,
as opposed to the more accurate method of optical encoding, proved to be the
primary limiting factor in the resolution of the final output.

The choice was made

based on costs, as potentiometers are considerably cheaper than optical encoders

and the premise of the project was to provide a reliable low-cost tangible solution
t o r o b o t i c f a b r i c a t i o n . H o w e v e r, i f c o s t s w e r e n o t a f a c t o r, t h e f i d e l i t y o f t h e p r o j e c t
could be greatly enhanced by using more accurate sensors for the joint angles.
S e c o n d l y, t h e d e s i g n o f t h e p h y s i c a l p r o t o t y p e w a s a f i r s t p a s s a t t h e d e s i g n o f
a sturdy five-axis digitizing arm.

During design development, some mechanical

aspects of the design were discovered which could be improved upon in future
iterations. Oiled bronze bearings were used for each joint on the digitizing arm, but
the connection detail for joint 4, or the wrist, deteriorated to a small degree over time
and became a potential source of error in the system.

Likewise, the stabilization

springs located on the top and back of the digitizing arm added too much tension
to the system; making it somewhat difficult to move the arm into various positions.
While they did provide the appropriate amount of force to leave the arm in a 90
r e s t i n g p o s i t i o n w i t h o u t f a l l i n g o v e r, p e r h a p s a n o f f s e t c o u n t e r w e i g h t w o u l d b e a
more elegant solution.
7 Co ncl usi o n
This paper presents a new tangible controller and optimized workflow for robotic
fabrication.

Although there has been considerable progress made in the digital

tools used to control robots, there is an identifiable problem in the existing designto-fabrication process. The physical articulation of embodied input and output
through purpose-built tools for fabrication can allow for wider adoption and new
creative opportunities by architects and designers.
Fig. 8

In turn, this will help re-

establish the relationship between designers and the physical fabrication process.
The proposed system allows for a fast and intuitive way to create custom tool
paths, simulate robotic kinematics, and production of RAPID code for controlling an
ABB-IRB 140 industrial robot.
Ac kn owle d g ements
This project was developed as part of two graduate level courses taught at the
Graduate School of Design at Harvard University and the Media Lab at MIT and
could not have been realized without the tremendous support from the faculty and
teaching assistants at both universities.

I would like to thank Martin Bechthold,

Director of the Design Robotics Group at the GSD and Neil Gershenfeld, Director
of the Center for Bits and Atoms at the Media Lab for their guidance and support
d u r i n g t h e d e v e l o p m e n t o f t h i s p r o j e c t . L a s t l y, I w o u l d l i k e t o t h a n k P r o f . P a n a g i o t i s
Michalatos for his mathematical and programming assistance in the development of
the parametric interface.
R e fe re n c e s
Day, M. (2 009). Rh in o Grassh oppe r, AEC Magazin e , X3DMe dia, h ttp://ae c mag.c om .
E d g ar, B . L . (2008). A Sh ort Biograph y of KR150 L110. In Di gi tal Mate ri al i ty i n
A rchit ect ure e ds. F. Gramazio an d M. Koh le r, 49-56. Bade n : Lars M lle r Pu blish e rs .
FR ONT Sketc h Fu rn itu re . (2006.), h ttp://www.de sign fron t.org.
Figure 8. Various end effectors can be purchased
from robotic suppliers or you can make your own
such as this spring loaded pen attachment

Gramazio, F., and M. Kohler. (2008). Digital Materiality in Architecture. Lars Mller Publishers.
L a g i o s , K . , J . N i e m a s z , a n d C . F. R e i n h a r t . ( 2 0 1 0 ) . A n i m a t e d B u i l d i n g P e r f o r m a n c e

168
acadia 2011

_proceedings

integration through computation

Simulation

(ABPS)

Linking

Rhinoceros/Grasshopper

with

Radiance/Daysim,

A c c e p t e d f o r P u b l i c a t i o n i n t h e P r o c e e d i n g s o f S i m B u i l d 2 0 1 0 , N e w Yo r k C i t y,
August 2010.
P o t t m a n , H . , A . A s p e r l , M . H o f e r, a n d A . K i l i a n . ( 2 0 0 7 ) . A r c h i t e c t u r a l G e o m e t r y .
E x t o n , PA : B e n t l e y I n s t i t u t e P r e s s .
S h a e r, O . , a n d E . H o r n e c k e r. ( 2 0 1 0 ) . Ta n g i b l e U s e r I n t e r f a c e s : P a s t , P r e s e n t a n d
F u t u r e D i r e c t i o n s . I n F o u n d a t i o n s a n d Tr e n d s i n H u m a n C o m p u t e r I n t e r a c t i o n 3 ( 1 2): 1137.
S i c i l i a n o , B . , a n d O . K h a t i b . ( 2 0 0 8 ) . S p r i n g e r H a n d b o o k o f R o b o t i c s . S p r i n g e rVe r l a g N e w Yo r k I n c .
W i l l i s , K . D . D . , J . L i n , J . M i t a n i , a n d T. I g a r a s h i . ( 2 0 1 0 ) . S p a t i a l S k e t c h : B r i d g i n g
Between Movement & Fabrication. Paper presented at Proceedings of the Fourth
I n t e r n a t i o n a l C o n f e r e n c e o n Ta n g i b l e , E m b e d d e d , a n d E m b o d i e d i n t e r a c t i o n , 2 0 1 0 :
Cambridge, MA.
Willis, K. D. D., C. Xu, K. J. Wu., G. Levin, and M. D. Gross. (2011). Interactive
Fabrication: New Interfaces for Digital Fabrication. Paper presented at Proceedings
o f t h e F i f t h I n t e r n a t i o n a l C o n f e r e n c e o n Ta n g i b l e , E m b e d d e d , a n d E m b o d i e d
Interaction, 2011: Funchal, Portugal.

169
integrative tools and techniques