SDH-2 MULTI-FINGER GRIPPER AND KUKA ROBOT

ANIMATIONS FOR REAL-TIME SIMULATIONS OF ROBOT-

HAND-CONTROL STRATEGIES USING OPEN SOURCE
SOFTWARE BLENDER

Thomas Haase
Institute for Process Control and Robotics
Karlsruhe Institute of Technology (KIT)
D-76131 Karlsruhe, Germany
e-mail: [email protected]

Abstract
This Paper introduces the construction and the necessity of
simulation surroundings for the construction of robot-hand-control system
simulations with the Open Source software Blender. The advantages of the
given game engine and the runtime export are worked out. All designed
Blender animations can be integrated into Real-Time simulations.
Necessary communication interfaces that use the given Python API are
described for both RTAI Linux and Real-Time Windows.

Introduction
With the development of SCHUNKs Multi-finger gripper SDH-2 the first
robot hand which can achieve industrial demands is available. Current
research projects deal with the development of necessary software modules
to integrate this robot - hand system into first industrial applications. Due to
the financial worth of each robot hand and the fear of many developers to
damage the gripper in real experiments the developing speed is slow. With
suitable simulation environments this problem can be solved. There are
different reasons that result in a higher developing speed: 1. direct
interception of arithmetic and design errors. This leads to less down-time
due to repairs. 2. Fears of practical tests series are reduced, because the
system could be tested in its behavior. 3. In spite of missing hardware a
parallel developing becomes possible, so that the forthcoming development
work will not take many additional years. A high research speed is
necessary. The simulation environment introduced in this work should
achieve exactly this. The exchange of data from and to the simulation is
especially important. This determines the applicability and the value of the
simulation in the process of development. It also decides whether and how
these simulations can be integrated into Real-Time environments.
 Requirements and Options

The simulation environment will be part of the SCHUNK product SDH-2.

Due to the fact, that it is provided to each customer of SCHUNK, the
software has to fulfill some significant requirements. The software must not
cost anything.

Figure 1: KUKA and SDH-2 Simulation environment

She must be accessible to every customer freely. Many programmers use

Linux as an operating system, which also has to be kept in mind. It is
explicitly necessary that importing the original CAD data is feasible.
Therefore, an easy exchange of data by means of a standardized interface is
required. A high announcement speed as well as a clear service is
important. In the following all points are listed once more:
 Open source software
 availability for Linux and Windows operating systems
 CAD data import / data interface
 high announcement for use in Real-Time simulations
 clear service
Professional software solutions like Cinema4D, 3ds Max or Maya are not
an appropriate choice here, as their price might account for a significant
increase in overall costs. AC3D is very cheap professional software for
Linux and Windows operating systems, (Inivis). AV3D offers a socket
communication interface to manipulate object behaviors. The Real-Time
3D graphics engine provides quick views into the created 3D world.
Milkshape 3D (Ciragan) is a low-polygen modeler that is able to read in
usual CAD formats. Milkshape 3D is available for Windows only and
offers no possibilities to exchange data with other programs. Truespace7 is
a product of Caligari (Caligari Corporation), a company acquired by
Microsoft recently. TrueSpace7 offers CAD import, a Visual Basic or
Javascript interface and also a wide range of Real-Time visualization tools.
Unfortunately it is available for Windows only but not for Linux. Blender is
an open source 3D modeler that is available for all major operating systems
(Ton Roosendaal). Blender exhibits an integrated fully fledged Python
interpreter. Thus, all usual interfaces can be used. Blender is able to export
stand alone executables. The integrated game engine permits a quick build
of user controls. Additionally a ``Blender for Robotics'' community exists
(Blender For Robotics). It can be assumed that Blender will be developed
more and more in this research direction. Only Blender fulfils all
conditions. Hence, it will be used for the construction of the simulation
environment.

Goals

A Simulation environment has to be built that may be used on Windows

and Linux operating systems. A user of this simulation not necessarily has
to be able to serve these 3D graphics software. The real experiment set-up
should be illustrated very precisely. Possibilities for the communication and
the data transfer between the simulation and the developed software
modules must be found. Besides, it must be possible to import and export
data.

Blender simulation environment

Figure 1 presents the real system and the developed simulation

environment. One can recognize the representation of the table, the robot
and the SDH-2 gripper clearly. The simulation all together has got more
than 20 degrees of freedom. Just the robot and the hand have a total of 13
DOF. In addition it is possible to change the qualities of the lamps and the
scaffolding. This could be important for optical reactive grasping
simulations. With the programmable camera movement, videos can be
provided. The Robot is controlled in joint space. According to this the
knowledge of the inverse kinematics of the manipulator is essential. An
inverse kinematics within the simulation is imaginable. Unfortunately, the
inverse kinematic comprises no solution, one solution ore multiple
solutions. Sometimes there are several possible sets of joint angles which
could be used to attain a given position and orientation (Craig, 1997).
That's why it must be calculated outside the simulation. Otherwise
possibilities must be found for it so that the user can select the desired joint
angles. This is only hardly possible with Blender. The control of the robot
hand is realized in the same way. There exist two versions of the
simulation. With one it is possible to read in data. The other one can be
navigated with the keyboard and is able to export all data.

The Significance of the Game Engine

The Blender Game Engine is a component of Blender. The game engine

uses a system of mechatronic elements (sensor, controller and actor) to
respond to user inputs and manipulate object behaviors. The game engine
supports Python scripting. That could be used to build up own adjusted
controllers. In addition, a lot of predefined sensors are available, such as
mouse and keyboard inputs. The game engine is able to generate stand
alone executables from build-on models, Figure 2. In this way ready
models can be served without having any Blender knowledge. This is valid
for Windows as well as for Linux operating systems.

Figure 2: Runtime Export and Data Rate

Creating executables is advantageous as the Python scripts are completely

encapsulated and therefore not accessible by the user anymore. Among
other things the executables are able to identify and handle mouse and
keyboard inputs. In this manner a navigation within the 3D world was
realized. By means of the middle mouse key the view can be rotated. The
left and right mouse keys are used to zoom in and out. To use the zoom
capability one should be able to see the mouse pointer in the game. One
reaches this with the following order sequence:
1. import Rasterizer
2. Rasterizer.showMouse(1)

During a rotation the cursor is faded out. In addition, keyboard inputs are
used to change important values such as step sizes, velocities or zoom
levels. All changes and inputs are made visible in the terminal.

UDP Data Exchange

In Python it is feasible to set up socket interfaces. The following listing
demonstrates how to open and close a UDP socket.

The socket package is already a component of the usual python installation.

Lines 4 and 5 create and bind the socket to the specified IP and port. After
the Socket is installed, it is necessary to read in new data repeatedly. The
defined function dataUDP checks for new incoming data. It's recommended
to use a nonblocking socket (socket.setblocking(0)) if the Blender
simulation includes more than this socket script file. Incoming data sets are
stored in a global variable. The function ActDataset returns the actual data
set. This single function has the advantage that it can be called by many
scripts. It is not checked every time whether new data has arrived. This
ensures that all Python script files will work with the same record.

The call of these functions occurs in a file in the Blender model that is
executed in every animation step, Figure 3. In addition to these socket
interfaces, the Blenders Python API makes it possible to specify some
object and joint properties with the help of user defined variables. The
following listing demonstrates how to combine the incoming data with the
properties of a bone.
Line 1 reads in a new data set. Lines 2 and 3 extract a value from the record
and the new orientation matrix of the bone is calculated. The lines 4 and 5
assign the new orientation to the joint.
Summary of the data exchange

Each simulation opens a UDP port and animate given finger positions and
joint angles. The socket communication can be configured manually. Pre-
settings were defined for connecting to special application development
systems. The data rate is limited, Figure 2. Blender can indicate round
about 30 pictures per second. If more than these 30 datasets per second are
sent to Blender, they are queued. Delays in the representation will occur
then.

Figure 3: Call of the socket communication in the Game Engine of Blender

Experimental Results

At the IPR, Real-Time Matlab/Simulink is used to develop necessary

reactive grasping skills. A component of these developing environments is
the integration of the Blender simulation models. Figure 4 illustrates the
connection on Windows based operating systems. Real-Time capable
Simulink blocks were built which can be integrated by drag and drop into
every simulation. Predefined blocks of the Windows Real-Time Target
were used for it. In Figure 4 a version which can be used for the simulation
of the hand only is shown. In general, the reactive grasping Real-Time
simulations are running so fast, that Blender cannot keep track of all the
data updates. But with the help of a rate transition block the data transfer
can be configured arbitrarily in the Simulink model. A rate transition
handles the transfer from the output of a block operating at one rate to the
input of a block operating at a different rate (The Mathworks). Thereby the
data rate can be limited. In Non-Real-Time models the transmitted data rate
can be limited by using triggered subsystems. The Trigger must be attached
directly to the incoming data. By the use of the UDP socket communication
all Blender simulations can be swapped out to different PCs. By that more
computing power is available for the Real-Time PC. In addition, running
simulations can be monitored in many offices that have no direct access to
the Hardware. Thus, project partners at different and distant places can
participate in distant experiments. With the help of the integrated
movement abilities any user is able to adjust those 3D views that permit the
best illustration of a given problem. These simulation environments may
also be used for the construction and verification of certain software
modules. For instance, a Real-Time collision detection of the robot hand
could be built (Thomas Haase, 2009). Ideal joint positions were necessary
for the construction. With the help of the Simulation the finger positions
can also be calibrated to each other. Parallel developing work becomes
possible in this way.

Figure 4: Real-Time Windows Target Simulink Connection

Blender and Simulink in RTAI Linux

With RTAI Linux, Simulink models are able to run in Real-Time, Figure 5.
Therefore, Simulink models are compiled into C-Code that could be
executed in kernel space. A designed S-function makes it possible to export
data sets from kernel space into some existing shared memory, (Holger
Nahrstaedt, 2009). An additional software module in user space reads this
shared memory and transmits all data. As well as described on Windows
operating systems, a UDP data set is created and sent to some Blender
models. The remaining communication occurs as described for Windows
operating systems. All build up programs are identical for Linux and
Windows. Maybe it could be possible to read in the shared memory data
directly with the help of the Blender Python interface.

CONCLUSIONS AND FUTURE WORKS

Conclusions

In this paper a simulation environment for developing robot-hand control

systems was introduced. Blender was used to build up these software tools.
It was shown how to export this software into stand alone executables.
Thereby the user of these simulation tools does not necessarily have to be
familiar with Blender itself. As data interface a Socket connection was
chosen. This permits concurrent simulations on different PCs. The interface
also permits the construction of user defined data protocols. It was
introduced how the Blender simulation can work together with Simulink
simulations. This was shown for Windows as well as for RTAI Linux
operating systems.

Future Works

Currently the simulation environment is only used to illustrate and export

several joint angles. There is no real physical behavior included. Real
kinetic delays and interferences should be part of these simulations. If it
could be possible to integrate these requirements, the simulations could be
used to develop further control systems and strategies. Blender's game
engine offers possibilities regarding to collision detection, physical
behaviors and additional dynamic research. It should be checked which of
these possibilities can deliver sensible contributions. Furthermore there
should be some options to simulate tactile sensors. This could be possible
with the integrated collision detection. Therewith reactive grasping
modules could be tested. For the development it would be very helpful if
one could insert objects into the virtual world. By that one can control the
correctness of estimated robot and hand positions.
Figure 5: Design of RTAI Linux Contact

References
Blender For Robotics. (n.d.). Robotics:Index. Retrieved 11 20, 2009, from Blender:
http://wiki.blender.org/index.php/Robotics:Index
Caligari Corporation. (n.d.). trueSpace 7.6. Retrieved 12 08, 2009, from http://www.caligari.com/
Ciragan, M. (n.d.). ChumbalumSoft. Retrieved 11 20, 2009, from Milkshape 3D:
http://chumbalum.swissquake.ch/index.html
Craig, J. J. (1997). Introduction to Robotics, Mechanics and Control, Second Edition, . Addison-
Wesley.
Holger Nahrstaedt, T. H. (2009). SDH-2 and RTAI Simulink using shared memory for external
kernel and user-space communication and QRtaiLab for matrix visualization. Karlsruhe, Berlin:
Institute for Process Control and Robotics, Karlsruhe Institute of Technology (KIT), Fachgebiet
Regelungssysteme Technische Universität Berlin.
Inivis. (n.d.). Invis Limited 2009. Retrieved 11 19, 2009, from AC3D: http://www.inivis.com/
The Mathworks. (n.d.). Documentation Simulink. Retrieved 11 18, 2009, from Rate Transition:
http://www.mathworks.com/access/helpdesk/help/toolbox/simulink/slref/ratetransition.html
Thomas Haase, P. h. (2009). Real-Time Collision Detection for intrinsic safety of Multi-fingered
SDH-2. Karlsruhe: Institute for Process Control and Robotics, Karlsruhe Institute of Technology
(KIT).
Ton Roosendaal. (n.d.). http://www.blender.org/. Retrieved 11 20, 2009, from Blender:
http://www.blender.org/