Python in Robotics and Mechatronics Education
Python in Robotics and Mechatronics Education
Abstract—In robotics and mechatronics education tools for it is being used at leading universities in the world. The object
programming, simulation, visualization and control are common oriented language is well structured, blocks of code, such as
practice to clarify abstract theory, to support and validate designs in conditions, loops and functions, need to be indented and
and to perform experiments and process their results. In our
education the Python programming language together with its libraries (called modules in Python) can be nested in a very
rich ecosystem is playing a central and powerful role. This modular manner, each with its own namespace(s). Python is
paper presents examples of the use of Python in robotics and made freely available by the Python Software Foundation [5]
mechatronics education, including computer vision (OpenCV), and cross-platform, versions for Linux, Mac and Windows
3D dynamics (ODE/VPython), control (python-control), 3D kine- are available. Multiple programming styles, such as procedu-
matics and robotics (python-robotics) and machine learning
(scikit-learn). ral, functional, imperative and object oriented, are possible.
Furthermore, it is an interpreted programming language, that
I. I NTRODUCTION allows highly interactive programming. This enables tinkering
Throughout all design phases of mechatronic systems the with pieces of code, which is very convenient in learning to
mechatronic engineer should have a thorough understand- program, debugging and in experimenting for example with
ing of the whole system, how the individual components robotic systems, computer vision and data-analysis. Though
(sensors, actuators, mechanisms, controls, etc.) behave and being an interpreted language and thus in calculations not as
operate together, as well as the operation of the system in fast as e.g. C, Python allows very flexible access to software
its environment, human operators and/or other systems. The libraries written in C or C++; in addition Python code can
tool set of mechatronic engineers consist of various simula- be compiled as well, e.g., by Cython. Since its advent in
tion and visualization software tools for Systems Modeling 1991 a large ecosystem [4] has grown around Python with
(SysML), see e.g. [1], 3D Computer Added Design (CAD), powerful computational (e.g. numpy [6], scipy [7], sympy
rigid body simulation, Finite Element Modeling (FEM), Elec- [8], scikit’s [9]), communication (sockets, serial interfaces,
tronic Circuit Simulation, Computer Vision, Spreadsheets and webservers, etc.) and visualization tools (e.g. matplotlib
general purpose numerical tools and not at least programming [10], VPython [11], mayavi [12], etc.).
languages. A common trend in the abundance of commercial, This paper gives several examples of the use of Python in
closed- and open-source software tools is the integration the mechatronics program at THUAS, discusses their educa-
of functionalities such that each component as well as the tional value and outlines future developments. Our hope is, that
mechatronic system as a whole can be simulated. readers will find inspiration for their education or training in
In the mechatronics program at The Hague University of mechatronics and robotics and will also share their experience.
Applied Sciences (THUAS), Delft, The Netherlands, several of The paper is organized as follows. Section II presents and
these software tools are being used (e.g. Autodesk Inventor for evaluates the introduction to programming course. Section III
3D modeling, Qucs for circuit simulation, MATLAB Simulink and IV presents several Python modules and how these are
for block diagram based dynamic simulation), but after a being used in respectively the mechatronics basis program and
number of developments, the programming language Python is the international minor Robotics and Vision Design. Section V
more and more playing a central and integrating role in many gives a further discussion on the use of Python and outlines
courses and projects throughout the curriculum. further developments.
After positive experience with the use of Python and the
II. L EARNING TO PROGRAM IN THE FIRST YEAR
computer vision library OpenCV [2] in a second year project
(see Section III-A) we decided to start in the first year with an A. Historical background
introductory programming course based on Python as well (see In the early eighties of the previous century students learned
Section II). This decision has been based on own experience to program in the language Pascal. The language is very struc-
with programming Python, positive evaluation in literature [3] tured and the compiler gives accurate and well informative
and a growing amount of high quality numerical tools, see feedback on syntactical errors, which makes the language ex-
e.g., [4]. The Python language is easy to learn, even kids can cellent for educational purpose. In the nineties of the previous
learn, yet allows the development of very advanced software, century the C programming language became the industry
[10]. The goal of this course is to learn the student the basics nr=0
22 print("Mass[kg] Height[m] Energy[J]")
of programming, the algorithmic part, and the plotting of
while (nr<=H):
functions in a graph. 24 X.append(nr) # filling plot data
To introduce the Python language in the programming Energy = Epot(M, nr, g)
course, an example is used of plotting the potential energy of a 26 Y.append(Energy) # filling plot data
mass as a function of the height, see Fig. 1. As an illustration print(format(M,’.1f’),’\t’,format(nr,’4d’),
the Python code is discussed in the following. 28 ’\t’,format(Energy,’10.1f’))
nr = nr + 1
We urge the students to separate 1) input, 2) processing 30 title = ’’ # code is omitted
and 3) output from each other. In this example, the input is Plot(X,Y,title)
being done in a separate function, which argument is a string
that is being shown when asking for input. The input is being An extension to the program may be to adjust the code such
checked, and for invalid inputs the user gets a retry. that the step size can be determined by the user. The output
is a graph and a table. The heading of the table is printed on
1 def Input(text):
number = 0
line 22 and the rows of the table are formatted as specified on
3 while (number <= 0 ): line 27/28, i.e., first the mass, then the height and finally the
number = float(input(text)) potential energy.
5 if (number <= 0):
print ("The value has to be > 0.") C. Evaluation
7 print ("Try again: ") We observed, that the basic programming structures like
return(number) iterations, conditional statements and functions were learned
The potential energy is calculated in a separate function. In this almost without any effort. The students also came with own
case the function is very simple, and one has good reasons not ideas, e.g., to measure data with an Arduino and logging the
to make a separate functions. But we want to stress to separate data with Python using the pyserial module for making a
the processing from input and output. The plotting is also serial interface between an Arduino and Python. The attention
being done in a separate function, and involves some specifics was back on the job and not on the syntax. As a consequence,
for dealing with matplotlib graphs and is omitted here the results of the exams became better as well. For fast,
(note, that matplotlib has an excellent gallery of examples real-time processing mastering of the C language is needed,
[10]). for example for microcontroller programming and embedded
015
Authorized licensed use limited to: German Jordanian University. Downloaded on January 26,2025 at 10:38:55 UTC from IEEE Xplore. Restrictions apply.
Fig. 3. After warping and application of a color mask the pills are being
located using Canny edge detection and the Hough-circle transform, all in
OpenCV through Python bindings from the cv2 module.
016
Authorized licensed use limited to: German Jordanian University. Downloaded on January 26,2025 at 10:38:55 UTC from IEEE Xplore. Restrictions apply.
can be adjusted on the fly and their effect can be observed in
“real-time”.
After implementing the formulas for the digital PID-
controller, the parameters need to be tuned. Starting with a
P-action in the vertical direction the real-time visualization
demonstrates a strong oscillatory behavior motivating for D-
action. Because of the gravity there remains an offset error
in the vertical position control, motivating the use of I-
action. Also the controllers for the other directions are tuned
based on tuning insight and some trial and error; students
get to realize that the quadcopter is not a linear dynamic
system and the feedback loops influence each other (horizontal
movements suppose some pitch and/or roll). The nice thing
Fig. 4. Quadcopter simulation using pyode and python-visual. of the simulation is, that quick insight is obtained in the
dynamics of the quadcopter, without the effect of all kinds of
disturbances or actuator saturation. For example, gravity can
through MATLAB/Simulink, to get familiar with block-diagram be turned off, movements in certain degrees of freedom can be
modeling, something that is not (yet) possible in Python. studied in isolation and oscillatory or unstable behavior can be
In a practical connected to the control systems course, examined. Even students started to program their own signal
students will work with the Python Control Systems Library monitor functions to study step-responses to further optimize
[13] module, a kind of translation of the MATLAB control their feedback loops.
toolbox. They use this module for analyzing the systems
behavior in open- and closed-loop, both in the time-domain IV. P YTHON EXAMPLES IN THE INTERNATIONAL MINOR
(step-responses) and the frequency-domain (Bode-diagrams). ROBOTICS AND V ISION D ESIGN
Of course we could have chosen very well for the MATLAB In the mechatronics program at THUAS we offer an interna-
control toolbox, which has excellent functionality and ease tional minor Robotics and Vision Design for third and fourth
of use. However, we have chosen for the Python Control year engineering students. The goal of this minor is to learn the
Systems Library which students can use as well at (small) state of the art in robotics and vision technology and apply this
companies that cannot afford expensive software licenses, and knowledge in designing an intelligent robot to solve a problem
maybe even more important because of the good integration delivered by a company. In the courses of this minor we also
with other (excellent) Python tools. The control design in heavily make use of the Python programming language. Here
simulation is followed by control design experiments with we select two examples, the use of Python in modeling and
several lab setups, for which students have to tune P-, I- visualizing the kinematic behavior of arm type robots in the
and D-controller tuning knobs. We would like to train the course Robot Modeling and the application of Python machine
students more in digital controller implementations, e.g., using learning tools in the course Machine Learning.
microcontrollers, being inspired by [14]. For this, we are
developing the module python-avr-cfw a flexible and A. Coordinate frames and robot kinematics
adjustable framework based on a serial-interface for data- In the course robot modeling, along the lines of the first
acquisition and (real-time) control, that is available from three chapters of [18], students learn to work with several
Github [15]. coordinate frames, to model the forward kinematics of an arm-
Parallel to the courses, students have to control a quadcopter type robotic manipulator following the Denavit-Hartenberg
(Parrot ARDrone 2.0) using visual based servoing and have to notation, and to derive the inverse kinematics for a number of
perform some task (like inspection of buildings, following a common robot configurations. The theory is rather abstract but
target, etc.). To get them experienced with the stabilization the coordinate frames, and the robots can be visualized very
and position control of the quadcopter, we have developed a well. We got inspired by the MATLAB Robotics Toolbox of
quadcopter simulation tool [16], see Fig. 4, using Python bind- Peter Corke [19], but preferred to work in Python and we were
ings to the Open Dynamics Engine [17] for the 3D dynamics not satisfied with the Python version of this toolbox (not well
simulation and the Visual Python module python-visual developed and maintained). We could not find Python based
(or vpython) for the 3D visualization [11]. Like pyode alternatives that suited our needs, we already had experience
and python-visual the tool heavily makes use of object with python-visual as well as numpy and therefore
oriented programming, such that the students get more familiar decided to program a robotics module ourselves, resulting in
with this programming style and see the true benefits, espe- the python-robotics [20]. This is one of the nice things
cially when starting to create multiple quadcopters in their of using Python, you get powerful tools to develop programs
simulations. The simulation of the quadcopter dynamics and that suit your needs, and there is a huge amount of examples
the control is being performed in a separate thread using the on the Internet for inspiration and guidance (e.g. search for
Python module threading such that (control) parameters Python on https://github.com or http://stackoverflow.com).
017
Authorized licensed use limited to: German Jordanian University. Downloaded on January 26,2025 at 10:38:55 UTC from IEEE Xplore. Restrictions apply.
which students get an introduction into learning algorithms,
especially neural networks. In this course we use the book
[22] that illustrates the machine learning algorithms with
many examples in Python. In the course we focus on neural
networks, starting with the single Perceptron and then the
multi-layer Perceptron, based on the examples being discussed
in the book [22] and the accompanying code the students get
good insight in learning algorithms and their implementation.
Though we are satisfied with these examples, we also
find it important for the students to get experience with
larger machine learning tools, that are more and more getting
available (e.g. Facebook and Google have made their machine
learning libraries available for general use). We have selected
Scikit-Learn [23], a very well documented general purpose
machine learning tool, that gives access to a wide variety of
machine learning algorithms through a common programming
interface. With scikit-learn students learn to solve vari-
Fig. 5. Visualization of UR5 robot with coordinate frames using the ous type of linear and non-linear regression problems, feature
python-robotics tool that relies on numpy and python-visual. selection, Perceptron learning, data compression with Prin-
ciple Component Analysis (PCA) and learning to recognize
handwritten digits and faces using Support Vector Machines
The module provides a number of objects, such as so3 for (SVM). Though we go into detail on neural-networks, we do
3 × 3 rotation matrices (the Special Orthogonal group), se3 not have room in the course to discuss all these algorithms
for 4 × 4 homogeneous transformations (the Special Euclidian in detail as well. We explain the general ideas behind the
group), i.e., rotation together with linear displacement, frame algorithms and let the student get experience with using the
for specification and visualization of coordinate frames and algorithms, doing data pre-processing, think about the types
link for specification and visualization of robot joints and of algorithms to be selected, the tuning of parameters and
links, on the basis of Denavit-Hartenberg parameters. Figure 5 evaluating the learning results.
shows a visualization of the Universal Robotics UR5 robot that For now, the focus is mainly on data-analysis given some
has been modeled using 6 coupled link objects. benchmark data-sets, e.g. to predict diabetes on the basis
In a practical students first learn to work with rotations, of some body parameters, to determine the type of flower
for example, they have to write a Python program to rotate given information on e.g. steeple length, thickness, determine
a number of points about the z-axis, an axis parallel to digits on the basis of images, etc. Though these examples are
the z-axis and an arbitrary axis. Then, they learn to work relatively easy to use and yet very illustrative, we would like to
with homogeneous transformations and multiple coordinate also make the step to robot-learning, i.e. the robot will change
frames, that they will visualize using the frame object. For its actions on the basis of the learning outcome. This is left
example, they have to specify coordinate frames of a camera for further development.
mounted on a ceiling and a workplace at a table and derive
the homogeneous transformation from one to the other. After V. D ISCUSSION AND FURTHER DEVELOPMENTS
mastering to work with coordinate frames we apply this to Reflecting on the use of Python in our education we are very
the modeling of robotic manipulators based on the Denavit- satisfied. We observe that the students are enthusiastic about
Hartenberg notation. We start with simple 2 link manipulators. Python and use it a lot. We also have seen that companies start
After mastering this, they have to choose a 6 DOF robotic getting to use Python after positive experience with students
manipulator containing at least 3 revolute joints for which they from our department doing an internship. The use of Python
have to model the forward kinematics as well as the inverse enables to integrate results from various courses such as com-
kinematics. puter vision, dynamics, control, robotics, machine-learning,
In general students obtain good results in modeling the etc. in a very coherent manner. Python has a large ecosystem of
forward kinematics, but have difficulty with the inverse kine- accessible, well-documented and powerful software. Students
matics. We consider to let the students derive the analytic are discovering this better and better during their studies, and
inverse kinematics for a standard robot and to let them use grow in their programming skills.
generic inverse kinematic solvers for their own selected robotic Though we are enthusiastic about the power and versatility
manipulators, e.g. through OpenRAVE [21]. of Python, we are aware that there are situations where another
languages or software are better. Especially in programming
B. Machine learning real-time embedded controllers and other very time-critical
Another course in the international minor Robotics and applications, for which C or C++ is common practice. This
Vision Design at THUAS is the course machine learning, in is also the reason, why we also teach these programming
018
Authorized licensed use limited to: German Jordanian University. Downloaded on January 26,2025 at 10:38:55 UTC from IEEE Xplore. Restrictions apply.
languages. Python has some nice tools to link with software be shared. We would like to get in contact with educators and
written in C, C++ or other languages. Also if the timing of developers with shared interest and join forces together.
a Python application needs to be improved, one can turn on The Python programming language and the many high
a profiler to determine which parts in the software are most quality modules being developed in the Python ecosystem
time consuming, which than may be translated in C or C++ shows that open-source software can be professional and is
and called by Python, without the need of translating the whole an excellent means in education, that encourages students to
application. discover and develop their own creativity. Python has proved
We are also surprised about the power of the visualization to be a powerful Swiss-army knife for engineers.
tools. We are widely using matplotlib mainly for (2D)
R EFERENCES
graphs and python-visual for 3D drawing. Despite the
power of python-visual it is not comparable with modern [1] A. L. Mair and R. Fraanje, “Using SysML to teach Systems Engineering
skills,” in submitted 17th International Conference on Research and
3D CAD tools, that we teach to our students as well. However, Education in Mechatronic (REM), 2016.
it would be very convenient and powerful when modern 3D [2] G. Bradski, “OpenCV,” Dr. Dobb’s Journal of Software Tools, 2000,
CAD tools have a Python interface as well and 3D designs can [Online] http://opencv.org/ (retrieved on 29/02/2016).
[3] H. Fangohr, “A Comparison of C, MATLAB, and Python asTeaching
be parameterized with parameters being calculated by Python. Languages in Engineering,” Lecture Notes on Computational Science,
The 3D CAD program FreeCAD [24] is an example of such vol. 3039, pp. 1210–1217, 2004.
a CAD tool with Python integration. We also would like have [4] F. Pérez, B. Granger, and J. Hunter, “Python: An Ecosystem for
Scientific Computing,” Computing in Science & Engineering, vol. 13,
a whole 3D production toolkit, including 3D sensing, CAD no. 13, 2011, [Online] http://scitation.aip.org/content/aip/journal/cise/13/
design, G-code generation, validation and optimization, 3D 2/10.1109/MCSE.2010.119 (retrieved on 29/02/2016).
printing control, etc. [5] Python Software Foundation (PSF), [Online] http://www.python.org/psf
(retrieved on 29/02/2016).
This paper has illustrated some possibilities of Python and [6] NumPy, [Online] http://www.numpy.org (retrieved on 29/02/2016).
has demonstrated its integrating capacity in our curriculum. [7] SciPy, [Online] http://www.scipy.org (retrieved on 29/02/2016).
We haven’t discussed the use of sympy [8], the symbolic [8] SymPy, [Online] http://www.sympy.org (retrieved on 29/02/2016).
[9] Scikits, [Online] https://scikits.appspot.com/scikits (retrieved on
algebra module for Python. Currently we are using this mod- 29/02/2016).
ule, together with the Python Dynamics module pydy [25] in [10] Matplotlib, [Online] https://matplotlib.org (retrieved on 29/02/2016).
our course on biomechanics. We also consider to use Python [11] VPython, [Online] https://vpython.org (retrieved on 29/02/2016).
[12] Mayavi, [Online] http://code.enthought.com/projects/mayavi/.
in our mathematics education, e.g. with sympy for symbolic [13] R. Murray, “Python Control Systems Library,” [Online] https://
calculus, numpy for numerical calculus and matplotlib sourceforge.net/projects/python-control/ (retrieved on 29/02/2016).
for visualization. To get a quick impression of what’s possible [14] M. Jouaneh and W. Palm III, “Control Systems Take-Home Experi-
ments,” IEEE Control Systems Magazine, vol. 58, no. 8, 2013, [Online]
with sympy we recommend to have a look at the webtools http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6557459 (re-
Sympy Gamma [26] and Sympy Live [27]. trieved on 29/02/2016).
For research on and development of intelligent robotics [15] “Python-AVR Control FrameWork,” [Online] https://github.com/
prfraanje/python-avr-cfw (retrieved on 29/02/2016).
applications the Robot Operating System (ROS) is a powerful [16] “Python Quadcopter Simulation,” [Online] https://github.com/prfraanje/
framework, that we are using in our education and research. quadcopter (retrieved on 29/02/2016).
The programming of ROS applications with Python is very [17] “PyODE - Python bindings for the Open Dynamics Engine,” [Online]
http://pyode.sourceforge.net/ (retrieved on 29/02/2016).
well supported. Here we cannot go into further detail, but the [18] M. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and
interested reader is referred to [28]. Control. Wiley, 2006.
Finally, there are various integrated user development en- [19] Matlab Robotics Toolbox, [Online] http://petercorke.com/Robotics
Toolbox.html (retrieved on 29/02/2016).
vironments for Python. Standard is the idle interface, which [20] “Python Robotics module,” [Online] https://github.com/prfraanje/
is very basic and we use it in our first your programming python-robotics (retrieved on 29/02/2016).
course. In the higher years we mainly use Spyder which has [21] “OpenRAVE,” [Online] http://openrave.org (retrieved on 29/02/2016).
[22] S. Marsland, Machine Learning – An Algorithmic Perspective. CRC
the powerful ipython integrated shell. This ipython shell Press, 2009.
has some nice features, like auto-completion on tabs, scrolling [23] Scikit-Learn, [Online] http://scikit-learn.org/stable (retrieved on
in the command history with arrow keys, etc., which ease 29/02/2016).
[24] FreeCAD, [Online] http://www.freecadweb.org/ (retrieved on
the interaction and experimenting with code. Finally, in some 29/02/2016).
courses we also use the ipython notebook, nowadays [25] PyDy: Multibody Dynamics with Python, [Online] http://www.pydy.org/
also called jupyter notebook [29]. This development is (retrieved on 29/02/2016).
[26] SymPy Gamma, [Online] http://gamma.sympy.org/ (retrieved on
very convenient for mixing theory and code, in a browser 29/02/2016).
like environment using a Markup language. For some example [27] SymPy Live, [Online] http://live.sympy.org/ (retrieved on 29/02/2016).
notebooks see [30]. [28] rospy, [Online] http://wiki.ros.org/rospy (retrieved on 29/02/2016).
[29] Jupyter, [Online] http://jupyter.org/ (retrieved on 29/02/2016).
[30] Jupyter nbviewer — a simple way to share Jupyter Notebooks, [Online]
VI. C ONCLUSIONS http://nbviewer.jupyter.org/ (retrieved on 29/02/2016).
In this paper we have sketched some examples of the use
of Python in our mechatronics and robotics education. It is
our hope that these examples inspire the reader and more use
cases of Python in mechatronics and robotics education will
019
Authorized licensed use limited to: German Jordanian University. Downloaded on January 26,2025 at 10:38:55 UTC from IEEE Xplore. Restrictions apply.