Industry 4.0 - Digital Twins and OPC UA: Aksel Øvern
Industry 4.0 - Digital Twins and OPC UA: Aksel Øvern
Industry 4.0 - Digital Twins and OPC UA: Aksel Øvern
Aksel Øvern
Trondheim, 11-06-2018
Aksel Øvern
i
Acknowledgement
I would like to thank my supervisors, Olav Egeland and Eirik Njåstad, for providing
me with an interesting project and for guidance and help during the work done.
I will especially like to express my gratitude to Eirik, for not only taking time
to answer my many questions but also for good discussions, continuously sharing
your experience on the robots in the lab and for helping me in my experimental
work.
During the work on this project I was allowed to attend ICNSC 2018, the 15th
IEEE International Conference on Networking, Sensing and Control in Zhuhai,
China. Attending this conference and seeing what work is currently being done
within the field of Digital Twins in the industry was a great motivation and pro-
vided me with ideas that have been implemented in this project.
I would like to thank Olivier Roulet-Dubonnet at SINTEF Raufoss Manufacturing
AS for taking time for a meeting about his work on FreeOpcUa, and Torstein
Anderssen Myhre for helping me with a setup in the lab related to his work on
KUKA RSI.
Also, I would like to thank the other students at the department working with
their Master’s thesis in the same facilities as me, for academic cooperation and
keeping the morale up.
Last, I would like to thank the department for over the years expanding the robot
lab, to facilitate such opportunities for students.
A.Ø.
ii
Summary
This project explores the term Industry 4.0 (I 4.0) and the use of Digital Twins
(DTws) as an asset in this modern industrial revolution. A DTw can be described
as a digital replica of a physical system including data about this systems in-
teraction with its environment. The goal of this project has been to develop a
DTw for a robot cell at MTP Valgrinda, NTNU, and investigate what benefits
could be gained from introducing the technology in this system and the domain of
automated robotic systems in general.
The DTw was developed using the OPC UA communication architecture. OPC
UA is called the pioneer of I 4.0 [1] as it is a communication architecture aiming
at the standardization of communication in industry. Three different visualization
software solutions were compared. It was concluded that Visual Components 4.0
(VC 4.0) was the strongest candidate for developing a visual representation of the
robot cell. Using VC 4.0 and OPC UA a DTw of the robot cell was created.
Most of the work done in this project revolved around creating communication
modules able to connect the physical robot cell to the virtual representation in VC
4.0, through the use of OPC UA. The result of this work is a communication library
containing the virtual representation of the robot cell and the different communi-
cation modules able to give the DTw various functionalities. This library is made
open-source and can be found in the digital appendix B.1 and GitHub1 .
The software included in this library enables the DTw to do real-time mirroring
of the physical robot’s movements, plotting of robotic sensor data, and controlling
the robots from the DTw. A graphical user interface was developed to organize
the functionalities. Figure 3 illustrates the current communication architecture for
the DTw.
The DTw was tested in a fixed case. This was a multi-robot case including mir-
roring of movements, plotting of sensor data and control from VC 4.0. It was
made a video from the case, found in the digital appendix B.2, and published
online2 .
It was concluded that OPC UA was a good solution for use in this DTw as it
is possible to implement on any platform, and is enabling a more flexible and
structured way of communicating than traditional communication software used
in client/server based systems. The benefits found with the use of the DTw in this
automated robotic system included visibility to operations and a better foundation
for statistical analysis to predict future states and for optimizing characteristic
parameters associated with the robot cell. Finally, it was concluded that the DTw
act as a good foundation for managing a complex system, something that could be
beneficial as this specific system is used in training and professional development
at the institute.
1 https://github.com/akselov/digital-twin-opcua
2 https://youtu.be/xlQhQPmJwlA
iii
Figure 1: Physical robot cell Figure 2: Digital twin
iv
Sammendrag
Dette prosjektet utforsker begrepet Industri 4.0 (I 4.0) og bruken av Digitale
Tvillinger (DTws) som en ressurs i denne moderne industrielle revolusjonen. En
DTw kan beskrives som en digital kopi av et fysisk system, inkludert data for-
bundet til systemet og den interaksjon det har med sine omgivelser. Målet med
dette prosjektet har vært å utvikle en DTw for en robotcelle på MTP Valgrinda,
NTNU, og utforske hvilke fordeler denne teknologien kan gi for dette systemet og
automatiserte robotsystemer generelt.
Det ble utviklet en DTw som tar i bruk OPC UA som kommunikasjonsarkitektur.
OPC UA kalles en pionér innenfor I 4.0 da dette er en arkitektur som tar sikte
på å standardisere kommunikasjon i industri. Tre forskjellige programvarer for
visualisering ble sammenlignet. Det ble konkludert med at Visual Components
4.0 (VC 4.0) var den beste kandidaten for utvikling av en visuell representasjon
av dette systemet. Ved bruk av VC 4.0 og OPC UA ble det utviklet en DTw for
robotcella.
Majoriteten av arbeidet utført i dette prosjektet dreide seg om utvikling av kom-
munikasjonsmoduler som kunne koble den fysiske robotcella til den digitale mod-
ellen i VC 4.0, ved bruk av OPC UA. Resultatet er et bibliotek som inneholder
den virtuelle modellen og kommunikasjonsmodulene som gir dette systemet ulike
funksjoner i kontakt med den fysiske robotcella. Dette biblioteket har åpen kildekode
som finnes i det digitale vedlegget B.1 og på GitHub3 .
Programvaren i dette biblioteket gjør det mulig for DTw-en å utføre speiling av
de fysiske robotbevegelsene i sanntid, plotting av sensordata og ekstern styring av
robotene. Et grafisk brukergrensesnitt ble utviklet for å bedre organiseringen av
funksjonalitetene. Figur 3 viser nåværende kommunikasjonsarkitektur.
Systemet ble testet i et case. Dette inkluderte speiling av bevegelser i sanntid, plot-
ting av sensordata og styring fra VC 4.0. Det ble laget en video som viser systemet
i bruk. Denne finnes i det digitale vedlegget B.2 og er publisert online4 .
Det ble konkludert med at OPC UA var en god løsning for bruk i denne DTw-en
da dette er en arkitektur som er mulig å implementere uavhengig av plattform.
Dette gir en mer fleksibel og strukturert kommunikasjon sammenlignet med tradis-
jonelle klient-/serverbaserte systemer. Fordelene som ble trukket frem ved bruk
av en DTw i dette automatiserte robotsystemet var en forbedret oversikt over op-
erasjoner, samt et bedre grunnlag for statistisk analyse for prediksjoner av frem-
tidig tilstand og optimalisering av karakteristiske parametere forbundet til robot-
cella. Til slutt ble det konkludert med at DTw-en fremstår som et godt grunnlag
for administrering av et komplekst system, noe som kan være fordelaktig da denne
robotcella er brukt i opplæring og profesjonell utvikling på instituttet.
3 https://github.com/akselov/digital-twin-opcua
4 https://youtu.be/xlQhQPmJwlA
v
vi
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Sammendrag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
1 Introduction 1
1.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Theory 5
2.3 OPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 OPC UA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
vii
2.6 Digital twins with OPC UA . . . . . . . . . . . . . . . . . . . . . . 16
2.7.2 KUKA.Sim . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.10.1 KUKAVARPROXY . . . . . . . . . . . . . . . . . . . . . . 26
3.3.1 Communication . . . . . . . . . . . . . . . . . . . . . . . . . 43
viii
3.3.8 Read time analysis of the improved OPC UA server . . . . 56
3.8.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4 Result 79
4.3 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.4 Functionalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.1.2 Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . 92
ix
5.4 Safety and security . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6 Conclusion 109
Bibliography 114
Appendix 125
x
List of Figures
xi
33 Visual Components [10] . . . . . . . . . . . . . . . . . . . . . . . . 38
34 First sketch of robot cell . . . . . . . . . . . . . . . . . . . . . . . . 39
35 Model of robot cell environment, made in VC 4.0 . . . . . . . . . . 40
36 Comparison of first sketch and final model of the robot cell . . . . 41
37 Comparison of the physical and final model of the robot cell . . . . 42
38 Communication without OPC [25] . . . . . . . . . . . . . . . . . . 44
39 Communication with OPC [25] . . . . . . . . . . . . . . . . . . . . 44
40 Client-Server architecture [26] . . . . . . . . . . . . . . . . . . . . . 45
41 Proposed communication architecture . . . . . . . . . . . . . . . . 46
42 Screenshot of cmd from PC containing the DTw . . . . . . . . . . 48
43 OPC UA server running in Linux . . . . . . . . . . . . . . . . . . . 49
44 OPC UA client GUI running in Linux . . . . . . . . . . . . . . . . 50
45 OPC UA server running in Windows . . . . . . . . . . . . . . . . . 51
46 Creating variable pairs in VC 4.0 . . . . . . . . . . . . . . . . . . . 51
47 Information flow 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
48 List of connected variables from VC 4.0 . . . . . . . . . . . . . . . 54
49 Read time, opcua-server01.py . . . . . . . . . . . . . . . . . . . . . 55
50 Read time, opcua-server02.py . . . . . . . . . . . . . . . . . . . . . 56
51 Sensor data from KUKA KR 16-2 . . . . . . . . . . . . . . . . . . 58
52 Sensor data from KUKA KR 16-2 . . . . . . . . . . . . . . . . . . 59
53 Screenshot of .csv file containing data on motor current of KR 16-2 60
54 GUI structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
55 Functions of OPC UA modeling tools [7] . . . . . . . . . . . . . . . 64
56 VC 4.0: Connected variables in OPC UA client . . . . . . . . . . . 65
57 Information flow 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
58 Proposed communication architecture with RSI . . . . . . . . . . . 67
59 Screenshot of first RSI communication with Windows PC . . . . . 68
60 Screenshot of first successful RSI communication in Linux . . . . . 69
61 RSI Read time, rsi-opcua-server.py, no movement . . . . . . . . . . 70
62 RSI Read time, rsi-opcua-server.py, with movement . . . . . . . . 71
63 Functionalities studied in fixed case . . . . . . . . . . . . . . . . . . 73
64 AprilTag [27] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
65 Camera stream from KR 16-2 . . . . . . . . . . . . . . . . . . . . . 76
66 Illustration of PTP path . . . . . . . . . . . . . . . . . . . . . . . . 76
67 Fixed case setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
xii
76 Break down of the control of KR 120 R2500 from VC 4.0 . . . . . 90
77 Axis values from VC 4.0 . . . . . . . . . . . . . . . . . . . . . . . . 90
78 Axis values from physical KR 120 R2500 . . . . . . . . . . . . . . . 91
79 Values from VC 4.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
80 Values from KR 120 R2500 . . . . . . . . . . . . . . . . . . . . . . 92
81 Git branching model [28] . . . . . . . . . . . . . . . . . . . . . . . . 93
82 KUKA acquires Visual Components [29] . . . . . . . . . . . . . . . 94
83 Some of the robot brands available in VC 4.0 [30] . . . . . . . . . . 95
84 Time-delay analysis for use of JOpenShowVar and KVP [26] . . . . 98
85 Visualization of improved information flow . . . . . . . . . . . . . . 100
86 Point cloud combined with VC 4.0 model, made by Eirik Njåstad . 103
87 EC monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
88 The three i’s of I 4.0 innovation [31] . . . . . . . . . . . . . . . . . 107
xiii
xiv
List of Tables
1 Software comparison . . . . . . . . . . . . . . . . . . . . . . . . . . 23
xv
Terms and Abbreviations
CPS - Cyber-Physical System, system which connects real (physical) objects and
processes with information processing (virtual) objects and processes via
information networks which are open, partially global, and at any time con-
nected [36].
IoT - Internet of Things, The network of physical devices, vehicles, and other
items embedded with electronics, software, sensors, actuators, and network
connectivity which enable these objects to collect and exchange data [41].
xvi
I 4.0 - Industry 4.0, A name for the current trend of automation and data ex-
change in manufacturing technologies [43].
RAMI 4.0 - Reference Architectural Model Industry 4.0, A three-dimensional
map showing how to approach the issue of I 4.0 in a structured manner
[9].
SOA - Service-Oriented Architecture, A style of software design where services
are provided to the other components by application components, through a
communication protocol over a network [44].
STS - Socio-Technical system, A system that considers requirements spanning
hardware, software, personal, and community aspects, and applies an un-
derstanding of the social structures, roles and rights (the social sciences) to
inform the design of systems that involve communities of people and tech-
nology [45].
CAD - Computer-aided design, The use of computer systems (or workstations) to
aid in the creation, modification, analysis, or optimization of a design [46].
VC 4.0 - Visual Components 4.0, A commercial 3D simulation software, com-
bines three engineering disciplines in one platform, including, material flow,
robotics simulation, and controls validation [47].
.NET - (pronounced dot net), A software framework developed by Microsoft [48].
API - Application Programming Interface, Aa set of routines, protocols, and tools
for building software applications [49].
UI - User interface, Space where interactions between humans and machines occur
in the industrial design field of human–computer interaction [50].
SCADA - Supervisory Control and Data Acquisition, A control system archi-
tecture that uses computers, networked data communications and graphical
user interfaces for high-level process supervisory management [51].
KRL - KUKA Robot Language, Programming language used in KUKA robots.
RSI - Robot Sensor Interface, Software used for configuration of signal processing
in a real-time system of KUKA robots [19].
KVP - KUKAVARPROXY, A TCP/IP server that runs on a KUKA robot con-
troller, originally created by Massimiliano Fago [52].
cmd - Command Prompt, The command-line interpreter on Windows operating
systems.
GUI - Graphical User Interface, A type of user interface that allows the use of
icons or other visual indicators to interact with electronic devices, rather
than using only text via the command line [53].
xvii
MDA - Model-Driven Architecture, An approach to software design, development
and implementation providing guidelines for structuring specifications, which
are expressed as models [54].
xviii
Chapter 1
Introduction
The project description that was given by the supervisors was the following:
In order to respond quickly to unexpected events and new demands without ex-
tensive system changes, future production systems must be able to work more
independently. There is a need for intelligent machines that perform complex
tasks without detailed programming and without human interaction. Autonomous
systems know their own abilities (which are modeled as "skills") and their state.
They are able to choose between a set of possible actions, orchestrating and per-
form their skills. To succeed, the autonomous systems need to have realistic models
of the current state of the production process and the system’s own behaviour in
interaction with its external environment - usually called a digital twin.
OPC Unified Architecture (OPC UA) is a platform-independent protocol for
machine-to-machine communication for industrial automation developed by the
OPC Foundation. OPC focuses on accessing large amounts of real-time data at
the same time as system performance is affected to a minimum. OPC UA has the
potential to become an important foundation in the future industrial environment
where machines deliver "production as a service", and all machines and sensors in
production are online (Internet of Things).
In this task, implementation of a digital twin will be studied. A solution must
be developed that provides seamless communication between robots, PLCs, and
other relevant control systems that can be part of an industrial production system,
linked to one digital representation of the system. The system must be tested at
the institute Robot Laboratory.
1
b) Examine the advantages and disadvantages of using Visual Components 4.0
or similar simulation software for the digital twin.
c) Use Visual Components 4.0 or similar simulation software to model and
simulate the robot cell at the Institute.
d) Examine the advantages and disadvantages of using KUKAVARPROXY and
KUKA RSI Ethernet as middleware between the KUKA KR C4 robot con-
trollers and the digital twin.
e) Present a solution for a digital twin of the Institute’s Robot Laboratory with
use of OPC UA for communication.
f) Try out the system in a fixed case. Evaluate the results.
1.2 Background
"Unlike computers, humans do not process information, at least not in the sense
of sequential step-by-step processing that computers do. Instead, humans look at a
situation and conceptualize the problem and the context of the problem. Humans
take in all the data about the situation there interested in. They then conceptualize
the situation, seeing in their mind’s eye its various aspects. While they can do this
looking at tables of numbers, reports, and other symbolic information, their most
powerful and highest bandwidth input device is their visual sight.
What currently happens is that humans take visual information, reduce it to sym-
bols of numbers and letters, and then re-conceptualize it visually. In the process,
we lose a great deal of information, and we introduce inefficiencies in time. The
capability of the digital twin lets us directly see the situation and eliminate the
inefficient and counterproductive mental steps of decreasing the information and
translating it from visual information to symbolic information and back to visually
conceptual information."
- Dr. Michael Grieves [55]
The concept of digital twins was named one of Gartner’s Top 10 Strategic Tech-
nology Trends for both 2017 and 2018 [56]. Thomas Kaiser, SAP Senior Vice
President of IoT, put it this way: “Digital twins are becoming a business impera-
tive, covering the entire lifecycle of an asset or process and forming the foundation
for connected products and services. Companies that fail to respond will be left
behind.” [57]
It is estimated that within three to five years, billions of things will be represented
by digital twins [58]. Using physics data on how the components of a thing operate
and respond to the environment as well as data provided by sensors in the physical
world, a digital twin can be used to analyze and simulate real-world conditions,
responds to changes, improve operations and add value.
2
Figure 4: Chris O’Connor (IBM) at IoT Genius of Things summit 2017 [2]
The term "digital twin" has in the last years gained a lot of attention and hype. It is
a technology that is said to have a wide range of benefits in the modern industry.
This project aims at analyzing what benefits could be gained from introducing
digital twins in the domain of industrial automated robotic systems. The industry
has for years used robots to automate single tasks and production lines. How could
these processes be improved by the use of digital twins, and is this technology
indeed a step in the evolution of industry towards the new industrial revolution,
named Industry 4.0.
Multiple papers have been published on how digital twins could benefit produc-
tion systems. This includes papers such as About The Importance of Autonomy
and Digital Twins for the Future of Manufacturing by Rosen, von Wichert, Lo
and Bettenhausen [59], and A Review of the Roles of Digital Twin in CPS-based
Production Systems by Negri, Fumagalli and Macchi [60]. However, after doing a
literature study on the subject, it was found that little work has been done to make
a digital twin for a given system and observe the benefits this provides.
In this project a digital twin of the robot cell at the department of MTP Val-
grinda is made, and its performance is studied. Much of the work done is related
to the software needed for such a system to provide functionalities that make the
system valuable. While the Unified Modelling Language (UML) is established as
the major modeling language in software development, there are several standards
currently being used in the manufacturing environment [7]. For each of them, the
model and the code must be written and maintained separately, which requires
3
high manual effort. OPC UA is called the pioneer of the 4th industrial revolution
[1], as it is a communication architecture aiming at the standardization of com-
munication in industry. This project investigates why OPC UA is beneficial for
use in modern industrial systems and uses this architecture in the development of
a communication system for the digital twin.
4
Chapter 2
Theory
This chapter will present theory relevant to the work done in this project. From
general terms used, to specific theory related to the components that constitute
the robot cell at MTP Valgrinda.
The term Industry 4.0 (I 4.0) became publicly known in 2011, when an initiative
named Industrie 4.0 – an association of representatives from business, politics, and
academia supported the idea as an approach for strengthening the competitiveness
of the German manufacturing industry [61]. In later time numerous academic
publications, articles, and conferences have tried to approach and describe the
topic in a variety of ways. The term is to this date still new to both industry and
media, and as a result, a generally accepted understanding of I 4.0 has not been
published so far. As a result, discussing the topic on an academic level is difficult,
and so is implementing I 4.0 scenarios.
The term I 4.0 is mostly used for describing what is taught to be the next indus-
trial revolution, which is about to take place right now [62][63]. Like illustrated
in figure 5 this industrial revolution has been preceded by three other industrial
revolutions in modern history. The first industrial revolution was the introduction
of mechanical production facilities starting in the second half of the 18th century
and being intensified throughout the entire 19th century. From the 1870s on, elec-
trification and the division of labour led to the second industrial revolution. The
third industrial revolution, also called the digital revolution, set in around the 1970s
[61], when advanced electronics and information technology developed further the
automation of production processes.
5
Figure 5: Industrial revolutions including apriori prediction [3]
6
2. Managing complex systems: Due to products and manufacturing systems
becoming more and more complex, appropriate planning and explanatory
models should provide a basis for managing this growing complexity. Engi-
neers should, therefore, be equipped with the methods and tools required to
develop such models.
3. A comprehensive broadband infrastructure for industry: Reliable,
comprehensive and high-quality communication networks are a key require-
ment for I 4.0. Broadband Internet infrastructure, therefore, needs to be
expanded on a massive scale.
4. Safety and security: Safety and security are critical to the success of smart
manufacturing systems. It is important to ensure that production facilities
and the products themselves do not pose a danger either to people or to the
environment while at the same time, both production facilities and products,
in particular, the data and information they contain, need to be protected
against misuse and unauthorized access.
5. Work organization and design: In smart factories, the role of employees
will change significantly. Increasingly real-time oriented control will trans-
form work content, work processes, and the working environment. Emphasis
on implementation of an approach to socio-technical systems (STS) to work
organizations that will offer workers the opportunity to enjoy greater respon-
sibility and enhance their personal development.
6. Training and continuing professional development: I 4.0 will radically
transform worker’s job and competency profiles. It will, therefore, be nec-
essary to implement appropriate training strategies and to organize work in
a way that fosters learning, enabling lifelong learning where digital learning
techniques should be investigated.
7. Regulatory framework: While the new manufacturing processes and busi-
ness networks found in I 4.0 will need to comply with the law, existing legis-
lation will also need to be adapted to take account of new innovations. The
challenges include the protection of corporate data, liability issues, handling
of personal data and trade restrictions.
8. Resource efficiency: Manufacturing industry’s consumption of large
amounts of raw materials and energy poses a number of threats to the envi-
ronment and security of supply. I 4.0 will deliver gains in resource produc-
tivity and efficiency. It will be necessary to calculate the trade-offs between
the additional resources that will need to be invested in smart factories and
the potential savings generated.
The report emphasize that the journey towards I 4.0 will be an evolutionary pro-
cess. Current basic technologies and experience will have to be adapted to the
specific requirements of manufacturing engineering and innovative solutions for
new locations and new markets will have to be explored [65].
7
2.2 Digital Twins
The idea of a Digital Twin (DTw) is said to originate from NASA’s Apollo pro-
gram, where at least two identical space vehicles were built to allow mirroring the
conditions of the space vehicle during a mission [66]. The specific term “Digital
Twin” was however first defined by Dr. Michael Grieves at the University of Michi-
gan around 2001-2002 [67][55]. He originally defined the term in the context of
Product Lifecycle Management. In his paper, he introduced the concept of “Digital
Twin” as a virtual representation of what has been manufactured. He promoted
the idea of comparing a DTw to its engineering design to better understand what
was produced versus what was designed.
The term DTw is still what can be categorized as a buzzword without any precise
definition. However, to conceptualize this technology both industry and academia
have tried to define the term in several different ways for years. A publication
from Massachusetts Institute of Technology [68] defines a DTw as an integrated
model of an as-built product including physics, fatigue, lifecycle, sensor informa-
tion, performance simulations, that is intended to reflect all manufacturing defects
and be continually updated to include wear-and-tear sustained while in use. US
Department of Defense (DoD) defines a DTw as an integrated multi-physics, multi-
scale, probabilistic simulation of an as-built system, enabled by Digital Thread,
that uses the best available models, sensor information, and input data to mirror
and predict activities/performance over the life of its corresponding physical twin
[69]. The more general definition used in media and various industrial publication
is the definition of a DTw as a digital replica of physical assets, processes, and
systems that can be used for various purposes [70]. One can think of a DTw as a
bridge between the physical and the digital world.
8
According to the German Association for Information Technology, Telecommu-
nications and New Media (BITKOM), DTws in the manufacturing industry will
have a combined economic potential of more than e78 billion by 2025. However,
this potential can only be achieved if future systems are not only networked, but
are also capable of seeing, understanding, and self-optimizing in order to adapt to
future changes [71].
The concept of the DTw is a very powerful one. As this is not a specific technology,
it is hard to describe the concrete benefits this concept provides. However, most
DTws have characteristics that give some typical benefits [67]:
• Visibility: The DTw allows visibility in the operations of the machines as
well as in the larger interconnected systems such as a manufacturing plant
or an airport.
• Predictive: Using various modeling techniques (physics-based and math-
ematics based), the DTw model can be used to predict the future state of
the machines and optimize characteristic parameters for production – from
energy consumption to error rates, maintenance and cleaning cycles [71].
• What if analysis: Through properly designed interfaces, it is easy to inter-
act with the model and ask the what-if questions to the model to simulate
various conditions that are impractical to create in real life.
• Documentation and communication mechanism to understand and
explain behaviour: A DTw can be used as a communication and docu-
mentation mechanism to understand as well as explain the behaviour of an
individual machine or a collection of data from several similar systems.
• Connect disparate systems such as backend business applications:
If designed correctly, the DTw model can be used to connect with the backend
business applications to achieve business outcomes in the context of supply
chain operations including manufacturing, procurement, warehousing, trans-
portation and logistics, field service, etc.
9
2.2.1 Digital Twins and Internet of Things
The typical benefits provided by DTws listed in section 2.2 is linked to the idea of
I 4.0 and connected systems. Markets dealing with the use of Internet of Things
(IoT) have therefore taken much interest in the concept of DTws. In a nutshell,
IoT is the concept of connecting any device (so long as it has an on/off switch) to
the Internet and to other connected devices [72]. Every device that is activated
and registered with an IoT platform has two categories of data [73].
The first category is the metadata that doesn’t change very often. It includes de-
tails that describe the device such as the serial number, asset identifier, firmware
version, make, model and year of manufacturing. The second type of data asso-
ciated with the device represents the dynamic state. It contains context-specific,
real-time data unique to the device. In this dynamic state the system can benefit
from the implementation of real-time data connected with a DTw. Often this is
accomplished by extensive use of sensor technology. Today almost every IoT plat-
form has implemented some sort of DTw capabilities [67], of course with varying
degrees of complexity and apparent differences regarding their maturity and vision.
An IoT platform can bring operational and performance data into the DTw and
thus provide data on how the product is performing for example in comparison
with a theoretical calculated performance. This can be valuable information that
closes the loop from the operations group back to the people doing design and
R&D.
Without this closed loop, product simulation models used in design lack knowledge
of the physical product. With the closed loop, a product simulation model allows
analysis of real-time product data and monitoring of product assets to improve
product design and operation. As suitable software analyzes data from the closed
loop this could potentially prevent downtime, expose new opportunities and in gen-
eral give a better foundation for taking operational and strategic decisions.
10
2.3 OPC
Today the acronym OPC stands for Open Platform Communications [33]. OPC
was first defined by a number of players in automation together with Microsoft
back in 1995. Over the following ten years, it became the most used versatile way
to communicate in the automation layer in all types of industry [6]. Over the years
it has evolved to have more extensive functionality and reach.
Originally OPC stood for OLE (Object Linking and Embedding) in Process Con-
trol. OLE was a technology developed by Microsoft that allows embedding and
linking to documents and other objects. Initially, the OPC standard was restricted
to the Windows operating system [33]. The specifications of this version of the
OPC is today known as OPC Classic. The OPC Classic specifications are based on
Microsoft Windows technology using the COM/DCOM for the exchange of data
between software components on networked computers [74].
Like illustrated in figure 8, OPC is client/server based communication which means
that it exists one or more servers that wait for several clients to make requests.
The server is connected to components for example PLC and Control devices and
performs the requested actions. In OPC it is the client that decides when and
what data the server will fetch from the underlying systems [6].
The OPC protocols are completely self-sustained and have nothing in common. In
the OPC Classic you have the following protocols [6]:
• DA - Data Access: The most basic protocol of the OPC stack that gets data
out of the control systems into other systems on the shop floor
11
• AE - Alarm & Events: Subscription based service where the clients gets all
the time stamped events that comes in
• HDA - Historical Data Access: Contains historical data and supports long
record sets of data for one or more data points
• DX - Data eXchange
2.4 OPC UA
• Robust security
12
2.4.1 CPS and CPPS
To get an understanding of the importance of this possibility, the term CPS (cyber-
physical system) should first be explained. CPS is defined as a system which con-
nects real (physical) objects and processes with information processing (virtual)
objects and processes via information networks which are open, partially global,
and at any time connected [36]. The introduction of CPS into the manufactur-
ing environment lead to Cyber-Physical Production Systems (CPPS). CPPS work
with conventional production and information communications technology, allow-
ing machines and products to exchange information, trigger actions and control
other components autonomously [7].
OPC UA is mostly used in higher automation levels for the purpose of monitoring
and control, it also increases device connectivity via standard communication in
lower automation levels. OPC UA abstracts data from the network technology
and software application and offers a generic communication interface. In this
way, it represents a technology for a transparent data representation/transmission
between heterogeneous system components. Here are some key concepts important
to CPPS that OPC UA can help realize [36]:
13
2.4.2 Information model design and communication
The OPC UA specification consists of 13 parts. The first seven parts are related
to the core specifications, e.g. the concept, security model, address space model,
services, information model, service mappings, and profiles. The parts eight to
thirteen are related to access type specifications like data access, alarms and con-
ditions, programs, historical access, discovery and aggregate [7]. Figure 9 shows
the most essential parts of OPC UA necessary for user-specific information mod-
els.
The advantage here is that the handling of data structures can be done in a
way that data always is grouped and handled as one piece. This is important in
many applications where it is essential that the data set is taken at the same time
[6].
For securing the flow of information above the standard transport layers, the com-
munication between client and server consist of two layers. One that handles the
session and one to establish a secure channel between the client and server. This
is illustrated in figure 10.
14
2.5 The role of OPC UA in Industry 4.0
Through digitalization and integrated networking I 4.0 turns simple value chains
into fully digitalized value networks and creates new business models. This
means IoT services and production objects also communicate with each other
autonomously across companies. Several R&D topics arise from the design and
implementation of CPPS and the standardization of interfaces for vertical and
horizontal data exchange through networks [7].
To create a uniform basis for I 4.0, the German companies ZVEI, VDMA and
BITKOM have teamed up to develop RAMI 4.0, the Reference Architecture Model
for Industrie 4.0 [77]. According to ZVEI, RAMI 4.0 is the first and most advanced
reference architecture model for I 4.0 [78]. The model is a three-dimensional map
showing how to approach the issue of I 4.0 in a structured manner that ensures
that all participants involved in I 4.0 discussions understand each other [9].
15
exchange of information between two or more I 4.0-components requires clear and
unambiguous semantics. Here RAMI 4.0 suggest OPC UA because it can be used
for both communication as well as to define virtual representations [7]. OPC UA is
in fact listed as the one and only recommendation for realizing the communication
layer of the model [77].
16
identifying the node, while the ”Description” is optional [81]. Data modeling is a
fundamental part of OPC UA. The base principals are as follows [7]:
1. Using object-oriented techniques with type hierarchies and inheritance.
2. Type information is provided and can be accessed the same way as an in-
stance.
3. Full meshed network of nodes allows connecting the information in different
ways.
4. Extensibility of the type hierarchies as well as the references types between
nodes.
5. No limitations of modeling in order to allow an appropriate information
model design.
6. OPC UA information modeling is always done on the server.
In a Model-driven Architecture (MDA) models are the central elements of the
software development process. The aim is to derive platform-specific models from
platform independent models using a highly automated process. This reduces the
effort of software development and the adaptation to new technologies. Any ad-
ditions and changes to the model will be automatically reflected the next time
the code is generated. MDA models can then be used for the production of doc-
umentation, acquisition specifications, system specifications, technology artifacts
(e.g. source code) and executable systems. MDA defines three levels of abstraction
[82]:
1. The Computer Independent Model (CIM): Shows what the system
will do, but do not respect technology-specific information, to remain inde-
pendent of the system implementation.
2. The Platform Independent Model (PIM): Specifies the system in more
detail but still maintains a sufficient degree of generality to allow its mapping
to one or more platforms
3. The Platform Specific Model (PSM): Combines the specifications de-
fined in the PIM with the details required to stipulate how a system uses a
particular type of platform
17
Figure 12: MDA approach to OPC UA information model design [7]
Special tools are often used for OPC UA information model design to generate
code. The tools use different versions of XML-schemata. A modeler such as this
was used in this project. These tools not only speed up the implementation, but
they also increase the software quality by producing well structured and error-
free code. As implementation is reduced to modeling and the code is generated
automatically, even complex models can be implemented quickly. However, the use
of such tools can lead to the developed information models not being compatible
with other solutions than the specific system from which the model is created. A
solution to this is the MDA approach illustrated in figure 12. The blue arrows show
the suggested workflow creating a virtual representation of systems with OPC UA.
The yellow arrows represent knowledge and actions done by humans [7].
In the modeler used in this project, and as required for OPC UA information
modeling, each node in OPC UA has the mandatory attributes NodeId, NodeClass,
BrowseName, and DisplayName. The use of this modeler and the steps that were
done for the OPC UA communication in this project to comply with requirements
can be found in section 3.5.
18
2.7 Visualization software
To give the user a visual representation of the DTw a variety of software is avail-
able. For a mathematical representation of the physical body of the system, three-
dimensional (3D) models can be used. 3D models are widely used anywhere in 3D
graphics and Computer-Aided Design (CAD). Here a model can be displayed as a
two-dimensional image through a process called 3D rendering used in a computer
simulation. Examples of software used for CAD is Simens NX, Solid Works, and
AutoCAD.
The development of the DTws connectivity with the physical world is essential.
This includes the input of, e.g. sensor data to be interpreted by the software to
get the closed loop as discussed in section 2.2.1, and open the possibility for real-
time monitoring. Software used for CAD has traditionally not been able to input
sensor data to the 3D model and process simulation [83]. However, solutions as
Siemens PLM are beginning to integrate this function through an OPC connection
[84]. The software solutions considered for use in this project is described in the
following.
19
and collisions, and defining robot logic and postures with control flow statements.
For custom CAD models it is possible to import files directly into the 3D world.
Visual Components supports CAD file types from many of the leading CAD ven-
dors, making it compatible with 3D models made by software from Autodesk,
Dassault, PTC and Siemens [88].
VC 4.0 is built on the .NET framework with a Python API. This version also
redefined and re-organized the core services layer, so it is more accessible and
designed to allow for easy customization, from the UI to simulation behaviors [89].
Visual Components is used by manufacturing companies, systems integrators, and
machine builders. Several companies are also using the software as a platform for
their simulation software. Examples include the KUKASim software developed by
KUKA and Octopuz developed by In-House Solutions [91].
2.7.2 KUKA.Sim
20
• I/O mapping for controlling compo-
nents by way of signals
Like VC 4.0 KUKA.Sim also has an electronic catalogue with grippers, conveyors,
safety fences and other components. Also, almost all recognized CAD formats can
be imported [12]. The software uses graphical programming in a virtual environ-
ment much in the same way as VC 4.0.
KUKA.Sim Pro can be used for offline programming of KUKA robots and enables
a real-time connection to the KUKA robot controller. The KUKA.Sim Pro pack-
age also includes OPC UA PLC interface for Beckhoff TwinCat, CODESYS or
SIEMENS PLCSIM Advanced (TIA Portal) [92].
As the KUKA.Sim software is built on Visual Components platform the two has
a lot of the same functionality. The main difference is the restriction on the
KUKA.Sim library to only contain KUKA products. As mentioned in section
2.7.1 the VC 4.0 library contains 30 of the largest brands.
Since 2010 KUKA has relied on EtherCAT technology as a system bus in all
KUKA robot controllers. All internal communication and the communication to
the lower-level I/O level take place via standard Ethernet or EtherCAT [93]. When
it comes to the connectivity to OPC UA, KUKA on their web-pages advocates
the standardization and openness of interfaces to allow all production “things” to
communicate with the cloud via the same edge. To this end, KUKA promotes OPC
UA as a standardized communication protocol [94]. Performing a comparison of
VC 4.0 and KUKA.Sim regarding connectivity with OPC UA is however difficult
due to this plug-in being a feature that is still under development.
21
2.7.3 Siemens SIMATIC WinCC
SIMATIC WinCC is scalable for simple and complex tasks. Together with the
integrated process database, the software represents information exchange cross-
company. Through what Siemens call Plant Intelligence it provides transparency
in production [96].
22
2.7.3.1 Visual Components 4.0 versus Siemens SIMATIC WinCC
An overall comparison of the use of VC 4.0 and SIMATIC WinCC shows that
larger enterprises use the latter as a complete operation and management software
with properties including digital planning and integrated engineering operation
[97]. While this in theory also is possible in VC 4.0, this software is mostly used
for simulation and visualization of production lines, with a lower threshold for
doing changes in a system environment consisting of robots and components used
in production.
23
2.8 Offline robot programming
Offline programming is a robot programming method that does not interfere with
production as the program for the robot is written outside the production process
on an external PC [98]. This shifts the work of programming the robot from
the shop floor to the office. The benefits of this were mentioned in Njåstads thesis
Robotic welding with corrections from 3D-camera [14]. Here it was also stated that
it is beneficial to develop and configure a virtual model of a robot cell, providing
the ability to simulate a process while doing offline programming. A DTw could
constitute such a virtual model.
24
2.9 Robot Cell at MTP Valgrinda
The physical system used as the basis for the development of the DTw in this
project is a robot cell at MTP Valgrinda consisting of the following:
• 1x Siemens PLC
• 2x KUKA KR 120 R2500 pro, robot manipulator
• 1x KUKA KR 16-2, robot manipulator
• 1x KUKA KR 5 arc, robot manipulator
• 4x KUKA KR C4, robot controller w/smartPAD
• 1x Fronius TransSteel 5000 welding machine
Figure 20: KUKA KR 5 arc [17] Figure 21: KUKA KR 16-2 [18]
The four KUKA robots are individually connected to their own KUKA KR C4 con-
troller w/smartPAD. The controllers are then connected to a network together with
a Siemens PLC. The smartPADs have touch-screen technology used for program-
ming and controlling the robots manually. The Fronius TransSteel 5000 welding
machine is connected to the controller of the KUKA KR 5 arc, with the welding
torch on the robot manipulator.
The PLC has a variety of safety features to prevent damage to robots and people
inside the robot cell. The PLC also has modules for digital and analogue I/O,
used for controlling grippers, sensors and other signals and is accompanied by a
stationary PC.
25
2.10 Robot software
The robot manufacturer traditionally makes the software used in KUKA robots
such as the ones included in the cell at MTP Valgrinda. After the robots being
delivered from the manufacturer, the robot is installed and programmed according
to its environment. The programming and integration costs of such work can be
more than half the up-front cost of a new robot cell [99]. The extended use for
robots in academia and modern industry has created a demand for more accessible
programming tools and software. Both software from the manufacturers, third
party software from other companies, and open source software have in the later
years evolved to satisfy these needs. Two different open source software solutions
have been used in this project, KUKAVARPROXY and RSI.
2.10.1 KUKAVARPROXY
KUKAVARPROXY (KVP) is created by the IMTS S.r.L. Company [100], and was
first released together with OpenShowVar on Sourceforge. Some sources claim that
Massimiliano Fago was the person first to develop KVP and OpenShowVar in C++
[52]. KVP is a server to be run on the KUKA robot controller communicating
with the internal system. OpenShowVar is a client that can be run externally to
communicate with KVP over a TCP/IP connection. The KVP server supports a
maximum of 10 simultaneous connections [26].
KVP runs as a background application on the robot controller and is able to
communicate with the VxWorks side of the controller via KUKA CrossCom [101].
KVP has been used successfully in numerous research projects and has also been
used in earlier projects in the robot cell at MTP Valgrinda by Bjørlykhaug and
Raknes, and co-supervisor for this project, Eirik Njåstad.
KVP is, according to [100], said to be released on GitHub under an open license
in the future.
26
4. Configuration of real-time data exchange between the robot controller and
an external system via Ethernet.
5. Influence on the robot motion or path planning by means of the data ex-
change via Ethernet.
The robot controller communicates with the sensor system via a field bus. The
sensor data and signals are read by the field bus. KUKA RSI accesses the data
and signals and processes them. If the data exchange is configured via Ether-
net: the robot controller communicates with the external system via a real-time-
capable network connection. The exchanged data are transmitted via the Ethernet
TCP/IP or UDP/IP protocol as XML strings. Signal processing is established us-
ing RSI objects with a functionality and signal inputs and/or outputs as illustrated
in figure 22.
For message formatting and communication with KUKA RSI, it was used software
based on the principles of an open-source KUKA RSI communicator. The software
is called kuka-rsi3-communicator, is made by C. Eren Sezener, and available on
GitHub. This communicator uses the UDP protocol for controlling KUKA robots
manipulators in real-time. It communicates with the robot controller in a loop by
sending XML strings through UDP. If the controller does not receive any strings,
it gives a timeout error. It accepts position update commands through another
socket; then sends the update commands to the robot controller in the correct
XML format.
27
is used by applications that require a faster stream of data. This is done by avoid-
ing error-checking as illustrated in figure 23. Both of the protocols are built on
top of the IP protocol.
OPC Toolbox provides access to live and historical OPC data directly from MAT-
LAB and Simulink [103]. This software was not used in this project but could
easily be used for extracting and analyzing data from an OPC UA server con-
nected to a KR C4 robot controller. This toolbox includes read, write, and log
OPC data from devices, such as distributed control systems, supervisory control
and data acquisition systems, and programmable logic controllers. This simplifies
working with data from live servers and data historians that conform to the OPC
Data Access (DA) standard, the OPC Historical Data Access (HDA) standard,
and the OPC Unified Architecture (UA) standard [103].
Using this toolbox opens up the possibility of doing simultaneous data logging and
numerical processing, and simultaneous connections to multiple OPC servers. This
makes it easier to access historical data for analysis and statistical processing. With
the toolbox, it is possible to browse for available OPC UA servers and connect to an
OPC UA server by creating an OPC UA Client object. Also, the toolbox provides
functions that enable browsing and search of nodes in the server namespace to
determine available data nodes. Interaction with multiple nodes at the same time
is possible by creating an OPC UA node array. When reading the current value
for a node or node array, the value, timestamp, and an estimate of the quality of
the data is received, and it is possible to determine whether the data is a raw or
interpolated value [104].
28
2.11 Robot kinematics
A rigid body is completely described in space by its position and orientation with
respect to a reference frame. This is illustrated in figure 25. Let O − xyz be
the orthonormal reference frame and x, y, z be the unit vectors of the frame axes.
The position of a point O0 on the rigid body with respect to the coordinate frame
O − xyz is expressed by the relation:
29
Figure 25: Position and orientation of a rigid body [22]
By adopting the compact notation these three unit vectors, describing the body
orientation with respect to the reference frame can be combined in a Rotation
matrix :
h i
R = x0 y0 z0 (2.4)
30
Elementary rotations about the x, y and z coordinate axis are described by the
rotation matrices Rz (α), Ry (β), Rx (γ):
cos α − sin α 0
Rz (α) = sin α (2.5)
cos α 0
0 0 1
cos β 0 sin β
Ry (β) = 0 (2.6)
1 0
− sin β 0 cos β
1 0 0
Rx (γ) = 0 − sin γ (2.7)
cos γ
0 sin γ cos γ
31
The coordinate transformation of a vector with the coordinate vA in reference
frame A to the vector vB in reference frame B is then given by:
vA = RA
Bv
B
(2.9)
With RA
B ∈ SO(3) being the rotation matrix from A to B.
KUKA robots use A, B, C as Euler angles about the current axis. Euler an-
gles are a representation of orientation in terms of three independent parameters,
constituting a minimal representation:
32
Using the specific information on a KUKA robot’s links and joints together with
Roll-Pitch-Yaw angles, a kinematic structure can be generated. This is illustrated
in figure 28 and figure 29.
33
A KUKA robot consists of a series of links connected through joints. Joints can
be essentially of two types: revolute and prismatic. These two principles are
illustrated in figure 30. The whole structure forms a kinematic chain.
b b b b
ne (q) se (q) ae (q) pe (q)
Tbe (q) = (2.12)
0 0 0 1
34
Figure 31: Description of position and orientation of the end-effector frame [22]
Ti−1
i = Rotz (θi )Transz (di )Rotx (αi )Transx (ai ) (2.13)
35
36
Chapter 3
In section 2.1 it was stated that the journey of getting to I 4.0 should start with
the following steps:
1. A reference architecture to provide a technical description of standards and
facilitate their implementation.
2. Engineers equipped with the methods and tools required to develop appro-
priate planning and explanatory models.
This chapter will present the method used to develop a DTw for the robot cell at
MTP Valgrinda, and the solutions found along the way. It includes the selection of
software to be used, the development of a virtual representation of the robot cell
and the development of software capable of combining the virtual representation
and the physical robot cell. The final part of this chapter is a description of a fixed
case that was performed to test the system in action.
37
3.1 Selection of simulation software
Goal c) of this project listed in section 1, was to Use Visual Components 4.0 or
similar simulation software to model and simulate the robot cell at the institute.
For this task three simulation and visualization software packages were considered:
1. VC 4.0
2. KUKA.Sim
3. Siemens SIMATIC WinCC
These three are described and compared in section 2.7. As the DTw of the robot
cell should be implemented with OPC UA, connectivity to this platform was es-
sential when choosing simulation and visualization software. Using OPC UA for
communication is supported by this being the only recommendation for realizing
the communication layer of RAMI 4.0 as stated in section 2.5.
Access to pre-defined components was also an essential factor. Both VC 4.0 and
KUKA.Sim includes all the KUKA components in the robot cell at MTP Valgrinda.
This saves time in the development of the DTw were every component needs to be
modeled with associated features.
After comparison of VC 4.0 to Siemens SIMATIC WinCC, summarized in table
1, it was decided that Siemens SIMATIC WinCC would not be suitable for use in
this project. As presented in the comparison of VC 4.0 and KUKA.Sim in section
2.7.2.1, these two are very similar when it comes to functionality and use. Both
satisfied the conditions that were necessary for use in this project. However, the
updated version of KUKA.Sim, KUKA.Sim 4.0, was not available for use during
this project.
Solution: Visual Components 4.0 will be used for modeling and simulation
38
3.2 Making a virtual representation
With the simulation software decided the development platform for the DTw was
set. The first step that was done to make the actual DTw was to make a vir-
tual representation of the robot cell that would work as the digital face of the
simulations and give a visual representation to the user.
To make the model as accurate and detailed as possible, it was done several mea-
surements of the physical robot cell. However, since all of this was done by hand
and without any form or digital equipment there was a significant uncertainty
related to the measurements. With the use of the average value of three measure-
ments, the uncertainty was estimated to be ±1cm. An elaboration of the various
uncertainties can be found in section 5.5.
The significant uncertainty related to the digital model was accepted because the
only critical measurements of the system were the measurements of the distance
between the KUKA robots and their orientation in the cell. The orientation of the
robots would, in any case, be needing a calibration for the robots to be able to
39
work together with sufficient precision. This would come at a later stage as this
step just included a first rough model of the system.
After the initial measurements were done, a 3D model based on these measure-
ments was created using VC 4.0. First, a model of the robot cell environment was
made as illustrated in figure 35.
The second stage was to include the components inside the robot cell. Since this
DTw is a first version focused around the KUKA robots and their controllers, the
Fronius TransSteel 5000 welding machine was excluded as a part of the digital
representation of this robot cell. This could be included in a later version. For
insertion of the KUKA robots and their controllers, the advantages with the VC
4.0 software were evident. The pre-defined library included all these components
including their associated features. Inserting these components in the environment
resulted in the final model shown in comparison with the first sketch in figure 36
and the physical robot cell in figure 37.
40
(a) First sketch
Figure 36: Comparison of first sketch and final model of the robot cell
41
(a) Physical
Figure 37: Comparison of the physical and final model of the robot cell
42
3.3 Simulating robot cell
Now the visual representation of the robot cell was made it was time to connect
the twins and make them communicate. OPC UA would be used for the commu-
nication between the physical robot cell and the visual representation.
3.3.1 Communication
The basis for the communication is built on the principle that the KUKA KR C4
robot controller works as a server and the computer which is running the DTw
works as a client. This is in compliance with the OPC UA standard illustrated in
figure 8 in section 2.3.
The KUKA KR C4 robot controllers used in the robot cell at MTP Valgrinda
did not have OPC UA server functionality implemented as a standard. Thus,
this functionality had to be implemented as an add-on. After some research, it
turned out KUKA has made an add-on technology package named KUKA.OPC
Server 4.1. This server supports OPC Classic which was described in section 2.3.
It makes it possible to communicate and exchange data with the robot controller
from a local or remote OPC client. This includes features like [25]:
• Reading and writing of system and program variables of the industrial robot
• Reading of robot controller messages
• Reading of keys from the registration database
• Reading of variables from other OPC servers via the proxy interface
Without OPC, two machines or applications must know each other’s communica-
tions options in order to exchange data. This is illustrated in figure 38.
43
Figure 38: Communication without OPC [25]
44
In previously completed projects in the robot cell at MTP Valgrinda, an open-
source communication interface to KUKA KR C4 controllers has been used named
JOpenShowVar and KUKAVARPROXY (KVP). This is mentioned by among oth-
ers Bjørlykhaug, Raknes on their project on Robotic welding without fixture super-
vised by professor Olav Egeland and Eirik Njåstad, and Bredvold on his master
thesis Robotic welding of Tubes with Correction from 3D Vision and Force Control
also supervised by professor Olav Egeland [105]. Co-supervisor for this master
thesis, Eirik Njåstad, also used this interface in his master thesis Robotic welding
with correction from 3D camera [14].
The reason for the use of JOpenShowVar is that the majority of industrial ma-
nipulators does not have an open control interface. For KUKA, the standard
programming language is the KUKA Robot Language (KRL). This language is
text-based and offers the possibility of declaring data types, specifying simple mo-
tions, and interacting with tools and sensors via I/O operations. A KRL program
can only run on a KUKA Robot Controller, in this robot cell the KR C4, where
it is executed according to real-time constraints. The KRL is very limited when it
45
comes to research purposes. This is because the KRL is custom made to the un-
derlying controller and, as a consequence, it only offers a fixed, controller-specific
set of instructions. There is no mechanism for including third-party libraries. Due
to this design, it is very difficult to extend the KRL with new instructions and
functionalities and no external input devices can be directly used [26].
The standard workaround for expanding the robot’s capabilities consists of using
additional software packages provided by KUKA. Some examples of such pack-
ages are the KUKA.RobotSensorInterface (KUKA RSI), which makes it possible
to influence the manipulator motion or program execution via sensor data, and
KUKA.Ethernet KRL XML, a module that allows the robot controller to be con-
nected with external systems. However, these additional software packages have
several drawbacks such as limited I/O, a narrow set of functions and often re-
quire major capital investments [26]. None of these were available for use in this
project.
JOpenShowVar and KVP work as a middleware between the user program and
the KRL and thus opens a backdoor for an alternative implementation of an OPC
UA communication on the KR C4 controllers. The proposed communication ar-
chitecture is illustrated in figure 41.
For this to work, KVP has to run on each robot controller in the cell. This was
done in the project by Bjørlykhaug, Raknes by merely transferring the open-source
files to the KUKA smartPADs. The program was put in the startup folder in the
Windows XP environment, so it will always start and run when resetting the
robot controller. The version of KVP used in this project can be found in the
forked GitHub repository https://github.com/akselov/JOpenShowVar together
with a detailed description of how to set up JOpenShowVar and KVP made by
the Mechatronics Lab at Ålesund University College [106].
When KVP is running, the Java code needed in order to connect is:
1 CrossComClient connection = new CrossComClient(ipAddress, port);
This piece of code creates a new Thread, meaning the control system will be a
multi-threaded system, with a thread for each connection. To read and write
to the robot controllers, CrossComClient has two principal methods, readVariable
(KRLVariable) and writeVariable (KRLVariable), where a KRLVariable can be any
global variable on the robot controller [105].
46
3.3.3 Setting up JOpenShowVar and KUKAVARPROXY
As mentioned in section 3.3.2 KVP already was running on all of the four robot
controllers in the cell before this project. Therefore, getting a connection from
the client side with JOpenShowVar was the first step that had to be done for the
communication to be established. JOpenShowVar was compiled using Eclipse to
a .jar file that could be run from the Windows PC hosting VC 4.0 and containing
the DTw.
This connection was successfully established through command prompt (cmd), and
variables were successfully read and written to the robot controllers through this
connection. Figure 42 show screenshots of initial connection, reading and writing
variables from the client side to the KR C4 robot controller controlling the KR
16-2.
A goal of this project listed in section 1, was to Present a solution for a digital
twin of the Institute’s Robot Laboratory with use of OPC UA for communication.
The variables that were needed from the robot controllers were now successfully
being communicated from the controller, but this communication did not satisfy
the requirements of the OPC UA client running in VC 4.0. To fulfill the proposed
communication system illustrated in figure 41 it was needed an open source OPC
UA server that could work together with JOpenShowVar extracting the desired
robot variables.
47
(a) Communication between KR C4 and PC established
48
• Methods
• Basic user implementation
• Encryption
• Certificate handling
• Removing nodes
• History support for data change and events
OPC UA is said to be free, but doing some research, it was found that it is still
difficult to find a sufficient open source library. FreeOpcUa by Roulet-Dubonnet
was therefore chosen to be the foundation for the OPC UA communication in
this DTw. The version of the python-opcua library used in this project can be
found in the forked GitHub repository https://github.com/akselov/python-
opcua.
After downloading this library, an initial client-server communication was set up
using Ubuntu Linux. Figure 43 shows a screenshot of the example server in the
library, uaserver, running and listening to 0.0.0.0:4840 which is all IPv4 addresses
on the local machine [109], port 4840.
For the client side, it was used a simple GUI client also made by Olivier Roulet-
Dubonnet and published in the FreeOpcUa library. This client is named opcua-
client-gui, and the version used in this project can be found in the forked GitHub
repository https://github.com/akselov/opcua-client-gui. Figure 44 shows
the GUI connected to localhost, port 4840.
49
Figure 44: OPC UA client GUI running in Linux
After successfully establishing the client-server connection in Linux, the next step
was to implement a similar client-server connection in Windows with VC 4.0 work-
ing as the OPC UA client. Using PIP, a package management system used to
install and manage software packages written in Python [110], the installation of
both the uaserver and the opcua-client-gui can be done easily through the pip3
install command in cmd. This installation, together with a running uaserver, is
found in the screenshot in figure 45. A local connection to this server through the
VC 4.0 OPC UA client was successful.
50
Figure 45: OPC UA server running in Windows
Running this server locally on the Windows machine running VC 4.0 resulted in
"MyObject" and "MyVariable" being available in VC 4.0. Figure 46 shows VC
4.0s variable pairing functionality, where "MyVariable" is being paired with the
A1 axis parameter of the KR 16-2. This resulted in the KR 16-2 slowly rotating
about the A1 axis as the server incremented "MyVariable".
51
3.3.5 Transforming JOpenShowVar into an OPC UA server
To communicate with the KVP server on the robot controller, the JOpenShowVar
client must specify two parameters in the message: the desired type of function
and the variable name. As mentioned, the communication protocol relies on the
TCP/IP protocol. In particular, on top of the TCP/IP layer, specially formatted
text strings are used for message exchanges and KVP actively listens on TCP port
7000.
When reading a specific variable, the type of function must be identified by the
character “0”. For instance, if the variable to be read is the system variable
$OV_PRO, which is used for the speed of the robot, the message that the client
has to send to the server will have the format shown in table 2.
Field Description
00 Message ID
09 Length of the next segment
0 Type of desired function
07 Length of the next segment
$OV_PRO Variable to be read
In detail, the first two characters of this string specify the message identifier (ID)
with an integer number between 00 and 99. The answer from the server will contain
the same ID so that it is always possible to associate the corresponding answer to
each request even if the feedback from the server is delayed. The two successive
reading message characters specify the length of the next segment in hexadecimal
units. In this specific case, 09 accounts for one character specifying the function
type, two characters indicating the length of the next segment and seven characters
for the variable length. The fifth character 0 in the message represents the type
of the desired function - reading, in this case. Subsequently, two more characters
are interacting the variable length (in hexadecimal units), and finally, the variable
itself is contained in the last section of the message [26].
To avoid cross programming language communication between the Java JOpen-
ShowVar and the Python OPC UA server, JOpenShowVar was begun translated
into Python. However, after talking to co-supervisor Eirik Njåstad, it turned
out that this already had been done by Ahmad Saeed in his GitHub reposi-
tory https://github.com/ahmad-saeed/kukavarproxy-msg-format. The ver-
sion used in this project can be found in the forked GitHub repository https:
//github.com/akselov/kukavarproxy-msg-format.
52
Based on the minimal server from the FreeOpcUa library and the trans-
lated JOpenShowVar, it was created an OPC UA server able to extract the
axis parameters from the KR C4 robot controller and broadcast them to an
OPC UA client. This code can be found in the project’s GitHub reposi-
tory https://github.com/akselov/digital-twin-opcua/blob/master/opcua-
server/opcua-server01.py.
With KVP running on the KR C4 and the OPC UA server running on the Windows
PC, the axis variables of the KR 16-2 would be available as connected variables
in VC 4.0. Figure 48 shows a screenshot from VC 4.0 where the axis variables are
paired with the axis values of the KR 16-2 in the DTw. This causes the digital
KR 16-2 to mirror the movements of the physical KR 16-2 actively.
53
Figure 48: List of connected variables from VC 4.0
After setting up the first OPC UA server, opcua-server01.py, it was done a read
time analysis to investigate the time from sending the axis variables from the
KR C4 robot controller to receiving them in VC 4.0. The internal time delay
related to KVP extracting the axis variables in the robot controller was not inves-
tigated.
A simple Point-To-Point (PTP) code was run on the robot controller of the KR
16-2 robot while recording the read time in the OPC UA server. This test code
can be found in listing 3.1.
1 INI
2
3 PTP P8 Vel=100 % PDAT8 Tool[1]:ballfresen Base[0]
4
5
6 PTP P12 Vel=100 % PDAT12 Tool[1]:ballfresen Base[0]
7
8 PTP P13 Vel=100 % PDAT13 Tool[1]:ballfresen Base[0]
9
10
11 PTP P14 Vel=100 % PDAT14 Tool[1]:ballfresen Base[0]
12
13 PTP P9 Vel=100 % PDAT9 Tool[1]:ballfresen Base[0]
Running this code for 60 seconds resulted in the read times displayed in figure
49.
54
Figure 49: Read time, opcua-server01.py
This test resulted in an average read time of the opcua-server01.py of 25.69ms. The
maximum read time that was obtained in this 60-second interval was 41.13ms.
55
separate double values, the number readings required to update the robot position
was reduced by a factor of 6.
The source code for this improved OPC UA server, opcua-server02.py,
can be found in appendix A.1, and in the project’s GitHub reposi-
tory https://github.com/akselov/digital-twin-opcua/blob/master/opcua-
server/opcua-server02.py.
The same PTP code used in the read time analysis for opcua-server01.py, found
in listing 3.1, was run on the robot controller to record the read time of the opcua-
server02.py. Running this code for 60 seconds resulted in the read times displayed
in figure 50.
This test resulted in an average read time of the opcua-server02.py of 8.75ms. The
maximum read time that was obtained in this 60-second interval was 15.04ms.
56
3.4 Expanding the functionality of the DTw
In section 2.2 it was stated that most DTws have characteristics that give some
typical benefits [67]. This included visibility, predictive analysis, what-if analysis,
documentation to understand behavior and connection to backend applications.
opcua-server02.py makes the axis variables of the robots in the physical robot cell
available in the DTw. With this information, the DTw gains visibility to the state
of the physical robot cell, by that all the robots in the cell are mirroring their
physical counterpart. However, the DTw has little information about the state of
the robots. With only information about the robot’s movement, benefits such as
predictive-, what-if analysis, and documentation would be limited.
A study of available sensor information on the KR C4 robot controller was done
to expand the functionality of the DTw and make it more suitable for predictive-,
what-if analysis, and documentation. This study was done with a base in the doc-
ument [113] describing all KUKA System Variables. It was chosen four variables
that would improve the information available on the state of the robots in the
robot cell:
1. $VEL_AXIS_ACT[X] - Velocity of motor controlling axis X
2. $TORQUE_AXIS_ACT[X] - Torque on motor controlling axis X
3. $CURR_ACT[X] - Current to motor controlling axis X
4. $MOT_TEMP[X] - Temperature of motor controlling axis X
With the opcua-server02.py as a basis it was developed four OPC UA servers with
the functionality described above. This additional information about the KUKA
robot is being plotted by the server in real-time for all six axes simultaneously.
The four servers can be found in the project’s GitHub repository:
1. Velocity: opcua-server-velocity.py
https://github.com/akselov/digital-twin-opcua/blob/master/
opcua-server/opcua-server-velocity.py
2. Torque: opcua-server-torque.py
https://github.com/akselov/digital-twin-opcua/blob/master/
opcua-server/opcua-server-torque.py
3. Current: opcua-server-current.py
https://github.com/akselov/digital-twin-opcua/blob/master/
opcua-server/opcua-server-current.py
4. Temperature: opcua-server-temperature.py
https://github.com/akselov/digital-twin-opcua/blob/master/
opcua-server/opcua-server-temperature.py
57
The four servers were run on the Windows PC working as middleware between
the KR C4 robot controller and the Windows PC running VC 4.0 and hosting the
DTw. To test the servers, they were run while the PTP code listed in listing 3.1
was run on the robot controller of the KR 16-2 robot. Figure 51 and 52 shows
screenshots of the real-time plotting done by the servers of the sensor data from
the KR C4 robot controller.
58
(a) Motor current
Plots of sensor data can be a good source of information for predictive-, what-if
analysis, and documentation. However, the raw data supplying the plots with in-
formation is often the element used when doing an actual analysis. In consultation
with co-supervisor, Eirik Njåstad, it was decided that it would also be beneficial
with the server functionality of writing sensor data to files. The format most often
used in such analysis is comma-separated values, .csv, files.
59
With the opcua-server02.py as a basis it was developed four OPC UA servers with
the functionality of writing sensor data to .csv files. The four servers can be found
in the project’s GitHub repository:
Figure 53: Screenshot of .csv file containing data on motor current of KR 16-2
Like shown in figure 53, every line in the .csv file contains data on the current
delivered to the six motors controlling the robot’s axes. Every value is separated
by a comma as required in the .csv format.
60
3.4.1 Structuring the expanded functionality in a GUI
61
Figure 54: GUI structure
62
3.5 Standardization to comply with OPC UA
The OPC UA server functionality that until now had been developed was based
on the specifications of the KUKA KR 16-2 robot. Using these servers for the
remaining robots in the cell would not be a problem as these also was of the
KUKA brand and used the same KUKA KR C4 robot controllers. However, one
of the principles in the OPC UA protocol is that it should be a module based
communication system where all specific information about the physical parts is
being obtained from XML-files. Extensible Markup Language (XML) is a language
used in computing, which defines a set of rules for encoding documents in a format
that is both human-readable and machine-readable [34].
As stated in section 2.6, model-driven techniques which are already used to design
complex software systems, tremendously simplify the process of information model
design. The main concept of a model-driven approach is to separate the functional
description and the implementation. Thus, it is not necessary to repeat the process
of defining an application or the system’s functionality and behavior each time a
new communication technology comes along. The model-driven approach behind
OPC UA was further discussed in section 2.6.1.
The modularization of the server functionality should also include the KVP com-
munication module. The opcua-server02.py, which a lot of the functionality to
this point was built on, had the KVP communication module included as a part
of the OPC UA server. At a later stage, to satisfy the "IPO_FAST" requirement
of a sensor cycle rate of 4ms as discussed in section 3.3.7, the OPC UA server
would need a faster communication module, such as RSI. If RSI easily could be in-
cluded as a communication module in parallel with KVP, without doing significant
changes in the OPC UA server, this would be beneficial.
To sum up, the standardization to comply with OPC UA standards boiled down
to two steps:
1. An OPC UA server that can obtain specifications about a physical asset from
an XML-file and make custom OPC UA objects based on these specifications.
2. Make the OPC UA server ready for more than one communication module.
Make KVP a separate module that is only being included if the OPC UA
client requests this.
Also, but not as a part of the standardization, it was concluded that it should be
done an attempt at including RSI as a separate communication module to satisfy
the "IPO_FAST" requirement of a sensor cycle rate of 4ms [112].
63
3.5.1 Standardization: Step 1
To extract the relevant information from the XML-file a server capable of creating
custom OPC UA objects had to be developed. This server was created with a base
in a server from the FreeOpcUa library called server-custom-object.py. This server
can be found in the forked GitHub repository at https://github.com/akselov/
python-opcua/blob/master/examples/server-custom-object.py.
The server developed sets the two Boolean values, KUKAVARPROXY and RSI,
to be writable by OPC UA clients. In this project, it was used VC 4.0 as an
OPC UA client, and thus the desired communication has to be set from here. The
64
server also retrieves the IP address of the robot controller from the XML-file and
establish a connection to this robot controller through the chosen communication
protocol, KVP or RSI. The server sets the robot axis variables as Double 9.99 by
default and to be changed by the connection with the KR C4 robot controller.
The connection, with KUKAVARPROXY chosen by the client as communication
protocol (TRUE) and axis values set to the default value, is seen in the screenshot
from VC 4.0 in figure 56.
To make the OPC UA server ready for more than one communication module the
KVP configuration and communication had to be modularized and extracted to
a separate communication module. This was simply done by extracting the KVP
configuration and communication into a separate Python file to be launched from
the OPC UA server. The server is set up so that it listens to the OPC UA client
to check if communication over KVP or RSI is desired.
The OPC UA server developed after standardization step 1 and 2 can be found
in appendix A.4 and in the project’s GitHub repository at https://github.com/
akselov/digital-twin-opcua/blob/master/xml/opcua-server-xml.py.
The extracted KVP communication module can be found in appendix A.5 and
in the project’s GitHub repository at https://github.com/akselov/digital-
twin-opcua/blob/master/xml/KUKAVARPROXY_communication.py. With stan-
dardization step 1 and 2 completed the system has a more modularized structure.
The new information flow in the DTw is illustrated in figure 57.
65
Figure 57: Information flow 2
66
3.6 Including RSI communication
To satisfy the "IPO_FAST" requirement of a sensor cycle rate of 4ms as discussed
in section 3.3.7, it was clear that the OPC UA server in some cases would need a
faster communication module, such as RSI. RSI uses UDP for communication, as
described in section 2.10.2. The idea was to make a communication architecture
similar to the one created around KVP and make this run in parallel with the sys-
tem created up until now. The basis of the communication architecture developed
for KVP was described in figure 41. The proposed communication architecture for
the system using RSI is illustrated in figure 58.
67
3.6.1 Setting up RSI communication
Eren Sezener has a thorough description of how to set up KR3C in the GitHub
repository. The configuration of the KR C4 robot controller was followed with two
exceptions:
1. Instead of copying the file RSI_Ethernet.src to C:\KRC\ROBOTER\KRC
\R1\Program, the file RSI_AxisCorr2016v2.src was copied to the same
folder. This file can be found in the project’s GitHub repository at:
https://github.com/akselov/digital-twin-opcua/blob/master/rsi/
RSI_AxisCorr2016v2.src
68
While setting up the file to print the received data from the robot controller, it
was encountered a problem. The connection broke down every time after precisely
100 iterations. This was most likely caused by the received data packages arriving
too late from the robot controller, and after a buffer size of 100 late packages
the communication shut down. This is a part of the RSI’s more strict real-time
requirements.
After consulting with Torstein Anderssen Myhre the same python code was instead
run on a Linux PC. Here the data was received successfully after 100 iterations.
The reason for this probably lies in the way the two OSs are designed and how
they use their cores to run the code. Screenshot of the received data from the
robot controller to the Linux PC can be found in figure 60.
69
3.6.2 Combine RSI communication with an OPC UA server
70
Figure 62: RSI Read time, rsi-opcua-server.py, with movement
The tests resulted in an average read time of the rsi-opcua-server.py of 4.00ms both
with and without robot movement. The maximum read time that was obtained
in this 60-second interval was 5.04ms for the test without robot movement and
5.12ms with robot movement.
71
3.7 Controlling robots from DTw
As a last property of the DTw, it was developed the functionality of controlling
the robots in the robot cell from the DTw. This could be used when wanting
to run a robot program directly in VC 4.0 and make the physical robots execute
the commands simultaneously as the digital robots. This system was developed
based on work done by Adam Leon Kleppe (http://folk.ntnu.no/adamleon/) on
controlling KUKA robots from an external PC using KVP.
For controlling the KUKA robots through KVP, the KR C4 robot controller has
to run a program listening to updates on position. With minor changes, this
program was made to listen to axis variables. The program consists of two files
that have to be copied to C:\KRC\ROBOTER\KRC\R1\Program on the KR C4
robot controller:
1. opcua_ptp_movement.dat
https://github.com/akselov/digital-twin-opcua/blob/master/
opcua-control/opcua_ptp_movement.dat
2. opcua_ptp_movement.src
https://github.com/akselov/digital-twin-opcua/blob/master/
opcua-control/opcua_ptp_movement.src
Selecting and running the opcua_ptp_movement.src results in the robot listening
to axis values from a KVP client. This is a similar setup to the
RSI_AxisCorr2016v2.src used for the RSI communication. Based on the stan-
dardized opcua-server-xml.py it was created an OPC UA server able to create a
custom OPC UA object from specifications in an XML-file, extract axis variables
from an OPC UA client, here VC 4.0, and control a KUKA robot by its axis
values. This file can be found in appendix A.7 and in the project’s GitHub reposi-
tory https://github.com/akselov/digital-twin-opcua/blob/master/opcua-
control/opcua-control-server.py.
With this setup, the KR C4 is listening to the axis variables from VC 4.0 and
updating the physical robot position to mirror the robot in the DTw. To avoid
the robots crashing with other components in the robot cell while being controlled
from the DTw, it was tested using working envelopes for limiting the working area
of the robots. This was configured from the smart pad the following way:
1. Log on Administrator
2. Navigate to Miscellaneous -> Monitoring working envelope -> Configuration
3. Choose "Axis specific"
4. Set minimum and maximum values for all axis
5. Choose: Mode = "OUTSIDE_STOP"
When this setting is activated, the robot will stop with an error message as soon
as it moves outside the set parameters.
72
3.8 Fixed case
A goal of this project listed in section 1.1, was to Try out the system in a fixed case.
Evaluate the results. The development of this DTw has resulted in several parallel
communication systems, all with different functionalities. The fixed case was tried
constructed in such a way that this would test as many of these functionalists as
possible. Together with co-supervisor, Eirik Njåstad, a test was created with the
goal of studying the functionalities listed in figure 63.
73
3.8.1 Setup
The aim of the code created in ViSP was to have a vision-based control of the KR
16-2 robot so that the robot would always work towards keeping the AprilTag in
the centre of the camera mounted to its end effector. In this way, the distance
from the centre is an error that has to be minimized. The error is typically defined
by [114]:
For a 3D point with coordinates X = (X, Y, Z) in the camera frame, which projects
in the image as a 2D point with coordinates x = (x, y) we have:
74
(
x = X/Y = (u − u0 )/px
(3.2)
y = Y /Z = (v − v0 )/py
Here m = (u, v) gives the coordinates of the image point expressed in pixel units,
and a = (u0 , v0 , px , py ) is the set of camera intrinsic parameters: u0 and v0 are the
coordinates of the principal point, while px and py are the ratio between the focal
length and the size of a pixel [114].
ṡ = Lx vc (3.3)
−1/Z 0 x/Z xy −(1 + x2 ) y
Lx = (3.4)
0 −1/Z y/Z 1 + y2 −xy −x
Considering vc as the input to the robot controller, and if we would like for instance
to try to ensure an exponential decoupled decrease of the error, we obtain:
vc = −λL+
xe (3.5)
Eirik Njåstad developed a code based on this library and combined it with the same
KVP controlling principle used in opcua-control-server.py. The code was developed
in C++ and can be found in his GitHub repository https://github.com/eirikn/
basicVisualServoing.
Running this code resulted in the KR 16-2 tracking the AprilTag mounted to the
KR 120 and mirrored its movements in real-time. A screenshot of the camera
stream from the KR 16-2 is displayed in figure 65.
75
Figure 65: Camera stream from KR 16-2
A simple PTP program containing four points was created in the DTw in VC
4.0 moving the KR 120s end effector 500mm along the y-, x- and z-axis. This is
conceptually illustrated in figure 66 with the movements represented as the vectors
ay , ax and az .
76
As the KR 16-2 would mirror the movement of the KR 120 its axis variables
would be sent to VC 4.0 so that the DTw could in real-time track its movement.
To illustrate the time delay between the KR 120 and the KR 16-2 mirroring its
movements, the Windows PC working as middleware between the KR C4 and
the PC hosting the DTw would do real-time plotting of the Cartesian x-, y- and
z-values from both the KR 120 and Kr 16-2.
The setup of the fixed case is shown in figure 67. The case was filmed using:
1. Overview camera showing the movement of KR 120 and KR 16-2 in robot
cell (top left)
2. Camera mounted to KR 16-2 (top middle)
3. Screen capture of PC hosting the DTw in VC 4.0 (bottom left)
4. Screen capture of Windows PC doing the real-time plotting (top and bottom
right)
The video footage was put into one frame to show how the DTw works with the
physical robot cell in real-time. The final video showing the fixed case described
can be found in the digital appendix B.2, and is also published online at
https://youtu.be/xlQhQPmJwlA.
77
78
Chapter 4
Result
This project is a study on how a DTw can be implemented using OPC UA. A
DTw for the robot cell at MTP Valgrinda was developed. Using this technology
for expanding the functionality of the robot cell gave various benefits to the system.
The general benefits usually seen when introducing a DTw was presented in section
2.2. It is analyzed the specific benefits this DTw provides for the robot cell in
section 5.6, and its value for I 4.0 in section 5.7.
The use of VC 4.0 was measured up against similar simulation software. VC 4.0
was considered the strongest candidate for this project and was used to make the
virtual representation of the robot cell. The modeling of the cell was thoroughly
described in section 3.2.1. Most of the work done in this project revolved around
the simulation and communication system connecting the DTw to the physical
robot cell. As a DTw can have multiple different use cases and therefore require
a lot of different functionality, it was decided that it would not be sufficient with
one single communication system, but rather a library of communication systems
suited for different tasks. All developed files, both for VC 4.0 and communication,
is described in chapter 3 and can be found in appendix B.1 and the project’s
GitHub repository
https://github.com/akselov/digital-twin-opcua
In all the different communication systems developed it was used a Windows PC
as middleware between the KR C4 controllers in robot cell and the Windows PC
running VC 4.0 and hosting the DTw. KUKA Norway was contacted to gain access
to software that would make the use of middleware unnecessary. This software
was not received and tested, but as discussed further in section 5.3, the Institute
of MTP is already in contact with KUKA Norway regarding the possibility for
updating the KR C4 controllers in the robot cell to KRC ROBOTstar, which has
OPC UA built in [115].
79
The result of the work done in this project is divided into 4 categories:
1. DTw in VC 4.0
2. Communication library
3. The setup
4. Functionalities
In addition, data from the fixed case described in section 3.8 is presented.
80
4.2 Communication library
81
4.3 Setup
The final setup of the DTw used in the fixed case described in section 3.8 is
illustrated in figure 70. Here the DTw is running in VC 4.0 and displayed on a
Windows PC that is located on the side of the robot cell. The DTw is connected
to a Windows laptop working as middleware between the Windows PC hosting the
DTw and the KR C4 robot controllers in the robot cell. This laptop is plotting
the relevant sensor information in real-time.
82
4.4 Functionalities
The GitHub library contains OPC UA servers giving the DTw the following func-
tionalities:
1. Programming, planning, and control in VC 4.0
2. Mirror movements of physical robot cell in real-time using KVP,
average read time 8.75ms
3. Mirror movements of physical robot cell in real-time using RSI,
average read time 4.00ms
4. Plot sensor data for every KUKA robot:
• Velocity of the motors controlling all axes
• Torque on the motors controlling all axes
• Current to the motors controlling all axes
• Temperature of the motors controlling all axes
5. Write sensor data for every KUKA robot to .csv files:
• Velocity to file
• Torque to file
• Current to file
• Temperature to file
6. GUI for access to functionalities
7. Extraction of physical robot properties through XML
8. Controlling physical robots from VC 4.0 using KVP
How to set up the DTw for the different functionalities is described in chapter
3. To illustrate some of the functionalities the DTw was tested in a fixed case
described in section 3.8. The result of this case was a video showing how the DTw
works with the physical robot cell in real-time. This video can be found in the
digital appendix B.2 and is also published online at
https://youtu.be/xlQhQPmJwlA
83
4.5 Data from fixed case
In the fixed case the Cartesian axes of the KR 16-2 and the KR 120 were plotted
in real-time as illustrated in the video. These plots are found in figure 72 and 73.
As the two OPC UA servers plotting these values were not started at the same
point in time, the timestamps of the data from KR 16-2 and the KR 120 is not
coherent. Comparing the timestamps between the data collected shows that the
timestamp for the KR 120 is lagging 2.6 seconds behind the KR 16-2.
84
It can be seen, from comparing the two plots, that the positive Cartesian movement
of the KR 120 along the robot’s y-axis results in the KR 16-2 responding with a
negative Cartesian movement along its respective y-axis. This is a result of the
robots facing each other in the robot cell and mirroring each other’s movements.
Likewise, the positive Cartesian movement of the KR 120 along the robot’s x-axis
results in the KR 16-2 responding with a negative Cartesian movement along its
x-axis. For the final positive Cartesian movement along the z-axis, the KR 16-2
responds with a similar positive movement along its z-axis.
From the two plots, it can also be seen that while the KR 120 performs linear
movement between the points, the KR 16-2 tries to follow this movement, but the
visual control is not fast enough for it to completely mirror the linear motion. This
results in the movement of the KR 16-2 being more smoothed out. It can also be
noticed that the correction of the KR 16-2 seems to be faster along the robot’s
x-axis than along the y-axis.
85
86
Chapter 5
This chapter contains an analysis of the system developed and discussion of soft-
ware, data acquisition, and communication architecture. This includes a discussion
of the uncertainties, safety features, and improvements that should be done to take
this DTw to the next level. The chapter is concluded in a final discussion of the
benefits the DTw provides, both for the institute and for the evolution towards I
4.0.
87
ing for documentation on implementations of software and the software itself. Peo-
ple had done much of this work while being in the same position as me, also in
the same laboratory working on the same robot cell. Therefore, together with
supervisor Olav Egeland, it was decided that a lot of effort should be put into
gathering relevant documentation and libraries that would in the future be useful
when working with the robot cell. This is the reason why the GitHub repository
was created.
This project is unique in that way that the product is not fixed, but rather a DTw
with a communication library where the user hopefully has the tools needed to set
up a customized DTw satisfying the requirements for the task at hand. As a DTw
can be used for a wide range of different applications, this was seen as the best
solution for future use.
The goal of the fixed case was to illustrate a scenario where a DTw would be
useful. It was not focused on doing the tracking algorithm based on the ViSP
library as efficient as possible. The algorithm used was based on position-based
control which could, with not too much effort, be improved to include a velocity
based correction system.
During the case, it was discovered that using the opcua-control-server.py for con-
trolling the KR 120 from VC 4.0 had some drawbacks that need improvement.
Even though a program with linear PTP movements along the robots axis was
created in VC 4.0, the physical robot did not keep a constant speed between the
set points. This was a visual observation. To establish that the KR 120 robot
was undoubtedly not moving with constant speed along the pre-planned paths the
speed of the motors controlling the axes of the robot was plotted. This was done
using GUI -> Display sensor data -> Display Motor Speed monitor. The result
of this test can be found in figure 74. It was clear from this plot that the velocity
of the motors was not constant. The same test was done with the KR 16-2 that
was not controlled from VC 4.0. The plot of this test is displayed in figure 75.
Comparing the two plots, it is clear that the velocity of the motors controlling the
KR 16-2s axes is also not constant, but is varying much less than the motors of
the KR 120, resulting in the KR 16-2 running more smoothly.
88
Figure 74: Motor speed of KR 120 from fixed case
89
An analysis was done to find out why the control from VC 4.0 resulted in the
KR 120 robot not moving with constant velocity. The opcua-control-server.py was
investigated to break down the steps from creating axis values in VC 4.0 from the
PTP program to executing the axis values on the robot controller. The process
was divided into the four steps displayed in figure 76.
Figure 76: Break down of the control of KR 120 R2500 from VC 4.0
As step 2, 3 and 4 were executed in a similar way for controlling the KR 16-2,
step 1 had to be the root of this problem. After consulting with co-supervisor
Eirik Njåstad, it was made a hypothesis that this was a result of VC 4.0 not being
able to produce a sufficiently constant stream of points when doing a simulation.
The server developed uses the points created by the simulation in real-time. Even
though the values were read from VC 4.0 fast enough the simulation would not be
able to deliver a sufficient amount of points per time interval.
This hypothesis was tested using a simple program only controlling axis 1 of the
KR 120 and moving it with constant speed while the stream of points from VC
4.0 was recorded and plotted. This plot can be found in figure 77.
90
Analyzing this plot confirmed the hypothesis. It can be seen that in a 10-second
interval the points that make up the linear movement of axis 1 is for more pro-
longed periods constant and do not follow the planned linear movement. The most
extended interval with a constant axis value delivered form VC 4.0 was between
0.23s and 0.44s, amounting to a total of 210ms.
The effect this had on the physical KR 120 robot was investigated by plotting the
axis value of axis 1 when running the same program controlling the robot from
VC 4.0. The plot of this recording can be found in figure 78.
Comparing the two plots 77 and 78 it can be seen that the motor which is con-
trolling axis 1 works as a low-pass filter for the stream of points. This results in
the movement being smoothed out. Still, the non-linear movement could be seen
by observing the robot when performing the test.
It is worth mentioning that the stream of points from VC 4.0 also does deviate
from a line representing the ideal linear movement as illustrated in figure 79 and
80.
91
Figure 79: Values from VC 4.0 Figure 80: Values from KR 120 R2500
The maximum deviation from ideal behavior is 0.20 degrees in the values from
VC 4.0 and 0.18 degrees in the values from KR 120 R2500. After consulting with
co-supervisor, Eirik Njåstad, it was concluded that this probably was a result of
the robot performing a PTP movement. This will result in the robot performing a
close-to-linear movement of a trajectory with variable velocity profile through the
movement.
5.1.2 Improvements
Regarding the software developed this would need gradually to be improved step
by step. GitHub is a platform where software development is made easy by the
idea of branching and merging of code. This idea is built on the principle of a
master branch with forked development branches in parallel. This is illustrated
in figure 81. Developing software in this way allows it always to be a functioning
master branch that can be used when the system needs to be up and running. It
is encouraged for people interested in the work done in this project to fork code
that has been developed and customize and improve for specific needs.
92
Figure 81: Git branching model [28]
93
controllers in the cell. As the MTP faculty is in the process of modernizing
the robot cell at Valgrinda this could potentially be fixed by switching to the
new generation of KUKA robot controllers, KUKA KRC ROBOTstar, which
have OPC UA built in as add-on technology [115]. This is further discussed
in section 5.3.
4. Remove lag in control of robots through VC 4.0 using KVP. This
phenomenon was discussed in section 5.1.1. The root of the problem was
found to be the stream of axis variables from the simulation in VC 4.0. A
similar control server should be developed that uses more robust variables
from VC 4.0 as output for controlling the robots.
5. Development of an OPC UA server controlling robots through VC
4.0 using RSI. This work was begun in the last stage of this project but
was not finished. The work was based on an RSI communication architec-
ture made by Torstein Anderssen Myhre that can be found in his GitHub
repository: https://github.com/torstem/examplecode-kukarsi-python.
In section 3.1 VC 4.0 was selected to be used as simulation software for this project.
KUKA.Sim was considered as an alternative, but due to this software not being
available for use in this project it was never actually tested as an alternative to VC
4.0. As KUKA.Sim is developed by KUKA; it was thought this could be better
suited for connections to KUKA.OPC Server 4.1 as discussed in section 3.3.1, and
future servers from KUKA supporting OPC UA. However, as of December 2017,
KUKA acquired Visual Components [29]. This supports VC 4.0 as a solution for
the future of interconnections with KUKA systems.
94
The significant advantage of VC 4.0 is that it opens up the possibility for the in-
tegration of robots from other manufacturers than KUKA into the existing DTw.
This includes major brands like ABB and Kawasaki. This is important when it
comes to scaling. If the faculty of MTP should decide to include components of
other brands than KUKA, these could also be imported as objects in the DTw.
95
5.3 Data acquisition and communication
In section 2.2.1 it was stated that every device that is activated and registered
with an IoT platform has two categories of data. The first being the metadata
that includes details to describe the device. In this project’s DTw, much of the
metadata is included in the VC 4.0 library. This could again be accompanied by
specific data on every component, such as the serial number, an asset identifier,
firmware version, make, model, and year of manufacturing. Data such as this
should be included in XML-files for easier configuration. An implementation of
OPC UA objects being configured through the use of XML-files was done in section
3.5. This complies with the standardization required in the OPC UA architecture
described in section 2.6.1. However, the XML files created for the components in
this robot cell only contains basic information about the robot’s IP address and
the joint variables. This could be extended to include more specific data.
The second type of data associated with the device represents the dynamic state.
It contains context-specific, real-time data unique to the device. In this dynamic
state, the system can benefit from the implementation of real-time data connected
with a DTw. Through the communication library developed for information ex-
change between the physical robot and digital model, real-time data should be
more easily available. In this project it was mainly focused on five sources of
data:
1. Axis values and thereby the position of the robots end-effectors
2. Velocity of motors controlling all robots axes
3. Torque of motors controlling all robots axes
4. Current of motors controlling all robots axes
5. Temperature of motors controlling all robots axes
This data can provide information about how the different components are per-
forming and interacting in the physical world, for example in comparison with a
theoretical calculated performance or historical data gathered from the component.
Analyzing the behavior and comparing this to the behavior of similar components
could also be possible through uploading sensor data to commonly used databases.
The base of analysis such as Big Data analytics is good and available sources of
data. Such analysis could potentially prevent downtime, expose new opportuni-
ties and in general give a better foundation for taking operational and strategic
decisions.
96
When deciding the communication architecture that should be chosen for use in
this DTw, it was first considered using an already developed OPC server made
by KUKA, named KUKA.OPC Server 4.1. This server supports OPC Classic as
described in section 3.3.1. KUKA.OPC Server 4.1 was not suited for this project
as it was not supporting OPC UA, but it was seen as attractive for testing as
a reference to the OPC UA servers that had to be developed. As this software
was not available online, KUKA Norway was contacted about the possibility of
gaining access. After talking to co-supervisor Eirik Njåstad it turned out that the
Institute of MTP is already in contact with KUKA Norway regarding a potential
update of the KR C4 controllers in the robot cell to KRC ROBOTstar, which
has OPC UA built in and the possibility for all components to be connected to
the robot controller for data exchange [115]. However, this upgrade is not fully
verified. The latest update on the situation, as of June 2018, is that the institute
probably has to replace the whole robot park in order to get this new robot con-
troller. This backs up the conclusion that it was still applicable to implement a
solution for communication with OPC UA on the KR C4 controllers currently in
the robot cell, and for all other KUKA controllers not currently supporting OPC
UA connectivity.
KVP was the alternative first used to establish a communication between the phys-
ical robots and the digital model. The reasoning behind the use of this structure
was done in section 3.3.2. KVP was also used as a basis for most of the OPC UA
servers developed in section 3.3. It should be emphasized that the use of KVP for
an alternative implementation of OPC UA is not an ideal solution. Even though
the JOpenShowVar/KVP structure is tested and applied by various projects, there
is no guarantee for this software to work as described. This software is open-source
and has not officially been verified by any robotics manufacturer. Using KVP as a
middleware can compromise the safety of the system and opens the possibility for
malfunctions not experienced in a server purely run as an OPC UA server made
by authorized software developers.
Use of the JOpenShowVar/KVP client/server architecture also includes a time-
delay described by Sanfilippo, Hatledal and Zang in their publication JOpenShow-
Var: An open-source cross-platform communication interface to KUKA robots [26].
This analysis is carried out for Cartesian movement of an end-effector and is dis-
played in figure 84. In a 20-second interval, an average access time of 4.27 ms was
obtained. A similar time-delay analysis was done in section 3.3.8 for the opcua-
server02.py. With simple Cartesian movement, an average read time of 8.75ms
in a 60-second interval was obtained. In section 3.6.2 it was done the same time-
delay analysis for the rsi-opcua-server.py with an average time-delay of 4.0ms in
a 60-second interval.
97
Figure 84: Time-delay analysis for use of JOpenShowVar and KVP [26]
Comparing the time-delay obtained by Sanfilippo, Hatledal and Zang with the
time delay obtained in the analysis of the opcua-server02.py, we find a 119% in-
crease. The reason for this is that while the JOpenShowVar/KVP client/server
architecture only performs reading and writing of variables from or to the robot
controller, the opcua-server02.py also does reading and writing of variables from
or to an OPC UA client. In this case VC 4.0. This amounts to more than a
doubling of the time-delay. Replacing the middleware and the use of KVP with
a local OPC UA server, as discussed as an improvement in section 5.1.2, would
presumably reduce the read-time and thereby lower the time-delay.
As the four KUKA KR C4 robot controllers in the robot cell did not include
OPC UA as an add-on technology, two different types of middleware were tested.
A variety of OPC UA servers based on KVP and one server based on the RSI
communication architecture had been developed. Comparing the two types of
middleware it was found that RSI would have a time-delay to VC 4.0 that was in
average about 46% (RSI: 4.0ms, KVP: 8.75ms) of the KVP OPC UA server with
the same functionality.
The reason for this can be found in the way the RSI communication is built, as
described in section 2.10.2. KVP uses TCP and RSI UDP. The difference is also
due to RSI including harder real-time requirements than KVP. Because of these
hard real-time requirements, the RSI server had to be run on a Linux machine
as described in section 3.6.1. After using the two systems in parallel, it was seen
that the RSI server had a higher possibility of breaking down while running due to
these real-time requirements. As a result, KVP could be seen as a solution more
reliable, but with more time-delay.
98
The advantages of using KVP in comparison with RSI as middleware included
that the KVP server on the KR C4 robot controllers was always running in the
background. This resulted in the possibility to run other tasks and programs on
the smartPAD without this server being in the way. RSI had to be run as a
specific program on the robot controllers losing the possibility of running other
tasks and programs. However, this difference was not the case with the function
of controlling the robots in the cell from VC 4.0. Here both KVP and RSI would
need a specific program to be run on the KR C4 controllers for the robot to listen
to updates on position.
OPC Toolbox was described in section 2.10.4. Co-supervisor Eirik Njåstad discov-
ered this software in the later stages of the project. It was therefore not tested as
an alternative to VC 4.0. Through the use of this software, a simple implementa-
tion of an OPC UA client could have been developed. As this toolbox opens up
the possibility of doing simultaneous data logging and numerical processing, and
simultaneous connections to multiple OPC servers, this is an easy way to get DTw
functionalities directly into MATLAB.
Using this software could simplify data acquisition and plotting of sensor data.
MATLAB has numerous functions for processing data and performing a statisti-
cal analysis which could streamline the handling of data from the KR C4 robot
controllers.
99
5.4 Safety and security
Safety and cyber security are becoming more and more critical in the intercon-
nected systems used in modern manufacturing. This was mentioned in section 2.1
as one of the key areas where the German Industrie 4.0 Working Group believes
that action is needed. Cyber security, computer security or IT security is the pro-
tection of computer systems from the theft and damage to their hardware, software
or information, as well as from disruption or misdirection of the services they pro-
vide [118]. This is especially the case for systems that are directly interconnected
with a physical system such as a DTw. A breach of the security related to a DTw
could potentially lead to unwanted personnel gaining access to data from the phys-
ical system, and even worse, loss of control of the physical components. This could
lead to systems out of control, and dangerous situations could occur.
You can only achieve a truly safe system if all existing risks are recognized in
advance and, where necessary, reduced as quickly as possible. A DTw used in the
manufacturing industry will also face the challenges of meeting current standards
and legislation, such as IEC 62061, ISO 13849-1, ISO 14121 [119]. With regards
to safety, OPC-UA is already an IEC standard (IEC 62541), and tools and test
laboratories for testing and certifying conformity are available. Additionally, vari-
ous validations regarding data security and functional safety have been performed
by external test and certification bodies [1].
100
Using the open-source FreeOpcUa created by Olivier Roulet-Dubonnet does also
constitute a safety threat since it exists just a small amount of documentation on
this project, and it still being work in progress. The version used in this project
does also not support security such as authentication and certificates handling
[120]. This software is not thoroughly tested and should like the KVP system not
be implemented as a permanent solution in a DTw used for professional purposes,
without further research.
Using the DTw for running real-time applications also constitute a general safety
threat as the robot cell at MTP Valgrinda contains four large robots with the ca-
pability of imposing potentially hazardous situations. The main hazards identified
in this robot cell is:
1. Collision: With itself, other components in the robot cell, the environment
surrounding the cell or people working in the cell.
2. Electrocution: Mainly from the current supplying the robots and the KR
C4 robot controllers. Especially when performing maintenance or doing
change in the hardware connections to the KR C4s. Opening the KR C4
cabinet to access Ethernet ports was done several times during the work
with this project.
3. Unwanted behaviour: Such as ejection of a tool during operation or pro-
grammed algorithms choosing unwanted paths when performing PTP move-
ment. This was especially evident in the last part of the project when trying
to develop a control server using RSI.
To minimize the risk of robot-human collisions the robot cell is fenced in and has
a sensor connected to the door of the cell activating an emergency stop signal
to all four KR C4 robot controllers when opened. Still, people could close the
door from inside the cell, and the robots could keep running in AUT mode. This
should be avoided. To avoid robots colliding with themselves, robot-robot collision
and robot-environment collision, it was set up a working envelope when using the
KR 16-2 robot. This limits the movement of the robot to a fixed space. This
configuration was explained in detail in section 3.7 and could also be activated for
the remaining robots in the cell.
101
5.5 Uncertainties
Uncertainty is a central concept quantifying the dispersion one may reasonably
attribute to a result. Uncertainties could also be the source of error. The creation
of a digital representation of a physical system includes many uncertainties because
it is impossible to replicate something physical in a digital environment fully. The
DTw is merely an imitation of the actual system, with as similar characteristics as
possible.
The main contributors to uncertainties identified are:
1. Uncertainties related to the individual components in the physical robot cell
2. Measurements of the physical robot cell
3. Data transferring and communication
4. Use of KVP and RSI module
5. Use of FreeOpcUa
The uncertainties related to the individual components in the physical robot cell
can be broken down into several sources. One source is the deviation due to
changes in temperature. The KUKA robots in the cell has an allowed ambient
temperature range of approximately 10 °C to 50 °C [121] [122]. Using the robots
outside this range of temperature could cause their operation not being as in their
documentation. As the digital representations of the components in the VC 4.0
library are built on the documentation of the robots, this could cause an error
in the digital representation. Changes in temperature between the components
ambient temperature is also a source of error. Extraction of information from
temperature sensors connected to the motors controlling the axes of the robots
was done in this project. These measurements could be used as information to the
digital twin about the operating state of the robots. However, it is still difficult
to calculate how the temperature differences in the robot affect the expansion and
contraction of the materials which constitutes the robots for this to be included in
the a more precise represented DTw.
Position repeatability of the individual components can also be characterized as a
source of error. In the KUKA robots in the robot cell this repeatability lies between
±0, 04mm [121] and ±0, 06mm [122]. This is a deviation that is difficult for the
digital representation to take into account as VC 4.0 does path planning assuming
ideal behaviour. This problem is however evident regardless of the robot being con-
trolled from VC 4.0 or programs run on directly on the KR C4 smart pads. Using
RSI for controlling the robots, real-time adjustments and path-corrections would
be possible. This is one of the main reasons why RSI also should be developed as
a parallel communication to KVP as mentioned in section 5.1.2.
The measurements of the physical robot cell, done in section 3.2.1 is also a source
of error. This measurement was done by hand using a tape measure. With the use
102
of the average value of three measurements, the uncertainty was estimated to be
±10mm. These measurements give uncertainty about the orientation of the com-
ponents in the robot cell. To reduce this error, digital equipment for measurement
could be used. As argued in section 3.2.1 the measurements done was accepted
because the only critical measurements of the system were the measurements of
the distance between the KUKA robots and their orientation in the cell. The ori-
entation of the robots would, in any case, be needing a calibration for the robots
to be able to work together in tasks that demand a high level of precision. Co-
supervisor Eirik Njåstad tested using camera measurements creating a point cloud
of the robot cell. This could potentially improve the accuracy of the orientation
of the components in the cell. The point cloud is illustrated in figure 86.
Figure 86: Point cloud combined with VC 4.0 model, made by Eirik Njåstad
The uncertainty included in the use of KVP and FreeOpcUa was mentioned in
section 5.4. This uncertainty is related to the data transferring and the communi-
cation between the physical and the digital twin. As none of these has been verified
to be completely bug-free, uncertainty about the data transferring is therefore also
a contributing factor in this system.
103
5.6 Robot cell as DTw
In section 2.2 it was presented certain characteristics related to DTws, which gives
some common benefits. These characteristics will be used as the basis for the
discussion on the benefits of the DTw for the robot cell at MTP Valgrinda.
1. Visibility: The DTw gives improved visibility and overview of the robot cell.
This would especially be beneficial when performing multi-robot planning
and control.
4. What if analysis: With a DTw that is able to imitate the physical robot
cell closely, it is possible to run simulations that would not be able in reality.
Such simulations would give insight to the limitations of the system and
how the interconnection between the components in the cell would react to
scenarios that could not be performed in the real world.
104
(a) Physical to virtual monitoring
(b) Self-monitoring
The fact that it was used OPC UA as a standard for communication would also
open up the possibility for connecting this DTw to other similar systems running
on OPC UA. In this way, this robot cell would be more equipped for future changes
and in line with the vision for I 4.0.
105
5.7 Value for I 4.0
As discussed in section 2.1 it is not clearly defined what the fourth industrial
revolution is, as this revolution is a prediction done apriori. It is therefore not
clear exactly what value a DTw will have in the revolutionized industry. However,
the work done in this project can be seen as in line with the recommendations
from the German Industrie 4.0 Working Group presented in section 2.1, which
includes the vision and basic technologies promoted by the idea of I 4.0. The
project delivers value to the following key areas presented by the group:
1. Standardization and reference architecture
As mentioned in section 2.5, OPC UA is listed as the one and only recom-
mendation for realizing the communication layer of RAMI 4.0. Using this
standard make it possible for this DTw to connect to other systems and ex-
pand its functionality as more of the industrial communication is done with
OPC UA. I 4.0 will involve networking and integration of systems across
different industries. For this DTw, this networking could include a common
platform for communication with other academical institutions or companies
which has implemented OPC UA in their systems.
2. Managing complex systems
The increasing complexity of systems used in industry presents a need for
management that is different from the systems traditionally used. In these
systems robots no longer just do repetitive work, but work autonomously
with the help of artificial intelligence and the use of sensor technology such
as computer vision. The DTw can facilitate and organize a complex system
that is continuously changing. Through the processing of data from the
physical robot cell, a DTw could work as a control centre for the user and
make handling the system easier.
6. Training and continuing professional development
As the robot cell at MTP Valgrinda is part of an educational system, it will
quickly change users and undergo personnel with varying degree of relevant
competence. An operating DTw with a library developed for future use could
help users of the robot cell in their respective work. Also, a robot cell that
is easier to use through control and analysis in VC could facilitate users in a
way that is beneficial for the educational program at the faculty. This could
potentially lead to more students at NTNU gaining insights to the use of this
robot sell and its use for I 4.0.
106
8. Resource efficiency
As discussed in section 5.6, the robot cell could potentially save energy by
using the DTw to simulate scenarios and testing code. EC could also be
decreased by comparing the physical data to the digital and historical data.
Testing in a digital environment before testing in the physical robot cell
could also reduce wear on the components in the cell and thereby extend the
lifetime of the components while at the same time reduce the risk associated
with trying out new software on the system.
107
108
Chapter 6
Conclusion
In this thesis, implementation of a DTw was studied. This study was done with
a focus on how a DTw can be implemented using OPC UA, according to goal a)
section 1.1. Using a DTw as a virtual representation for the robot cell at MTP
Valgrinda described in section 2.9 was considered to have various benefits. The
use of Visual Components 4.0 was measured up against similar simulation software
according to goal b) section 1.1. VC 4.0 was seen as the strongest candidate for
use in this project, and the software was used to make a virtual representation of
the robot cell, according to goal c) section 1.1.
Most of the work done in this project revolved around creating communication
modules able to connect the physical robot cell with the virtual representation in
VC 4.0, through the use of OPC UA. The result of this work was a communication
library that has been made open-source on GitHub. The library contains the vir-
tual representation of the robot cell as well as the different communication modules
able to give the DTw various functionalities. This system is the first version of a
DTw for the robot cell at MTP Valgrinda, according to goal e) section 1.1. How-
ever, it is emphasized that this is an alpha version open for future development,
and not a finished product.
As the four KUKA KR C4 robot controllers in the robot cell do not include OPC
UA as an add-on technology, two different types of middleware were tested, ac-
cording to goal d) section 1.1. A variety of OPC UA servers based on KVP and
one server based on the RSI communication architecture were developed. When
comparing the two types of middleware, it was found that RSI would have a time-
delay to VC 4.0 that was about 46% of the KVP OPC UA server with the same
functionality. It was concluded that KVP could be seen as a solution more reliable,
but with more time-delay, than RSI.
109
The system was tried out in a fixed case, according to goal f) section 1.1. This was
a multi-robot case including two of the four robots in the cell. The KUKA KR 120
R2500 was controlled from VC 4.0 while the KR 16-2 was controlled externally
and mirrored in VC 4.0. From working with this case, it was concluded that
the DTw gave improved visibility to the operation and made it easier to analyze
the different sensor data from the robots. From this data, it was found that the
control from VC 4.0 poses the problem of the simulation not being able to deliver
a sufficient stream of iterative points, something that made the robot lag slightly
when moving. This should be analyzed more closely and improved in further work
with the DTw.
An analysis of the safety and security of the robot cell was done. It was concluded
that the system potentially could impose serious harm to both humans and envi-
ronment. As some of the functionalities in the DTw directly control physical assets
with allot of power, the DTw should be used with caution. It is also emphasized
that as the system is open-source and not approved or tested by any authorized
personnel, a permanent solution for professional purposes should be thoroughly
tested.
Before developing the DTw, it was stated that the general benefits for such systems
include visibility to operations of machines and systems, modeling techniques used
to predict future states, and a foundation for optimizing characteristic parameters
and improve the use of the physical asset that is modeled. After developing the
DTw, it was concluded that these benefits would also be valid for the robot cell at
MTP Valgrinda, as well as the DTw being a good foundation for statistical and
predictive analysis with Energy Consumption (EC) as one of the main areas of
interest. Analysis of EC data could both be done by comparing actual EC data
with simulated EC data, and by comparing real-time EC data with historical EC
data.
Concerning I 4.0, it was concluded that the DTw would represent one step in an
evolutionary process and give benefits such as a new foundation for managing a
complex system that can be used in training and professional development at the
institute. This would be beneficial for NTNU as OPC UA is listed as the one
and only recommendation for realizing the communication layer of the Reference
Architecture Model for Industrie 4.0 (RAMI 4.0) created by the German companies
ZVEI, VDMA and BITKOM, and will most likely be an essential part of the
industry of the future.
110
6.1 Further work
As mentioned in section 5.3 the Institute of MTP is already in contact with KUKA
Norway regarding the possibility for updating the KR C4 controllers in the robot
cell to KRC ROBOTstar, which has OPC UA built in [115]. This could improve
the communication flow of the system by removing middleware as illustrated in
figure 85. With a base in the hardware currently in the robot cell, further essential
work revolves around software development and improving the communication
modules that have been developed in this project. The first steps could be the
following:
111
As described in section 5.5 the visual representation of the robot cell in VC 4.0
include uncertainty about the orientation of the components, estimated to be
±10mm. Further work on this DTw should include a more accurate calibrated
model of the robot cell. This could be done for example by creating a detailed
point cloud of the cell with an accuracy below 10mm, and base the visual repre-
sentation in VC 4.0 on measurements from the point cloud.
Figure 90: Point cloud combined with VC 4.0 model, made by Eirik Njåstad
112
Figure 91: Future vision for OPC UA [32]
113
114
Bibliography
115
[11] Hannovermesse. 3d simulation with visual components. http:
//www.hannovermesse.de/product/3d-simulation-with-visual-
components/2238209/C171973. Accessed: 06-11-2017.
[12] KUKA. Technology kuka.sim. https://www.kuka.com/-/media/
kuka-downloads/imported/d5d0533e621841d296c8cc5ef289258d/
kuka_sim_en.p. Accessed: 12-11-2017.
[13] Siemens. Scada system simatic wincc v7. http://w3.siemens.com/mcms/
human-machine-interface/en/visualization-software/scada/Pages/
Default.aspx. Accessed: 06-11-2017.
[14] Eirik Bjørndal Njåstad. Robotic welding with corrections from 3d-camera.
https://brage.bibsys.no/xmlui/handle/11250/2351215. Accessed: 06-
06-2018.
[15] Direct Industry. Articulated robot / 6-axis / surface treatment / for
assembly. http://www.directindustry.com/prod/kuka-roboter-gmbh/
product-17587-1439831.html. Accessed: 08-11-2017.
[16] RobotWorx. Kuka kr c4 controller. https://www.robots.com/kuka/
controller/kr-c4. Accessed: 08-11-2017.
[17] Direct Industry. Articulated robot / 6-axis / surface treatment / for
assembly. http://www.directindustry.com/prod/kuka-roboter-gmbh/
product-17587-417160.html. Accessed: 08-11-2017.
[18] Direct Industry. Articulated robot / 6-axis / surface treatment / for
assembly. http://www.directindustry.com/prod/kuka-roboter-gmbh/
product-17587-417172.html. Accessed: 08-11-2017.
[19] KUKA. Kuka.robotsensorinterface 2.3. http://vip.gatech.edu/wiki/
images/3/3c/KUKARobotSensorInterface.pdf. Accessed: 05-12-2017.
[20] Oodles. Tcp vs udp. https://www.oodlestechnologies.com/blogs/Why-
UDP-is-preferred-for-Live-Streaming. Accessed: 06-06-2018.
[21] ResearchGate. Joints and links of robot. https://www.researchgate.net/
figure/Joints-and-Links-of-Robot_fig4_301895257. Accessed: 07-06-
2018.
[22] Villani Siciliano, Sciavicco and Oriolo. Robotics. Springer-Verlag London
Limited, London, England, 2010.
[23] Kevac Djuric, Filipovic. Graphical representation of the signifi-
cant 6r kuka robots spaces. https://pdfs.semanticscholar.org/a012/
0bf2649ac0b9db1080988ed8819e4c01a6a9.pdf. Accessed: 07-06-2018.
[24] ResearchGate. Denavit-hartenberg kinematic parameters. https:
//www.researchgate.net/figure/Denavit-Hartenberg-kinematic-
parameters-4_fig1_257067449. Accessed: 07-06-2018.
116
[25] KUKA. Kuka.opc server 4.1. http://supportwop.com/IntegrationRobot/
content/6-Syst%C3%A8mes_int%C3%A9grations/OPC-Server/
KST_OPC_Server_41_en.pdf. Accessed: 20-11-2017.
[26] F. Sanfilippo, L. I. Hatledal, H. Zhang, M. Fago, and K. Y. Pettersen. Jopen-
showvar: An open-source cross-platform communication interface to kuka
robots. In 2014 IEEE International Conference on Information and Au-
tomation (ICIA), pages 1154–1159, July 2014.
[27] April robotics laboratory. Apriltag. https://april.eecs.umich.edu/
software/apriltag.html. Accessed: 31-05-2018.
[28] Vincent Driessen. A successful git branching model. https://nvie.com/
posts/a-successful-git-branching-model/. Accessed: 01-06-2018.
[29] KUKA. Kuka invests in the factory of the future. https://www.kuka.com/
en-de/press/news/2017/12/kuka-akquiriert-visual-components. Ac-
cessed: 31-05-2018.
[30] SoliCAD. Visual components - robotická simulace. http://solicad.com/c/
visual-components-robotizace. Accessed: 02-12-2017.
[31] BDO. The middle market manufacturer’s roadmap to industry
4.0. https://www.bdo.com/insights/industries/manufacturing-
distribution/the-middle-market-manufacturer-s-roadmap-to-in-
(1)/the-middle-market-manufacturer-s-roadmap-to-indust. Ac-
cessed: 02-06-2018.
[32] Kudzai Manditereza. How opc ua works and its role in industrial
iot. https://www.linkedin.com/pulse/how-opc-ua-works-its-role-
industrial-iot-kudzai-manditereza/. Accessed: 02-12-2017.
[33] OPC Foundation. What is opc? https://opcfoundation.org/about/
what-is-opc/, 2017. Accessed: 05-10-2017.
[34] IGI Global. What is xml. https://www.igi-global.com/dictionary/
application-semantic-web-technology-business/32919, 2017. Ac-
cessed: 07-10-2017.
[35] OPC Foundation. Analyzer device integration
(adi). http://wiki.opcfoundation.org/index.php/
Analyzer_Device_Integration_(ADI), 2017. Accessed: 06-10-2017.
[36] Miriam Schleipen, Syed-Shiraz Gilani, Tino Bischoff, and Julius Pfrommer.
Opc ua & industrie 4.0 - enabling technology with high diversity and vari-
ability. Procedia CIRP, 57:315–320, 2016.
[37] IGI Global. What is ict. https://www.igi-global.com/dictionary/
enhancing-autonomy-persons-intellectual-impairments/13620, 2017.
Accessed: 07-06-2018.
117
[38] Microsoft. The component object model. https://msdn.microsoft.com/en-
us/library/windows/desktop/ms694363(v=vs.85).aspx, 2018. Accessed:
07-06-2018.
[39] Microsoft. Distributed component object model. https:
//msdn.microsoft.com/en-us/library/6zzy7zky.aspx, 2017. Accessed:
01-06-2018.
[40] Microsoft. Object linking and embedding. https://msdn.microsoft.com/
en-us/library/19z074ky.aspx, 2018. Accessed: 03-05-2018.
[41] ITU. Internet of things global standards initiative. http://www.itu.int/
en/ITU-T/gsi/iot/Pages/default.aspx, 2016. Accessed: 19-10-2017.
[42] Cisco. What is the internet of everything? http://
ioeassessment.cisco.com/learn/ioe-faq, 2017. Accessed: 06-11-2017.
[43] S. F. Su, I. J. Rudas, J. M. Zurada, M. J. Er, J. H. Chou, and D. Kwon.
Industry 4.0: A special section in ieee access. IEEE Access, 5:12257–12261,
2017.
[44] Microsoft. Understanding service-oriented architecture. https://
msdn.microsoft.com/en-us/library/aa480021.aspx, 2017. Accessed: 03-
06-2018.
[45] Interaction Design Foundation. Socio-technical systems. https:
//www.interaction-design.org/literature/topics/socio-technical-
systems, 2017. Accessed: 06-06-2018.
[46] M.M.M. Sarcar K. Lalit Narayan, K. Mallikarjuna Rao. Computer Aided
Design in Manufacturing. Prentice-Hall of India Private Limited, New Delhi,
India, 2008.
[47] ABCO Automation. 3d simulation software: Visual components 4.0. https:
//goabco.com/3d-simulation-software/, 2017. Accessed: 06-11-2017.
[48] Microsoft. .net. https://www.microsoft.com/net/. Accessed: 06-11-2017.
[49] Vangie Beal. Api - application program interface. https://
www.webopedia.com/TERM/A/API.html. Accessed: 06-06-2018.
[50] Arun Singh. The active impact of human computer interaction (hci) on
economic, cultural and social life. The IIOAB Journal, 8:141–146, 09 2017.
[51] Dzineia. Scada systems. http://www.dzineia.com/scada-systems/. Ac-
cessed: 06-06-2018.
[52] Massimiliano Fago. openshowvar. https://github.com/cyberpro4/
openshowvar/blob/master/src/openshowvardock.cpp. Accessed: 25-11-
2017.
118
[53] Computer Hope. Graphical user interface. https://www.computerhope.com/
jargon/g/gui.htm. Accessed: 10-05-2018.
[54] OMG. Mda - the architecture of choice for a changing world. https://
www.omg.org/mda/, 2018. Accessed: 06-06-2018.
[55] Dr. Michael Grieves. Digital twin: Manufacturing excellence through virtual
factory replication. http://innovate.fit.edu/plm/documents/doc_mgr/
912/1411.0_Digital_Twin_White_Paper_Dr_Grieves.pdf, 2014. Accessed:
19-10-2017.
[56] Gartner. Gartner top 10 strategic technology trends for 2018.
https://www.gartner.com/smarterwithgartner/gartner-top-10-
strategic-technology-trends-for-2018/. Accessed: 03-06-2018.
[57] Forbes. What is digital twin technology. https://www.forbes.com/sites/
bernardmarr/2017/03/06/what-is-digital-twin-technology-and-
why-is-it-so-important/#40fb27fa2e2a, 2017. Accessed: 18-10-2017.
[58] Kasey Panetta. Gartner’s top 10 strategic technology trends for
2017. https://www.gartner.com/smarterwithgartner/gartners-top-
10-technology-trends-2017/, 2016. Accessed: 05-11-2017.
[59] Roland Rosen, Georg von Wichert, George Lo, and Kurt D. Bettenhausen.
About the importance of autonomy and digital twins for the future of man-
ufacturing. IFAC-PapersOnLine, 48(3):567 – 572, 2015. 15th IFAC Sympo-
sium onInformation Control Problems inManufacturing.
[60] Elisa Negri, Luca Fumagalli, and Marco Macchi. A review of the roles of dig-
ital twin in cps-based production systems. Procedia Manufacturing, 11:939
– 948, 2017. 27th International Conference on Flexible Automation and
Intelligent Manufacturing, FAIM2017, 27-30 June 2017, Modena, Italy.
[61] Mario Hermann, Tobias Pentek, and Boris Otto. Design principles for in-
dustrie 4.0 scenarios. In System Sciences (HICSS), 2016 49th Hawaii Inter-
national Conference, pages 3928–3937. IEEE, 2016.
[62] Department of Industry Innovation Australian Government and Science.
Industry 4.0. https://industry.gov.au/industry/Industry-4-0/Pages/
default.aspx, 2018. Accessed: 11-06-2018.
[63] Forbes. What everyone must know about industry 4.0. https:
//www.forbes.com/sites/bernardmarr/2016/06/20/what-everyone-
must-know-about-industry-4-0/#37a8375c795f, 2016. Accessed:
11-06-2018.
[64] Rainer Drath and Alexander Horch. Industrie 4.0: Hit or hype? [industry
forum]. Industrial Electronics Magazine, IEEE, 8(2):56–58, June 2014.
[65] Dr. Johannes Helbig Prof. Dr. Henning Kagermann, Prof. Dr. Wolf-
gang Wahlster. Recommendations for implementing the strategic initiative
119
industrie 4.0. In Securing the future of German manufacturing industry.
acatech – National Academy of Science and Engineering, 2013. Accessed:
06-11-2017.
[66] Roland Rosen, Georg Von Wichert, George Lo, and Kurt D. Bettenhausen.
About the importance of autonomy and digital twins for the future of man-
ufacturing. IFAC PapersOnLine, 48(3):567–572, 2015.
[67] Oracle. Digital twins for iot applications. http://www.oracle.com/
us/solutions/internetofthings/digital-twins-for-iot-apps-wp-
3491953.pdf, 2017. Accessed: 19-10-2017.
[68] Donna H. Rhodes Jack B. Reid. Digital system models: An investigation of
the non-technical challenges and research needs. Massachusetts Insttute of
Technology, 2016.
[69] US Department of Defense (DoD) w/Gary Hagan. Glossary: Defense acqui-
sition acronyms and terms. http://seari.mit.edu/documents/preprints/
REID_CSER16.pdf, 2015. Accessed: 05-11-2017.
[70] GE Digital. Minds + machines: Meet a digital twin. https://
www.youtube.com/watch?v=2dCz3oL2rTw, 2016. Accessed: 19-10-2017.
[71] Siemens. Twins with potential. https://www.siemens.com/customer-
magazine/en/home/industry/digitalization-in-machine-building/
the-digital-twin.html, 2017. Accessed: 18-10-2017.
[72] IBM. What is the internet of things? https://www.ibm.com/blogs/
internet-of-things/what-is-the-iot/, 2016. Accessed: 11-06-2018.
[73] Janakiram MSV. The role of device twins in industrial iot solutions.
https://www.forbes.com/sites/janakirammsv/2017/10/30/the-
role-of-device-twins-in-designing-industrial-iot-solutions/
#3b47a86820ac, 2017. Accessed: 05-11-2017.
[74] OPC Foundation. Classic. https://opcfoundation.org/about/opc-
technologies/opc-classic/, 2017. Accessed: 05-10-2017.
[75] OPC Foundation. Unified architecture. https://opcfoundation.org/
about/opc-technologies/opc-ua/, 2017. Accessed: 05-10-2017.
[76] Sven Goldstein OPC Foundation. Opc ua information model for ad-
vanced manufacturing (industry 4.0). https://www.hiindustryexpo.com/
activities/digitalisation-conference, 2017. Accessed: 05-05-2018.
[77] OPC Connect. Opc ua in the reference architecture model rami
4.0. http://opcconnect.opcfoundation.org/2015/06/opc-ua-in-the-
reference-architecture-model-rami-4-0/, 2017. Accessed: 05-11-2017.
[78] ZVEI Die Elektroindustrie. Zvei explains rami 4.0. https://
www.youtube.com/watch?v=NO4qWRVvMiU.
120
[79] D. A. Cruse. Meaning in Language: An Introduction to Semantics and Prag-
matics. Oxford University Press, 2004.
[80] GE w/Matt Wells. The industrial internet and interoperability. https:
//www.ge.com/digital/blog/opc-standard-offers-new-horizons, 2017.
Accessed: 05-11-2017.
[81] Wolfgang Mahnke Stefan-Helmut Leitner. Opc ua – service-oriented archi-
tecture for industrial applications. http://pi.informatik.uni-siegen.de/
stt/26_4/01_Fachgruppenberichte/ORA2006/07_leitner-final.pdf,
2006. Accessed: 06-06-2018.
[82] OMG. Object management group model driven architecture (mda), mda
guide rev. 2.0. http://www.smallake.kr/wp-content/uploads/2016/04/
ormsc-14-06-01-2.pdf, 2014. Accessed: 06-06-2018.
[83] Visual Components. A highly developed ready-made solution. http:
//www.visualcomponents.com/insights/case-studies/kuka/, 2014. Ac-
cessed: 06-11-2017.
[84] Siemens. Process simulate virtual commissioning. https:
//www.pmcorp.com/Portals/5/_Downloads/b07_Process%20Simulate%
20Virtual%20Commissioning.pdf, 2017. Accessed: 11-06-2018.
[85] Jot Automation Group. Annual report. http://web.lib.hse.fi/FI/
yrityspalvelin/pdf/1999/Eelektrobit1999.pdf, 1999. Accessed: 06-11-
2017.
[86] United States MI. Visual components company profile. https:
//www.robotics.org/company-profile-detail.cfm/Supplier/Visual-
Components/company/833, 2017. Accessed: 06-11-2017.
[87] Visual Components. Introducing visual components 4.0.4. http:
//www.visualcomponents.com/insights/blog/introducing-visual-
components-4-0-4/. Accessed: 04-12-2017.
[88] Visual Components. Essentials. http://www.visualcomponents.com/
products/visual-components-4-0/essentials/. Accessed: 06-11-2017.
[89] Visual Components. Visual components 4.0. http://
www.visualcomponents.com/products/visual-components-4-0/, 2017.
Accessed: 06-11-2017.
[90] Visual Components. Robots ecatalog. http://www.visualcomponents.com/
insights/blog/new-robots-ecatalog/, 2017. Accessed: 06-11-2017.
[91] Tyler Yanta. In-house solutions introduces octopuz robotic and sim-
ulation software. http://www.inhousesolutions.com/2014/01/house-
solutions-introduces-octopuz-robotic-simulation-software/, 2014.
Accessed: 06-11-2017.
121
[92] KUKA. Kuka.sim software. https://www.kuka.com/en-de/products/
robot-systems/software/planning-project-engineering-service-
safety/kuka_sim. Accessed: 12-11-2017.
[99] Manufacturing Institue. The new hire: How a new generation of robots
is transforming manufacturing. https://www.nist.gov/sites/default/
files/documents/mep/data/TheNewHire.pdf. Accessed: 06-06-2018.
[101] Ivar Eriksen. Setup and interfacing of a kuka robotics lab. https:
//github.com/aauc-mechlab/JOpenShowVar. Accessed: 09-05-2018.
[105] Simen Hagen Bredvold. Robotic welding of tubes with correction from
3d vision and force control. https://brage.bibsys.no/xmlui/bitstream/
handle/11250/2404424/15620_FULLTEXT.pdf?sequence=1. Accessed: 20-
11-2017.
122
[106] Mechatronics Lab at Ålesund University College. Jopenshowvar.
https://brage.bibsys.no/xmlui/bitstream/handle/11250/2490915/
16482_FULLTEXT.pdf?sequence=1&isAllowed=y. Accessed: 06-06-2018.
[107] Sintef. Sintef employees. https://www.sintef.no/en/all-employees/
employee/?empId=4104. Accessed: 22-11-2017.
[108] Olivier Roulet-Dubonnet. python-opcua. https://github.com/FreeOpcUa/
python-opcua. Accessed: 10-05-2018.
[109] Bradley Mitchell. 0.0.0.0. https://www.lifewire.com/four-zero-ip-
address-818384. Accessed: 10-05-2018.
[110] Python Software Foundation. Installing packages. https:
//packaging.python.org/tutorials/installing-packages/. Accessed:
10-05-2018.
[111] S. Kudrle, M. Proulx, P. Carrières, and M. Lopez. Fingerprinting for solving
a/v synchronization issues within broadcast environments. SMPTE Motion
Imaging Journal, 120(5):36–46, July 2011.
[112] KUKA. Kuka.robotsensorinterface 3.1. http://supportwop.com/
IntegrationRobot/content/6-Syst%C3%A8mes_int%C3%A9grations/
RobotSensorInterface/KST_RSI_31_en.pdf. Accessed: 15-05-2018.
[113] KUKA. Kuka system variables. http://www.wtech.com.tw/public/
download/manual/kuka/krc4/KUKA%20System%20Variables%208.1%
208.2%208.3.pdf. Accessed: 13-05-2018.
[114] ViSP. Visual servoing platform. http://visp-doc.inria.fr/doxygen/
visp-daily/index.html. Accessed: 31-05-2018.
[115] KUKA. Krc robotstar: a star among controllers. https://www.kuka.com/
en-de/products/robot-systems/robot-controllers/krc-robotstar.
Accessed: 05-12-2017.
[116] Visual Components. The importance of plc validation in manu-
facturing. http://www.visualcomponents.com/insights/articles/plc-
validation-manufacturing/. Accessed: 04-12-2017.
[117] KUKA. Introducing visual components 4.1. https://
www.visualcomponents.com/insights/blog/introducing-visual-
components-4-1/. Accessed: 31-05-2018.
[118] Nikola Zlatanov. Computer security and mobile security
challenges. https://www.researchgate.net/publication/
298807979_Computer_Security_and_Mobile_Security_Challenges,
12 2015. Accessed: 05-06-2018.
[119] Siemens. Safety for the manufacturing industry – functional safety
services. https://www.industry.siemens.com/datapool/industry/
123
industrysolutions/services/en/Safety-manufacturing-industry-
en.pdf. Accessed: 29-11-2017.
[120] Olivier Roulet-Dubonnet. Freeopcua. https://github.com/FreeOpcUa/
freeopcua. Accessed: 22-11-2017.
[121] KUKA. Kr cybertech. https://www.kuka.com/en-in/products/robotics-
systems/industrial-robots/kr-cybertech. Accessed: 19-11-2017.
[122] KUKA. Kr quantec pro. https://www.kuka.com/en-de/products/robot-
systems/industrial-robots/kr-quantec-pro. Accessed: 19-11-2017.
[123] KUKA. Kr c4. http://www.wtech.com.tw/public/download/manual/kuka/
krc4/KUKA%20KR%20C4%20Operating%20Instructions.pdf. Accessed: 19-
11-2017.
[124] OPC Foundation. Opc ua. https://opcfoundation.org/wp-content/
uploads/2014/05/OPC-UA_Overview_EN.pdf. Accessed: 03-12-2017.
124
Appendix A
Source Code
125
A.1 OPC UA Server, opcua-server02.py
1 ’’’
2 Source code for an OPC UA server able to extract axis variables
3 from a KUKA KR C4 robot controller and send them to an
4 OPC UA client.
5
17 import sys
18 import socket
19 sys.path.insert(0, "..")
20 import time
21 from opcua import ua, Server
22
23 class KUKA(object):
24 # Open socket
25 # KUKAVARPROXY actively listens on TCP port 7000
26 def __init__(self, TCP_IP):
27 try:
28 client.connect((TCP_IP, 7000))
29 except:
30 self.error_list(1)
31
126
46 msg.append((len(val) & 0xff00) >> 8)
47 msg.append((len(val) & 0x00ff))
48 msg.extend(map(ord, val))
49 temp.append(bool(val))
50 temp.append(((len(var)) & 0xff00) >> 8)
51 temp.append((len(var)) & 0x00ff)
52 temp.extend(map(ord, var))
53 msg = temp + msg
54 del temp[:]
55 temp.append((msgID & 0xff00) >> 8)
56 temp.append(msgID & 0x00ff)
57 temp.append((len(msg) & 0xff00) >> 8)
58 temp.append((len(msg) & 0x00ff))
59 msg = temp + msg
60 except :
61 self.error_list(2)
62 try:
63 client.send(msg)
64 return client.recv(1024)
65 except :
66 self.error_list(1)
67
127
93 self.error_list(2)
94
95 # Close socket
96 def disconnect (self):
97 client.close()
98
99 # In case of error
100 def error_list (self, ID):
101 if ID == 1:
102 print ("Network Error (tcp_error)")
103 print ("Check your KRC’s IP address on the network,
104 and make sure kukavarproxy is running.")
105 self.disconnect()
106 raise SystemExit
107 elif ID == 2:
108 print ("Python Error.")
109 print ("Check the code and uncomment the lines related to
110 your python version.")
111 self.disconnect()
112 raise SystemExit
113 elif ID == 3:
114 print ("Error in write() statement.")
115 print ("Variable value is not defined.")
116
128
140 A2 = myObject.add_variable(idx, "A2", 6.7)
141 A3 = myObject.add_variable(idx, "A3", 6.7)
142 A4 = myObject.add_variable(idx, "A4", 6.7)
143 A5 = myObject.add_variable(idx, "A5", 6.7)
144 A6 = myObject.add_variable(idx, "A6", 6.7)
145
146 A1.set_writable()
147 A2.set_writable()
148 A3.set_writable()
149 A4.set_writable()
150 A5.set_writable()
151 A6.set_writable()
152
156 try:
157 while(True):
158 time.sleep(0.1)
159
164 A1.set_value(output_elements[2])
165 A2.set_value(output_elements[4])
166 A3.set_value(output_elements[6])
167 A4.set_value(output_elements[8])
168 A5.set_value(output_elements[10])
169 A6.set_value(output_elements[12])
170
171 finally:
172 # Close OPC UA client connection, remove subcsriptions, etc
173 server.stop()
129
A.2 OPC UA Server GUI, opcua-server-gui.py
1 ’’’
2 Source code for a GUI able to execute different OPC UA servers
3 with a variety of functionalities for a KUKA robot:
4 − Send axis parameters to Visual Components 4.0
5 − Display sensor data of motors controlling each axis
6 (real−time plotting):
7 # Motor velocity
8 # Motor torque
9 # Motor current
10 # Motor temperature
11 − Write sensor data to .csv files
12
17 import tkinter as tk
18 from tkinter import ttk
19 from tkinter import messagebox
20 from tkinter import *
21 LARGE_FONT= ("Verdana", 12)
22 import sys
23 import subprocess
24
34 container = tk.Frame(self)
35 container.pack(side="top", fill="both", expand = True)
36 container.grid_rowconfigure(0, weight=1)
37 container.grid_columnconfigure(0, weight=1)
38
44 canvas_id = canvas.create_text(432,25)
45 canvas.itemconfig(canvas_id, text = "akselov@stud.ntnu.no")
130
46 canvas.insert(canvas_id, 12, "")
47
48 self.frames = {}
49
55 self.show_frame(StartPage)
56
61 class StartPage(tk.Frame):
62 def __init__(self, parent, controller):
63 tk.Frame.__init__(self,parent)
64 label = tk.Label(self, text="KUKA KR 16−2 Monitoring Application",
font=LARGE_FONT)
65 label.place(x=44,y=10)
66
81 close_button.place(x=65,y=140)
82
83 class PageOne(tk.Frame):
84 def __init__(self, parent, controller):
85 tk.Frame.__init__(self, parent)
86 label = tk.Label(self, text="Axis parameters to Visual Components
4.0", font=LARGE_FONT)
131
87 label.place(x=20,y=10)
88
97 class PageTwo(tk.Frame):
98 def __init__(self, parent, controller):
99 tk.Frame.__init__(self, parent)
100 label = tk.Label(self, text="Write data to file", font=LARGE_FONT)
101 label.place(x=120,y=10)
102
132
StartPage))
131 button0.place(x=65,y=50)
132
133
167 subprocess.call([sys.executable, ’server_gui_motorTemperature.py’, ’
argument1’, ’argument2’])
168
134
A.3 XML-file, KUKA_KR16_2_object.xml
1 <!−−
2 Source code for a XML file including information about
3 variables relevant for a KUKA KR 16−2 robot.
4
135
41 </UAObject>
42 <UAVariable BrowseName="0:KUKAVARPROXY" DataType="Boolean" NodeId="i
=20006" ParentNodeId="i=20001">
43 <DisplayName>KUKAVARPROXY</DisplayName>
44 <Description>KUKAVARPROXY</Description>
45 <References>
46 <Reference IsForward="false" ReferenceType="HasProperty">i=20001</
Reference>
47 <Reference ReferenceType="HasTypeDefinition">i=68</Reference>
48 </References>
49 <Value>
50 <uax:Boolean>false</uax:Boolean>
51 </Value>
52 </UAVariable>
53 <UAVariable BrowseName="0:RSI" DataType="Boolean" NodeId="i=20007"
ParentNodeId="i=20001">
54 <DisplayName>RSI</DisplayName>
55 <Description>RSI</Description>
56 <References>
57 <Reference IsForward="false" ReferenceType="HasProperty">i=20001</
Reference>
58 <Reference ReferenceType="HasTypeDefinition">i=68</Reference>
59 </References>
60 <Value>
61 <uax:Boolean>false</uax:Boolean>
62 </Value>
63 </UAVariable>
64 <UAVariable BrowseName="0:A1" DataType="Double" NodeId="i=20008"
ParentNodeId="i=20001">
65 <DisplayName>A1</DisplayName>
66 <Description>A1</Description>
67 <References>
68 <Reference IsForward="false" ReferenceType="HasComponent">i=20001</
Reference>
69 <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
70 </References>
71 <Value>
72 <uax:Double>9.99</uax:Double>
73 </Value>
74 </UAVariable>
75 <UAVariable BrowseName="0:A2" DataType="Double" NodeId="i=20009"
ParentNodeId="i=20001">
76 <DisplayName>A2</DisplayName>
77 <Description>A2</Description>
78 <References>
79 <Reference IsForward="false" ReferenceType="HasComponent">i=20001</
Reference>
136
80 <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
81 </References>
82 <Value>
83 <uax:Double>9.99</uax:Double>
84 </Value>
85 </UAVariable>
86 <UAVariable BrowseName="0:A3" DataType="Double" NodeId="i=20010"
ParentNodeId="i=20001">
87 <DisplayName>A3</DisplayName>
88 <Description>A3</Description>
89 <References>
90 <Reference IsForward="false" ReferenceType="HasComponent">i=20001</
Reference>
91 <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
92 </References>
93 <Value>
94 <uax:Double>9.99</uax:Double>
95 </Value>
96 </UAVariable>
97 <UAVariable BrowseName="0:A4" DataType="Double" NodeId="i=20011"
ParentNodeId="i=20001">
98 <DisplayName>A4</DisplayName>
99 <Description>A4</Description>
100 <References>
101 <Reference IsForward="false" ReferenceType="HasComponent">i=20001</
Reference>
102 <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
103 </References>
104 <Value>
105 <uax:Double>9.99</uax:Double>
106 </Value>
107 </UAVariable>
108 <UAVariable BrowseName="0:A5" DataType="Double" NodeId="i=20012"
ParentNodeId="i=20001">
109 <DisplayName>A5</DisplayName>
110 <Description>A5</Description>
111 <References>
112 <Reference IsForward="false" ReferenceType="HasComponent">i=20001</
Reference>
113 <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
114 </References>
115 <Value>
116 <uax:Double>9.99</uax:Double>
117 </Value>
118 </UAVariable>
119 <UAVariable BrowseName="0:A6" DataType="Double" NodeId="i=20013"
ParentNodeId="i=20001">
137
120 <DisplayName>A6</DisplayName>
121 <Description>A6</Description>
122 <References>
123 <Reference IsForward="false" ReferenceType="HasComponent">i=20001</
Reference>
124 <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
125 </References>
126 <Value>
127 <uax:Double>9.99</uax:Double>
128 </Value>
129 </UAVariable>
130 <UAVariable BrowseName="0:IP" DataType="String" NodeId="i=20016"
ParentNodeId="i=20001">
131 <DisplayName>IP</DisplayName>
132 <Description>IP</Description>
133 <References>
134 <Reference IsForward="false" ReferenceType="HasProperty">i=20001</
Reference>
135 <Reference ReferenceType="HasTypeDefinition">i=68</Reference>
136 </References>
137 <Value>
138 <uax:String>192.168.250.16</uax:String>
139 </Value>
140 </UAVariable>
141 </UANodeSet>
138
A.4 OPC UA Server XML, opcua-server-xml.py
1 ’’’
2 Source code for an OPC UA server able to create a custom OPC UA
3 object from specifications in a XML−file, estract axis variables
4 from a KUKA KR C4 robot controller and send them to an OPC UA client.
5
17 import sys
18 sys.path.insert(0, "..")
19 import time
20 import KUKAVARPROXY_communication
21 from opcua import ua, Server
22
23 if __name__ == "__main__":
24 # Setup OPC−UA Server
25 server = Server()
26 server.set_endpoint("opc.tcp://192.168.250.200:50895")
27
28 # Setup namespace
29 uri = "http://examples.freeopcua.github.io"
30 idx = server.register_namespace(uri)
31
139
45 variables_all = kuka_object.get_children()
46
59 try:
60 print("Select KUKAVARPROXY or RSI communication configuration in
OPC UA Client")
61 while True:
62 time.sleep(0.1)
63
69 if(variable_kukavarproxy.get_value()):
70 print("Launching KUKAVARPROXY communication")
71 KUKAVARPROXY_communication.run(variables_axis, ip_adress)
72 break
73
74 if(variable_rsi.get_value()):
75 print("Launching RSI communication")
76 #Run RSI communication module
77
78 finally:
79 # Close OPC UA client connection, remove subcsriptions, etc
80 server.stop()
140
A.5 Module, KUKAVARPROXY_communication.py
1 ’’’
2 Source code for a KUKAVARPROXY communication module being used
3 by the opcua−server−xml.py for communication with a KUKA KR C4
4 robot controller. The module extracts axis variables from the
5 robot controller.
6
16 import socket
17 import time
18
22 class KUKA(object):
23
24 # Open socket
25 # KukaVarProxy actively listens on TCP port 7000
26 def __init__(self, TCP_IP):
27 try:
28 client.connect((TCP_IP, 7000))
29 except:
30 self.error_list(1)
31
141
46 msg.append((len(val) & 0xff00) >> 8)
47 msg.append((len(val) & 0x00ff))
48 msg.extend(map(ord, val))
49 temp.append(bool(val))
50 temp.append(((len(var)) & 0xff00) >> 8)
51 temp.append((len(var)) & 0x00ff)
52 temp.extend(map(ord, var))
53 msg = temp + msg
54 del temp[:]
55 temp.append((msgID & 0xff00) >> 8)
56 temp.append(msgID & 0x00ff)
57 temp.append((len(msg) & 0xff00) >> 8)
58 temp.append((len(msg) & 0x00ff))
59 msg = temp + msg
60 except :
61 self.error_list(2)
62 try:
63 client.send(msg)
64 return client.recv(1024)
65 except :
66 self.error_list(1)
67
142
92
93 # Close socket
94 def disconnect (self):
95 client.close()
96
97 # In case of error
98 def error_list (self, ID):
99 if ID == 1:
100 print ("Network Error (tcp_error)")
101 print ("Check your KRC’s IP address on the network, and make
sure kukaproxyvar is running.")
102 self.disconnect()
103 raise SystemExit
104 elif ID == 2:
105 print ("Python Error.")
106 print ("Check the code and uncomment the lines related to your
python version.")
107 self.disconnect()
108 raise SystemExit
109 elif ID == 3:
110 print ("Error in write() statement.")
111 print ("Variable value is not defined.")
112
125 i = 0
126 for axis in variables_axis:
127 axis.set_value(output_elements[2+i])
128 i+=2
143
A.6 RSI OPC UA Server, rsi-opcua-server.py
1 ’’’
2 Source code for an OPC UA server able to extract axis variables
3 from a KUKA KR C4 robot controller through RSI communication
4 and send them to an OPC UA client.
5
15 import socket
16 import numpy
17 import command
18 import time
19 import time_keeper
20 from opcua import ua, Server
21
22 tiden = time_keeper.time_keeper()
23
24 def simple_joint_correction_command(from_kuka):
25 #Correction example (not used)
26 A1 = 10.0 * numpy.sin(tiden.get() / 10.0)
27 joint_desired = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
28 return command.joint_correction_command(from_kuka, joint_desired)
29
30 if __name__ == "__main__":
31 # Initialize OPC−UA Client connection
32 client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
33
38 # Setup namespace
39 uri = "http://examples.freeopcua.github.io"
40 idx = server.register_namespace(uri)
41
144
45 # Populating address space
46 myObject = objects.add_object(idx, "KUKA_KR16_2")
47
56 A1.set_writable()
57 A2.set_writable()
58 A3.set_writable()
59 A4.set_writable()
60 A5.set_writable()
61 A6.set_writable()
62
72 try:
73 while True:
74 # Buffer size is 1024 bytes
75
76 #time.sleep(0.1)
77 received_data, socket_of_krc = sock.recvfrom(BUFFER_SIZE)
78 received_data = received_data.decode("utf−8")
79 output_elements = received_data.split(’ ’)
80 raw_axis_variables = output_elements[14:20]
81 axis_variables = []
82
86 print(axis_variables)
87
88 A1.set_value(axis_variables[0])
89 A2.set_value(axis_variables[1])
90 A3.set_value(axis_variables[2])
91 A4.set_value(axis_variables[3])
145
92 A5.set_value(axis_variables[4])
93 A6.set_value(axis_variables[5])
94
95 kuka_command = simple_joint_correction_command(received_data)
96 reply = bytes(kuka_command, ’utf−8’)
97 sock.sendto(reply, socket_of_krc)
98
99 finally:
100 # Close OPC UA client connection, remove subcsriptions, etc
101 server.stop()
146
A.7 KVP Control Server, opcua-control-server.py
1 ’’’
2 Source code for an OPC UA server able to create a custom OPC UA
3 object from specifications in a XML−file, extract axis variables
4 from an OPC UA client and control a KUKA robot by its axis values.
5 Based on work by:
6
13 To be done:
14 − Custom setup of axis variables from OPC UA object
15 − Include RSI control module
16
21 import sys
22 import socket
23 sys.path.insert(0, "..")
24 import time
25 from xml.dom import minidom
26 from opcua import ua, Server
27
28 class KUKA(object):
29 # Open socket
30 # KukaVarProxy actively listens on TCP port 7000
31 def __init__(self, TCP_IP):
32 try:
33 client.connect((TCP_IP, 7000))
34 except:
35 self.error_list(1)
36
147
46 try:
47 msg = bytearray()
48 temp = bytearray()
49 if val != "":
50 val = str(val)
51 msg.append((len(val) & 0xff00) >> 8)
52 msg.append((len(val) & 0x00ff))
53 msg.extend(map(ord, val))
54 temp.append(bool(val))
55 temp.append(((len(var)) & 0xff00) >> 8)
56 temp.append((len(var)) & 0x00ff)
57 temp.extend(map(ord, var))
58 msg = temp + msg
59 del temp[:]
60 temp.append((msgID & 0xff00) >> 8)
61 temp.append(msgID & 0x00ff)
62 temp.append((len(msg) & 0xff00) >> 8)
63 temp.append((len(msg) & 0x00ff))
64 msg = temp + msg
65 except :
66 self.error_list(2)
67 try:
68 client.send(msg)
69 return client.recv(1024)
70 except :
71 self.error_list(1)
72
148
93 if val != (""):
94 return self.__get_var(self.send(var,val,msgID))
95 else:
96 raise self.error_list(3)
97 except :
98 self.error_list(2)
99
120
149
138
171 try:
172 # Give user information about communication that is being used
173 print("Kuka: ", variable_kukavarproxy.get_value())
174 print("RSI: ", variable_rsi.get_value())
175 print("Select KUKAVARPROXY or RSI control configuration in OPC UA
Client")
176
150
182 raise ValueError("KUKAVARPROXY and RSI can not both be
selected")
183 SystemExit
184
185 if(variable_kukavarproxy.get_value()):
186 print("Launching KUKAVARPROXY control")
187 robot.write("TARGET_AXIS","{E6AXIS: A1 " + str(A1_var.
get_value()) + ", A2 " + str(A2_var.get_value()) + ", A3 " + str(
A3_var.get_value()) + ", A4 " + str(A4_var.get_value()) + ", A5 " +
str(A5_var.get_value()) + ", A6 " + str(A6_var.get_value()) + "}")
188
189 if(variable_rsi.get_value()):
190 print("Launching RSI control")
191 #Run RSI control module
192
193 finally:
194 # Close OPC UA client connection, remove subcsriptions, etc
195 server.stop()
151
152
Appendix B
Digital Appendix
153