Digital Twin For Human-Robot Collaboration

Computers & Industrial Engineering 187 (2024) 109764
Contents lists available at ScienceDirect
Computers & Industrial Engineering

journal homepage: www.elsevier.com/locate/caie
Digital twin for human-robot collaboration enhancement in manufacturing

systems: Literature review and direction for future developments
Alessio Baratta a, Antonio Cimino b, Francesco Longo a, *, Letizia Nicoletti c
a
University of Calabria, Department of Mechanical, Energy and Management Engineering, Italy
b
University of Salento, Department of Innovation Engineering, Italy
c
Cal-Tek Srl, Italy
A R T I C L E I N F O A B S T R A C T
Keywords: The integration of robots in manufacturing systems known as Human Robot Collaboration (HRC) can provide
Human-Robot Collaboration relevant opportunities in terms of productivity, safety and quality control although there are still many chal
Industry 4.0 lenges such as integration costs, complexity, safety and workforce concerns, skills gaps and regulatory compli
Digital Twin
ance. Digital Twins (DTs), which are virtual replicas of physical systems, are being explored as a promising
Simulation software
Manufacturing systems
solution to address these challenges. Therefore, this paper delves into the potential DTs-based approaches for
HRC in manufacturing systems with a focus on their ability to provide a realistic simulation environment for
testing and optimizing collaboration strategies. The first part of the study involves a comprehensive literature
review devoted to understand how the scientific community perceives the application of DTs for HRC in
manufacturing systems and the main trends of this research area are identified. The second part of the study
presents a detailed analysis of the currently available commercial simulation software solutions for developing
DT models in manufacturing systems for HRC scenarios. The analysis encompassed various aspects such as: the
software’s capabilities in human and robot simulation, ergonomic and time measurements analysis, data ex
change and interoperability with the physical world as well as with other software. The results of the analysis are
meant to provide practical insights to researchers and practitioners interested in implementing DTs to enhance
HRC in manufacturing systems.
1. Introduction By integrating human-robot collaboration into smart factories, manu

facturers can achieve a balance between automation and the human
Industry 4.0 has facilitated the development of smart factories workforce, taking advantage of the strengths of both. This approach not
through the integration of various technologies such as robotics, artifi only mitigates the risk of neglecting human factors but also leverages
cial intelligence, and the Internet of Things, which has led to highly human skills, creativity, and problem-solving abilities to optimize
optimized, automated and interconnected manufacturing processes (Lu, manufacturing processes and drive innovation. HRC represents a subset
2017). The integration of these technologies in Industry 4.0 has led to of the broader concept of Human-Robot Interaction (HRI) but with a
the creation of highly efficient, agile, and adaptable manufacturing higher level of integration between the human operator and the robot
environments. Smart factories can respond quickly to changing market (Müller et al., 2017). As previously described by Hoffman and Breazeal
demands, reduce waste, improve product quality, and enhance overall (2004), the main features that define HRC include: shared activity
productivity, making them a key driver of innovation and competi (Searle, 1990; Bratman, 1992), joint intention (Cohen and Levesque,
tiveness in modern manufacturing industries. However, with the 1991), common ground (Clark, 1996), and goal-oriented action (Hoff
increased level of automation, the risk of neglecting human-related man and Breazeal, 2004; Bratman, 1992; Cohen and Levesque, 1991). In
factors in manufacturing processes arises (Selvaggio et al., 2021). In HRC, the human operator and the robot agent are in close contact, and a
this framework, HRC has become even more necessary as it involves the leader–follower relationship may naturally occur, with the leader taking
interaction between humans and robots working together in a shared responsibility for executing a given task (Wang et al., 2019; Wang et al.,
workspace to complete a task (Huang et al., 2021; Boschetti et al., 2023). 2017). Moreover, HRC is often expressed in the form of a complex
* Corresponding author at: Via Pietro Bucci 45C, Rende, CS 87036, Italy.
E-mail address: [email protected] (F. Longo).
https://doi.org/10.1016/j.cie.2023.109764
Received 11 May 2023; Received in revised form 11 October 2023; Accepted 16 November 2023
Available online 24 November 2023
0360-8352/© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
A. Baratta et al. Computers & Industrial Engineering 187 (2024) 109764
system in which the job can consist of a set of repetitive tasks (as can Table 1
happen in assembly scenarios) and high responsiveness is needed to Comparison between our paper and other literature reviews.
ensure human operator’s safety. Here, Iterative Learning Control (ILC) Reference Reviewed topics
algorithm optimization, can enhance robots performances in tasks that
Grieves (2014) Early perspective on digital twins and their
require precision and repeatability when collaborating with humans applications in manufacturing
(Zhou et al., 2022; Zhuang et al., 2023). In addition, robots may need Saddik (2018) The role of multimedia technologies in the
adaptive control in response to changing environments or human ac development and use of digital twins
tions. It can be achieved by neural networks and adaptive control Huang et al. (2021) AI-driven DTs in smart manufacturing and advanced
robotics
methods that can help robots to adjust their behavior and interactions Leng et al. (2021) Digital twin-based smart manufacturing
with humans, making the collaboration safer and more efficient (Song Rathore et al. (2021) Overview of digital twins and their integration with
et al., 2023). In this framework, the feasibility of supporting HRC big data analytics
through the use of digital twins has shown relevant potential benefits. Wilhelm et al. (2021) Main applications of DT-based HMI in manufacturing
and definition of roles which both humans and DTs
Recalling that a digital twin is a virtual representation or digital coun
adopt in manufacturing systems
terpart of a physical object, system, or process (Tao et al., 2019), DTs García et al. (2022) DT learning applications in manufacturing systems
enable dynamic, real-time simulation of physical entities and this allows Liang et al. (2022) DT enabling technologies for manufacturing and
monitoring, analysis, and optimization. The pivotal importance of DT robotics
has been so widely recognized to the extent that the need for a standard Sharma et al. (2022) Overview of the state-of-the-art in digital twin
technology, applications, challenges, and future
definition of the DT concept has emerged to lay the foundation upon directions
which the digital twin ecosystem can grow, evolve, and mature while Ramasubramanian et al. Approaches used to create DTs for HRC in
ensuring consistency and alignment across different sectors and appli (2022) manufacturing
cations (Coelho et al., 2021). Thus, DT may be defined as “a virtual Stavropoulos and Mourtzis Role of digital twins in Industry 4.0 and their
(2022) applications across various domains
representation of manufacturing elements such as personnel, products,
Correia et al. (2023) Data integration aspects of digital twins and how data
assets and process definitions, a living model that continuously updates from various sources are used to create and maintain
and changes as the physical counterpart changes to represent status, digital twins
working conditions, product geometries and resource states in a syn Our contribution DTs for HRC enhancement in manufacturing systems:
chronous manner” (ISO 23247, 2021). (Lu et al., 2020). When it comes main trends, key requirements and commercial
software benchmark
to HRC-based DTs, virtual models of human operators are used to
analyze their interaction with robots in a virtual environment (Lu, 2017)
considering ergonomic aspects of the workspace and how to optimize The reminder of the paper is organized as follows: Section 2 provides
HRC scenarios. As a matter of facts, ergonomics is crucial in HRC sce an overview of the materials and methodologies employed in the study;
narios (Gualtieri et al., 2021) where the workspace design should reduce Section 3 offers the primary findings, encompassing the results of the
the physical and mental workload of the human operator as much as systematic literature review and the identification of key prerequisites,
possible and avoid any physical strain or injury. Here, DTs allow for tools, and limitations for improving Human-Robot Collaboration
planning, simulating, monitoring, and optimizing human-robot collab through Digital Twins in manufacturing systems; Section 4 e delves into
oration (Wang et al., 2024). Besides, in conjunction with sensory data, the importance and implications of the proposed research while the last
big data analytics, AI and machine learning, Digital Twins can support section draws conclusions and future research directions.
diagnostics, prognostics, and reducing accidents and injuries (Zaccaria
et al., 2018; Cai et al., 2017). 2. Materials and methods
Several literature reviews investigating the use of DTs in
manufacturing systems have been carried out as summarized in Table 1. The study includes two complementary research streams. The first
The background summarized in Table 1 shows that DTs have been involves a systematic exploration of the scientific literature related to
widely investigated in terms of applications, technologies and data the implementation of Digital Twins for Human-Robot Collaboration in
integration but there is a lack of research in the area of DTs for HRC manufacturing systems (RQ1, RQ2). The second entails an analysis of
enhancement. In fact, the state-of-the-art shows that a proper consid the most frequently used solutions, tools and approaches for simulation
eration of human-related factors is a compelling research issue DTs models within a Digital Twin framework including both human and
design (Lu et al., 2020). Given that, the proposed research is aimed at robotic models (RQ3, RQ4).
investigating the potential for digital twin technology to enhance The literature review has been carried out using the PRISMA meth
human-robot collaboration. While detecting and analyzing meaningful odology (Moher et al., 2009; Page et al., 2021), as depicted in Fig. 1 and
applications of DTs for HRC enhancement and future research di summarized hereafter:
rections, a particular attention is paid on the latest tools and best
practices for successful implementation of DT in HRC scenarios. These (1) Identification: Carry out a detailed search for scientific docu
considerations following research questions (RQs): ments in databases like Scopus and PubMed, based on research
questions set out in Section 1. The process includes selecting
- RQ1: What are the most recent and impactful applications of Digital databases, developing search queries with relevant keywords,
Twins in enhancing Human-Robot Collaboration across different executing searches, and finding additional documents from
industries? various sources.
- RQ2: What are the essential factors that must be considered when (2) Screening: Filter the documents to get those pertinent to the
developing Digital Twins for enhancing Human-Robot Collaboration research questions. This involves applying predefined inclusion
in manufacturing systems? and exclusion criteria, screening titles and abstracts first, and
- RQ3: Which solutions and tools are appropriate for developing and keeping records of the screening process.
deploying Digital Twins specifically designed for Human-Robot (3) Eligibility and Inclusion: Evaluate the filtered documents for
Collaboration in the context of manufacturing systems? quality, remove duplicates, and screen full texts to select final
- RQ4: Which constraints and obstacles are to be considered when documents for data extraction and analysis.
implementing Digital Twins in manufacturing systems to enhance
Human-Robot Collaboration (HRC)? For software analysis, a technical approach was used to evaluate
2
Fig. 1. Literature Review Methodology.
different tools’ abilities to integrate simulation models for HRC contexts Ergonomic Analysis, Time Measurement Analysis, and Data Ex
(see Fig. 2). The analysis included structured testing, reviewing support change and Interoperability. The outcomes of this rigorous evalua
documentation, and engaging in community forums. The goal was to tion process were compiled into a report, which included a
identify and document each tool’s features, advantages, and limitations categorization and discussion of the discerned features and re
without ranking them, providing an impartial assessment of the avail quirements of the selected software.
able options.
The methodology used for the software review was structured into This methodological approach has supported a systematic assess
three main steps: ment of software tools within the context of Digital Twins for collabo
rative human-robot scenarios. It provided valuable insights into their
1. Selection: In the initial step, we identified the most pertinent soft capabilities and areas for enhancement.
ware tools based on the existing scientific literature. Our focus was
on tools that had gained recognition within the academic commu 3. Results
nity, which closely aligns with the literature review conducted in the
first part of this study. Section 3.1 provides a systematic literature review of existing
2. Exploration: This multifaceted phase consisted of three pivotal sub- research on DTs for HRC enhancement in manufacturing systems. In
phases. In the first sub-phase, named ’Testing,’ we pinpointed and Section 3.2, the key requirements for developing simulation models in a
tested the core features of each chosen software to establish a DT framework for HRC in manufacturing systems are highlighted. Sec
foundational understanding of their capabilities and functionalities. tion 3.3, explores the potentials of currently available tools for DTs
The second sub-phase, ’Manufacturer Support Material,’ involved aimed at HRC enhancement in manufacturing systems. Finally, section
the analysis of support materials offered by the software providers, 3.4 presents a comparative analysis of software tools detected in Sec
including manuals, tutorials, and case studies. These resources pro tions 3.3. based on the key requirements outlined in Section 3.2.
vided valuable insights into the software’s optimal usage and po
tential applications. The third sub-phase, ’Online Communities,’ 3.1. DT for HRC in manufacturing system – A literature review
encompassed delving into various dedicated forums, YouTube
channels, and discussions on third-party platforms. This exploration As outlined in Section 2, the systematic literature review in this study
offered a broader perspective on the user community’s perspectives, adheres to the PRISMA methodology. The Scopus database was selected
experiences, and challenges with the software, enriching our un as the primary source for this review due to its comprehensive collection
derstanding of its practical implications and limitations. of peer-reviewed literature, including scientific journals, books, and
3. Evaluation: It was carried out with a focus on the identified key conference proceedings, with a total of 87 million documents and 17
requirements for designing and developing Digital Twins in Human- million authors (https://www.elsevier.com/solutions/scopus/how-sco
Robot Collaboration (HRC). These requirements encompassed areas pus-works/content).
such as Human Operator Simulation, Robotic Agent Simulation, To analyze the relevant scientific literature, three major research
Fig. 2. Software Review Methodology.
3
areas, namely “Manufacturing Systems,” “Human-Robot Collaboration,” Manufacturing Systems are the most frequent sources for publications
and “Digital Twin,” were cross-referenced. This approach facilitated the related to the use of DT for HRC enhancement in manufacturing systems.
identification and examination of relevant literature from diverse dis However, it should be noted that many sources have started to take an
ciplines and sources, ensuring a comprehensive review. interest in this area by publishing papers on this topic. The authors have
Table 2 provides an overview of the scientific productivity of three also included papers from conferences in their literature analysis due to
areas of research both when searched independently and when cross- the recent development of this research field. In fact, the contribution of
referenced. The results from the independent search revealed that preliminary studies on this topic is also relevant to understanding how
each area of research generates a substantial number of publications, this research trend is developing within the scientific community.
reflecting the high level of interest and research activity in these areas. Document analysis – the authors utilized the initial pool of 59 articles
However, when the three areas of research are combined, the number of as a starting point to conduct the document analysis. Each of these ar
publications decreases significantly, as evidenced by the mere 59 doc ticles was thoroughly reviewed, and only 19 articles were deemed
uments retrieved by the cross-referenced search. This suggests that there appropriate for this study, as they are aligned with the scope and re
may be limited overlap between the three areas of research as well as the quirements of this research. Specifically, the chosen articles examine
importance of using cross-disciplinary approaches to foster innovation HRC scenarios by utilizing DTs within manufacturing systems and by
and breakthroughs. incorporating a case study. These articles have been classified in Table 4
The query employed to cross-reference the three areas of research is based on the type of operations that the robot performs in collaboration
provided below: with the operators, the simulation environment type which refers to the
presence (controlled) or not (real) of approximations or assumptions in
- (TITLE-ABS-KEY(“digital twin” OR “digital mirror” OR “digital the development of the DT, the scope for conducting the simulations,
shadow”) AND TITLE-ABS-KEY(“human-robot collaboration” OR and the software employed for the development of the DTs.
“human-robot interaction”) AND TITLE-ABS-KEY(“manufact*”)) The document analysis reveals that HRC in manufacturing systems is
AND (LIMIT-TO(LANGUAGE,“English”)). primarily analyzed for assembly/disassembly, material handling, pick &
place and welding operations. Fig. 5 shows exactly the distribution of
It is noteworthy that the only exclusion criterion was language, with operation Types within each simulation environment.
only English-language articles selected for the analysis. The analysis reveals that these studies are often carried out in
By focusing on the 59 articles identified through the cross-referenced controlled environments, which means that, even though the simula
search, the authors have gained valuable insights into the current state tions are intended to capture real-world case studies, approximations
of knowledge related to the use of DTs for HRC enhancement in are needed to make the experiments possible; in fact, very often, phys
manufacturing systems. The analysis has considered several key factors ical environments are either specially recreated in the laboratory or
including publication frequency, global scale distribution, and source deliberately simplified to limit the number of variables to be handled for
analysis. Additionally, an in-depth document analysis was carried out to the scope of the research study. However, this practice should not be
examine the articles content and trends. viewed as an absolute constraint on the application of this paradigm.
Publication frequency – it expresses the number of scientific studies This is supported by the growing research interest, particularly in recent
published per year. Fig. 3 shows that the research topic under investi years, and in response to the disruptions in real industrial operations
gation is relatively new, with the first article published in 2017. How caused by the Covid-19 pandemic.
ever, there is a clear upward trend in the number of annual publications, As for the simulation scope of the studies being considered, partic
indicating a growing interest in this area among the scientific commu ular attention has been paid on safety levels, production plans, ergo
nity. It is important to specify that the discontinued line between years nomics as well as on collaborative spaces management. Safety levels are
2022 and 2023 means that data referring to 2023 are not definitive as identified as a top priority, as the potential risks associated with the
the study was carried out at the beginning of this year. interaction between human operators and robots in dynamic work en
Global scale distribution – this refers to the number of published vironments cannot be underestimated. Production plans are evaluated
documents for each country where the study has been conducted. The based on task allocation between humans and robots, with a focus on
results of the analysis indicate that, despite the increasing trend in optimizing efficiency and productivity. Ergonomics is also considered,
publishing on this topic, only a few countries have dedicated efforts to ensure that the work environment is conducive to human health and
toward investigating the use of DTs for HRC enhancement in well-being. Finally, collaborative spaces are managed to promote
manufacturing systems. As illustrated in Fig. 4, the countries with the effective communication and coordination between human operators
highest number of published papers in this area are China, Germany, and robots. Last but not least, the document analysis also reveals that the
Greece, and the United Kingdom. Italy, the United States, Denmark, and most prevalent software for developing the DTs is Unity 3D.
Sweden also have significant contributions. Several other countries have The synthesis of literature reveals a concerted progression in the field
only one published paper in this field. These findings suggest that there of HRC. Horváth & Erdὅs (2017) lay foundational work, introducing
is potential for increased research collaboration and knowledge ex controls for Cyber-Physical Systems within collaborative environments,
change between countries to advance the understanding of this underpinning subsequent developments in virtual modeling and gesture
emerging field. recognition facilitated by Microsoft Kinect and Wolfram Mathematica.
Source analysis – it is used to identify the number of published doc Malik & Bilberg (2018) underscore the indispensability of digital twins,
uments per research source. Table 3 presents the most prolific sources in proposing a framework notably illustrated through a case study
the research area in terms of published documents. The results indicate involving Universal Robots UR-5 and UR-3 in assembly workstations.
that Computer & Industrial Engineering and the Journal of This emphasis on virtual representations is further echoed by Oyecan
et al. (2019), who not only delve into Virtual Reality (VR) digital twins
but also introduce a Kinetic Energy Ratio metric, registering a note
Table 2 worthy 73 % accuracy rate in classifying human reactions to robot
Scientific productivity overview.
motions. The narrative of virtual integration continues with Malik et al.
Keyword Published documents (2019), who weave VR into human-robot simulations, utilizing a spec
“Manufacturing Systems” 49.537 trum of technologies including AWS Sumerian and HTC Vive for varied
“Digital Twin” 12.112 applications such as assembly tasks and collision analysis.
“Human-Robot Collaboration” 2.947 In a similar way, Arnarson et al. (2021) explore VR’s potential in
“Cross-referenced search” 59
robot programming and interaction, emphasizing simplified processes
4
Fig. 3. Publication trends.
Fig. 4. Documents per country.
Dimitropoulos et al. (2021), yielding improvements in cycle time (6 %),

Table 3
ergonomic factors, and operator satisfaction in industrial contexts.
Number of published documents per source.
Safety remains paramount, with Douthwaite et al. (2021) introducing a
Source name Published framework dedicated to safety–critical digital twins in collaborative
documents
robotics, while Maruyama et al. (2021) provide a proof of concept for
Computers & Industrial Engineering 4 integrating Digital Humans (DH) into Digital Twin-driven HRC systems
Journal of Manufacturing Systems 4
for enhanced ergonomic evaluations and efficiency.
ASME International Mechanical Engineering Congress And 3
Exposition Proceedings Imece
The trajectory of development also witnesses the advent of
International Journal Of Computer Integrated Manufacturing 3 Augmented Reality (AR) in the sector, as illustrated by Aivaliotis et al.
Procedia CIRP 3 (2023), who unveil an AR suite significantly optimizing operational and
Robotics And Computer Integrated Manufacturing 3 recovery tasks. In a methodological expansion, Gallala et al. (2022)
Other journals
outline a six-step framework for HRI, validated through use-case studies,
<3
whereas Gleirscher et al. (2022) spotlight a software approach

and enhanced safety. Pisanelli et al. (2022) contribute to the discourse increasing productivity by 40 % through optimal safety controller
with the introduction of ROSIE Adapter, a pivotal tool facilitating the synthesis.
creation of digital twins, receiving accolades for user-friendliness and From architectural advancements showcased by Koukas et al. (2022)
high satisfaction rates among users during a UK-based event. with the OpenFlow architecture, to comparative studies like those by
Digital twins are integrated with AI systems, as seen in the work of Kuts et al. (2022) evaluating Digital Twin VR interfaces, the literature
5
Table 4
Document insights.
Reference Sector Operation Simulation environment Simulation DT Development Software
Type Scope
Horváth & Erdὅs (2017) Process Not specified Controlled Gesture recognition Wolfram Mathematica
Malik & Bilberg (2018) Not specified Assembly Controlled What-if analysis Process Simulate
Malik et al. (2019) Assembly Assembly Controlled Workstation Layout design Process Simulate
Oyecan et al. (2019) Process Not specified Controlled Safety assessment Unity 3D
Arnarson et al. (2021) Not specified Not specified Controlled Robot Programming Visual Components
Dimitropoulos et al. (2021) Process Material Handling Real Ergonomic assessment Gazebo
Douthwaite et al. (2021) Process Welding Controlled Safety assessment Unity 3D
Maruyama et al. (2021) Not specified Pick & Place Controlled Ergonomic assessment DhaibaWorks(in-house software)
Gallala et al. (2022) Not specified Pick & Place Controlled Communication enhancement Unity 3D
Gleirscher et al. (2022) Process Not specified Controlled Safety Unity 3D
Koukas et al. (2022) White Goods Assembly Controlled Task Planning OpenFlow(in-house software)
Kuts et al. (2022) Not specified Material Handling Controlled Safety assessment Unity 3D
Lu et al. (2022) Food Packaging Controlled Task Allocation Not Specified
Pisanelli et al. (2022) Not specified Not specified Controlled Path Movement Unity 3D
Schmidt et al. (2022) Not specified Material Handling Controlled Cognitive Ergonomic assessment Unity 3D
Yi et al. (2022) Not specified Not specified Controlled Safety assessment Visual Components
Zhu et al. (2022) Not specified Not specified Controlled Task Allocation –
Aivaliotis et al. (2023) Automotive Assembly Controlled Safety assessment Unity 3D
Li et al. (2023) Not specified Disassembly Controlled Safety assessment Unity 3D
Fig. 5. Distribution of Operation Types within Simulation Environment.
captures a field in dynamic evolution. Notably, the works of Lu et al. of the studies is controlled, which may not accurately reflect the com
(2022), Schmidt et al. (2022), and Yi et al. (2022) respectively offer plexities and unpredictabilities of real-world operational environments.
modularized HRC frameworks, visualization tools for robotic cell opti This limitation in the simulation environment restricts the external
mization, and depth camera-based human recognition systems for effi validity of these studies, as the findings may not be generalizable to
cient collaboration. actual industrial settings with dynamic and unpredictable elements.
Zhu et al. (2022) add to this rich tapestry with their dynamic Additionally, there is a wide range of DT development software used
reconfiguration optimization method, whereas Li et al. (2023) introduce across the studies, with Unity 3D being the most commonly used.
an innovative Human-Robot Interaction framework with AR devices, However, the use of different software tools, including in-house devel
with a validation experiment noting a 93 % success rate in collision oped solutions, might hinder the interoperability and integration of DTs
detection. Together, these studies coalesce into a multifaceted under across different systems and platforms. The inconsistency in the software
standing of the ongoing innovations and challenges within HRC, each tools used also makes it challenging to compare and consolidate findings
contributing a unique perspective and empirical finding to the bur across different studies. Furthermore, the simulation scope varies across
geoning field. studies, ranging from safety assessment to ergonomic assessment, to task
Table 4 also suggests several limitations in the current research on planning and allocation, without a clear focus or prioritization of
DTs in HRC. Firstly, many studies do not specify the sector and operation research efforts in addressing the critical challenges and needs in the
type investigated, which significantly hampers the understanding and application of DTs in HRC.
applicability of the research to real-world scenarios. This lack of speci On the other hand, considering Table 5, there’s a noticeable lack of
ficity limits the transferability of the findings across different industrial standardization across studies in both the types of robots and sensors
sectors and operation types. The simulation environment in the majority employed, which hinders the comparability and generalizability of the
6
Table 5 outcomes. The absence of algorithmic details may also hinder the
Documents comparative analysis. development of integrated frameworks that seamlessly combine various
Reference Robot Sensors Algorithms Standards algorithms for enhanced DT-HRC functionality. This variability in
technological and methodological approaches across the literature ne
Horváth & Not Microsoft Proximity Query Not
Erdὅs (2017) specified KinectTM Package specified cessitates a more coordinated and standardized research agenda to
Malik & UR-5/UR- Microsoft Robotiq Not advance the field of DTs in HRC effectively.
Bilberg 3 KinectTM specified
(2018)
3.2. DT design for HRC - software key requirements
Malik et al. UR-5 Microsoft Amazon Not
(2019) KinectTM/ Sumerian specified
Photosensors Developing DTs for HRC enhancement in manufacturing systems
Oyecan et al. ABB IRB Microsoft Savitzky–Golay Not requires careful consideration of the software key requirements. Simu
(2019) 2600 KinectTM filtering method specified lation studies are essential for a variety of reasons, and the motivations
12–1.85
for using DTs in HRC enhancement are many. Given the delicate nature
Arnarson et al. SCARA Not specified Not specified IEC 62,541
(2021) Adept of HRC, which involves the coexistence of human operators and robots
604-S/ in the same workplace, appropriate integration studies are necessary.
KUKA KR- DTs enable the verification of space management and interference,
30–3/
guaranteeing efficiency and safety in shared work environments. The
Nachi
MZ07
ergonomics of human operators and the productivity of manufacturing
Dimitropoulos COMAU Kinect Azure Convolutional Not systems are also critical factors that need to be considered when
et al. (2021) AURA Neural Networks specified developing DTs for HRC. These factors enable the evaluation of task
(Actions management and assignment between human operators and robots, the
Perception
delegation of tasks to the robot, and the selection of the best HRC sce
Module)
Douthwaite Not LIDAR Not specified Not nario based on predefined key performance indicators (i.e., ergonomic
et al. (2021) specified specified parameters, production time, etc.). Moreover, data exchange and
Maruyama Not OptiTrack Optimization- Not interoperability between the DT and the physical manufacturing system
et al. (2021) specified based motion specified
are crucial for effective communication and coordination between the
estimation
Gallala et al. KUKA Microsoft Deep Not
digital and real-world environments in HRC scenarios, thus enabling the
(2022) IIWA Hololens 2 Conventional specified two systems to influence each other, and allowing for adjustments and
Neural Network improvements to be made to the manufacturing system.
Gleirscher et al. Not Microsoft Not specified ISO Following the systematic literature review presented in section 3.1
(2022) specified KinectTM/ 10,218/
and considering the aforementioned factors, this study has identified the
LIDAR/Light ISO/TS
barrier 15,066 following five key requirements as essential for a software to be used to
Koukas et al. UR-10 Microsoft Not specified Not develop DTs for HRC enhancement in manufacturing systems:
(2022) HoloLens 2 specified
Kuts et al. Not Head- Not specified Not 1. Human operator simulation: the software should have the capability to
(2022) specified Mounted specified
Display
accurately and reliably replicate the actions of a real operator,
Lu et al. (2022) ABB IRB uEye XS Not specified Not ensuring precision and accuracy in HRC scenarios. Developing a
2600–20 Camera specified digital human model for HRC scenarios is a challenging task, as the
(12)/1.65 human body consists of a complex system of approximately 230
Pisanelli et al. KUKA Not specified Google Assistant Not
degrees of freedom, controlled by about 630 muscles (https://www.
(2022) LBR IIWA specified
14/ ifittraining.co.uk/insights/movement-variability-degrees-freedom/
Iconsys #:~:text=In%20total%20therefore%2C%20we%20have,we%
iAM-R 20each%20have%20inside%20us). Due to the vast range of move
Schmidt et al. ABB IRB Microsoft NSGA-II Not ments and postures that a human operator can assume, it is crucial to
(2022) 4600/ HoloLens 2 specified
ABB IRB
accurately replicate the motions of the real human operator in the
2600 simulation. Customization of anthropometric parameters is also
Yi et al. (2022) Not ASUS Xtion Not specified IEC 61,499 important to ensure that the digital human operator closely re
specified Pro cameras/ sembles the real operator involved in the case study, especially for
Kinect v2.0
highly specialized work (Chaffin, 2005).
Zhu et al. Not Not specified NSGA-II Not
(2022) specified specified 2. Robotic agent simulation: the software should have the capability to
Aivaliotis et al. Mobile SICK Not specified Not accurately and reliably replicate the actions of a real robot, enabling
(2023) Robot microScan3 specified efficient and effective collaboration between humans and robots.
Platform /Microsoft The implementation of robot models for HRC simulations is a com
HoloLens 2
Li et al. (2023) UR-5 Microsoft Proximal Policy Not
plex task that requires specialized software tools. Some of these
HoloLens 2 Optimization specified tools’ features, which are particularly useful for the development of
(PPO) DTs for HRC enhancement, include the ability to import different
types of CAD files without loss of information, to build kinematic
chains from scratch, to create robot models using simple geometry
findings. The utilization of various robots and sensors without consistent
modeling functions, to access a library of pre-packaged robots, and to
reference to specific standards suggests that the research landscape is
implement the Robot Operating System (ROS) (https://wiki.ros.org/
fragmented (Fig. 6). Moreover, the standards referenced in the studies
ROS/Introduction).
are few and inconsistently applied, pointing out a potential gap in the
3. Ergonomic analysis: the software should be capable of performing
establishment of universal protocols and benchmarks for evaluating and
ergonomic analysis which supports the assessment and optimization
deploying DTs in HRC contexts. There’s also an evident lack of trans
of human operators’ ergonomics. Several ergonomics methods can
parency and specificity regarding the algorithms deployed in some
also be integrated into the DT to analyze and optimize human op
studies, making it challenging to reproduce and validate the research
erator’s working conditions. These include the Ovako Working
7
Fig. 6. Relationship between Robots and Operation Types.
Posture Analysis (OWAS) (Kharu et al., 1977; Kharu et al., 1981), the can be used to develop DTs for HRC enhancement in manufacturing
Static Strength Prediction Program (3D SSPP/SSP) (Chaffin et al., systems. The software evaluated in this analysis were selected based on
2006), the Muscle Fatigue Analysis (MFA) (Rodgers and Suzanne, the document analysis conducted in section 3.1 (see last column in
1988; Rodgers and Suzanne, 1992), the Lower Back Analysis (LBA) Table 3), with a focus solely on commercially available options, thus
(Herrin et al., 1986), the Cumulative Back Load analysis (CBL) excluding In-house software. To ensure comprehensiveness, the authors
(Coenen et al., 2012; Coenen et al., 2014), the Strain Indexes Anal further expanded the list of software by conducting additional searches
ysis for Distal Upper Extremity Disorders (Moore and Garg, 1995), through major search engines. Following a complete list of the software
Rapid Upper Limb Assessment (RULA) (McAtamney and Corlett, is reported, along with a concise description of each software’s key
1993), the NIOSH 81 method (National Institute for Occupational features.
Safety and Health, 1981), the NIOSH 91 method (Waters et al., 1994) Wolfram Mathematica – it is a computational software program
and the GARG analysis (Garg, 1976). The presence of these methods developed by Wolfram Research (https://www.wolfram.com/mathemat
within the DT enables the most accurate studies possible of HRC ica/). It provides a comprehensive set of tools for numerical and sym
scenarios concerning the ergonomic conditions of the human bolic computations, visualization, and programming across a wide range
operator. of disciplines. Some of its key features include symbolic computation,
4. Time measurement analysis: the software should be capable of per numerical computation, data analysis and visualization, programming,
forming time measurement analysis which plays a crucial role in integration with other technologies and access to curated data.
optimizing the overall manufacturing system’s productivity. Several Process Simulate – it is a software developed by Siemens that enables
time measurement methods can be integrated within a DT, such as the design, analysis, simulation, and optimization of manufacturing
the Methods-Time Measurement (MTM) (Maynard et al., 1948) and processes, from the factory level down to lines and work cells (https://pl
the Maynard Operation Sequence Technique (MOST) (Zandin, m.sw.siemens.com/en-US/tecnomatix/products/process-simulate-s
2003). These methods allow for the accurate measurement and oftware/). Some of its key features include visualization for displaying
evaluation of the time required to complete a particular task, which and analyzing simulation results, human modeling and simulation for
can be used to identify inefficiencies in the manufacturing system analyzing human-centered operations and robotic simulation to simu
and improve the productivity of human operators. late robot movements and optimize robot paths for maximum efficiency.
5. Data Exchange and Interoperability: the software should be able to Unity 3D – it is a cross-platform game engine developed by Unity
exchange input and output data with the physical world as well as Technologies. This software is widely used for developing video games,
other software, facilitating efficient and effective communication Augmented Reality (AR) and Virtual Reality (VR) applications, and
and coordination between different components in the other interactive 3D experiences (https://unity.com/). Some of its key
manufacturing system. The data exchange enables real-time moni features include a powerful editor that enables developers to create and
toring, identification of issues, optimization of the manufacturing manipulate 3D objects, a physics engine for simulating realistic move
process, leading to improved productivity, reduction of waste, and ment and collisions, a scripting engine for creating custom behaviors
increased efficiency. Additionally, it enhances better collaboration and game logic, and support for a wide range of platforms and devices.
between humans and robots, improving manufacturing system safety Visual Components – it is a 3D simulation software developed by Vi
as well as human operator’s ergonomics. sual Components Oy (https://www.visualcomponents.com/products
/premium/). The software is used to model and simulate production
3.3. DT development in HRC - commercial software overview lines, robots, machines, and other industrial equipment for optimizing
material flow, reducing cycle times, and improving the overall efficiency
This section presents an overview of the commercial software that of manufacturing and logistics systems. Some of its key features include
8
drag-and-drop 3D modeling to create 3D models of manufacturing sys Wolfram Mathematica

tems, a flexible simulation engine to enable the users to simulate com
plex manufacturing processes and data visualization to monitor the - Human operator simulation: although this software does not offer a
performance of the simulated system. dedicated module for human modeling and simulation, it is still
Gazebo – it is an open-source software currently maintained by the possible to create an approximation of a human model by using joints
Open-Source Robotics Foundation (OSRF) (https://classic.gazebosim. and body segments. Furthermore, kinematic functions such as
org/tutorials?cat = guided_b&tut = guided_b1). Gazebo is a 3D dy AngleVector or AngleMatrix enable users to specify joint angles over
namic simulator that is capable of accurately and efficiently simulating time and define the motion of the human model.
populations of robots in complex indoor and outdoor environments. - Robotic agent simulation: this software integrates a module, called
Some of its key features of Gazebo include multiple physics engines for Mathematica’s Robotics module, which provides a set of tools and
the simulation of rigid-body dynamics, contacts, and collisions, a rich functions for modeling, analyzing, and controlling robots. With this
library of robot models and environments, and a wide variety of sensors. module, end-users can access built-in functions that simplify the
CoppeliaSim (V-rep) – it is a robot simulator by Coppelia Robotics AG process of working with common robotic structures such as manip
(https://www.coppeliarobotics.com/helpFiles/index.html). This soft ulators, end-effectors, and sensors. Moreover, the software allows
ware can be used both as a stand-alone application or easily embedded users to generate dynamic simulations and visualizations of robot
into a main client application. Some of its key features include simula motion, and to perform both kinematic and dynamic analyses of
tion of factory automation systems, remote monitoring, hardware con robot behavior. These capabilities enable users to predict a robot’s
trol, fast prototyping and verification, safety monitoring, fast algorithm performance under different conditions, and to optimize its func
development, robotics-related education, and product presentation. tionality accordingly.
Delmia (3DExperience) - it is a software developed by Dassault Sys - Ergonomic analysis: the software does not provide a module for er
tems, which provides a wide range of software applications for simu gonomic measurement analysis but functions can be implemented
lating manufacturing processes, production systems, and supply chain through coding in the supported languages such as Wolfram Lan
management (https://www.3ds.com/products-services/delmia/). Some guage, C, C++, Java,.NET and Python.
of its key features include manufacturing process planning and optimi - Time measurement analysis: the software does not provide a module
zation, human modeling and ergonomics, robotics and automation. for time measurement analysis but functions can be implemented
Morse – it is an open-source robotics simulation software developed through coding in the supported languages such as Wolfram Lan
by LAAS-CNRS (https://www.openrobots.org/morse/doc/stable/wh guage, C, C++, Java,.NET and Python.
at_is_morse.html#performances). The software provides realistic 3D - Data Exchange and Interoperability: the software provides users with
simulations of environments ranging from small to large, whether in a range of built-in interfaces for interoperable communication with
door or outdoor, using autonomous robots. Some of its key features external software such as LabVIEW, Python, Java,.NET, C, and
include modularity and therefore easy to be extended with new func FORTRAN. Additionally, standard protocols such as HTTP, FTP, and
tionalities, multi-robot simulation to simulate robots and their in SMTP can be used to achieve interoperability with other software. In
teractions, and a realistic simulation environment supported by 3D addition, Wolfram Mathematica also supports various import and
graphics and virtual reality. export file formats, such as CSV, HDF5, and FITS, which allows for
easy transfer of data to and from other programs.
3.4. DT design for HRC - software comparison
Process Simulate
In this section, the authors present the comparative analysis of the
commercial software discussed in Section 3.3, based on the key re - Human operator simulation: the software offers a dedicated module
quirements outlined in Section 3.2. It is important to note that the for human modeling and simulation. This module includes highly
analysis does not aim to rank or promote any particular software tool accurate and reliable human models, such as “Jack” and “Jill,” that
over another; rather, the goal is to offer an objective report on the can be customized to match specific user requirements. Users can
software tools currently available in the market to develop DTs for HRC adjust the anthropometry of the models allowing for realistic simu
enhancement in manufacturing systems. To facilitate this comparison, lation of human motion during a task as well as the evaluation of
Table 6 presents the lists of the commercial software and indicates human factors such as ergonomics and task feasibility before
whether each identified key requirement is fully satisfied (✓), partially implementation in the real manufacturing system.
satisfied (-) or not satisfied (x) by the respective software. This approach - Robotic agent simulation: the software provides the users with a
provides a comprehensive overview of the strengths and limitations of module for robot modeling and simulation. This module allows users
each software tool, allowing researchers and practitioners, who are to create primitive geometries and joints with attached kinematic
seeking to implement DTs for HRC enhancement in manufacturing chains to create simple mechanisms that can be implemented in
systems, to make informed decisions on the software selection based on simplified simulation models. It also enables the import of CAD
their specific requirements. models through a CAD translator that is installed during software
Following, the authors present for each software the information that installation. The CAD translator converts generic CAD models into.jt
was used to populate Table 4. format, the only format supported by Process Simulate (please note
Table 6
Software comparison.
Software Human operator simulation Robotic agent simulation Ergonomic analysis Time measurement analysis Data Exchange and Interoperability
Wolfram Mathematica – ✓ – – ✓
Process Simulate ✓ ✓ ✓ ✓ –
Unity 3D – – – – ✓
Visual components – ✓ X – ✓
Gazebo – ✓ X X ✓
CoppeliaSim – ✓ – – ✓
Delmia ✓ ✓ ✓ ✓ –
Morse – ✓ X X ✓
9
that a license for the import format is required to convert the model module in the current version of the software only offers approxi
to.jt format). mate simulations and is not suitable for detailed analysis of specific
- Ergonomic analysis: the software allows users to perform ergonomic manual operations.
analysis such as OWAS, 3D SSPP/SSP, MFA, LBA, CBL, RULA, - Robotic agent simulation: the software provides a module specif
NIOSH81, NIOSH91 and Garg analysis. These analyses are integrated ically for robot modeling and simulation. This module offers a library
into the software and provide insights into the physical efforts of pre-built robot models, featuring popular brands such as ABB,
required of the workers in performing a task. Fanuc, and KUKA, along with generic robot models that can be
- Time measurement analysis: the software integrates the MTM anal customized to meet specific requirements. In addition, various pro
ysis tool, thus enabling opportunities for process improvement and gramming languages, including ABB’s RAPID, KUKA’s KRL, and
optimization by measuring the time required to complete specific FANUC’s TP can be used to program the robot models. For users who
tasks and evaluating the motions involved in those tasks. This allows prefer a graphical approach, the software offers a drag-and-drop
for the identification of unnecessary or inefficient motions that can visual programming interface, enabling them to create robot pro
be eliminated or improved to reduce cycle time and increase grams without the need for coding. Once programmed, the robots
productivity. can be simulated in the virtual production environment, facilitating
- Data and Information Exchange: the software allows the connection the testing and optimization of their behavior and performance. The
of virtual models with real Programmable Logic Controller (PLC) software offers a suite of tools for testing collision detection, motion
code and hardware (HW) over OPC (OLE for Process Control) planning, and path optimization, as well as analysis tools for evalu
interface. This enables users to simulate the behavior of a production ating the robot’s cycle times and other performance metrics.
line and test the PLC code before implementing it in the real - Ergonomic analysis: the software does not provide a specific module
manufacturing system. However, the software’s complete interop for ergonomic analysis.
erability is not assured as it can only interface with Siemens Product - Time Measurement analysis: the software does support time mea
Lifecycle Management Software. This is a suite of software applica surement analysis, thus enabling users to perform time studies,
tions, developed by Siemens for managing the product’s entire life measure cycle times, and analyze production line efficiency.
cycle, including its design, engineering, manufacturing, and - Data Exchange and Interoperability: the software provides several
maintenance. built-in interfaces for interoperable communication with external
software, including Python, C#, and.NET. It also supports a variety
Unity 3D of import and export file formats, including STEP, IGES, and
COLLADA.
- Human operator simulation: although the software lacks a specific
module for human simulation, it offers a simulation environment Gazebo
where CAD models can be integrated into the scene to recreate sce
narios with human operators. Human models can be used as - Human operator simulation: although this software lacks a dedicated
dummies to which specific programming codes can be attached for module for human simulation, it is possible to recreate a human
evaluations and analyses. model using the “actor” function of the software (He et al., 2020),
- Robotic agent simulation: although the software lacks a specific which also allows to addition of animations to a joint model. How
module for robotic agent simulation, it allows to import CAD models ever, in order to accurately control the movement of the human
including robot models and provides tools for creating kinematic model, it is necessary to create the structure of the joints (with
chains, joints, and scripts to control robotic components. attached hierarchy) by using an additional tool called EMA - Editor
- Ergonomic analysis: although the software does not provide a mod for Manual Work Activities. Additionally, the same EMA tool must be
ule for ergonomic analysis, it allows users to integrate mathematical used to program the workflow and actions to be performed by the
functions through scripts that can be attached to specific components model.
or joints. By leveraging this functionality, users can perform analyses - Robotic agent simulation: this software can generate basic geometric
of human operator ergonomics within the simulation environment. models and import pre-existing 3D models, including robot models,
- Time Measurement analysis: no specific module for time measure into simulation scenarios. Users can create and configure joints
ment analysis is integrated within the software. However, Unity 3D within the software, and adjust parameters such as gravity, density,
allows users to integrate mathematical functions through scripts that kinematics, and mass to meet specific requirements.
can be attached to specific components or joints, thus enabling time - Ergonomic analysis: this software does not offer the capability to
measurements within the simulation environment. carry out ergonomic analysis.
- Data Exchange and Interoperability: the software supports interop - Time Measurement analysis: this software does not support time
erable communication with other software through several built-in measurement analysis.
interfaces and APIs. Additionally, Unity supports communication - Data Exchange and Interoperability: Gazebo provides the ability to
with external software through plugins and extensions, which can be exchange data via Gazebo Transport [69]. Additionally, the software
written in languages such as C++, C#, and Java. Furthermore, Unity also supports several plugin APIs, including the C++ plugin API, the
3D has built-in support for several industry standards and protocols ROS plugin API, and the Ignition Gazebo plugin API. These APIs
such as OpenVR, OpenXR, and WebGL, which enable interopera allow to extend the functionality of Gazebo by creating custom
bility with a wide range of hardware and software platforms. In plugins that can interact with other software or perform specialized
terms of data exchange, Unity supports various file formats for simulation tasks. Gazebo also supports a variety of import and export
importing and exporting assets, including 3D models, animations, file formats, including COLLADA, URDF, and SDF and gives the op
textures, and audio. Among the others, these formats include FBX, portunity to incorporate simulative scenarios with several sensors,
OBJ, Collada, WAV, and MP3. including monocular or depth cameras to get input data.
Visual Components CoppeliaSim
- Human operator simulation: the software offers a module for - Human operator simulation: the software has no dedicated module
including a human model in the simulation environment, which can for human operator simulation, but it does offer some representative
be programmed to perform specific operations. However, the human models of humans in its rich model library. However, these models
10
do not have specific features for implementing HRC-efficient human Morse

simulations. The human models can be handled as if they were ro
bots, but it would be necessary to handle all degrees of freedom of the - Human operator simulation: while this software does not have a
model. Alternatively, the human models can be handled as if they dedicated module for human simulation, it does provide the option
were animations to make approximate movements to briefly illus to place a human model within the simulated environment. How
trate the action to be represented. ever, these models do not have specific features to enable efficient
- Robotic agent simulation: the software offers a vast library of robot HRC simulations; in fact, the human models can be manipulated as if
models that can be easily imported into the virtual environment to they were robots by managing all degrees of freedom of the model.
create different simulation scenarios. It allows for dynamic analysis, - Robotic agent simulation: this software offers capabilities for robotic
inverse and forward kinematics, collision detection and minimum agent simulation by either providing the end-user with a library
distance calculation, the implementation of visual and proximity including some pre-packaged models or directly importing the CAD
sensors as well as the planning of paths and motions. model within the simulative environment. It provides a range of
- Ergonomic analysis: the software does not offer any feature set for features for configuring the robot’s physical properties, such as mass,
performing ergonomic analysis; however, it allows users to attach joint parameters, etc. and supports the integration of various sensors
scripts to objects in the scene, which could be programmed to and actuators.
perform ergonomic analyses. - Ergonomic analysis: this software does not offer the capability to
- Time Measurement analysis: the software does not offer any feature carry out ergonomic analysis.
set for performing time measurement analysis; however, as for the - Time Measurement analysis: this requirement is not supported by the
ergonomic analysis, CoppeliaSim allows users to attach scripts to software.
objects in the scene, which could be programmed to calculate process - Data Exchange and Interoperability: Morse supports six open-source
times. middlewares, including ROS, YARP, Pocolibs, MOOS, HLA, and
- Data Exchange and Interoperability: CoppeliaSim enables the ex Mavlink to facilitate communication and data exchange with
change of data with the outside world through various means, external devices, systems and software. Users can choose the mid
including signals, custom data blocks, plugin and script functions, dleware that best suits their needs and integrate Morse with various
and WebSocket. Additionally, CoppeliaSim guarantees interopera other tools and platforms, allowing for greater flexibility, interop
bility through the use of several built-in interfaces for communica erability and data exchange. Additionally, this software provides a
tion with external software, including ROS, MATLAB, Python, Java, socket-based protocol that enables easy integration with other lan
C++, and Lua. These interfaces allow to send and receive data be guages and toolboxes, simplifying the process of exchanging data
tween CoppeliaSim and other software, as well as control simulation and information.
parameters and perform simulations programmatically.
4. Discussion
Delmia (3DExperience)
This section explores the significance of the results of the work by
- Human operator simulation: this software allows the implementation directly answering the RQs presented in Section 1. Below, each RQ is
of customizable manikins based on anthropometric standards, which reiterated, and its corresponding answer is provided.
can be tailored to the simulation needs. This feature makes Delmia an RQ1: What are the most recent and impactful applications of Digital
ideal tool for modeling human operators. Twins in enhancing Human-Robot Collaboration across different industries?
- Robotic agent simulation: this software offers capabilities for robotic The use of DTs for enhancing HRC in manufacturing systems is a
agent simulation, including the ability to model simple 3-D robot relatively new research topic, with the first article published in 2017.
models and assign inverse kinematics to them. It also enables However, there has been a significant increase of publications in this
defining joints for imported models that lack these characteristics. area as a result of a growing level of interest and engagement within the
Delmia’s advanced features include validation of welding, riveting, scientific community. Despite this trend, only a few countries have
and drilling operations, calculating interference zones and interlocks dedicated efforts to investigating the use of DTs for HRC enhancement:
in multi-robot work cells, and offline programming of multiple ro China, Germany, Greece, and the United Kingdom. The studies primarily
bots from different companies while validating specific robot syn focus on assembly/disassembly, material handling, pick & place, and
taxes. Although Delmia does not have a library of pre-packaged robot welding operations, with safety, production plans, ergonomics, and
models, it allows importing robot CAD models in various formats. collaborative spaces as the top priorities in the simulation scope. In
- Ergonomic analysis: this software offers a wide range of ergonomic addition, although many of these studies are conducted in controlled
analysis options, such as vision, reach, space, and posture analysis, environments, it should not be considered an absolute constraint on the
biomechanical analysis, push, pull, and carry analysis, lifting and application of the DT paradigm. The growing research interest in this
lowering analysis, rapid upper limb assessment, and smart posturing. area, especially in recent years and in response to the disruptions caused
It is worth noting that the software utilizes Delmia standards for by the Covid-19 pandemic, suggests that this approach could be prom
ergonomic analysis, which may differ from international standards. ising to face real-world challenges in industrial operations.
- Time Measurement analysis: this software includes tools for RQ2: What are the essential factors that must be considered when
analyzing cycle times, lead times, and other time-based metrics, as developing Digital Twins for enhancing Human-Robot Collaboration in
well as tools for optimizing these metrics. manufacturing systems?
- Data Exchange and Interoperability: DELMIA provides support for The present study has identified five key requirements for software
data import and export in the “.xml” format during simulation. used to develop DTs in manufacturing systems to enhance HRC. Firstly,
Additionally, it offers plugin APIs such as the CAA V5/V6 API and the the software should accurately replicate human operators’ actions, as
DELMIA Automation API, which can be utilized to expand the the human body is complex and consists of a vast range of movements
functionality of the software by creating custom plugins that can and postures. Customization of anthropometric parameters is also
perform specialized simulation tasks or interact with other software. important. Secondly, the software should replicate the actions of a real
It is important to note that the interoperability of DELMIA is robot, enabling effective collaboration between humans and robots.
currently limited to software developed by Dassault System, Specialized software tools are required to implement robot models for
including CATIA, ENOVIA, SIMULIA, and DELMIA Process Engineer. HRC simulations. Thirdly, the software should enable ergonomic and
time measurement analyses, which play a crucial role in optimizing
11
human operators’ ergonomics and the overall manufacturing system’s with human simulation often taking a backseat. Furthermore, only a
productivity. Finally, the software should be able to exchange input and limited number of software packages incorporate ergonomic and time
output data with the physical world as well as with other software measurement analysis to their full extent. While most offer some degree
enabling efficient and effective communication and coordination. of data exchange and interoperability capabilities, they often do not
Beside, deploying DTs in HRC environments necessitates careful fully meet the holistic needs of HRC.
consideration of practical and technical aspects to ensure a seamless This study is proposed as a resource for both researchers and prac
implementation. Beginning with data acquisition, appropriate sensors titioners committed to advancing HRC in the manufacturing domain
such as Kinect, Kinect Azure, LIDAR, uEye XS Camera, or OptiTrack are through the implementation of digital twins. By delving into software
indispensable to capture motion and environmental data necessary for benchmarking within this field, the analysis allows readers to access
robotic and human entities modeling. Moreover, the selection of data crucial information about the most widely used software options for
acquisition devices, that can efficiently collate real-time data from these developing digital twins, and how this software have been employed in
sensors while maintaining compatibility and ease of integration, is of scientific literature in this field of interest. This insight serves as a
utmost importance. compass, guiding individuals towards the most suitable software solu
The selected sensors need to be fed into a robust communication tions that align with their research objectives and practical applications.
infrastructure that supports high-speed, low-latency data transmission, Implementing DTs in HRC environments poses notable challenges
facilitated by high-performance routers and switches. The employment and necessitates careful consideration. Integration complexity is a sig
of communication protocols like MQTT, CoAP, or WebSockets that nificant hurdle due to the imperative for data consistency between the
promote efficient data transfer, synchronization between the DT and its physical and virtual entities and the need for seamless interoperability
physical counterpart, and effective collaboration between human op among diverse devices and technologies in HRC systems. Ensuring ac
erators and robots is also necessary for minimizing latency in real-time curacy and precision in both the modeling of the physical object in the
applications, thus optimizing the effectiveness of HRC. DT and the sensors’ data, which are crucial for the DT’s continuous
On the computational side, data processing requires high- update, is challenging (Eichler et al., 2017). Addressing real-time per
performance computing hardware equipped with powerful CPUs and formance issues, such as minimizing latency in data transmission and
GPUs to run complex algorithms and facilitate real-time simulation and processing and achieving synchronization between the DT and its
interaction within the DT environment. It is also important to consider physical counterpart, is crucial for the effectiveness of real-time appli
algorithmic libraries and frameworks that not only support but also cations (Wu et al., 2023). The DT-HRC system should also be scalable
optimize the chosen algorithms for data analysis, such as TensorFlow or and flexible, able to expand without substantial performance degrada
PyTorch for neural networks or other appropriate machine learning or tion and adapt to various robotic hardware, sensors, and application
data analysis libraries. requirements. Security and privacy concerns are paramount, given the
As the data flows through the system, a suitable simulation envi continuous data exchange, with the need to secure communication
ronment becomes essential. These platforms offer diverse tools for channels and data while also respecting the privacy of human operators
developing, testing, and validating DTs in HRC, serving as the bedrock in the environment (Tabassum et al., 2023). Usability and user experi
for DT development and modeling. Within these environments, there ence are vital, requiring the design of intuitive user interfaces and
could be the need to employ additional modeling tools and AI/ML effective user feedback mechanisms. Reliability and maintenance of the
frameworks (like TensorFlow, Keras, Scikit-learn) for creating accurate DT-HRC system under various operational conditions need proactive
representations and functionalities of robots and human operators and strategies for troubleshooting and minimizing downtime. Continuous
enabling intelligent, adaptive interactions between them. validation and verification of the DT’s accuracy and the entire system’s
In the future it will be also necessary to ensure that DTs adhere to performance are essential for maintaining integrity and expected
relevant industry standards and compliance protocols, such as IEC functionality.
62541, ISO 10218, or other applicable standards. In essence, this research equips readers with the knowledge to make
RQ3: Which solutions and tools are appropriate for developing and informed decisions when selecting software for their specific needs. It
deploying Digital Twins specifically designed for Human-Robot Collaboration not only illuminates the shortcomings of existing commercial software
in the context of manufacturing systems? but also highlights the distinctive features and limitations of each option
The list of software identified in this research study is available in available in the HRC realm. As a result, readers can tailor their software
Table 6 and is reported in the following: Wolfram Mathematica, Process choices to precisely match the demands of their research studies or in
Simulate, Unity 3D, Visual Components, Gazebo, CoppeliaSim, Delmia dustrial projects.
and Morse. Furthermore, our findings emphasize the critical necessity for the
RQ4: Which constraints and obstacles are to be considered when imple development of dedicated software solutions tailored explicitly to the
menting Digital Twins in manufacturing systems to enhance Human-Robot requirements of HRC in manufacturing systems. These solutions should
Collaboration (HRC)? seamlessly integrate all the essential components of HRC, bringing them
The examination of software benchmarking has shed light on a together within a unified and coherent tool. This call to action un
critical research gap within the realm of DTs for HRC in manufacturing derscores the urgency of bridging the gap in the current software
systems. This study has brought to the forefront the glaring absence of landscape, ultimately paving the way for more efficient and effective
comprehensive software solutions tailored to digital twin development digital twin development in the context of HRC.
for HRC. The integration of DTs into Human-Robot Collaboration HRC sce
The analysis indicates that current commercial software solutions do narios also raises several profound ethical, societal, and legal consid
not fully meet the critical requirements identified as necessary for erations that warrant careful examination. Ethically, the use of DTs in
optimizing Human-Robot Collaboration (HRC) within manufacturing HRC introduces questions about the boundaries of human-robot inter
systems from a Digital Twin (DT) perspective. This suggests that there action and control. As DTs enable a high degree of real-time monitoring
may be a gap between the capabilities of existing software and the and simulation, there is the potential for invasive surveillance of human
specific needs or objectives of HRC in manufacturing systems. As a workers. Ensuring that the data collected and utilized by DTs respects
result, organizations may need to consider custom development or the individuals’ privacy and autonomy is paramount (Braun, 2021).
potential for emerging solutions to address these requirements effec Furthermore, issues of transparency and accountability arise when de
tively. Many of these software programs are developed with a modular cisions made by robots, informed by DTs, impact human safety and well-
approach, primarily emphasizing robot simulation. This has been the being (Wullenkord and Eyssel, 2020). Ethical guidelines must be
dominant trajectory in research and development in recent decades, established to govern the responsible use of DTs in HRC, emphasizing
12
transparency, informed consent, and fairness. and refine proposed models, contributing to the advancement of DTs in
From a societal perspective, the widespread adoption of DTs in HRC HRC.
may have significant implications for the workforce. Automation and
enhanced collaboration between robots and humans facilitated by DTs Declaration of Competing Interest
can lead to shifts in employment dynamics. While it may create new
opportunities and improve efficiency, it also raises concerns about job The authors declare that they have no known competing financial
displacement and the need for retraining and upskilling to adapt to interests or personal relationships that could have appeared to influence
changing roles in an HRC setting. There is a pressing need for proactive the work reported in this paper.
policies and strategies to address these potential disruptions, ensuring
that the benefits of DT-enabled HRC are shared equitably among society. Data availability
On the legal front, the use of DTs in HRC introduces complex liability
issues. Determining responsibility in the event of accidents or errors The data that has been used is confidential.
becomes more intricate when human actions and decisions are inter
twined with those of robots informed by DTs (Weber-Lewerenz, 2021). Acknowledgements
Legal frameworks must evolve to establish clear lines of accountability
and liability. Additionally, intellectual property rights may come into This research work was partly supported by the FELICE Project –
play when DTs are used to model and optimize collaborative processes. Flexible Assembly Manufacturing with Human-Robot Collaboration and
Legal frameworks will need to adapt to address issues related to data Digital Twin Models – funded from the European Union’s Horizon 2020
ownership, data sharing, and intellectual property protection in HRC Research and Innovation program under grant agreement No.
scenarios involving DTs (Kendzierskyj et al., 2019). Achieving a balance 101017151.
between fostering innovation and safeguarding individual rights within
a legal framework will be a key challenge. Overall, the ethical, societal,
References
and legal implications of implementing DTs in HRC underscore the
importance of comprehensive and thoughtful governance frameworks Aivaliotis, S., Lotsaris, K., Gkournelos, C., Fourtakas, N., Koukas, S., Kousi, N., &
that promote responsible and beneficial deployment. Makris, S. (2023). An augmented reality software suite enabling seamless human
robot interaction. International Journal of Computer Integrated Manufacturing, 36(1),
3–29. https://doi.org/10.1080/0951192X.2022.2104459
5. Conclusion Arnarson, H., Solvang, B., and Shu, B. (2021, January). The application of virtual reality
in programming of a manufacturing cell, 2021 IEEE/SICE International Symposium on
System Integration (SII), Iwaki, Fukushima, Japan, pp. 213-218, doi: 10.1109/
This paper explores the potential of using DTs to support HRC in
IEEECONF49454.2021.9382657.
manufacturing systems, with a specific focus on the ability of DTs to Boschetti, G., Faccio, M., Granata, I., & Minto, R. (2023). 3D collision avoidance strategy
provide a realistic simulation environment for testing and optimizing and performance evaluation for human–robot collaborative systems. Computers &
collaboration strategies. The study includes a systematic literature re Industrial Engineering, 179, Article 109225. https://doi.org/10.1016/j.
cie.2023.109225
view devoted to understand how the scientific community perceives the Bratman, M. (1992). Shared cooperative activity. The Philosophical Review, 101(2),
application of DTs to support HRC in manufacturing systems and iden 327–341. https://doi.org/10.2307/2185537
tify the main trends in this research area. While the review indicates that Braun, M. (2021). Represent me: Please! Towards an ethics of digital twins in medicine.
Journal of Medical Ethics, 47(6), 394–400. https://doi.org/10.1136/medethics-2020-
there is a growing interest, further research efforts are needed to expand 106134
the existing knowledge base and to address the challenges and limita Cai, Y., Starly, B., Cohen, P. H., & Lee, Y. (2017). Sensor data and information fusion to
tions in the implementation of DTs for HRC in manufacturing systems. construct digital-Twins virtual machine tools for cyber-physical manufacturing.
Procedia Manufacturing, 10, 1031–1042. https://doi.org/10.1016/j.
These might include issues related to data accuracy, real-time integra promfg.2017.07.094
tion, human-robot interaction dynamics, and more. Chaffin, D. B. (2005). Improving digital human modelling for proactive ergonomics in
Security is another significant concern. Manufacturing systems design. Ergonomics, 48(5), 478–491. https://doi.org/10.1080/00140130400029191
Chaffin, D. B., Andersson, G. B. J., & Martin, B. J. (2006). Martin Occupational
typically involve sensitive data and critical operations, so safeguarding Biomechanics, 376 (4th ed.). New York, USA: J. Wiley & Sons, Inc.
the DT and the associated data from cyber threats and unauthorized Clark, H. H. (1996). Using Language. Cambridge University Press, Cambridge, UK,. https://
access is paramount. Moreover HRC introduces its own set of com doi.org/10.1017/CBO9780511620539
Coelho, F. C., Relvas, S., & Barbosa-Póvoa, A. P. (2021). Simulation-based decision
plexities. Ensuring that the DT accurately models and simulates these
support tool for in-house logistics: The basis for a digital twin. Computers & Industrial
interactions, considering factors such as worker safety and ergonomics, Engineering, 153, Article 107094. https://doi.org/10.1016/j.cie.2020.107094
is challenging. Coenen, P., Kingma, I., Boot, C. R. L., Bongers, P. M., & van Dieën, J. H. (2014).
The second part of the work focused on the definition of the main key Cumulative mechanical low-back load at work is a determinant of low-back pain.
Occupational and Environmental Medicine, 71, 332–337. https://doi.org/10.1136/
requirements that software should have to develop proper DTs for HRC oemed-2013-101862
in manufacturing systems along with an overview on commercial soft Coenen, P., Kingma, I., Boot, C. R. L., Twisk, J. W. R., Bongers, P. M. B., & van
ware and benchmark. The results of this benchmark reveal that no Dieën, J. H. (2012). Cumulative low back load at work as a risk factor of low back
pain: A prospective cohort study. Journal of Occupational Rehabilitation, 23, 11–18.
software can currently and efficiently integrate all the identified key https://doi.org/10.1007/s10926-012-9375-z
requirements for developing DTs for HRC. Future research initiatives Cohen, P. R., & Levesque, H. J. (1991). Teamwork. Noûs, 35, 487–512. https://doi.org/
should consider addressing several pivotal areas to further enhance the 10.2307/2216075
CoppeliaSim (V-rep) by Coppelia Robotics AG. Retrieved from: https://www.
efficacy and integration of DTs in collaborative environments. Firstly, coppeliarobotics.com/helpFiles/index.html. Accessed February 13, 2023.
developing innovative algorithms for real-time synchronization be Correia, J. B., Abel, M., & Becker, K. (2023). Data management in digital twins: A
tween DTs and physical counterparts is crucial to reduce latency and systematic literature review. Knowledge and Information Systems, 65(8), 3165–3196.
https://doi.org/10.1007/s10115-023-01870-1.
improve system responsiveness, facilitating smoother human-robot Delmia (3DExperience) by Dassault Systems. Retrieved from: https://www.3ds.com/
interaction. Secondly, constructing robust security frameworks is vital products-services/delmia/. Accessed February 11, 2023.
to safeguard data integrity and confidentiality within DT-HRC systems, Dimitropoulos, N., Togias, T., Zacharaki, N., Michalos, G., & Makris, S. (2021). Seamless
Human-Robot Collaborative Assembly Using Artificial Intelligence and Wearable
protecting both robotic and human elements. Attention should also be
Devices. Applied Sciences, 11(12), 5699. https://doi.org/10.3390/app11125699
directed towards creating adaptive DTs that can autonomously optimize Douthwaite, J. A., Lesage, B., Gleirscher, M., Calinescu, R., Aitken, J., Alexander, R., &
performance in response to environmental and task changes, and Law, J. (2021). A Modular Digital Twinning Framework for Safety Assurance of
developing user-friendly interfaces to broaden DT-HRC technology Collaborative Robotics. Frontiers in Robotics and AI, 8, Article 758099. https://doi.
org/10.3389/frobt.2021.758099
accessibility and adoption across various industries. Conducting T. Eichler, S. Draheim, C. Grecos, Q. Wang and K. von Luck, “Scalable context-aware
empirical studies in diverse real-world scenarios is essential to validate development infrastructure for interactive systems in smart environments,” 2017
13
IEEE 13th International Conference on Wireless and Mobile Computing, Networking Maruyama, T., Ueshiba, T., Tada, M., Toda, H., Endo, Y., Domae, Y., … Suita, K. (2021).
and Communications (WiMob), Rome, Italy, 2017, pp. 147-150, doi: 10.1109/ Digital Twin-Driven Human Robot Collaboration Using a Digital Human. Sensors, 21
WiMOB.2017.8115848. (24), 8266. https://doi.org/10.3390/s21248266
Gallala, A., Kumar, A. A., Hichri, B., & Plapper, P. (2022). Digital Twin for Human-Robot Maynard, H. B., Stegemerten, G. J., & Schwab, J. L. (1948). Methods-time Measurement.
Interactions by Means of Industry 4.0 Enabling Technologies. Sensors, 22(13), 4950. New York: McGraw-Hill Book Company.
https://doi.org/10.3390/s22134950 McAtamney, L., & Corlett, E. N. (1993). RULA: A survey method for the investigation of
García, Á. L., Bregon, A., & Martínez-Prieto, M. A. (2022). Towards a connected Digital work-related upper limb disorders. Applied ergonomics, 24(2), 91–99.
Twin Learning Ecosystem in manufacturing: Enablers and challenges. Computers & Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. (2009).
Industrial Engineering, 171, Article 108463. https://doi.org/10.1016/j. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA
cie.2022.108463 Statement. PLOS Medicine, 6(7). https://doi.org/10.1371/journal.pmed.1000097.
Garg, A. (1976). A Metabolic Rate Prediction for Manual Materials Handling Jobs. Moore, J. S., & Garg, A. (1995). The Strain Index: A proposed method to analyze jobs for
Dissertation. University of Michigan, DOI: 10.1080/0002889778507831. risk of distal upper extremity disorders. American Industrial Hygiene Association
Gazebo by the Open-Source Robotics Foundation (OSRF). Retrieved from: https://classic. Journal, 56, 443–458. https://doi.org/10.1080/15428119591016863
gazebosim.org/tutorials?cat=guided_b&tut=guided_b1. Accessed January 28, 2023. Morse by LAAS-CNRS. Retrieved from: https://www.openrobots.org/morse/doc/stable/
Gleirscher, M., Calinescu, R., Douthwaite, J. A., Lesage, B., Paterson, C. R., Aitken, J. M., what_is_morse.html#performances. Accessed February 15, 2023.
… Law, J. (2022). Verified synthesis of optimal safety controllers for human-robot Müller, R., Vette, M., & Geenen, A. (2017). Skill-based Dynamic Task Allocation in
collaboration. Science of Computer Programming, 218, Article 102809. https://doi. Human-Robot-Cooperation with the Example of Welding Application. Procedia
org/10.1016/j.scico.2022.102809 Manufacturing, 11, 13–21. https://doi.org/10.1016/j.promfg.2017.07.113
Grieves, M. (2014). Digital twin: Manufacturing excellence through virtual factory National Institute for Occupational Safety and Health, (1981). NIOSH Technical Report,
replication. White Paper, 1(2014), 1–7. Work Practices guide for Manual Lifting. DHHS (NIOSH) Publication Number 81-122,
Gualtieri, L., Rauch, E., & Vidoni, R. (2021). Emerging research fields in safety and Cincinnati, OH, USA.
ergonomics in industrial collaborative robotics: A systematic literature review. Oyekan, J. O., Hutabarat, W., Tiwari, A., Grech, R., Aung, M. H., Mariani, M. P., …
Robotics and Computer-Integrated Manufacturing, 67, Article 101998. https://doi.org/ Dupuis, C. (2019). The effectiveness of virtual environments in developing
10.1016/j.rcim.2020.101998 collaborative strategies between industrial robots and humans. Robotics and
He, L., Glohowski, P., Lemmerz., K., Kuhlenkötter, B., Zhang, W. (2020, Februar). Computer-Integrated Manufacturing, 55, 41–54. https://doi.org/10.1016/j.
Method to Integrate Human Simulation into Gazebo for Human-robot Collaboration. rcim.2018.07.006
2020 IOP Conf. Ser.: Mater. Sci. Eng, DOI 10.1088/1757-899X/825/1/012006. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D.,
Herrin, G. D., Jaraiedi, M., & Anderson, C. K. (1986). Prediction of overexertion injuries … McGuinness, L. A. (2021). The PRISMA 2020 statement: An updated guideline for
using biomechanical and psychophysical models. American Industrial Hygiene reporting systematic reviews. British Medical Journal, 372(71). https://doi.org/
Association Journal, 47, 322–330. https://doi.org/10.1080/15298668691389829 10.1136/bmj.n71
Hoffman, G., Breazeal, C., (2004, September). Collaboration in Human-Robot Teams. Pisanelli, G., Tymczuk, M., Douthwaite, J. A., Aitken, J. M., Law, J. (2022, August)).
AIAA 1st Intelligent Systems Technical Conference, Chicago, Illinois https://doi.org/ ROSIE: A ROS Adapter for a Modular Digital Twinning Framework. 2022 31st IEEE
10.2514/6.2004-6434. International Conference on Robot and Human Interactive Communication (RO-MAN),
Horváth, G., & Erdős, G. (2017). Gesture Control of Cyber Physical Systems. Procedia Napoli, Italy, 1297-1304, doi: 10.1109/RO-MAN53752.2022.9900842.
CIRP, 63, 184–188. https://doi.org/10.1016/j.procir.2017.03.312 Process Simulate by Siemens. Retrieved from: https://plm.sw.siemens.com/en-US/
Huang, J., Pham, D. T., Li, T., Qu, M., Wang, Y., Kerin, M., … Zhou, Z. (2021). An tecnomatix/products/process-simulate-software/. Accessed February 10, 2023.
experimental human-robot collaborative disassembly cell. Computers & Industrial Ramasubramanian, A. K., Mathew, R., Kelly, M., Hargaden, V., & Papakostas, N. (2022).
Engineering, 155, Article 107189. https://doi.org/10.1016/j.cie.2021.107189 Digital Twin for Human-Robot Collaboration in Manufacturing: Review and
Huang, Z., Shen, Y., Li, J., Fey, M., & Brecher, C. (2021). A Survey on AI-Driven Digital Outlook. Applied Sciences, 12(10), 4811. https://doi.org/10.3390/app12104811
Twins in Industry 4.0: Smart Manufacturing and Advanced Robotics. Sensors, 21(19), Rathore, M. M., Shah, S. A., Shukla, D., Bentafat, E., & Bakiras, S. (2021). The role of AI,
6340. https://doi.org/10.3390/s21196340 machine learning, and big data in Digital twinning: A Systematic literature review,
ISO, ISO/AWI 23247 - Automation systems and integration — Digital twin framework for Challenges, and opportunities. IEEE Access, 9, 32030–32052. https://doi.org/
manufacturing, (2021). 10.1109/access.2021.3060863.
Kendzierskyj, S., Jahankhani, H., Jamal, A., & Ibarra, J. (2019). The transparency of big Rodgers, S. H., & Suzanne, H. (1988). Job evaluation in worker fitness determination.
data, data harvesting and digital twins. In In Advanced sciences and technologies for Occupational Medicine, 3, 219–239.
security applications (pp. 139–148). https://doi.org/10.1007/978-3-030-11289-9_6 Rodgers, S. H., & Suzanne, H. (1992). A functional job evaluation technique. Occupational
Kharu, O., Kansi, P., & Kuorinka, I. (1977). Correcting working postures in industry: A Medicine, 7, 679–711.
practical method for analysis. Applied Ergonomics, 8, 199–201. https://doi.org/ ROS. Retrieved from: http://wiki.ros.org/ROS/Introduction. Accessed February 17,
10.1016/0003-6870(77)90164-8 2023.
Kharu, O., Harkonen, R., Sorvali, P., & Vepsalainen, P. (1981). Observing working Saddik, A. E. (2018). Digital Twins: The convergence of multimedia technologies. IEEE
postures in industry: Examples of OWAS application. Applied Ergonomics, 12, 13–17. MultiMedia, 25(2), 87–92. https://doi.org/10.1109/mmul.2018.023121167.
https://doi.org/10.1016/0003-6870(81)90088-0 Schmidt, B., De Ocãna Torroba, A. S., Grahn, G. V., Karlsson, I., & Ng, A. H. C. (2022).
Koukas, S., Kousi, N., Aivaliotis, S., Michalos, G., Bröchler, R., & Makris, S. (2022). ODIN Augmented Reality Approach for a User Interface in a Robotic Production System.
architecture enabling reconfigurable human – robot based production lines. Procedia Advances in Transdisciplinary Engineering. https://doi.org/10.3233/atde220143
CIRP, 107, 1403–1408. https://doi.org/10.1016/j.procir.2022.05.165 Searle, J. R. (1990). Collective intentions and actions. In P. R. Cohen, J. Morgan, &
Kuts, V., Marvel, J. A., Aksu, M., Pizzagalli, S. L., Sarkans, M., Bondarenko, Y., & Otto, T. M. E. Pollack (Eds.), Intentions in Communication, chapter 19 (pp. 401–416).
(2022). Digital Twin as Industrial Robots Manipulation Validation Tool. Robotics, 11 Cambridge, MA: MIT Press.
(5), 113. https://doi.org/10.3390/robotics11050113 Selvaggio, M., Cognetti, M., Nikolaidis, S., Ivaldi, S., & Siciliano, B. (2021). Autonomy in
Leng, J., Wang, D., Shen, W., Li, X., Liu, Q., & Chen, X. (2021). Digital twins-based smart Physical Human-Robot Interaction: A Brief Survey. IEEE Robotics and Automation
manufacturing system design in Industry 4.0: A review. Journal of Manufacturing Letters, 1–1. https://doi.org/10.1109/lra.2021.3100603
Systems, 60, 119–137. https://doi.org/10.1016/j.jmsy.2021.05.011. Sharma, A., Kosasih, E., Zhang, J., Brintrup, A., & Calinescu, A. (2022). Digital Twins:
Li, C., Zheng, P., Yin, Y., Pang, Y. M., & Huo, S. (2023). An AR-assisted Deep State of the art theory and practice, challenges, and open research questions. Journal
Reinforcement Learning-based approach towards mutual-cognitive safe human- of Industrial Information Integration, 30, 100383, ISSN 2452-414X, https://doi.
robot interaction. Robotics and Computer-integrated Manufacturing, 80, Article org/10.1016/j.jii.2022.100383.
102471. https://doi.org/10.1016/j.rcim.2022.102471 Song, X., Sun, P., Song, S., & Stojanović, V. (2023). Quantized neural adaptive finite-time
Liang, X., Xiao, R., & Zhang, J. (2022). Dezember). A Review on Digital Twin for Robotics preassigned performance control for interconnected nonlinear systems. Neural
in Smart Manufacturing. In 2022 IEEE 17th Conference on Industrial Electronics and Computing and Applications, 35(21), 15429–15446. https://doi.org/10.1007/s00521-
Applications (ICIEA). https://doi.org/10.1109/iciea54703.2022.10006119 023-08361-y
Lu, Y. (2017). Industry 4.0: A survey on technologies, applications and open research Stavropoulos, P., & Mourtzis, D. (2022). Chapter 10 - Digital twins in industry 4.0.
issues. Journal of Industrial Information Integration, 6, 1–10. https://doi.org/10.1016/ Design and Operation of Production Networks for Mass Personalization in the Era of
j.jii.2017.04.005 Cloud Technology. Elsevier, 2022, 277–316. https://doi.org/10.1016/B978-0-12-
Lu, Y., Liu, C., Wang, K. I., Huang, H., & Xu, X. (2020). Digital Twin-driven smart 823657-4.00010-5.
manufacturing: Connotation, reference model, applications and research issues. Tabassum, N., Naeem, H., & Batool, A. (2023). The Data Security and multi-cloud
Robotics and Computer-Integrated Manufacturing, 61, Article 101837. https://doi.org/ Privacy concerns. International Journal for Electronic Crime Investigation, 7(1),
10.1016/j.rcim.2019.101837 49–58. https://doi.org/10.54692/ijeci.2023.0701128.
Lu, X., Li, X., Wang, W., Chao, K. M., Xu, L., De Vrieze, P., & Jing, Y. (2022). October). A Tao, F., Zhang, H., Liu, A., & Nee, A. Y. C. (2019). Digital Twin in Industry: State-of-the-
generic and modularized Digital twin enabled human-robot collaboration. In 2022 Art. IEEE Transactions on Industrial Informatics, 15(4), 2405–2415. https://doi.org/
IEEE International Conference on e-Business Engineering (ICEBE). https://doi.org/ 10.1109/tii.2018.2873186
10.1109/ICEBE55470.2022.00021 Unity 3D by Unity Technologies. Retrieved from: https://unity.com/. Accessed February
Malik, A. A., & Bilberg, A. (2018). Digital twins of human robot collaboration in a 9, 2023.
production setting. Procedia Manufacturing, 17, 278–285. https://doi.org/10.1016/j. Visual Components by Visual Components Oy. Retrieved from: https://www.
promfg.2018.10.047 visualcomponents.com/products/premium/. Accessed January 29, 2023.
Malik, A. A., Masood, T., & Bilberg, A. (2019). Virtual reality in manufacturing: Wang, X. V., Kemény, Z., Váncza, J., & Wang, L. (2017). Human-Robot Collaborative
Immersive and collaborative artificial-reality in design of human-robot workspace. Assembly in Cyber-Physical Production: Classification Framework and
International Journal of Computer Integrated Manufacturing, 33(1), 22–37. https://doi. Implementation. CIRP Annals Manufacturing Technology, 66(4), 5–8. https://doi.org/
org/10.1080/0951192x.2019.1690685 10.1016/j.cirp.2017.04.101
14
Wang, L., Gao, R., Váncza, J., Krüger, J., Wang, X. V., Makris, S., & Chryssolouris, G. Wullenkord, R., & Eyssel, F. A. (2020). Societal and ethical issues in HRI. Current Robotics
(2019). Symbiotic human-robot collaborative assembly. CIRP Annals - Manufacturing Reports, 1(3), 85–96. https://doi.org/10.1007/s43154-020-00010-9
Technology, 68, 701–726. https://doi.org/10.1016/j.cirp.2019.05.002 Yi, S., Liu, S., Xu, X., Wang, X. V., Yan, S., & Wang, L. (2022). A vision-based human-
Wang, S., Zhang, J., Wang, P., Law, J., Călinescu, R., & Mihaylova, L. (2024). A deep robot collaborative system for digital twin. Procedia CIRP, 107, 552–557. https://doi.
learning-enhanced Digital Twin framework for improving safety and reliability in org/10.1016/j.procir.2022.05.024
human–robot collaborative manufacturing. Robotics and Computer-Integrated Zandin, K. B. (2003). MOST Work Measurement Systems. New York City: Marcel Dekker.
Manufacturing, 85, Article 102608. https://doi.org/10.1016/j.rcim.2023.102608 Zaccaria, V., Stenfelt, M., Aslanidou, I., & Kyprianidis, K. (2018). Fleet monitoring and
Waters, T.R., Vern, P.A., Garg, A. (1994). Application Manuals for the Revised NIOSH Diagnostics framework based on digital twin of Aero-Engines. Volume 6: Ceramics;
Lifting Equation. In Cincinnati, OH: US Department of health and human services, Controls, Diagnostics, and Instrumentation; Education; Manufacturing Materials and
National Institute for Occupational Safety and Health. DHHS (NIOSH) Publication, Metallurgy. https://doi.org/10.1115/gt2018-76414.
USA, p. https://doi.org/10.26616/NIOSHPUB94110revised092021. Zhou, C., Tao, H., Chen, Y., Stojanović, V., & Paszke, W. (2022). Robust point-to-point
Weber-Lewerenz, B. C. (2021). Corporate digital responsibility (CDR) in construction iterative learning control for constrained systems: A minimum energy approach.
engineering—ethical guidelines for the application of digital transformation and International Journal of Robust and Nonlinear Control, 32(18), 10139–10161. https://
artificial intelligence (AI) in user practice. SN. Applied Sciences, 3(10). https://doi. doi.org/10.1002/rnc.6354
org/10.1007/s42452-021-04776-1 Zhu, Q., Huang, S., Wang, G., Moghaddam, S. K., Lu, Y., & Yan, Y. (2022). Dynamic
Wilhelm, J., Petzoldt, C., Beinke, T., & Freitag, M. (2021). Review of Digital Twin-based reconfiguration optimization of intelligent manufacturing system with human-robot
Interaction in Smart Manufacturing: Enabling Cyber-Physical Systems for Human- collaboration based on digital twin. Journal of Manufacturing Systems, 65, 330–338.
Machine Interaction. International Journal of Computer Integrated Manufacturing, 34 https://doi.org/10.1016/j.jmsy.2022.09.021
(10), 1031–1048. https://doi.org/10.1080/0951192x.2021.1963482 Zhuang, Z., Tao, H., Chen, Y., Stojanović, V., & Paszke, W. (2023). An optimal iterative
Wolfram Mathematica by Wolfram Research. Retrieved from: https://www.wolfram. learning control approach for linear systems with nonuniform trial lengths under
com/mathematica/. Accessed January 31, 2023. input constraints. IEEE Transactions on Systems, Man, and Cybernetics, 53(6),
Wu, J., Chen, W., & Ephremides, A. (2023). Achieving extremely low latency: 3461–3473. https://doi.org/10.1109/tsmc.2022.3225381
Incremental coding for Real-Time applications. IEEE Transactions on Communications,
71(8), 4453–4467. https://doi.org/10.1109/tcomm.2023.3278313
15

Digital Twin For Human-Robot Collaboration

Uploaded by

Copyright:

Available Formats

Digital Twin For Human-Robot Collaboration

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Digital Twin For Human-Robot Collaboration

Uploaded by

Copyright:

Available Formats

Computers & Industrial Engineering 187 (2024) 109764

Contents lists available at ScienceDirect

Computers & Industrial Engineering