World Academy of Science, Engineering and Technology

International Journal of Computer and Information Engineering

Vol:11, No:10, 2017

An Overview of Evaluations Using Augmented

Reality for Assembly Training Tasks
S. Werrlich, E. Eichstetter, K. Nitsche, G. Notni

 design an AR-based application for assembly training and

Abstract—Augmented Reality (AR) is a strong growing research which advantages result in the comparison to an expert based
topic in different training domains such as medicine, sports, military, training?
education and industrial use cases like assembly and maintenance We make the following contributions:
tasks. AR claims to improve the efficiency and skill-transfer of
 In Section II, we describe our literature research process.
training tasks. This paper gives a comprehensive overview of
Four online databases are used for the research using the
International Science Index, Computer and Information Engineering Vol:11, No:10, 2017 waset.org/Publication/10007977

evaluations using AR for assembly and maintenance training tasks

published between 1992 and 2017. We search in a structured way in keywords “augmented reality” and “training”. The search
four different online databases and get 862 results. We select 17 returned 862 papers, with a select 54 papers in the field of
relevant articles focusing on evaluating AR-based training AR-based training for assembly and maintenance tasks.
applications for assembly and maintenance tasks. This paper also
indicates design guidelines which are necessary for creating a
successful application for an AR-based training. We also present five
scientific limitations in the field of AR-based training for assembly
tasks. Finally, we show our approach to solve current research
problems using Design Science Research (DSR).

Keywords—Assembly, augmented reality, survey, training.

I. INTRODUCTION

A SSEMBLY training of novice employees plays a crucial

role in the car manufacturing industry. Processes and
products become more and more complex, which means a
higher responsibility and challenge for every single assembly
employee. Therefore, a good assembly training can prevent Fig. 1 AR-based engine assembly training using a HMD
errors and should be efficiently designed. A common method
is face-to-face training where an expert is teaching the novice  In Section III, we present the research results and describe
employee new assembly procedures. This method is time the papers found. We give background information about
consuming and expensive because it requires an always the participants in the studies and the results of the
available expert for the trainee. A new technology for training, evaluations. Section III ends with the presentation of
which is discussed in several studies, is AR. AR claims to design guidelines for AR, especially for AR-based
improve the efficiency and skill-transfer of assembly tasks by assembly and maintenance training tasks.
visualizing computer generated 3D information in the real  Based on the existing research, we indicate five
environment (Fig. 1). There has been no structured literature limitations in Section IV: the participants, the assembly
review in that field so far and therefore, this paper aims to give tasks, the comparisons, the KPIs and the technology.
a comprehensive overview of AR based assembly studies.  In Section V, we describe our approach to solve the
This paper focuses on AR-based training applications for indicated research problems. The DSR framework is best
assembly tasks, especially on evaluations and guidelines suitable for our future research. The methodology to
because we want to answer the following question: How to address the seven DSR guidelines is presented at the end
of this section.
 The summary of the paper is given in Section VI.
S. Werrlich is a PhD candidate at BMW Group Munich, 80788 München.
He is also with the Group for quality assurance and industrial image
processing at the Technical University Ilmenau, 98693 Ilmenau, Germany (e- II. LITERATURE RESEARCH PROCESS
mail: [email protected])
E. Eichstetter is an intern at the BMW Group Munich, 80788 München. Most current scientific research follows an unstructured
She is also with the Hochschule München, 80335 München, Germany (e-mail: approach; it is not systematic, and therefore, lacks in
[email protected]). completeness. This paper presents research with a more
K. Nitsche is the head of the quality management department for electrified
powertrains at BMW Group Munich, 80788 Munich, Germany (e-mail: structured approach. Before starting the search, we define our
[email protected]). keywords, which are necessary for the research in online
G. Notni is professor and the head of the Group for quality assurance and databases. We use the keywords “augmented reality” and
industrial image processing, Technical University Ilmenau, 98693 Ilmenau,
Germany (e-mail: [email protected]). “training” for our research and limit our research to abstracts,

International Scholarly and Scientific Research & Innovation 11(10) 2017 1129 scholar.waset.org/1307-6892/10007977
 World Academy of Science, Engineering and Technology
International Journal of Computer and Information Engineering
Vol:11, No:10, 2017

titles and keywords. We use online databases which are maintained a Boeing 737 engine bleed air system with the
recommended in previous summaries for AR [1]. Those support of either a tablet or written instructions. Results
databases are: IEEE Xplore, ACM Digital Library, Science showed a time improvement of 17% percent and 24% quality
Direct and Web of Science. In IEEE Xplore and ACM Digital increase when using AR.
Library it is only possible to search either for abstracts, titles Another study by [11] focused on the benefits of AR in
or keywords. A combined search is not possible. A combined gender. A LEGO model was used for the assembly procedure.
search is possible in Science Direct and Web of Science. The Twenty eight students participate in the study. They were
search returned 862 papers (Fig. 2). divided into two groups with seven males and seven females
in each group. The first group used a computer monitor to
train the assembly process and the second group used a 3D
printed manual. After finishing one LEGO assembly cycle, the
participants had to repeat the assembly procedure without any
assistance. The results show the AR helps both male and
female trainees to learn the assembly procedure faster. AR-
International Science Index, Computer and Information Engineering Vol:11, No:10, 2017 waset.org/Publication/10007977

based training is also more effective for male and female

Fig. 2 Results for "augmented reality" AND training
trainees than a 3D manual.
Horejsi et al. [12] compared paper based instructions with
It the next step, we read all document titles and abstracts AR, using a monitor. Twenty volunteers were asked to
and decided if the paper is relevant to the present research. We assemble a gully trap using paper based instructions. Another
are interested in user studies and design guidelines for AR- 20 participants assembled with AR support. Results indicate
based training because this is still an unexplored field [2]. This that trainees learn 43% faster, using AR.
paper aims to create a status quo in order to present further A study from the late 90s [13] compared VR with AR as a
research directions. training tool. In five groups with five students each, a water
Scientific contributions from other domains like medicine pump should be assembled. The study compares 2D drawing,
[3], sports [4] and military [5] are excluded because this is not a Desktop solution, stereoscopic glasses, immersive VR using
scope of this paper. Also excluded are paper from the field of a head-mounted display (HMD) and context free AR.
virtual reality (VR) training [6]. Research contributions Participants were faster when using AR compared to all other
focusing only on the description of AR-based systems [7] (e.g. methods.
new tracking algorithms) for training but not on evaluation Valimont et al. [14] used four learning paradigms. The
were excluded as well. study compared a printed manual, a video tape, text
In [8], it is mentioned that a training application must differ annotations by mouse interaction and AR using a monitor.
from a guidance application because a step-by-step solution is Participants were encouraged to learn the assembly procedure
not sufficient for training. An application with a strong from an oil pump as well as the functions of single parts and
guidance has the danger that the trainee becomes dependent on the different part locations. The experiment indicated that AR
the support and therefore might be not able to perform the task performed better in the short-term and the long-term memory
without any assistant. Therefore, this paper is not focusing on test compared to the other three methods.
AR-based guidance applications [9] for assembly or Another study by Pathomaree et al. [15] focused on the skill
maintenance tasks. transfer in assembly tasks, using a 2D and a 3D puzzle.
We indicate 17 relevant papers, focusing on user evaluation Twenty participants were exposed to one of four experiments.
and seven design recommendations for user interface design, Each participant was asked to assemble the 2D or 3D puzzle
including those for AR-based training applications. In order to either with or without AR. All instructions were shown on a
create a comprehensive overview, it is necessary to check the monitor. The assembly procedure was shown once. Trainees
references of the indicated paper. Based on the titles in the had to assemble the puzzle afterwards without any support. It
references, we decide if a paper fits in our category of AR was shown that AR helps to increase training efficiency.
based assembly and maintenance training. The indicated titles Trainees also needed less assembly steps to finish the task,
are searched in the online databases and added to our final which indicates a better skill-transfer when using AR.
database if an article fits the determined category. We present Macchiarella et al. [16] measured the impact of short and
our results in the Section III. long-term memory and compared different presentation
modes: print-based, video-based, AR-based and interactive
III. AR-BASED ASSEMBLY TRAINING AR-based. A total of 96 participants, all students, were
divided in four groups and had to study an aircraft oil pump
In this section, we present user evaluations using AR for
with support from one presentation mode. Trainees were asked
assembly and maintenance training tasks as well as design
afterwards if they remember the names of major components
guidelines for AR-based training.
as well as several functionalities in order to determine how
A. Evaluations much knowledge they acquired in the training session. The
Rios et al. [10] compared a tablet-based solution with long-term recall test occurred seven days later where the same
written instructions. Seventeen participants (all students) questions were asked. Results from the immediate and long-

International Scholarly and Scientific Research & Innovation 11(10) 2017 1130 scholar.waset.org/1307-6892/10007977
 World Academy of Science, Engineering and Technology
International Journal of Computer and Information Engineering
Vol:11, No:10, 2017

term recall indicate that trainees forget less over time when Participants made also fewer errors with support of the AR
using AR and interactive AR, compared to print and video- system. A higher mental workload was measured when
based learning. participants used the manual in comparison to the AR system
Webel et al. [8] did a skill transfer evaluation and compared for the assembly process. The second experiment aimed to
two learning approaches. Twenty participants, all technicians measure the learning curve for the two scenarios in order to
with at least two years of assembly experience attended to the find a difference between the two systems. Participants had to
study. They had to perform an assembly task with 25 steps. In assemble the Lego model once. They were encouraged to
Group 1, 10 trainees had to watch an instructional video remember the assembly procedure. After 5 min relaxing, they
showing the assembly task of an electro-mechanical actuator. had to assemble the model again without any assistance. The
The second group, consisting of 10 participants, performed the results show that the AR-based training reduces the learning
physical task with AR support using a tablet. All trainees did curve and the number of errors.
the training in the morning. They had to perform the same In [20], a monitor-based system was used. For the assembly
physical task without any assistance in the afternoon. The training task, they used 12 parts of a RV-10 aircraft. Seven
evaluation showed that trainees perform 20% slower and made engineering students (all male) with no prior knowledge of
International Science Index, Computer and Information Engineering Vol:11, No:10, 2017 waset.org/Publication/10007977

77% less un-solved errors when trained with AR. assembling took part in the experiment. All of them had to
Gavish et al. [17] compared VR-based assembly training perform six assembly trials, measuring the time, errors and the
with AR-based assembly training. 40 technicians were questions users ask to the supervisor who was available at all
randomly assigned to four groups: VR training, AR training times. Results indicate a shorter assembly time for an AR-
and two control groups. They used an electronic actuator with assisted assembly compared to a traditional manual.
25 assembly steps for the experiment. The VR and VR control Peniche et al. [21] combined AR and VR. In the first
group performed the assembly task twice with support of the training session, participants had to perform the assembly task
VR training platform. The AR and control AR group in VR five times and continued in the second session with AR
performed the assembly task with the AR tablet-based three times. The advantage of VR-based training is the save of
platform just once. After training, participants had to perform resources because trainees learn with virtual parts. Additional,
the task again without any support, after a break of several users continue the training with an AR-based system where
hours. They determined that the VR and AR groups had they come into direct contact with the real parts. The task was
significantly longer training times compared to the control to assemble a milling machine. Before they start the two
groups. Results also indicated a higher user satisfaction and training conditions, participants had to perform the task twice
usability when using the AR platform for training. in order determine their initial skills. The control group
Furthermore, trainees made less assembly errors when using performed the assembly task eight times using the
the AR platform. conventional methodology. They measured the time to
Another study compared a video-see through solution with complete the task and found out that their AR system lowered
face-to-face training [18]. In this evaluation, an Oculus Rift the performance in session seven, when the switched from
VR Headset with a mounted pair of cameras to the HMD is VR-based training to the AR-based training. It was also shown
used. Some 24 volunteers, twelve in each group, had to learn a that VR was able to replace a major part of the training and
maintenance task of an aircraft door. Group 1 had to observe transferred skills in a significant manner.
what the trainer was doing and try to remember as much as Rios et al. [22] compared three different methodologies for
possible. Group 2 used the HMD for the training. After the transfer of knowledge of complex training task in
training, trainees were asked to complete a knowledge aeronautical processes. He compared printed instructions with
retention test, which was a written multiple choice test with an audiovisual tool and an AR application using a laptop. The
eight questions and a knowledge interpretation test. This test training duration was four hours. Equal to [20], they measured
evaluated whether the assembly procedure was captured. the time, errors and questions asked during the training to the
Results indicated no significant difference for the knowledge supervisor. The same assembly task, as well as seven male
retention test and no significant differences for the knowledge engineering students, was used. Trainees needed more time
interpretation test between the two conditions. Trainees spent completing the task when using the AR application. There
more time to complete the training when using the HMD. were a 4.5min time difference between AR and the
Hou et al. [19] compared an animated AR-system, using a audiovisual tool. The longest training was with the traditional
big screen, with a paper-based manual system. He used a Lego training method. The error and question comparison showed a
model for the experiments. 50 participants, all students, were difference between AR and the other methodologies. Trainees
conducted to compare both systems. None of them had made up to four times fewer errors using the AR-system
experience in using AR. The first experiment aimed to compared to the other methodologies and needed less help
compare the cognitive workload of both systems using the from the supervisor to complete the task.
NASA-TLX questionnaire. They also measured the time to Westerfield et al. [23] used a HMD to compare a traditional
complete the assembly task and the number of errors AR-based system using a step-by-step guidance with an
(selection error, assembly sequential error, installation error intelligent AR solution which gives feedback to the users. The
and help from the AR system or manual). The first experiment aim was to determine the difference in knowledge retention
indicated a shorter completion time using the AR system. between the two approaches. Sixteen randomly assigned

International Scholarly and Scientific Research & Innovation 11(10) 2017 1131 scholar.waset.org/1307-6892/10007977
 World Academy of Science, Engineering and Technology
International Journal of Computer and Information Engineering
Vol:11, No:10, 2017

participants (11 male and five female students) performed one [28]. Their similarities are listed below:
of the conditions. They had no experience with computer  Simple Design: An application should have all the
hardware assembly. All of them had to assemble five required functionalities, not more and not less.
motherboard components. After performing the training once,  Feedback: Feedback from the system is necessary at
participants had to repeat the task without any assistance. Only every user interaction in order to inform the user about the
the names of the components were given and users had to current action.
correctly identify and install them. The completion time and  Preknowledge: The system should take users
errors were also measured during the process. There were no preknowledge into account, e.g. everyday life
significant differences in errors or completion time comparing symbolisms.
both systems. Trainees who learned with the intelligent AR  Consistency: The system should use standards and be
system scored 25% higher in the knowledge retention test and consistent throughout the entire application.
performed 30% faster. Giving user feedback is important and  User Mode: The system should have different modes
can significantly improve the learning outcome compared to (tutorial, beginner, intermediate, advanced) in order to
traditional approaches. allow advanced users to work faster.
International Science Index, Computer and Information Engineering Vol:11, No:10, 2017 waset.org/Publication/10007977

Vincenzi et al. [24] compared four different learning  User Control: The system should allow to go back,
methodologies: printed material, video tape, text annotations forward and to the starting point at any time.
activated by mouse interaction and AR. They performed an  Help: The system should recognize incorrect user actions
immediate and long-term recall test. The aim was to find the and inform then with a clear understandable solution.
best learning paradigm. The participants were all students and  Error prevention: The system should prevent errors e.g.
had no experience of an aircraft oil pump before. The four ask the user before acting.
groups performed an eight-minute session, learning the  Individualization: The system should be adaptable to
terminology, functions and locations of the oil pump parts. personal user demands, e.g. different languages.
After a short three-minute break, participants were asked to  Help & Documentation: Help should be available. The
answer 10 multiple choice questions and five matching system should give easy understandable help at the right
questions. They were asked again after seven days in order to time.
measure how much knowledge they retained after one week. These principles allow designing a user interface for
The results indicate that participants who received instruction standard devices such as smartphones or tablets and can be
through AR demonstrated significantly better immediate recall also used for AR devices. Regenbrecht et al. [30] presented
results than those who received print instruction or interactive suggestions especially for industrial AR after five years’
video instruction. Participants in the AR instructional group experience in 10 different projects which should be noticed
also scored significantly higher on the long-term recall test when designing AR applications for industrial use cases:
than the video group. However, the AR group failed to show  Real Data integration: Most of the demo scenarios work
significantly better performance than the video group for pretty well, but when it comes to the first real scenario
immediate recall, and the print and interactive video groups trial, systems mostly fail because of the data quantity and
for long-term recall. complexity. Therefore, the usage of real data from the
Haritos and Macchiarella [25] compared video-based, AR- beginning of the project is crucial for the project’s
based and print-based learning approaches. They had 36 success.
participants (all students) performing the two conditions. They  Acceptance: The initial application should be
were divided into three groups with 12 participants each. Each implemented very carefully with the help of key persons
group performed an eight-minute instructional session, who are accepted among their colleagues and work as
learning about the airplane propeller, its mounting bolts and close as possible in cooperation with AR researchers. He
safety instructions. The aim was to determine how much also prefers an “island setup” beside the production line
knowledge participants gained through the different training with single-user and a single-task setting.
sessions. An immediate and a long-term recall test after seven  Simplicity: According to [26], a simple solution is also
days was performed. No results were given in this paper. It is recommended instead of an application with the highest
not clear which method is the best for training. level of originality or novelty.
 Added Value: Its crucial for the project to get started to
B. Design Guidelines
outline the return of invest (ROI). If a researcher wants to
Users have high expectations regarding the quality and start a project in the automotive industry, using AR, they
functionality of an application. In order to meet this have to estimate factors like cost and time savings.
requirement, an app designer uses established design All mentioned design guidelines are general for user
guidelines (DG) for their user interface development. We interface design and can be also used for AR app
present three general DGs which are well-known in the field development. More specific design guidelines for AR-based
of user interface design. The first is the DIN EN ISO 9241- training for assembly and maintenance task should be noticed.
110 [26] which delivers seven heuristics for the user interface Until now, there are just a few guidelines available.
design. Further are the principles by Nielsen and Molich [27] Webel et al. [29] presented four design recommendations
and the well-known “8 Golden Rules of Interface Design” which help to train procedural skills. Procedural skills are the

International Scholarly and Scientific Research & Innovation 11(10) 2017 1132 scholar.waset.org/1307-6892/10007977
 World Academy of Science, Engineering and Technology
International Journal of Computer and Information Engineering
Vol:11, No:10, 2017

most important abilities in assembly and maintenance task results made with students are subjective.
because it is the human capability to follow a task step-by-  Limitation 2- Assembly Task: Researcher use simple
step. An AR-based application should rather focus on training assembly tasks such as LEGO models for their assembly
these skills than guiding a user through the task such as a task. The results, made with those easy assembly tasks are
navigation system. With a strong guidance, the user can not reliable because a repetition of the measurement in a
become dependent from the technology and may not be able to much more complex environment like an engine assembly
repeat the task without any assistance. The following line might be much worse or even not possible with the
recommendations can help to strengthen the procedural skill: used technology (e.g. projection-based AR).
 Visual Aids: Direct visual aids are permanent presented  Limitation 3- Comparisons: Researchers tend to
information such as 3D models superimposed on the compare their AR-based training systems mostly against
related real environment. Indirect visual aids are paper-based or video-based solutions. Until now, there are
additional information, only presented or available for the no comparisons between an AR-based assembly training
user when needed (e.g. text annotations, documentation). and face-to-face training, which is the current training
This concept allows to adapt information during the solution in the automotive industry [35].
International Science Index, Computer and Information Engineering Vol:11, No:10, 2017 waset.org/Publication/10007977

learning process. Clear and detailed instructions at the  Limitation 4- KPIs: Researchers mostly focus on
beginning are necessary for the trainee to understand and measuring the time, which is an insufficient variable for
perform the tasks. During the training, information can be training. There are few evaluations, measuring the quality
gradually reduced with each assembly cycle. This concept and the training transfer (immediate recall and long-term
is supported by Fitts and Posner [33] who divided the recall), which should be favored when evaluating AR-
learning process into three stages: cognitive, associative based training systems.
and autonomous. Different amounts of information are  Limitation 5- Technology: Hand-held devices such as
needed to perform the task in each of the learning stages. tablets and smartphones as well as monitors are used in
 Mental Model Building: The mental model of an most of the studies. When assembling, users need to work
assembly task describes the internal representation of an hands-free, which is not possible with such a device. A
entire task. Context information such as progress bars can possible solution would be a HMD. HMDs are becoming
help creating a mental model. It is also favorable for the less bulky and lighter. They can be used for assembly
mental model building to present pre- and post-conditions training, but until now, there has been no evaluation in an
of the task [34]. industrial environment. Future studies should investigate
 Passive Learning: There should be a part in the training the potential of HMDs such as the Microsoft HoloLens
where the trainee is not active and only receives for assembly training measuring the usability, the quality,
information about the task. This concept can help to gain the efficiency of the training, ergonomic aspects (e.g.
a global picture of the entire task. NASA-TLX) as well as the impact on short- and long-
 Haptic hints: A vibrotactile bracelet allows to give term memory.
feedback about current actions and can help to prevent
errors. Furthermore, this multimodal approach supports V. FUTURE RESEARCH
human’s memory and therefore the assembly training. We indicated five research limitations in Section IV. In
Another study [31] aimed to find the optimum amount of Section V, we present our approach to address these
information to be delivered at the same time for novice user. limitations and how we want to gain new knowledge. We use
The optimum amount was found to be from four to five the DSR for future research (Fig. 3). This framework is most
information pieces at the same time. notable in Computer Science and aims to develop and evaluate
new artifacts such as algorithms, new human-computer
IV. CURRENT LIMITATIONS interfaces or methodologies.
The presented studies in Section III show that AR can help
to improve the efficiency and quality of assembly training task
but all of them have several limitations which will be summed
up in this section.
 Limitation 1- Participants: Researchers use mostly
students, without any assembly experience, in their
assessments. Webel [32] mentioned that cognitive skills
such as the procedural memory as well as fine motor
skills are necessary to perform an assembly task. AR can
help train the procedural skills but not fine motor skills.
These skills were acquired through years of experience in
the assembly domain. Compared to a real assembly
worker with years of assembly experience, students might Fig. 3 DSR Framework
be much slower and their assembly quality worse, using
the same technology. Therefore, the existing evaluation

International Scholarly and Scientific Research & Innovation 11(10) 2017 1133 scholar.waset.org/1307-6892/10007977
 World Academy of Science, Engineering and Technology
International Journal of Computer and Information Engineering
Vol:11, No:10, 2017

Hevner et al. [36] presented seven guidelines for DSR We present our research contributions to the scientific
which will be addressed in our future research. Our aim is to community as well as to management audiences (guideline 7).
develop and evaluate an HMD-based training for assembly
tasks (guideline 1). Werrlich et al. [35] analyzed an assembly VI. CONCLUSION
training in an industrial environment and found out that the In this paper, we gave a comprehensive overview on
current assembly training is time consuming and stress evaluations and design guidelines for AR-based assembly
provoking for new employees. Furthermore, the pedagogical training tasks. We used a structured research method and
skill varies between different trainers and therefore the searched in four online databases. We used the keywords
training is not standardized. We try to solve these problems “augmented reality” AND training and got 862 results. We
with our artifact (guideline 2). In Section III, several design indicated 17 relevant user studies and seven design guidelines
guidelines which we use for the development of our first which we will apply for our future application development.
application were presented. Werrlich et al. [35] did semi- Based on our findings, we indicated five scientific limitations
structured interviews and found basic requirements for the such as participants, simple user tasks, lack of comparisons,
development of an HMD-based assembly training. We use that insufficient KPIs and the technology used in the studies. We
International Science Index, Computer and Information Engineering Vol:11, No:10, 2017 waset.org/Publication/10007977

knowledge for our application development and evaluate will address these limitations in our future work using DSR.
(guideline 3 and guideline 5) our artifact, regarding the utility We will develop an HMD-based engine assembly training
and usability, with assembly employees under field conditions using the Microsoft HoloLens and evaluate our application in
in an industrial environment using standardized evaluation a real industrial environment with skilled assembly workers.
techniques (e.g. System Usability Scale). We assess and Our future evaluations are divided in two phases. The first is
improve our artifact continuous (guideline 6). Therefore, this the evaluation phase, the formative evaluation, which focuses
research approach provides clear research contributions on iterative designing, assessing and improving our
through the assessments, measuring additionally the quality of application, while the second phase, the summative
the training as well as the short and long-term memory. We evaluation, focuses on the comparison between HMD-based
will also deliver the first comparison between a HMD-based training and traditional face-to-face training.
assembly training (Fig. 2) and a face-to-face training (Fig. 3)
and therefore address the fourth DSR guideline. ACKNOWLEDGMENT
We would like to thank the BMW Group for its assistanc3e
and for providing the tools, space and funding to realize and
complete the project successfully.

REFERENCES
[1] Dünser, Andreas; Grasset, Raphaël; Billinghurst, Mark (2008): A survey
of evaluation techniques used in augmented reality studies. In: Human
Interface Technology Laboratory New Zealand, S. 1–27. DOI:
10.1145/1508044.1508049.
[2] Wang, X.; Ong, S. K.; Nee, A. Y. C. (2016): A comprehensive survey of
augmented reality assembly research. In: Advances in Manufacturing, S.
1–22. DOI: 10.1007/s40436-015-0131-4.
[3] Rolland, J.; Meyer, C.; Davis, L.; Hamza-Lup, F.; Norfleet, J.;
Imielinska, C.; Kerner, K. F. (2002): Merging augmented reality and
anatomically correct 3D models in the development of a training tool for
endotracheal intubation. In: Biomedical Imaging, 2002. Proceedings.
2002 IEEE International Symposium on, S. 895–898. DOI:
Fig. 4 HMD-based engine assembly training
10.1109/ISBI.2002.1029405.
[4] Daiber, Florian; Kosmalla, Felix; Krüger, Antonio (2013): BouldAR –
Using AugmentedReality to Support CollaborativeBoulder Training. In:
CHI 2013: Changing Perspectives, Paris, France, S. 949–954. DOI:
10.1145/2468356.2468526.
[5] Stevens, Jon (2014): Augmented Reality Technology in U.S. Army
Training (WIP).
[6] Langley, A.; Lawson, G.; Hermawati, S.; D'Cruz, M.; Apold, J.; Arlt, F.;
Mura, K. (2016): Establishing the Usability of a Virtual Training System
for Assembly Operations within the Automotive Industry. In: Human
Factors and Ergonomics in Manufacturing & Service Industries, S. 667–
679. DOI: 10.1002/hfm.20406.
[7] Gorecky, Dominic; Worgan, Simon F.; Meixner, Gerrit (2011):
COGNITO - A Cognitive Assistance and TrainingSystem for Manual
Tasks in Industry. In: Proceedings of ECCE 2011 Conference, S. 53–56.
DOI: 10.1145/2074712.2074723.
[8] Webel, Sabine; Bockholt, Uli; Engelke, Timo; Gavish, Nirit; Olbrich,
Manuel; Preusche, Carsten (2013): An augmented reality training
platform for assembly and maintenance skills. In: Robotics and
Autonomous Systems, S. 398–403. DOI: 10.1016/j.robot.2012.09.013.
Fig. 5 Trainer-based engine assembly training [9] Loch, Frieder; Quint, Fabian; Brishtel, Iuliia (2016): Comparing Video

International Scholarly and Scientific Research & Innovation 11(10) 2017 1134 scholar.waset.org/1307-6892/10007977
 World Academy of Science, Engineering and Technology
International Journal of Computer and Information Engineering
Vol:11, No:10, 2017

and Augmented Reality Assistance in Manual Assembly. In: 2016 12th [27] Nielsen, J.; Molich, R. (1990): Improving a human-computer dialogue.
International Conference, S. 147–150. DOI: 10.1109/IE.2016.31. In: Communications of the ACM 33, 3 (March), S. 338–348.
[10] Rios, Horacio; González, Eduardo; Rodriguez, Ciro; Siller, Hector R.; [28] Shneiderman, Ben; Plaisant, Catherine; Cohen, Maxine S.; Jacobs,
Contero, Manuel (2013): A Mobile Solution to Enhance Training and Steven M.; Elmqvist, Niklas (2016): Designing the user interface:
Execution of Troubleshooting Techniques of the Engine Air Bleed Strategies for effective human-computer interaction. In: CEC Faculty
System on Boeing 737. In: Procedia Computer Science, S. 161–170. Books and Book Chapters.
DOI: 10.1016/j.procs.2013.11.020. [29] Webel, Sabine; Bockholt, Ulrich; Engelke, Timo; Gavish, Nirit; Tecchia,
[11] Hou, Lei; Wang, Xiangyu (2013): A study on the benefits of augmented Franco (2011): Design Recommendations for Augmented Reality based
reality in retaining working memory in assembly tasks: A focus on Training of Maintenance Skills. In: Alem L, Huang WD (eds) Recent
differences in gender. In: Automation in Construction, S. 38–45. DOI: trends of mobile collaborative augmented reality systems, S. 69–82.
10.1016/j.autcon.2012.12.007. DOI: 10.1007/978-1-4419-9845-3_5.
[12] Hořejší, Petr (2015): Augmented Reality System for Virtual Training of [30] Regenbrecht, H.; Baratoff, G.; Wilke, W. (2005): Augmented Reality
Parts Assembly. In: Procedia Engineering, S. 699–706. DOI: Projects in the Automotive and Aerospace Industries. In: IEEE
10.1016/j.proeng.2015.01.422. Computer Graphics and Applications, S. 48–56. DOI:
[13] Boud, A. C.; Haniff, D. J.; Baber, C.; Steiner, S. J. (1999): Virtual 10.1109/MCG.2005.124.
reality and augmented reality as a training tool for assembly tasks. In: [31] Yim, Ho Bin; Seong, Poong Hyun (2010): Heuristic guidelines and
Information Visualization, 1999. Proceedings. 1999 IEEE International experimental evaluation of effective augmented-reality based
Conference on, S. 32–36. DOI: 10.1109/IV.1999.781532. instructions for maintenance in nuclear power plants. In: Nuclear
[14] Valimont, R. B.; Vincenzi, D. A.; Gangadharan, S. N.; Majoros, A. E. Engineering and Design, S. 4096–4102. DOI:
International Science Index, Computer and Information Engineering Vol:11, No:10, 2017 waset.org/Publication/10007977

(2002): The effectiveness of augmented reality as a facilitator of 10.1016/j.nucengdes.2010.08.023.

information acquisition. In: Digital Avionics Systems Conference, 2002. [32] Webel, Sabine; Bockholt, Ulrich; Keil, Jens (2011): Design Criteria for
Proceedings. The 21st, S. 7C5-1-7C5-9. DOI: AR-Based Training of Maintenance and Assembly Tasks. In:
10.1109/DASC.2002.1052926. Proceedings of the 2011 international conference on virtual and mixed
[15] Pathomaree, Nattapol; Charoenseang, Siam (2005): Augmented reality reality: new trends - volume part 1, S. 123–132. DOI: 10.1007/978-3-
for skill transfer in assembly task. In: Robot and Human Interactive 642-22021-0_15.
Communication, 2005. ROMAN 2005. IEEE International Workshop [33] Fitts, P.M.; Posner, M.I. (1967): Human performance. In: Oxford,
on, S. 500–504. DOI: 10.1109/ROMAN.2005.1513829. England: Brooks/Cole.
[16] Macchiarella, N. D.; Vincenzi, D. A. (2004): Augmented reality in a [34] Taatgen, Niels A.; Huss, David; Dickison, Daniel; Anderson, John R.
learning paradigm for flight aerospace maintenance training. In: Digital (2008): The acquisition of robust and flexible cognitive skills. In: Journal
Avionics Systems Conference, 2004. DASC 04. The 23rd, S. 5.D.1-5.1- of experimental psychology. General, S. 548–565.
9. DOI: 10.1109/DASC.2004.1391342. [35] Werrlich, Stefan; Nitsche, Kai; Notni, Gunther (): Demand Analysis for
[17] Gavish, Nirit; Gutiérrez, Teresa; Webel, Sabine; Rodríguez, Jorge; an Augmented Reality based Assembly Training. In: PETRA- The 10th
Peveri, Matteo; Bockholt, Uli; Tecchia, Franco (2013): Evaluating International Conference, S. 416–422. DOI: 10.1145/3056540.3076190.
virtual reality and augmented reality training for industrial maintenance [36] Alan R. Hevner; Salvatore T. March; Jinsoo Park; Sudha Ram (2004):
and assembly tasks. In: Interactive Learning Environments, S. 778–798. Design Science in Information Systems Research. In: MIS Quarterly
DOI: 10.1080/10494820.2013.815221. Vol. 28 No. 1, S. 75–105.
[18] Gonzalez-Franco, Mar; Pizarro, Rodrigo; Cermeron, Julio; Li, Katie;
Thorn, Jacob; Hutabarat, Windo; Tiwari, Ashutosh; Bermell-Garcia,
Pablo (2017): Immersive Mixed Reality for Manufacturing Training. In:
Frontiers in Robotics and AI, S. 1. DOI: 10.3389/frobt.2017.00003.
[19] Hou, Lei; Wang, Xiangyu; Bernold, Leonhard; Love, Peter E. D. (2013):
Using Animated Augmented Reality to Cognitively Guide Assembly. In:
Journal of Computing in Civil Engineering, S. 439–451. DOI:
10.1061/(ASCE)CP.1943-5487.0000184.
[20] Suárez-Warden, Fernando; Mendívil, Eduardo González; Rodríguez,
Ciro A.; Garcia-Lumbreras, Salvador (2015): Assembly Operations
Aided by Augmented Reality: An Endeavour toward a Comparative
Analysis. In: Procedia Computer Science, S. 281–290. DOI:
10.1016/j.procs.2015.12.249.
[21] Peniche, A; Treffetz, H; Diaz, C et al (2012): Combining virtual and
augmented reality to improve the mechanical assembly training process
in manufacturing. In: Proceedings of the 2012 American conference on
applied mathematics, S. 292–297.
[22] Rios, Horacio; Hincapié, Mauricio; Caponio, Andrea; Mercado, Emilio;
González Mendívil, Eduardo (2004): Augmented Reality: An
Advantageous Option for Complex Training and Maintenance
Operations in Aeronautic Related Processes. In: Hutchison, Kanade et
al. (Hg.) 2011 – Virtual and Mixed Reality, S. 87–96. DOI:
10.1007/978-3-642-22021-0_11.
[23] Westerfield, Giles; Mitrovic, Antonija; Billinghurst, Mark (2015):
Intelligent Augmented Reality Training for Motherboard Assembly. In:
International Journal of Artificial Intelligence in Education, S. 157–172.
DOI: 10.1007/s40593-014-0032-x.
[24] Vincenzi, Dennis A.; Valimont, Brian; Macchiarella, Nickolas;
Opalenik, Christopher; Gangadharan, Sathya N. (2003): The
Effectiveness of Cognitive Elaboration Using Augmented Reality as a
Training and Learning Paradigm. In: Proceedings of the Human Factors
and Ergonomics Society 47th Annual Meeting, S. 2054–2058.
[25] Haritos, T.; Macchiarella, N. D. (2005): A Mobile Application of
Augmented Reality for Aerospace Maintenance Training. In: 24th
Digital Avionics Systems Conference 30-03 Oct. 2005, S. 5.B.3-1-5.B.3-
9. DOI: 10.1109/DASC.2005.1563376.
[26] DIN EN ISO 9241-110:2008-09: Ergonomie der Mensch-System-
Interaktion-Teil 110: Grundsätze der Dialoggestaltung (ISO 9241-
110:2006).

International Scholarly and Scientific Research & Innovation 11(10) 2017 1135 scholar.waset.org/1307-6892/10007977