Abstract
Maintenance decision errors can result in very costly problems. The 4th industrial revolution has given new opportunities for the development of and use of intelligent decision support systems. With these technological advancements, key concerns focus on gaining a better understanding of the linkage between the technicians’ knowledge and the intelligent decision support systems. The research reported in this study has two primary objectives. (1) To propose a theoretical model that links technicians’ knowledge and intelligent decision support systems, and (2) to present a use case how to apply the theoretical model. The foundation of the new model builds upon two main streams of study in the decision support literature: “distribution” of knowledge among different agents, and “collaboration” of knowledge for reaching a shared goal. This study resulted in the identification of two main gaps: firstly, there must be a greater focus upon the technicians’ knowledge; secondly, technicians need assistance to maintain their focus on the big picture. We used the cognitive fit theory, and the theory of distributed situation awareness to propose the new theoretical model called “distributed collaborative awareness model.” The model considers both explicit and implicit knowledge and accommodates the dynamic challenges involved in operational level maintenance. As an application of this model, we identify and recommend some technological developments required in augmented reality based maintenance decision support.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Erroneous maintenance decisions and their fatal consequences have been an ongoing concern. For example, maintenance decision error is a common denominator and appears in one form or another in nearly all aviation accidents (Kraus 2009; Marais and Robichaud 2012). The problem is that maintenance technicians are required to perform routine and non-routine complex tasks with different types of equipment, processes, and personnel (Raouf et al. 2006) under tight schedules often with little or no feedback (Liang et al. 2010) so they have difficulty developing adequate mental models about the consequences of their work (Endsley and Robertson 2000). Despite the emergence of many new technologies, the information flow at the technician level is mostly limited to conversation, job task cards, e-mails or whiteboards (Lall et al. 2017), and they have to deal with poorly designed interfaces, and outdated and confusing manuals (Webel et al. 2013). These findings emphasize the importance of maintenance decision support (MDS).
Maintenance actions can be seen as a combination of information sources, technologies, and physical tools, requiring maintenance data to be physically transferred between objects and systems (Tretten and Normark 2014). The operational level maintenance decisions require “Situation Awareness” or the knowledge state about dynamics in the environment (Endsley 2017) involved in system health, workplace safety, supporting resources, and the corporate requirements (Endsley and Robertson 2000; Golightly et al. 2013; Oliveira et al. 2014). The decision-making process comprises four phases: intelligence, design, choice, and implementation (Simon 1997). Decision support systems (DSSs) allow relationships, associations, and connections to be found within information and enable better decisions; they combine multiple decision-makers, including humans and non-humans; they differ from information systems by providing an interactive environment between the system and the decision-maker (Boukhayma and ElManouar 2015).
Various types of intelligent decision support systems have emerged and variously referred to as active decision support, knowledge-based decision-support, and expert systems (Zhou et al. 2008). With the rise of the 4th industrial evolution (Industry 4.0) Artificial Intelligence (AI) based MDS increasingly learns from experience, make sense out of ambiguous information, deal with perplexing situations, recognize the relative importance of different elements and autonomously provide failure predictions through data acquisition from sensors (Phillips-Wren 2012). Out of the four basic AI-based DSS models: symbiotic, expert, adaptive, and holistic (Mirchandani and Pakath 1999), the latter is the most advanced, as it has holistic problem recognition and processing capabilities (Alenljung 2008). Progress has been made on digitally simulating holistic recognition (e.g., Mishra 2018); however, many challenges remain in developing truly holistic systems (Persson 2015). Various studies make a valuable contribution to the understanding of the technical strengths of the emerging MDS. However, current studies lack a specific focus on the linkage between intelligent decision support systems and the technicians’ knowledge. Therefore, this study is set to assess emerging MDS for their linkages with the technicians’ knowledge and to recommend a new MDS model to bridge the gaps.
2 Two main streams of decision support studies
A review conducted on DSS a decade ago (Durand et al. 2008) finds only one-third of the systems rely on a theoretical framework, and most are not explicit about how theory has guided the design and evaluation of the system. We have concluded through thorough research of the literature that two main streams have emerged of late in connection to DSS. Below are two research streams and their limitations.
2.1 Decision support based on “distribution”
The distributed model of cognition (Shaft and Vessey 2006) (see Fig. 1), which is an extension of the cognitive fit theory (Vessey 1991) has gained attention in DSS in variety of domains including online customer behavior (Kamis et al. 2008), military affairs (Godé and Lebrat 2013), business intelligence (Bacic 2014), financial risk assessment (Song and Wright 2017), and maintenance (Park 2007). This model uses a systems view of cognition and focuses on developing an ensemble of distributed individuals and artifacts (Salomon 1997). In this model, a cognitive task is viewed as a system of distributed representations, with internal and external representations two indispensable parts (Zhang and Norman 1994). Internal representations are the knowledge structures in the problem solver’s head, that is, those that can be retrieved from memory (e.g., a set of symbols to accomplish a particular task, the rules that govern the use of those symbols, the processes for acting on them, mental images, etc.). External representations are the knowledge and structures in the environment (e.g., physical symbols, objects, dimensions, constraints, relations embedded in physical configurations, etc.). Internal and external representations, the task, and the interactions among them contribute to the mental representation of a task solution. In the MDS context, the task of understanding system behavior is influenced by the technician’s existing knowledge of the system (internal representation of the problem domain), guidance from the decision support system (external representation of the problem domain), and the task to be completed (problem-solving task). When these three factors are known, a mental representation of the task solution is created. For example, the fast growing augmented reality (AR) based MDS can be considered as an external problem representation.
2.2 Decision support based on “collaboration”
Studies stress the opportunities for human and intelligent systems to collaborate, allowing humans and intelligent systems to learn from each other and work together to achieve shared goals (Terveen 1995). For effective collaboration, MDS should ensure intelligent activities are explicable to the human (Hall 2017), and be able to understand the human cognitive processes. The concept of explainable AI is gaining attention for its ability to fulfill the user’s requirement to understand what is often seen as a black-box (Gunning 2017), appropriately trust (Phillips-Wren 2012), and effectively manage these systems. A cognitive interactive training environment can learn and improve with a human operator acting as a mentor for the system, and the system provides feedback to the human (Crowder and Carbone 2014). This process of feedback and interactions increases the efficiency of both the system and the human mentor and allows the human operator to develop trust in the system over time. A technology called Pi-Mind suggests a compromise between total human control and total AI control over decisions, bringing a middle layer of them (Terziyan et al. 2018).
In the MDS domain, intelligent systems are now capable of anomaly detection; however, an anomaly detected may not necessarily be associated with a fault; it could be an anomaly with no-fault or a false alarm. An anomaly in one given context is considered acceptable, while in another context, even with identical data, is considered unacceptable (Galar et al. 2015). Therefore, to verify an anomaly detected by the intelligent system, a manual inspection usually follows, and feedback is given to the system, of its accuracy (Chandola et al. 2009). Intelligent systems raise queries based upon the technicians’ responses (Engelke et al. 2015), combined with physical models of degradation, and allow for adaptive levels of decision support (Erkoyuncu et al. 2017). The Collaborative Control Theory (CCT) offers a collection of principles for supporting effective human-AI collaboration (Nof 2007): collaboration requirement planning; e-work parallelism; keep it simple system; conflict/error detection and prevention; fault tolerance by teaming; association/dissociation; and dynamic lines of collaboration and best matching.
3 Limitations of current decision support
This section discusses two major limitations we identified in the existing MDS and associated models.
3.1 Limited focus on operational dynamics
Because distributed cognition focuses on information representation, transformation, and distribution across individuals and representational media, some see it as state-focused (Fomin and Vaujany 2008), whereas the alternative can also consider dynamics (Endsley 2015). The knowledge elements required by operational level maintenance are dynamic (Illankoon et al. 2019a), and the decision support model must capture and accommodate this. For example, performing a successful diagnosis of an intermittent fault state requires capturing its dynamic behavior; in other words, the technician should know what has been happening, what is happening, and what will happen next. Moreover, work routines are constantly changing due to varying business demands and resource availability. Various dynamics in the working environment can also influence the safety of maintenance work. Already available context-aware MDS consider context as “any information that can be used to characterize the situation of an entity; a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves” (Dey 2001). These systems are able to address the information that involves different scenarios based on inherent complexities, prevailing machine conditions, and decisions on what maintenance action is best. However, context awareness does not necessarily facilitate the technician to be aware of the dynamics involved in maintenance actions, and how those dynamics are considered in providing decision support. Therefore, MDS modeling should incorporate a concept that can accommodate not only the state but also the dynamics involved in maintenance actions.
3.2 Limited focus on human perceptual-cognitive tasks
Current MDS models do not enough consider human perceptual-cognitive tasks such as anticipation, pattern recall, and judgments. The mechanisms driving the judgments of operational level maintenance personnel may be associated with the knowledge that is complex, diverse, and local. Regardless of the generally accepted four phases of decision-making mentioned previously (Simon 1997) the degree of certainty of the problem representation and solution can result in structured, unstructured, or semi-structured decisions (Aronson et al. 2005). Unstructured decisions are highly dependent on the preferences or experiences of the decision-maker. A significant part of most maintenance processes is relatively straightforward to model; however, human cognitive involvement with more diverse and complex tasks is almost impossible to model (Knauff and Wolf 2010), and this affects the determination of how the MDS should intervene. Studies strive to minimize the discrepancies between the human’s mental model of what is to be accomplished and the intelligent system’s understanding of the task (Chakraborti et al. 2017). In addition to all the technological challenges that have to be overcome with MDS, we have the further complication of understanding technicians’ perceptual and cognitive drives while making maintenance decisions.
4 A new model: distributed collaborative awareness model (DCAM)
Our discovery of the above limitations motivated our development of a new model for MDS, a Distributed, Collaborative Awareness Model (DCAM), as shown in Fig. 2. DCAM also supports the opportunities identified in a previous study (Illankoon et al. 2019c) to model the cognition of abnormal machine behavior using a combined approach along two dimensions: broad types of knowledge (i.e., explicit–implicit) and broad levels of agents (i.e., human-intelligent systems). In the next sections, we discuss how the DCAM can overcome the two major limitations of the present models: limited focus on dynamics, and limited focus on perceptual-cognitive tasks. We further discuss a potential application of the new model using the fast growing AR based MDS.
4.1 From state to dynamic operations
To accommodate the dynamics (Illankoon et al. 2019a), situation awareness (SA) seems most suitable. SA is the knowledge state about dynamic elements in the environment, helps to make sense of barrage of data in dynamic situations to maintain users informed about what action is expected to be executed, who is collaborating, and their actions (Endsley 2017). Applications of SA in the maintenance context in different domains, such as aviation (Endsley and Robertson 2000; Truitt and Ahlstrom 2000; Bridges 2013), oil and gas pipelines (Nadj et al. 2016), energy distribution (Salmon et al. 2017), railway (Walker et al. 2006) and manufacturing (Oliveira et al. 2014) have already been studied. Distributed situation awareness (DSA) (Salmon et al. 2008) focuses on exchanges of situation awareness between agents and considers the activated knowledge for a specific task within a system (Stanton 2006). As viewed by some researchers (Salmon 2017), in a distributed system, some agents engage in perception tasks, while others engage in comprehension and projection. Therefore, the system collectively holds the SA required for task performance. With the rise of intelligent systems that deal with a huge amount of data, we acknowledge DSA as a more pragmatic approach for modeling the human–intelligent systems collaboration required in the MDS.
In the DCAM, we elaborate on the problem-solving task as comprising four major knowledge elements required in operational level maintenance: diagnosis to prognosis, corporate environment, team synchrony, and work safety. The model recognizes the dynamics of these knowledge elements, as the task environment now connects with the internal problem representation and the external problem representation. These connections appear in both subtle and overt means that we discuss further in the next section. At implementation, the DCAM can benefit from number of SA interventions (Endsley 2016), for example see (Illankoon et al. 2019b, d; Illankoon and Phillip 2019).
4.2 From explicit to implicit collaboration
Previous applications of DSA in maintenance, those trace explicit communication between agents (Salmon et al. 2017), have been criticized for not accounting for implicit communication between agents (Endsley 2015). Explicit communication requires answering questions and explaining situations that the intelligent system might encounter or codify. However, more needs to be recognized by the models (del Amo et al. 2018); the ability to include implicit knowledge that is not verbalized (Dane and Pratt 2007) and its influence on situation awareness. A previous study (Illankoon et al. 2018) lays the foundation but does not develop the concept.
MDS model should consider both the different levels of automation (Parasuraman et al. 2000) offered by a decision support system and the wide modes of human cognition involved in gaining situation awareness of machine behaviors (Illankoon et al. 2019c). Accordingly, in the DCAM, the internal problem representation is identified as a continuum between two poles: explicit and implicit knowledge. However, appropriate technological support must be provided during implementation. Below, we discuss two central technological challenges in implicit knowledge collaboration. We also identify a way to move forward.
4.2.1 MDS should not hinder implicit learning about machine behavior
Technicians who are physically involved with machines have the access to subtle cues (Endsley 2016, 2017; Endsley et al. 2000) those comprise information gained directly through human senses about the machine behavior (Illankoon et al. 2016). Positive effects of subtle cues for implicit learning are empirically evident in many domains, such as medical diagnosis (Conci et al. 2013), artificial grammar learning (Shanks 2005), social intuition (Norman and Price 2012), motor control (Sanchez et al. 2010), and maintenance (Gardner et al. 1996; Amadi-Echendu and Smidt 2015; Ruiz et al. 2014). For such implicit learning, humans require only a few cognitive resources (Evans and Stanovich 2013; Patterson 2017). However, there is a trade-off between the access to explicit cues provided by the MDS, and the access to subtle cues (Endsley, 1999, 2016). For example, limited peripheral visibility of AR headsets (Novak-Marcincin et al. 2013; Fiorentino et al. 2014; Dini and Dalle Mura 2015) hinders the awareness of the surroundings, and headphones prevent access to subtle changes in noise emission. Design efforts are being taken to make subtle cues more explicit (Endsley 2016), thus promoting more explicit awareness about machine behavior. However, in attempting to make subtle cues explicit, technicians can be overloaded with too much information.
The cues used by MDS will mostly deplete visual and audible resources and limit the chance to convey awareness through other human senses. Operational level maintenance in Industry 4.0 will go beyond facilitating operations to serve real-time operations more directly and closely. For example, airbus real-time health monitoring (AiRTHM) is an advanced airbus service through which operators receive advice on optimized maintenance actions and real-time troubleshooting actions (services.airbus.com 2018). With the increased use of real-time condition monitoring and remote operations, human technicians will have less physical involvement with the system (for monitoring and inspection) and will be less exposed to subtle cues about the system behavior.
We recommend that emerging MDS technologies should preserve access to subtle cues by alternative and novel means. Although the previously mentioned Pi-Mind technology (Terziyan et al. 2018) attempts to capture the human creative cognitive capabilities, it does not consider provisions for implicit learning. Only a few studies have focused on this possibility. For example, Fountx commercial solution comprises bone-conducting headphones to remain alert to the surroundings (fountx.com 2018). Another study (Domova et al. 2017) presents a prototype haptic mouse to explore the re-introduction of physical user interfaces into industrial control rooms. An extended view of this challenge is how to give more access to subtle cues with MDS. Such an MDS strategy should carefully select a few critical cues, make them more salient, and replicate or produce even more subtle cues (of non-critical information) to get the benefit of humans’ implicit learning capability. In fact, some studies have already started paying attention to developing such technologies, for example, Eagleman’s “Sensory Substitution” project (Eagleman 2015), which aims to design devices to send any kind of information to the brain for sensory processing via atypical sensory modalities. The notion of Human Cyber-Physical Systems (Romero et al. 2016) suggests keeping human-in-the-loop using variously enriched and enhanced technologies to improve physical, sensing, and cognitive capabilities.
Contemporary intelligent systems seek more inspiration and try to mathematically model how people make successful implicit inferences (Tao and He 2009; Dundas 2011; Mishra 2018); however, this requires enormous computational power (Guszcza et al. 2017). An alternative is to learn directly from technicians’ implicit knowledge and how they use that knowledge to be situationally aware during maintenance work. Previous studies recognize both explicit and implicit interactions for supporting multimodal user interfaces (Gallardo et al. 2018); however, their focus is not to elicit implicit knowledge from those interactions. Contemporary MDS focus heavily on supporting physical manipulations, only a few studies focus on a potential reverse approach; how to capture and learn from human implicit involvements. For example, one study (Botelho et al. 2014) proposes a maintenance strategy that combines and coordinates human physical abilities and computational elements. Another study (Vasenev et al. 2013) uses virtual reality (VR) to follow machine operators’ physical activities (in road construction) and extract their implicit knowledge. A major drawback of these studies is the method used, as VR is not able to capture real-life scenarios, leaving out a wealth of implicit involvements.
4.2.2 Managing conflicts during collaboration
Previous sections offer a rather optimistic character of human implicit knowledge, suggesting ways for it to collaborate with intelligent MDS. However, heuristic biases and erroneous human judgments related to implicit knowledge are common problems (Kahneman 2011; Reason and Hobbs 2017). Worse yet, humans are not able to identify which cognitive mechanism comes into play when they employ thinking with less deliberation (Kahneman and Klein 2009). We have also discussed the less well-understood algorithms used in the fast-growing machine learning field. This raises a critical question: how to handle potential conflicts between the implicit component of human cognition and the less well understood intelligent systems. While collaborative control theory (CCT) offers high-level strategies, specific techniques are required. One promising step towards collaboration would be to provide facilities in the MDS for controlled experimentation and simulation (Taylor 2001) to give insight into dynamic interactions and to offer feedback on decision alternatives. Intelligent features of the MDS may facilitate simulation of what is captured from the implicit knowledge of human agents, and, as discussed, explainable AI helps humans better understand the reasoning used by AI. Taken together, these strategies can provide a better understanding of which decision alternative is more effective and pragmatic. Once such an understanding is obtained, evidence justifying or challenging alternatives will add new knowledge to MDS for retrieving in the future.
In sum, our distributed collaborative awareness model supports the collaboration of human and intelligent agents along two important dimensions. First, it accounts for different types of situation awareness (explicit–implicit) held by both human and intelligent systems. The human explicit–implicit continuum stretches from verbalized to intuitive knowledge; the intelligent system continuum stretches from rule-based to holistic approaches. Second, it takes into account the awareness distributed among different agents between humans and intelligent systems. We illustrate this in Fig. 3.
5 How to use the model: the case of augmented reality (AR) in maintenance
This section discusses how to apply and further improve the DCAM. Although fast-growing, the successful use of AR in maintenance is still in progress (Wang et al. 2016; Martinetti et al. 2017; Quandt et al. 2017); therefore, AR-based MDS is a potential use case. In what follows, we review the fast-growing AR-based MDS, assess their strengths and weaknesses against the DCAM. We make recommendations to improve AR-based MDS so that the DCAM can be implemented.
5.1 Methods
We conducted a structured literature review using Google Scholar, because it was easy to sort influential studies (ranked by the number of citations). Google Scholar has coverage to Open Access articles, other citation resources (e.g., Scopus, Emerald, Web of Science), and those contained in institutional sources such as Universities. We searched the string “augmented reality AND maintenance” in the title of the articles published from 2008 to 2018. This gave 195 results. Then we excluded the non-English articles and those not cited by others. The remaining 109 articles were reviewed and six literature reviews were excluded. Using thematic analysis, the decision support features discussed in each of the articles were categorized into 10 categories (see next section). This resulted in a count of the literature that addresses different decision support features. We then critically assessed those features in comparison to our DCAM.
5.2 Strengths and limitations of the existing AR based MDS
The literature analysis resulted in the identification of 10 key MDS features. Figure 4 presents the number of articles describing those features. Below is a brief review of those features.
-
AR applications are largely limited to operational level maintenance actions, providing support for physical manipulations and/or providing step-by-step instructions (75 articles).
-
23 Articles discuss providing support for maintenance diagnosis. 17 articles mention the use of AI, and 18 articles recognize the ability to be context-aware.
-
18 Articles focus on providing remote expert support. Those applications are available as experimental prototypes (e.g., Stricker and Bleser 2012; Martinetti et al. 2017) and commercial products, such as EASE-R3 (Zenati-Henda et al. 2014) and XMReality (Karlsson 2018).
-
17 Articles discuss ways to capture existing knowledge from users, mainly to assess the knowledge levels and provide customized solutions.
-
Five articles support team synchronization through coordination, cooperation, and communication.
-
Only three articles discuss the need to comprehend the environment in which an operator works to avoid accidents (global awareness).
-
The manufacturing sector attracts the most attention (48 articles), only 8 articles solely target the aviation industry.
5.3 Application of the DCAM using AR based decision support
Current AR solutions are able to guide the technicians, by providing visual and audible instructions, and giving tactile feedback (see black arrows in Fig. 5). As per our review, the present usage of AR-based MDS is largely limited to unidirectional information flow from intelligent systems to the technician. To meet one of the recommendations of the DCAM, more work should be done to present none critical information using subtle cues, for example, projected on the peripheral, audible alerts and tactile feedback provided in varying frequencies.
Although a significant number of the articles (n = 17) discuss ways to capture existing knowledge from technicians, that is largely limited to verbalized (explicit) knowledge, and the purpose is to assess the knowledge levels to provide customized solutions, but not to learn from the technician. On the other hand, a successful acquisition of knowledge and training an AI-based MDS requires a huge amount of data, demanding the acquisition of knowledge well beyond that can be verbalised. In this context, we identify opportunities for AR-based MDS for learning from human implicit situation awareness (implicit knowledge state about dynamic elements) using multiple strategies. Sensors for collecting data from machines are increasingly smaller and less expensive. We identify the use of sensors beyond the collection of data from machines. It is possible to outfit technicians with sensors to elicit the psychomotor and cognitive responses that are beyond what technicians can verbalize.
For example, there is a possibility of capturing psychomotor behaviors, including implicit moves and gestures using GPS devices, accelerometers, and video recording. Gestures can be seen as a reflection of underlying cognitive processes (Novack and Goldin-Meadow 2015), because gestures often reveal information that cannot be found in speech. Although not meant for the same purpose, studies discuss how gestures can improve interactions with AR (e.g., Zhao et al. 2014). Capturing psychomotor behaviors is already used for providing motion and position feedback for technicians. For example, some studies (e.g., Webel et al. 2013) superimpose the video recorded movements of assembly operation using VR; the objective is to train novice technicians in motor skills. A study with Boeing company (Richardson et al. 2014) captured technicians’ motions (head position and orientation), communication frequency (interface button presses), and video recorded movements to assess training effectiveness. While these are useful for improving the motor skills of novice technicians, the potential of these technologies to elicit implicit psychomotor responses of expert technicians is what we are most interested in.
The second opportunity lies in capturing implicit attention behaviors using eye-tracking. Although not mentioned as such by the authors, we find similar approaches in other domains. For example, a study (Baber et al. 2009) uses wearable technology to enhance crime scene examination by integrating the tasks of searching for, retrieving, and recording evidence. Using eye-tracking and verbal protocol from real-time crime scene examination, they show how the approach to searching a scene differs with experience. A fuzzy approach (Ahmadi et al. 2017) suggests capturing experts’ knowledge through real-time verbal protocols, which can produce greater advantages when combined with other methods that capture implicit behaviors (Gonzalez 2003).
Another opportunity is linked to implicit cognitive responses captured using heart-rate, skin conductance, and respiration. Studies demonstrate that the dynamic of heart-rate is rich enough to reveal relevant episodes of inner thoughts, such as the correctness of a choice (Mealla et al. 2011). Another study found initial evidence that skin conductance and respiration are sensitive to implicit memory (klein Selle et al. 2018). Adding such tracking functions to MDS can provide the opportunity to learn from humans in real-time. These sensory data can be combined with performance measures to let intelligent systems elicit useful implicit knowledge, as well as to learn about and warn against potentially improper implicit behavior. In sum, recognition of explicit to implicit knowledge can aid forming a collaborated awareness, which is distributed among human and intelligent systems as we described in the DCAM.
6 Conclusions and future work
We presented a new approach to modeling collaboration of situation awareness among technicians and intelligent systems. Almost all research concerning decision support has considered and applied collaboration and distribution as two important intertwined concepts, we identify that they can also be applied as two complementary concepts. Modeling decision support relates to but improved from the cognitive fit theory, which does not support collaboration in a dynamic context. In the application of the maintenance domain, we see that a concept that can address the dynamics inherent to it is necessary, thus the situation awareness concept is considered most relevant. Collaboration and distribution are presently limited to the transactions that can only be explicitly identified, such as verbalized knowledge, and not implicit knowledge. Therefore, we emphasize the importance of implicit transactions among the agents: human, decision support, and machines.
Since the technology of AR uses both distribution and collaboration concepts, we studied the relevant literature. AR solutions distribute knowledge between the user and what is built in the AR system. AR solutions also assist the user and help in collaborating with others. However, the development of the DCAM led us to identify two main gaps: collaboration has not been fully identified since AR does not learn from the user and AR does not assist the user to develop implicit knowledge, it may actually hinder it. We made recommendations on how new technologies, i.e. eye-tracking, can be added to AR to help overcome these gaps. The main implication of the development of AR beyond its role as a pedagogical tool is that the bidirectional information flow will elevate AR as a knowledge elicitation technology from the experts.
We presented the DCAM as a descriptive model and for future work, we recommend that it should be applied using formal notations, such as ontologies. Although we mention a few former studies supporting our arguments, a comprehensive study is recommended to validate all propositions of the DCAM.
References
Ahmadi M, Seneviratne D, Garmabaki A (2017) An approach to symbolic modelling: a railway case study for maintenance recovery level identification. In: Maintenance performance and measurement and management 2016 (MPMM 2016). November 28, Luleå, Sweden 2017 Luleå tekniska universitet. p. 187
Alenljung B (2008) Envisioning a future decision support system for requirements engineering: a holistic and human-centred perspective. Dissertation, Institutionen för datavetenskap
Amadi-Echendu J, de Smidt M (2015) Integrating tacit knowledge for condition assessment of continuous mining machines. 9th WCEAM Research Papers. Springer, Cham, pp 119–130
Aronson JE, Liang TP, MacCarthy RV (2005) Decision support systems and intelligent systems. Pearson Prentice-Hall, USA
Baber C, Smith P, Butler M, Cross J, Hunter J (2009) Mobile technology for crime scene examination. Int J Hum Comput Stud 67(5):464–474
Bacic D (2014) The role of cognitive effort in decision performance using data representations: a cognitive fit perspective. ETD Archive. Paper 19
Botelho SS, Rettberg A, Duarte Filho N, Hellingrath B, Amaral M, Espíndola D, Pereira CE, Cordes AK, Ventura R, Frazzon E (2014) Towards intelligent maintenance systems: rescuing human operator and context factors. IFAC Proc Vol 47(3):7110–7115
Boukhayma K, ElManouar A (2015) Evaluating decision support systems. In: Proceedings of the 5th International Conference on Intelligent Systems Design and Applications (ISDA), IEEE, Marrakech, Morocco
Bridges III DC (2013) A situation awareness design approach to the position of airline maintenance control in a simulated operations control center. In: 17th International Symposium on Aviation Psychology. p. 92
Chakraborti T, Kambhampati S, Scheutz M, Zhang Y (2017) AI challenges in human-robot cognitive teaming. arXiv preprint arXiv:1707.04775
Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):1–58
Conci M, Mueller HJ, von Mühlenen A (2013) Object-based implicit learning in visual search: perceptual segmentation constrains contextual cueing. J Vis 13(3):15
Crowder JA, Carbone JN (2014) Collaborative shared awareness: human-AI collaboration. In: Proceedings of the International Conference on Information and Knowledge Engineering (IKE). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp): 1
Dane E, Pratt MG (2007) Exploring intuition and its role in managerial decision making. Acad Manag Rev 32(1):33–54
del Amo IF, Erkoyuncu JA, Roy R, Palmarini R, Onoufriou D (2018) A systematic review of augmented reality content-related techniques for knowledge transfer in maintenance applications. Comput Ind 103:47–71
Dey AK (2001) Understanding and using context. Pers Ubiquit Comput 5(1):4–7
Dini G, Dalle Mura M (2015) Application of augmented reality techniques in through-life engineering services. Procedia CIRP 38:14–23
Domova V, Ralph M, Vartiainen E, Muñoz AA, Henriksson A, Timsjö S (2017) Re-introducing physical user interfaces into industrial control rooms. In: Proceedings of the European Conference on Cognitive Ergonomics. pp. 162–168.
Dundas J, Chik D (2011) Implementing human-like intuition mechanism in artificial intelligence. arXiv preprint arXiv:1106.5917
Durand MA, Stiel M, Boivin J, Elwyn G (2008) Where is the theory? Evaluating the theoretical frameworks described in decision support technologies. Patient Educ Couns 71(1):125–135
Eagleman DM (2015) Sensory substitution. https://www.eagleman.com/research/sensorysubstitution. Accessed 21 Jun 2019
Endsley MR (1999) Situation awareness in aviation systems. Handbook of aviation human factors. Lawrence Erlbaum Associates, USA, pp 257–276
Endsley MR, Garland DJ (eds) (2000) Situation awareness analysis and measurement. CRC Press, USA
Endsley MR (2015) Situation awareness misconceptions and misunderstandings. J Cogn Eng Decision Making 9(1):4–32
Endsley MR (2016) Designing for situation awareness: an approach to user-centered design. CRC Press, USA
Endsley MR (2017) Toward a theory of situation awareness in dynamic systems. Situational awareness. Routledge, UK, pp 9–42
Endsley MR, Robertson MM (2000) Situation awareness in aircraft maintenance teams. Int J Ind Ergon 26(2):301–325
Engelke T, Keil J, Rojtberg P, Wientapper F, Schmitt M, Bockholt U (2015) Content first: a concept for industrial augmented reality maintenance applications using mobile devices. In: Proceedings of the 6th ACM Multimedia Systems Conference. pp. 105–111
Erkoyuncu JA, del Amo IF, Dalle Mura M, Roy R, Dini G (2017) Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality. CIRP Ann 66(1):465–468
Evans JS, Stanovich KE (2013) Dual-process theories of higher cognition: advancing the debate. Perspect Psycholog Sci 8(3):223–241
Fiorentino M, Uva AE, Gattullo M, Debernardis S, Monno G (2014) Augmented reality on large screen for interactive maintenance instructions. Comput Ind 65(2):270–278
Fomin VV, Vaujany FX (2008) Theories of ICT design: where social studies of technology meet the distributed cognitive perspective. In: ICIS 2008 Proceedings. pp. 177
fountx.com (2018) Boosting the productivity and safety of your team. https://fountx.com/products/. Accessed 20 Oct 2018
Galar D, Thaduri A, Catelani M, Ciani L (2015) Context awareness for maintenance decision making: a diagnosis and prognosis approach. Measurement 67:137–150
Gallardo J, Bravo C, Molina AI (2018) A framework for the descriptive specification of awareness support in multimodal user interfaces for collaborative activities. J Multimodal User Interfaces 12(2):145–159
Gardner PH, Chmiel N, Wall TD (1996) Implicit knowledge and fault diagnosis in the control of advanced manufacturing technology. Behaviour Inform Technol 15(4):205–212
Godé C, Lebraty JF (2013) Improving decision making in extreme situations: the case of a military decision support system. Intern J Technol Human Interact (IJTHI) 9(1):1–7
Golightly D, Ryan B, Dadashi N, Pickup L, Wilson JR (2013) Use of scenarios and function analyses to understand the impact of situation awareness on safe and effective work on rail tracks. Saf Sci 56:52–62
Gonzalez C (2003) Verbal protocols in real-time dynamic decision-making. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting 47(3), pp. 293–296. Sage CA, Los Angeles, CA, SAGE Publications
Gunning G (2017) Explainable artificial intelligence (xai), Defense Advanced Research Projects Agency (DARPA), Available https://www.darpa.mil/attachments/XAIProgramUpdate.pdf. Accessed 01 Jun 2019
Guszcza J, Lewis H, Evans-Greenwood P (2017) Cognitive collaboration Why humans and computers think better together, Deloitte Review 20. Deloitte University Press. https://www2.deloitte.com/insights/us/en/deloitte-review/issue-20/augmented-intelligence-human-computer-collaboration.html. Accessed 20 Oct 2018
Hall L (2017) Explicable planning and replanning for human-in-the-loop decision support. https://www.nasa.gov/directorates/spacetech/esi/esi2016/Human-in-the-loop_Decision_Support. Accessed 18 Oct 2018
Illankoon P, Tretten P (2019) Judgemental errors in aviation maintenance. Cogn Technol Work 1–8
Illankoon P, Abeysekera J, Singh S (2016) Ergonomics for enhancing detection of machine abnormalities. Work 55(2):271–280
Illankoon P, Tretten P, Kumar U (2018) Identifying significance of human cognition in future maintenance operations. International conference on intelligent human systems integration. Springer, Cham, pp 550–556
Illankoon P, Manathunge Y, Tretten P, Abeysekara J, Singh S (2019a) Lockout and tagout in a manufacturing setting from a situation awareness perspective. Safety 5(2):25
Illankoon P, Tretten P, Kumar U (2019b) A prospective study of maintenance deviations using HFACS-ME. Int J Ind Ergon 74:102852
Illankoon P, Tretten P, Kumar U (2019c) Modelling human cognition of abnormal machine behaviour. Human Intell Syst Integr 1(1):3–26
Illankoon P, Tretten P, Singh S (2019d) Decision support system for flight maintenance technicians: issues and challenges. eMaintenance 88.
Kahneman D (2011) Thinking, fast and slow. Farrar, Straus and Giroux, NY
Kahneman D, Klein G (2009) Conditions for intuitive expertise: a failure to disagree. Am Psychol 80:237–251
Kamis A, Koufaris M, Stern T (2008) Using an attribute-based decision support system for user-customized products online: an experimental investigation. MIs Q 32:159–177
Karlsson J (2018) Smarta glasögon smörjer jättegruvan. https://www.di.se/nyheter/smarta-glasogon-smorjer-jattegruvan/. Accessed 21 Feb 2018
Klein Selle N, Ben-Shakhar G, Kindt M, Verschuere B (2018) Preliminary evidence for physiological markers of implicit memory. Biolog Psychol 135:220–235
Knauff M (2010) Wolf AG (2010) Complex cognition: the science of human reasoning, problem-solving, and decision-making. Cogn Process 11:99–102. https://doi.org/10.1007/s10339-010-0362-z
Kraus TL (2009) A quest for improved safety and security. a historical perspective 1903–2008. FAA, USA, pp 83–94
Lall M, Torvatn H, Seim EA (2017) Towards industry 4.0: increased need for situational awareness on the shop floor. IFIP international conference on advances in production management systems. Springer, Cham, pp 322–329
Liang GF, Lin JT, Hwang SL, Wang EM, Patterson P (2010) Preventing human errors in aviation maintenance using an on-line maintenance assistance platform. Int J Ind Ergon 40(3):356–367
Marais KB, Robichaud MR (2012) Analysis of trends in aviation maintenance risk: an empirical approach. Reliab Eng Syst Saf 106:104–118
Martinetti A, Rajabalinejad M, van Dongen L (2017) Shaping the future maintenance operations: reflections on the adoptions of augmented reality through problems and opportunities. Procedia CIRP 59(1):14–17
Mealla S, Väaljamäae A, Bosi M, Jordà S (2011) Listening to your brain: implicit interaction in collaborative music performances. NIME 11:149–154
Mirchandani D, Pakath R (1999) Four models for a decision support system. Inform Manage. https://doi.org/10.1016/S0378-7206(98)00074-3
Mishra AK (2018) ICABiDAS: intuition centred architecture for big data analysis and synthesis. Procedia Comput Sci 123:290–294
Nadj M, Jegadeesan H, Maedche A, Hoffmann D, Erdmann P (2016) A situation awareness driven design for predictive maintenance systems: the case of oil and gas pipeline operations. Prototypes 6, Available https://aisel.aisnet.org/ecis2016_prototypes/6. Accessed 18 Oct 2018
Nof SY (2007) Collaborative control theory for e-work, e-production, and e-service. Annu Rev Control 31(2):281–292
Norman E, Price MC (2012) Social intuition as a form of implicit learning: sequences of body movements are learned less explicitly than letter sequences. Adv Cogn Psychol 8(2):121
Novack M, Goldin-Meadow S (2015) Learning from gesture: how our hands change our minds. Educ Psychol Rev 27(3):405–412
Novak-Marcincin J, Barna J, Janak M, Novakova-Marcincinova L (2013) Augmented reality aided manufacturing. Proced Comput Sci 25:23–31
Oliveira A, Araujo R, Jardine A (2014) Human-centered interfaces for situation awareness in maintenance. In: International Conference on Human Interface and the Management of Information Springer, Cham, pp. 193–204
Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybernet Part A Syst Humans 30(3):286–297
Park YK (2007) Human cognitive task distribution model for maintenance support system of a nuclear power plant. PhD thesis, Korea Advanced Institute of Science and Technology, Daejeon (KR), https://inis.iaea.org/search/search.aspx?orig_q=RN:47114725. Accessed 10 Oct 2018
Patterson RE (2017) Intuitive cognition and models of human–automation interaction. Hum Factors 59(1):101–115
Persson M (2015) Challenges of implementing decision support systems: joining the views of consultant and customer. Master Thesis, School of Economics and Management Department of Informatics, LUND University, Sweden
Phillips-Wren G (2012) AI tools in decision making support systems: a review. Int J Artif Intell Tools 2:1240005
Quandt M, Ait Alla A, Meyer L, Freitag M (2017) Success factors for the development of augmented reality-based assistance systems for maintenance services. Schmitt, RH, Schuh G (Hrsg) 7:175–182
Raouf A, Duffuaa S, Ben‐Daya M, Dhillon, BS, Liu Y (2006) Human error in maintenance: a review. J Qual Maint Eng
Reason J, Hobbs A (2017) Managing maintenance error: a practical guide. CRC Press, USA
Richardson T, Gilbert SB, Holub J, Thompson F, MacAllister A, Radkowski R, Winer E, Davies P, Terry S (2014) Fusing self-reported and sensor data from mixed-reality training. In: Proc. Industrial and Manufacturing Systems Engineering. pp. 92
Romero D, Bernus P, Noran O, Stahre J, Fast-Berglund Å (2016) The operator 4.0: human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems. In: IFIP international conference on advances in production management systems. Springer, Cham. pp. 677–686
Ruiz PP, Foguem BK, Grabot B (2014) Generating knowledge in maintenance from experience feedback. Knowl-Based Syst 68:4–20
Salmon PM, Stanton NA, Walker GH, Baber C, Jenkins DP, McMaster R, Young MS (2008) What really is going on? Review of situation awareness models for individuals and teams. Theor Issues Ergon Sci 9(4):297–323
Salmon PM, Stanton NA, Jenkins DP (2017) Distributed situation awareness: theory, measurement and application to teamwork. CRC Press, USA
Salomon G (1997) Distributed cognitions: psychological and educational considerations. Cambridge University Press, Cambridge
Sanchez DJ, Gobel EW, Reber PJ (2010) Performing the unexplainable: implicit task performance reveals individually reliable sequence learning without explicit knowledge. Psychon Bull Rev 17(6):790–796
services.airbus.com (2018) Airbus Real Time Health Monitoring. https://services.airbus.com/maintenance/e-solutions/airbus-real-time-health-monitoring/airbus-real-time-health-monitoring. Accessed 18 Oct 2018
Shaft TM, Vessey I (2006) The role of cognitive fit in the relationship between software comprehension and modification. Mis Q 29–55
Shanks DR (2005) Implicit learning. Handbook of cognition. SAGE Publication Ltd, USA, pp 203–221
Simon H (1997) The new science of management decisions. Prentice-Hall, USA
Song Q, Chan SH, Wright AM (2017) The efficacy of a decision support system in enhancing risk assessment performance. Decis Sci 48(2):307–335
Stanton NA, Stewart R, Harris D, Houghton RJ, Baber C, McMaster R, Salmon P, Hoyle G, Walker G, Young MS, Linsell M (2006) Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology. Ergonomics 49(12–13):1288–1311
Stricker D, Bleser G (2012) From interactive to adaptive augmented reality. In: 2012 International Symposium on Ubiquitous Virtual Reality 2012, IEEE, pp. 18–21
Tao W, He P (2009) Intuitive learning and artificial intuition networks. In: 2009 Second International Conference on Education Technology and Training, IEEE, pp. 297–300
Taylor SJ (2001) Netmeeting: a tool for collaborative simulation modeling. Int J Simul Syst Sci Technol 1(1–2):59–68
Terveen LG (1995) Overview of human–computer collaboration. Knowl-Based Syst 8(2–3):67–81
Terziyan V, Gryshko S, Golovianko M (2018) Patented intelligence: cloning human decision models for Industry 4.0. J Manuf Syst 48:204–217
Tretten P, Normark CJ (2014) Human factors issues in aircraft maintenance activities: a holistic approach. In: Human Factors and Ergonomics Society Annual Meeting
Truitt TR, Ahlstrom V (2000) Situation awareness in airway facilities: replacement of maintenance control centers with operations control centers. William J Hughes Technical Center, USA
Vasenev A, Hartmann T, Doree AG (2013) Employing a virtual reality tool to explicate tacit knowledge of machine operators. In: ISARC Proceedings of the International Symposium on Automation and Robotics in Construction 30(1). IAARC Publications
Vessey I (1991) Cognitive fit: a theory-based analysis of the graphs versus tables literature. Decis Sci 22(2):219–240
Walker GH, Gibson H, Stanton NA, Baber C, Salmon P, Green D (2006) Event analysis of systemic teamwork (EAST): a novel integration of ergonomics methods to analyse C4i activity. Ergonomics 49(12–13):1345–1369
Wang Y, Anne A, Ropp T (2016) Applying the technology acceptance model to understand aviation students’ perceptions toward augmented reality maintenance training instruction. Intern J Aviat Aeronaut Aerosp 3(4):3
Webel S, Bockholt U, Engelke T, Gavish N, Olbrich M, Preusche C (2013) An augmented reality training platform for assembly and maintenance skills. Robot Auton Syst 61(4):398–403
Zenati-Henda N, Bellarbi A, Benbelkacem S, Belhocine M (2014) Augmented reality system based on hand gestures for remote maintenance. In: 2014 International Conference on Multimedia Computing and Systems (ICMCS), IEEE, pp. 5–8
Zhang J, Norman DA (1994) Representations in distributed cognitive tasks. Cogn Sci 18(1):87–122
Zhao S, Zhang Y, Zhou B, Ma D (2014) Research on gesture recognition of augmented reality maintenance guiding system based on improved SVM. In: 7th International Symposium on Advanced Optical Manufacturing and Testing Technologies: Optical Test and Measurement Technology and Equipment 9282: 92822L
Zhou F, Yang B, Li L, Chen Z (2008) Overview of the new types of intelligent decision support system. In: 2008 3rd International Conference on Innovative Computing Information and Control, IEEE, pp. 267–267
Funding
Open access funding provided by Lulea University of Technology. Luleå Railway Research Center (JVTC).
Author information
Authors and Affiliations
Contributions
Prasanna Illankoon: conceptualization, methodology, validation, formal analysis, investigation, writing—original draft. Phillip Tretten: conceptualization, validation, resources, writing—review and editing, supervision, funding acquisition.
Corresponding author
Ethics declarations
Conflict of interest
We wish to confirm that there are no known conflicts of interest associated with this manuscript and there has been no significant financial support for this work that could have influenced its outcome. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property.
Availability of data and material
The authors would be happy to supply the Supplementary Material on request.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Illankoon, P., Tretten, P. Collaborating AI and human experts in the maintenance domain. AI & Soc 36, 817–828 (2021). https://doi.org/10.1007/s00146-020-01076-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-020-01076-x