Managing A Team in The Operating Room: The Science of Teamwork and Non-Technical Skills For Surgeons

Current Problems in Surgery 59 (2022) 101172
Contents lists available at ScienceDirect
Current Problems in Surgery
journal homepage: www.elsevier.com/locate/cpsurg
Managing a team in the operating room: The

science of teamwork and non-technical skills
for surgeons
Robert D. Sinyard, MD,MBA a, Courtney M. Rentas, MPH b,
Eilidh GM Gunn, MBChB,MRCSEd c, James C. Etheridge, MD d,
Jamie M. Robertson, PhD,MPH d, Augustus Gleason, MD e,
Max S. Riley, MD d, Steven Yule, PhD f,
Douglas S. Smink, MD,MPH d,∗
Introduction
At 3:15 AM on an otherwise unremarkable Thursday morning, a routine appendectomy was

about to begin. Well, at least everyone thought it would be routine. Nothing about the 22-
year-old woman’s history was remarkable and her story as well as examination findings, labora-
tory results, and imaging studies all pointed directly at a diagnosis of acute appendicitis. Half-
heartedly and half-asleep, the various team members mechanically acknowledged their respec-
tive pieces of the preoperative checklist as the circulating nurse methodically moved through
the numerous items. Eventually, an incision was made and, shortly thereafter, insufflation of the
abdomen was initiated. Within 30 seconds, the otherwise healthy patient had experienced a
bradycardic arrest and was undergoing chest compressions as the team frantically tried to re-
spond. The situation could only be described as chaos.
“Why did no one tell me the patient was unstable while we were insufflating?!” the surgeon
shouted between compressions. “Is this a drug reaction? What antibiotics were given? Was the
patient bradycardic and atropine wasn’t given?!”
From the a Massachusetts General Hospital, Department of Surgery, Boston, MA; b University of Alabama at Birming-
ham Medical School, Birmingham, AL; c Department of Vascular Surgery, Royal Infirmary of Edinburgh, Edinburgh, Scot-
land; d Brigham & Women’s Hospital, Department of Surgery, Boston, MA; e Tufts Medical Center, Department of Surgery,
Boston, MA; and f The University of Edinburgh, College of Medicine and Veterinary Sciences, Edinburgh, Scotland
∗
Address reprint requests to Douglas S. Smink, MD, MPH, Department of Surgery, Brigham & Women’s Faulkner Hos-
pital, 1153 Centre Street, Suite 5D, Boston, MA 02130.
E-mail address: [email protected] (D.S. Smink).
https://doi.org/10.1016/j.cpsurg.2022.101172
0011-3840/© 2022 Elsevier Inc. All rights reserved.
Downloaded for Anonymous User (n/a) at Royal Australasian College of Surgeons from ClinicalKey.com.au by Elsevier on June
11, 2023. For personal use only. No other uses without permission. Copyright ©2023. Elsevier Inc. All rights reserved.
2 R.D. Sinyard, C.M. Rentas and E.G. Gunn et al. / Current Problems in Surgery 59 (2022) 101172
“Where is the code cart?” came an exasperated response from the nurse anesthetist at the
head of the bed.
“Where is the attending anesthesiologist? Is she still placing that epidural in the labor and
delivery unit?” cried the circulating nurse while scrambling to the find the defibrillator.
Situations like this one highlight the crucial need for teamwork in the operating room (OR).
Although common in other industries, the concepts of team training and non-technical skill
(NTS) development are relatively new to the perioperative environment. Where such techniques
have been applied, results have often been striking. Perhaps they would have been in this case
as well. Perhaps in an OR characterized by greater psychological safety, the overnight medical
student would have felt comfortable speaking up when he noticed that the patient’s heart rate
had suddenly dropped from 70 to 35 beats per minute. Perhaps greater attentiveness during
the pre-briefing would have created clarity about the antibiotics given prior to incision. Per-
haps learning names and role would have allowed someone to take a leadership role amidst the
chaos.
Even beyond these crises, however, the need for team training and NTS development has a
tangible impact on the quality of health care delivery in surgical contexts. In this monograph,
we trace the historical underpinnings of team training and NTS to the OR and provide a review
of the evidence regarding their effectiveness. We also provide interested readers with resources
for the development of individual and team NTS.
Origins of the science of teamwork
In the 1920s, Western Electric’s Hawthorne Plant, located just outside of Chicago, was a Bell
Telephone hardware manufacturing site. Intrigued by how environmental and interpersonal fea-
tures of the workplace might affect employee productivity, psychologists Elton Mayo and Fritz
Roethlisberger set up a series of extraordinarily simple but now famous experiments spanning
over a decade. Unbeknownst to employees, Mayo and Roethlisberger manipulated small vari-
ables in the workplace environment that could affect performance such as plant lighting. Re-
searchers were interested to see how employees would respond to such annoyances and what
impact this might have on their productivity. Much to the surprise of Mayo and Roethlisberger,
worker productivity actually increased over the study period. Follow-up focus groups and em-
ployee interviews demonstrated that the stressors of the factory changes had actually resulted
in improved communication and collaboration among employees. Employees had constructed a
process to communicate the issues with management and, when management responded favor-
ably to rectify the changes, teams felt empowered, motivated, and appreciated. Workers also re-
called that their shared experiences contributed to a sense of community within the workplace
for the first time. From these insights, Mayo and Roethlisberger concluded that the most criti-
cal determinant for performance improvement in the workplace was not the characteristics of
the work environment itself but rather the quality of relationships between workers, peers, and
management (leader, supervisor, or manager).1 , 2 So began the scientific inquiry into the nature
of work group dynamics and its impact on both individual and collective outcomes.3
World War II greatly accelerated the scientific interest in team dynamics as it was recognized
that team performance was an essential component of military effectiveness.4 In the decade
that followed, more than 20 0 0 studies of teamwork were conducted, laying the foundation for
modern teamwork research.5 While interest declined following the war years, applied industrial
organizational research interest led to a resurgence in the 1980s. In the aviation industry, stud-
ies on aircrews led to the development of crew resource management (CRM) training programs
in an attempt to foster the proven techniques of interpersonal communication, leadership, and
decision making skills required for high pressure environments where human error can be catas-
trophic.6 Insights gleaned from cockpit simulation proved to be so impactful that by the 1990s,
not only had CRM become a required training competency for all commercial flight crews in the
United States, it was also being modified and adapted in other high-stakes professions, such as
the military, as well as health care, and particularly surgery.7
R.D. Sinyard, C.M. Rentas and E.G. Gunn et al. / Current Problems in Surgery 59 (2022) 101172 3
Fig. 1. Taxonomy of team development interventions. Adapted from Lacerenza et al.8 .
Team development interventions
Most industries have seen an enormous growth in the nature of work that necessitates a
team of individuals. With this exponential growth of teams, so has grown the research focused
on interventions designed to enhance teamwork effectiveness.8 These are referred to as team
development interventions (TDIs), “systematic activities aimed at improving requisite team com-
petencies, processes, and overall effectiveness.”9 When considering the breadth of TDIs, a taxon-
omy and shared language is helpful (Fig 1).
Two major classes of interventions have been shown to improve both team performance and
outcomes across multiple industries: training interventions and process interventions. Training
interventions can be further subset by the group or individual to which they are applied, namely
a team leader or a host of individuals with a shared purpose (ie, a team). The resulting outputs
form the commonly referenced vernacular terms of “leadership training” and “team training,”
respectively. The other important aspect of TDIs are process interventions, which can be subset
based upon whether the intervention is applied to interpersonal processes or team processes
as a whole.8 These interventions produce the more commonly referenced “team building” and
“team debriefing” exercises, respectively. This nomenclature provides a helpful framework for
considering the vast array of interventions available for improving teamwork, particularly in a
complex field such as surgery. What follows is a brief overview of the most effective TDIs within
each of the 4 subcategories. We shall then offer an overview of how these have been applied to
health care and, more specifically, surgical teams and the OR environment.
Training interventions
Training interventions are planned activities designed to teach learners what they will need
to know, do, and feel to be successful in their role.9-11 Although the obvious goal of any training
intervention is for behaviors acquired through training to be implemented on-the-job after the
intervention ends,12 the extent to which this learning transfer occurs is influenced by trainee
characteristics (eg, motivation), characteristics of the work environment (eg, organizational sup-
port), and training design features (eg, content).8 , 13 , 14
Typically, training interventions start with a needs assessment to identify gaps between ex-
isting and needed knowledge, skills, or actions.15 A thorough needs assessment specifies the
critical work functions of a job and outlines associated task requirements.11 Based on identified
needs, a training curriculum is created both with the constraints of the team in mind as well
as an evaluative methodology. Effective training interventions include 4 core elements: informa-
tion, demonstration, practice, and feedback. Training interventions that convey the information,
demonstrate desired behaviors and attitudes, and create an opportunity for trainees to practice
via simulation or role-playing have been shown to result in improved team-performance.11 The
provision of feedback after training must be timely, constructive, and actionable to help trainees
reflect on their learning and guide remedial action.11 , 16 Lastly, studies have found that train-
ing interventions intentionally designed to make trainees commit errors actually lead to better
task performance as error-training “encourages a greater effort to learn, promotes a deeper un-
derstanding of tasks, and provides strategies and emotional-management tactics for handling
subsequent errors.”17 The 2 classes of training interventions are (1) leadership training and (2)
team training.
Leadership training (methods to improve leadership capabilities) refers to systematic interven-
tions designed to enhance leader knowledge, skills, and abilities.18 Although some individuals
may be more inclined towards behaviors that are traditionally ascribed to leadership roles, a
recent meta-analysis by Lacerenza and colleagues on leadership training programs noted that
participants made notable gains in cognitive, affective, and skills-based learning domains as well
as transfer these learnings into actual behaviors in the workplace.19 The change in intellectual
skills and knowledge acquisition that stem from cognitive learning result in enhanced problem
recognition, planning, creative thinking, and idea evaluation among leaders.20 Increased affective
learning, which focuses on understanding the attitudes and motivation of others, enhances the
leader’s ability to better connect with and influence others to reach shared goals.19 Advances in
skills-based learning result in further development of the technical skills that guide decision
making and problem resolution by leaders.19 Most importantly, leadership training has been
shown to improve performance at the level of the team–not just for the individual partaking
in training.21
Although interventions that teach trainees skills like problem-solving and data analysis are
easiest for members to learn and apply on-the-job, interpersonal skills have actually been
demonstrated to improve organizational and team-performance outcomes most significantly. As
such, leadership training interventions which help trainees develop skills like active listening
and adaptability to changing circumstances lead to the most substantial results.19 Additionally,
leadership training interventions delivered in multiple short sessions across time are more likely
to result in behavioral changes on the job than are extended retreats or workshops in which all
training content is aggregated and delivered together.19 Finally, leadership training that teaches
learners how to organize and execute tasks, clearly define roles for members of their team, and
establish well-defined patterns of communication tend to be most effective at improving team
performance.22
Team training (methods to improve the competencies of individual team members) is the sec-
ond type of TDI focused on individual development. The purpose of team training is to equip
an individual member of a team with the knowledge, skills, or actions required for that individ-
ual to effectively partake in the team’s performance.9 Team training aims to improve individual
competencies and then provide opportunities for practice in a simulated environment.23 Two
particularly important outcomes of effective team training are an individual’s perceived clarity
of understanding regarding role and associated responsibilities as well as an individual’s comfort
level in speaking up to identify and/or prevent team breakdowns.9
Several types of team training interventions have demonstrated effectiveness in improving
team outcomes. First, cross-training typically leads to increased awareness of the duties and re-
sponsibilities of other team members and builds a shared knowledge of the essential tasks and
responsibilities for the team as a whole.9 Secondly, team training interventions that enable in-
dividuals to diagnose communication and coordination breakdowns and then develop internal
solutions have demonstrated considerable improvements in long term team performance met-
rics.9 Experts suspect that much of this can be attributed to the establishment of a psycho-
logically safe environment. Psychological safety, or “the shared belief that the team is safe for
interpersonal risk taking,” facilitates the willing contribution of ideas to a shared team goal.24
When team members possess a “sense of confidence that the team will not embarrass, reject,
or punish someone for speaking up,” they are more inclined to openly disclose errors. A critical
component of successful team training, psychological safety thereby affords team members an
opportunity to review previous errors in an effort to prevent future recurrence.25 , 26
Process interventions
Unlike team training interventions which aim to improve individual team member competen-
cies needed to accomplish future tasks, process interventions teach intact teams how to collec-
tively function at a higher level.27 Process interventions help team members reflect on their past
interactions in order to devise more effective ways of working together in the future.27 Effective
process interventions typically result in the identification of underlying problems contributing
to weak performance.8
Process interventions are most effective when they uncover breakdowns in communication,
role clarification, team problem solving, and decision making.27 Most process interventions uti-
lize a facilitator, at least initially, to lead team members in exercises as well as discussions re-
garding the factors limiting their performance. Ultimately, teams are expected to be able to work
together to self-identify and correct weaknesses in order to improve subsequent team perfor-
mance. Interventions in which facilitators prematurely offer learners their own observations and
insights typically do not result in substantial nor sustained performance improvements.28
Team building (methods to enhance interpersonal competencies) is focused on improving inter-
personal relationships and social interactions between team members.23 Team building has been
shown to improve both affective outcomes, like trust and confidence, and process outcomes, like
communication and coordination.9 There are 4 basic components of successful team building
interventions which can be implemented independently or together: goal setting, interpersonal
relationship management, role clarification, and problem solving.
Although each of the 4 components of team building has demonstrated efficacy, role clarifica-
tion and goal setting result in the best team outcomes due to the development of shared mental
models among team members.29 Role clarity implies a shared understanding among team mem-
bers about how different roles should be performed and who is responsible for performing each
role. Effective team building interventions help members determine the specific situations that
may require individuals to assume another members’ role. This role flexibility, which is essen-
tial for teams that need to be highly adaptive, can lead to enhanced team performance.30 When
setting collective goals, teams must consider that which is immediately relevant to all members
and focus on team-level outcomes rather than individual performance.8 Lastly, it is critical that
team building interventions guide teams to develop tangible action plans that outline future
team performance objectives.8
Team debriefing (methods to enhance team processes) is a familiar yet highly effective process
intervention that offers teammates an opportunity to reflect on past performance by discussing
what occurred, what could be improved, and what can be done differently in the future.31 Team
debriefs encourage active-learning by establishing shared mental models regarding strengths and
weaknesses among team members.32 Although team debriefing interventions have been shown
to improve team effectiveness by up to 25% in less than 20 minutes, as with other TDIs, efficacy
is dependent on the inclusion of certain characteristics, namely the targeted use of constructive
criticism, an effective facilitator, identification of moments of importance, and a balance between
celebrating success with highlighting failure.8 , 31
First, effective use of constructive criticism typically focuses less on individual team members
and more on team dynamics. Presenting criticisms focused on task-related information, as op-
posed to person-oriented factors, avoids potential breakdown of the psychologically safe culture
created in a strong team environment.33 Second, effective third-party facilitation that fosters
Table 1
Summary of team development interventions (TDIs) to maximize performance.
Evidence-based Recommendations for Characteristics

Team Development Interventions
Leadership training
Target interpersonal skills Present learners with role play opportunities to practice refining their
skills, like effective listening and communication, followed by
behavioral feedback.
Space delivery of interventions Opt to deliver content across several temporally spaced sessions to
provide learners with an opportunity to process and retain the
information presented
Target task-oriented skills Teach learners how to organize and execute tasks, clarify team
member roles, and establish effective communication patterns.
Team training
Cross-train duties As possible, teach each member the duties of his/her teammates to
promote role clarity.
Develop self-correction capabilities Embed the ability to diagnose and internally resolve breakdowns
within individual team members through focused training and/or
directed readings.
Establish psychological safety Encourage an environment that minimizes perceived threats and
tolerates failure without retaliation, rejection or embarrassment.
Team building
Goal Setting Establish team goals that are relevant to all team members and focus
on collective outcomes.
Interpersonal relationship development Rely on a facilitator to help teams work together to identify and
resolve weak performance stemming from unresolved interpersonal
conflicts
Role clarification Ensure each member of the team knows the responsibilities of other
team members and what situations may necessitate an individual to
assume another’s role.
Problem solving Help team members work together to identify task-related problems
and implement solutions accordingly.
Establish accountability Develop tangible action plans to guide future performance.
Team debriefing
Effective feedback Focus on task-related information instead of person-oriented factors.
External Facilitation Utilize the skills of a trained facilitator to ensure all team members
have an opportunity to contribute and that conversation remains as
productive as possible.
Highlight failure and success Discuss both positive and negative examples of behavior.
open and honest communication is preferential to self-led debriefing sessions, although inclu-
sivity of all team members in the discussion and limited facilitator involvement seem to lead
to the most effective debriefs.34 Third, chronological debriefs have been shown to be more thor-
ough but less effective than focused debriefing sessions on the extremes of positive and negative
actions.35 Finally, rather than solely debriefing on performance failures, successful debriefing in-
terventions involve discussion of both successes and failures.36 See Table 1 for a summary of the
evidence for activities that maximize team performance.
Team development in health care
Health care environments place extraordinary demands on teams to perform complex, rapidly
changing tasks in high stress situations. A patient in a given inpatient health encounter may re-
quire the coordinated efforts of more than 10 clinicians, not to mention a rotating cycle of nurses
and ancillary health professionals.37 In the outpatient setting, the average Medicare patient vis-
its 7 physicians on an annual basis, not to mention the allied health professionals also required
for their care.38 Furthermore, unlike most industries, health care teams are often convened in
Table 2
Rush University Medical Center, geriatric interdisciplinary team training program.42
1. The team should have explicitly stated team goals

2. The patient and family are at the center of all team activities and are active team members.
3. Professional roles must be clearly defined and understood
4. All team members should contribute to team function through constructive individual behaviors, including leadership
5. There must be effective communication across all work settings
6. The team must have tools or strategies for the effective management of conflict
7. The team should have explicit rules about participation and decision making
8. The team must be adaptable, responding to new challenges and conditions as they develop over time.
Table 3
Broader principles of team-based care in health care from the National Academy of Medicine.38
1. Shared goals
2. Clear roles
3. Mutual trust
4. Effective communication
5. Measurable processes and outcomes
emergency settings and frequently consist of individuals who have never met up until the mo-
ment of team formation. Yet these teams are expected to deliver potentially lifesaving care in a
synchronized, effective manner. Given these circumstances, the Joint Commission’s finding that
the breakdown of teamwork is a root cause in more than 2 thirds of sentinel events39 is rel-
atively unsurprising. Given the significance of teamwork interventions in other disciplines, nu-
merous efforts have been made to apply the science of teamwork to the health care industry. In
this section, we review a working definition of teamwork in health care as well as the existing
literature on effective teamwork interventions within health care as a whole before exploring
the published interventions for surgical teams and ultimately those interventions that are most
specific to the OR.
Defining “team” within the context of health care delivery
The fluidity of teams within health care and lack of expectations have been cited as ma-
jor factors in the poor performance of health care teams both in terms of patient outcomes
as well as unnecessary waste and associated costs.38 Thus, although the vast majority of clin-
icians agree that team-based care improves care decisions, up to a third have cited cumber-
some team interactions and difficulties in sharing information across a multidisciplinary team
in a timely fashion.40 In order to promote clarity in this discussion, we adopt the definition for
team-based health care created by Naylor and colleagues and adopted by the National Academy
of Medicine:41
Team-based healthcare is the provision of health services to individuals, families, and or/their
communities by at least two health providers who work collaboratively with patients and their
caregivers–to the extent preferred by each patient–to accomplish shared goals within and across set-
tings to achieve coordinated, high-quality care.
Notable in this definition are the major principles for effective teamwork. Many organizations
have gone further to explicitly define the effective principles of team collaboration. For instance,
Tables 2 and 3 provide examples of broader principles of team-based care in health care. Both
institutions which produced the following principles have interactive guides for evaluating and
improving these competencies within a specific team context.
Lastly, there are unique aspects of health care teams, some of which explain difficulties in
improving teamwork via interventions that have worked for other industries. Specifically, Hughes
and colleagues note that health care teams have low temporal stability, short team lifespan,
functional role structures, high skill differentiation among team members, rotating leadership
structures, high authority differentiation, and high interdependence.43
Team development interventions (TDIs) within health care
Instead of attempting to cover the vast array of interventions that have been implemented in
a wide array of health care settings to improve teamwork, we use several systematic reviews of
team development interventions to assess the current level of evidence in a health care setting.
First, team training, team building, and team debriefing interventions are typically interwoven
within health care, thus the evidence for those interventions are discussed in aggregate. Sec-
ond, we provide an overview of the various measurement techniques developed to assess team
effectiveness in a health care setting.
Leadership development
Leadership development within health care often encompasses a broad range of training ef-
forts and curricula extending from performance improvement to organizational change efforts
to succession planning.44 Given that the focus of this monograph is on surgeon leaders in the
OR, we primarily frame the discussion of leadership development within the role of a leader
seeking to bring about performance improvement in a team context. In many ways, the model
of OR leadership differs from other industries, where an assigned hierarchy results in direct re-
ports for upper-level management. This is the focus of the later section entitled “Leadership in
an Intraoperative Context,” whereas what follows is a translation of some of the organizational
leadership strategies that still demonstrate efficacy within health care and surgical teams.
First, just as effective leadership has been linked to improved performance in the meta-
analysis by Lacarenza and colleagues, better clinical leadership has also been linked to improved
patient outcomes as well as the satisfaction of providers and patients in numerous health care
settings.45 The measured output of strong leadership includes successful implementation of pa-
tient safety initiatives, higher staff retention rates, and increased perception of psychological
safety. These differences in performance persist when policies and protocols are held equal. For
instance, in a study of hospitals with high- and low-quality measures for acute myocardial in-
farction, Curry and colleagues demonstrated that higher performing hospitals were more likely
to report a positive influence of physician leadership and team empowerment than those hospi-
tals performing poorly despite the fact that no difference in care protocols was found.46
Despite the impact of leadership development for care delivery and process improvement,
there is little consensus about what makes a successful clinical leadership development pro-
gram. In a 2014 systematic review of leadership development programs within health care, Frich
and colleagues found that virtually all studies reported positive outcomes, although the major-
ity measured pre/post evaluations as opposed to system-level effects (eg, quality improvement
measures, patient satisfaction).44 Yet the objectives and composition of these programs were
drastically different. Some programs employed seminars, group work, and lectures focusing pri-
marily on imparting conceptual knowledge, with a minority focusing on personal growth and
awareness. Notwithstanding these results, there do appear to be several principles for effec-
tive health care leadership development programs. First, programs that utilize established prin-
ciples of adult learning tend to demonstrate the most significant impact on both individuals and
organizational outcomes. Techniques such as developmental relationships (eg, mentors, coach-
ing), rotating assignments, and multi-source feedback processes tend to be most successful at
driving significant personal and team-based change.44 Second, effective health care leadership
development programs promote substantial personal leadership development in the domains
of emotional intelligence and self-awareness. Although numerous programs impart conceptual
knowledge to program participants, the application of knowledge and self-awareness developed
through action-based learning and simulation seem to be more effective as perceived both by
learners and independent raters. Third, multidisciplinary engagement, although expensive and
complex, is more likely to be linked to improvement in significant patient-oriented outcomes,
implying a substantial need for team training and the role of leadership within these teams.
Table 4
The “Big Five” or 5 core components of teamwork that promote team effectiveness. Adapted from Salas et al.10 , 11 , 48 .
Competency Exhibited Behaviors
Leadership Coordination of member activities to achieve the collective purposes of the team;
includes (1) development of knowledge, skills, and abilities of team members, (2)
motivation of team members, (3) culture creation, (4) performance assessment,
and (5) feedback
Performance monitoring Ability for team members to construct shared understanding of environment and
work roles as well as perceive the success/failure of member and overall team
functioning
Backup behavior Team member’s insight into other individual’s needs to anticipate upcoming gaps
Adaptability Adjustment of strategy based on changes in the environment or team composition
Team orientation Propensity to seek broad input and feedback in order to promote goals of team
over goals of individual team members
Team training, team building, and team debriefing
TDIs beyond individual leadership competencies have been highly funded and publicized
within health care, especially for physicians. Approximately 75% of American medical students
participate in a team training event prior to graduation, and, as demonstrated by a 2017 sys-
tematic review, more than three fourths of trainees within team training events are practic-
ing physicians.39 The majority of the interventions published involve situations and teams in
acute hospital settings, focusing primarily on NTS development. Team functioning is commonly
measured by the Safety Attitudes Questionnaire (SAQ) or Non-technical Skills Tool (NOTECHS,
[NOTECHS-II for the OR]), although some studies have included organizational and patient-level
outcomes as well.47 Simulation-based team development is the most common method for in-
tervention, followed closely by principle-based training such as CRM and Team Strategies and
Tools to Enhance Performance and Patient Safety (TeamSTEPPS). Finally, given the need, notable
interventions have also been made to improve role clarity as well as the effectiveness of team
debriefing. Overall, TDIs appear to be effective in terms of teamwork outcomes such as com-
munication, improved reactions, and skill transfer to the real clinical environment. Impact on
patient outcomes has been demonstrated with weaker, but still positive, effects in some stud-
ies. That being said, there is strong evidence that learning outcomes are predictive of improved
patient outcomes in the future.43
Simulation-based learning is the most prevalent team development intervention within
health care. Most studies of simulation for health care teams have demonstrated mild to mod-
erate improvement in team-oriented NTS, although they frequently do not comment on the per-
sistence of these improvements over time.47 Interestingly, despite a prevailing belief that high
fidelity simulation sessions mimicking clinical scenarios as closely as possible lead to higher
transfer of knowledge the skills to the workplace, the fidelity of simulation does not appear to
influence the effectiveness of team training interventions.43 Instead, simulations that mimic the
psychological stressors of clinical teamwork (eg, time pressure, urgent patient needs, etc.) are
more likely to lead to transfer of skills than simulations mimicking the physical environment.43
Principle-based team training, predominantly characterized by CRM and TeamSTEPPS, is one
of the more heavily researched interventions. Given the prevalence of these interventions within
surgery, a more in-depth history, description, and implementation guide can be found in the
following section. But the evidence for utilizing these tools extends far beyond the surgical spe-
cialties. Specifically, principle-based team training interventions utilize tailored training sessions
organized around established tenets (Table 4) of effective teamwork to bring about sustained
change. Similar to simulation, the evidence for these interventions usually consists of improve-
ment in NTS competencies. However, more studies of principle-based team training have utilized
rigorous study designs and typically demonstrate improvement in process measures such as er-
ror reporting or throughput time in addition to learning outcomes for trainees.47 These learning
outcomes merit the later section on surgery-specific application of principle-based team train-
ing.
Lastly, it is worth mentioning the effectiveness of interventions specifically focused on im-
proving role clarity and the utility of debriefing for any type of team training event. Regarding
role clarity, a systematic review from June, 2020 concluded that short interventions (half day
or less) designed to prioritize role clarity among providers and conducted in the workplace led
to improvements in technical skills of trainees but that NTS seemed to require greater invest-
ments of time. Other than patient satisfaction scores in one of those studies, no other patient
outcomes appeared to be affected, although very few were measured.49 This review seems to
provide mild evidence for brief, on-the-job team development interventions aimed specifically
at role clarification.
Regarding debriefing, many TDIs utilize debriefing as an adjunct to cement learning and pro-
mote transfer of knowledge as well as skills to the workplace. A systematic review of debriefing
following NTS simulation training sessions found that improvement in NTS by trainees who un-
derwent debriefing with a trained facilitator was similar to that of trainees who performed self-
debriefing (alone or in a team setting) or those who had a pre-programmed debrief prompted
by multimedia.50 This may indicate that skilled facilitators are less important to the debriefing
than previously assumed. Effective debriefing helps trainees interrupt their usual methods for
interpreting events in order to construct new modes of understanding and behavior, offers ac-
tionable feedback for transfer to the clinical workplace, reduces cognitive load to focus on key
learning, and effectively manages the balance between a trainee’s emotional response to the
simulation and cognition necessary for improvement.50 The hierarchical structure of health care
can lead to unintended consequences when it comes to the feedback and debriefing process.
Since most feedback is health care is provided by a senior provider, the high-power distance may
increase trainee anxiety and inhibit learning. Furthermore, this unidirectional feedback has been
demonstrated to increase trainee focus on self instead of task-related learning. These insights are
undoubtedly pertinent to the surgical field where hierarchy often reigns supreme within surgi-
cal and multidisciplinary teams. For these reasons, much work has been done to promote the
advocacy-inquiry technique and debriefing with “good judgment” in order to remove some of
the effects of a hierarchical professional system.51 , 52
The application of teamwork science to surgery
Now we will turn to the application of team development science to surgical teams and the
evidence for its effectiveness, including a discussion of techniques such as CRM and TeamSTEPPS.
We shall then discuss NTS, focusing on individual surgeon assessment and development, before
concluding with a discussion of the connections between team and individual NTS development.
Crew resource management
One of the most frequently employed surgical team training systems, CRM has its origins in
the aviation industry as a byproduct of several noteworthy commercial airline accidents in the
1970s, from which resulting analysis uncovered human error in areas such as task saturation,
group think, and deviating behavior.53 CRM encompasses the knowledge, skills, and attitudes
of effective communication, situational awareness, problem solving, decision making, and team-
work. The efficacy of CRM within the aviation industry led to spread into safety-oriented indus-
tries such as firefighting, emergency responders, and health care.54 Although many details are
industry specific, the overall principles of CRM are conserved across 3 categories and associated
subcategories (Table 5).53
CRM led to briefings in the OR and promoted techniques such as the “inquiry, advocacy, as-
sertion” model to enable non-confrontational sharing of concerns and closed loop communi-
cation. For example, in the OR, a brief example of inquiry, advocacy, and assertion might sound
Table 5
Crew resource management (CRM) techniques.
Categories Subcategories Description
Communication Process Briefing Setting the stage, establishing and confirming roles,
and Decision Behavior establishing open communication, addressing all safety
concerns, confirming all pre-procedural checks are
complete
Inquiry, advocacy, & assertion Advocacy - expressing concern or risk to safety
Inquiry - identifying and asking about a potential
concern or change in status
Assertion - providing a solution and obtaining
agreement.
Crew self-critique An internal debriefing among the crew regarding the
processes that were followed, including an assessment
of the communication of the team and resulting
outcomes.
Conflict resolution A formal process for resolving disagreements among
crewmembers in interpreting information or proposing
courses of action
Communications and decision Methodology for obtaining information through open
making communication in order to make informed decisions
Team Building and Leadership, followership, and Components of effective leadership including
Maintenance concern for tasks coordination of activities that maintain proper balance
between respect for authority and assertiveness of
junior team members.
Interpersonal relationships and Techniques for the acknowledgment of other
group climate crewmembers’ personalities and styles, including
mechanisms for maintenance of a friendly, relaxed, and
supportive yet task-oriented tone in the operating
room
Workload Management Preparation, planning, vigilance Techniques for monitoring tasks, responding to new
and Situational information, and pre-operative preparation for all
Awareness members of the crew
Workload distribution and Assignment of appropriate roles, workload, and the
distraction avoidance triage of tasks to minimize distractions
something like the following: “Dr. X, I noticed that the patient has not yet received their prophy-
lactic antibiotics. If you agree, I’d like give them prior to your incision.” CRM techniques enable
effective communication across the hierarchy of the OR while minimizing conflict and ultimately
have been demonstrated to improve patient safety.53 Some have applied CRM techniques in par-
allel with personality assessments (eg, Myers-Briggs, DiSC) to enable increased sensitivity to-
wards different personality types, as some personality types have been noted to perform worse
under beratement or aggressive leadership.53 Separately, training in workload management and
situational awareness allows teams to assign roles, distribute tasks appropriately, prevent work
overload, and anticipate upcoming needs. Often, when team members assist in portions of the
care process that are not specifically assigned to them (“boundary spanning behaviors”) in or-
der to promote team success, the ensuing psychological safety and camaraderie result in overall
team motivation. For instance, simple acts such as the attending surgeon remaining in the OR to
assist with moving the patient, retrieving the bed, or even placing postoperative orders in order
to reduce work overload for the remainder of the team may result in improved work ethic by
all team members.
The adoption of CRM-based training to the OR took place in the mid-20 0 0s and has since
demonstrated effectiveness for improving perceptions of communication and safety, reducing
rates of human error, increasing compliance with system policies, improving perioperative ef-
ficiency, and even reducing procedural morbidity and mortality. Catchpole and colleagues con-
ducted CRM-based training programs consisting of classroom team training followed by intraop-
erative observation for 3 surgical subspecialties in the UK. Observers documented significant
increases in time-outs performed, briefings, and debriefings.55 Similarly, Gore and colleagues
employed aviation-based CRM experts to guide the implementation of a curriculum focused on

patient safety, error reduction, and an establishment of preprocedural briefings. Post-curriculum
survey results demonstrated improved perceptions among surgeons, anesthesia, and nursing in
regard to adverse event reporting, patient safety climate, and teamwork.56 Medical team training
(MTT), a derivative of CRM, was enacted at San Francisco VA Medical Center in 2006 along with
an accompanying standardized OR briefing and debriefing model. After analysis of 4,800 cases,
it was noted that perceptions and safety attitudes towards teamwork climate, safety climate, job
satisfaction, stress recognition, perception of management, and working conditions significantly
increased.57 Even after 24 months, follow-up surveys demonstrated a significant decrease in pre-
operative delays, hand-off issues, equipment delays, and improved timing of prophylactic antibi-
otic administration.56 , 58 Additionally, the process resulted in the identification and correction of
a serious systems problem related to the delay of ordering and administering preoperative med-
ication and launched a nationwide initiative to adopt MTT at all VA medical centers. Data from
this initiative have demonstrated a significant reduction in rates of venous thromboembolism
and postoperative infections at institutions that implemented perioperative MTT as compared to
those that did not.
Clinical vignette:
Wakeman and colleagues describe a case in pediatric surgery in which a 3-year-old girl un-
derwent resection of a Wilms tumor, however no lymph nodes were received by the pathol-
ogist.59 No documentation existed about a lymph node harvest or tissue pass off to the
nursing staff. The authors assert that several aspects of CRM could be applied in order to
review the error and prevent repetition in the future. The authors use this framework to illus-
trate the different types of group interactions that occur in the hospital and operative setting
– crews, teams, and emergently formed ad hoc teams (eg, code team). Most ORs function
in the form of a crew–a senior constant presence (surgeon) with a rotating substitution of
competent members (eg, nurses, surgical technicians, anesthesiologists). Crew members
may come and go throughout the case due to time constraints but the senior leader (op-
erating surgeon) will stay throughout its entirety. This contrasts to a team, which will not
substitute during cases and often have a much more leveled hierarchy, better communi-
cation, coaching, and conflict resolution. Finally, an emergent ad hoc team is made up of
competent individuals serving specific roles who assemble rapidly for an emergency situ-
ation (eg, cardiac arrest). Using a crew format in the OR is practical given constraints with
shift work and allowing hospitals to efficiently allocate personnel and staff. Drawbacks to
crews include over reliance on hierarchy/leadership, lack of familiarity with crew members,
and inexperience of a crew member in a complex scenario. In the Wilms tumor case, CRM
analysis resulted in several identified issues. First, the senior leader failed to instruct other
crew members that they should expect a specimen pass off. Second, a lack of familiarity
among team members prevented advocacy such as a nurse speaking up to address lack of
specimen pass off. Finally, inexperienced crew members unfamiliar with the major goals
of the operation did not have the knowledge that extracting and sending lymph nodes to
pathology was an essential part of the case.59
More specific applications of CRM-based training programs have been created for improv-
ing particular aspects of the team. For example, to improve intraoperative communication be-
tween surgeon and anesthesiologist, Awad and colleagues utilize a series of didactic instructions,
role-play, training films, and clinical vignettes, which ultimately resulted in an improvement in
the frequency of preoperative briefings, increased objective measures of communication, and re-
sulted in an 11% increase in the rate of administration of antibiotics prior to incision as well as
an 8% increase in the application of sequential compression devices prior to induction.60
TeamSTEPPS
Heavily influenced by CRM, TeamSTEPPS was created in 2003 as a joint venture between the
Department of Defense (DoD) and the Agency for Health Care Research and Quality (AHRQ) to
provide health care institutions with a standardized framework for improving teamwork and
training.61 The curriculum is centered around 4 core competencies: (1) leading teams; (2) mu-
tual support; (3) communication; and (4) situation monitoring. Within each competency, clear
definitions of excellence are provided alongside the tools and strategies for team members to
improve, achieve, and measure proficiency.61 Within the realm of surgery, TeamSTEPPS has fre-
quently been applied to enhance the performance of trauma and resuscitation teams. For in-
stance, Cappella and colleagues demonstrated decreased time to the CT scanner and time to the
OR as well as improvements in perceived leadership, situational monitoring, team-based mu-
tual support, and communication following implementation of TeamSTEPPS training.62 Specifi-
cally, the most effective tactics employed in the TeamSTEPPS model have been team briefings,
task assignment prior to patient arrival, closed loop communication, and directed call-outs.62
Despite these additional components of an already complex perioperative process, TeamSTEPPS
techniques appear to increase, not decrease, operative efficiency. In one such study, researchers
evaluated OR efficiency and patient safety by comparing metrics of 1481 cases following im-
plementation of TeamSTEPPS and found that the mean case time decreased by 10.1% with first
case on-time-starts increasing by 21%. Simultaneously, the overall rate of patient safety issues
decreased from 15.8% to 6.2%.63 , 64 In other study, Forse and colleagues studied the effects of
TeamSTEPPS implementation for all perioperative staff at a single institution and noted signif-
icant improvements in OR communication and teamwork, with more punctual first cases and
improvements in appropriate administration of preoperative medication.65 Most notably, overall
surgical morbidity and mortality improved as well, with a persistent effect at a follow-up inter-
val of one year, although later follow up demonstrated potential longer-term decrement in this
effect.65 In a similar study, a 6-month audit of more than 1600 cases after TeamSTEPPS train-
ing and implementation showed statistically significant improvements in recognition of team
members by name and role, anticipation of case complexity, discussion of medical status, ac-
tive engagement, utilization of closed loop communication, “time-out” performance compliance,
team assembly, discussion of postoperative plan, and discussion of improvement opportunities.
Additionally, a 67% reduction in retained foreign body and wrong site/wrong person procedure
was observed.66 Because of these findings, many have argued that TeamSTEPPS should be a nor-
malized part of perioperative accreditation and that internal auditors should play an active role
in cases by gently providing real-time feedback and coaching.
Simulation-based team training in surgery
Although frequently used to assist with the development of technical skills and clinical de-
cision making, simulation has also become a crucial aspect of team training, particularly to al-
low interprofessional education and collaboration67 . In fact, most newly developed team train-
ing programs for health professionals utilize some form of simulation,68 specifically within
trauma69-71 , labor and delivery72 , 73 and intensive care.74 These programs are designed to bring
together providers across professions and disciplines with the goal of improving teamwork, es-
pecially for the purpose of responding to high-acuity, low-frequency events.
Team training programs frequently utilize simulation for OR scenarios, but training pro-
grams for surgeons have often lacked inclusion of other professions that are integral to the OR
team.75 , 76 This is especially true in courses designed for surgical residents, in which simulated
participants in scripted roles play anesthesiologists, scrub technicians, and other key roles. Al-
though the benefits of convenience are obvious, such interactions do not create the interprofes-
sional environment necessary to maximize the benefits of these programs. True interprofessional
training requires participants to learn both with and from one another to develop key team be-
haviors.77 This can only be achieved when members from multiple professions and disciplines
are learning together as equals in a program.
Programs designed to be interprofessional have frequently focused on the interactions be-
tween surgeons and anesthesiologists, often leaving out the nursing staff and other key members
of the OR team.75 , 76 A smaller number of programs have brought together the 3 key professions
that are found in the OR, surgeons, anesthesiologists, and nurses.78 Other members of teams
that have been included on rare occasions include perfusionists,79 , pharmacists,80 respiratory
therapists,80 and other allied health professionals.80-83
The vast majority of interprofessional OR team training simulation programs are designed
for surgical trainees across postgraduate year levels,80-82 , 84-87 with a much smaller number of
programs combining both trainees and attending surgeons,83 , 88 and a few designed specifically
for attending surgeons.79 , 89–91 There remains room to develop ongoing, longitudinal programs
targeting surgical learners across the novice to expert continuum.
Critically, active participation in any simulation experience must be driven by the intrinsic
and extrinsic motivation of the participants, as well as the physical and psychological fidelity.
Data from research studies surrounding OR team training with simulation programs have con-
sistently shown that participants enjoy the experience, feel that the simulation program was
beneficial, and believe that the simulation has helped prepared them to handle potential crises
in the OR.79-84 , 87 , 88 , 91 Data regarding the impact of simulation-based OR team training on out-
comes are limited, but a few studies have looked at the overall impact of these training pro-
grams on outcomes related to patient safety in the hospital and have shown statistically sig-
nificant reductions in surgical morbidity92 , 93 and mortality92 , 94 following implementation of OR
team training programs.
Reviews of the literature have highlighted key features of team training programs that cre-
ate effective programs. The most effective programs provide learners with readiness assessments
and pre-work activities that help prime learners for the experience. Similarly, continued coach-
ing and support after team training also helps to enhance the learner experience and transfer
to skills.68 Although high-fidelity simulation utilizing advanced simulators and software are ap-
pealing and can draw participants into the training, studies have not shown that such programs
are any more effective in providing training than those using low-fidelity models and systems.68
Other features of successful programs include a focus on specified key skills,68 scripted scenar-
ios, and use of trained faculty debriefers.95
Barriers to conducting full OR team simulations include time, scheduling, space, equipment,
and buy-in from key leaders.91 Although such programs can be conducted either in a dedicated
simulation laboratory or in situ, creating realistic scenarios that provide appropriate fidelity for
all team members usually requires advanced simulators that may be cost prohibitive in some
programs. Finding available team members who can be free from clinical obligations for the
duration of the course can be challenging, especially when departments are already understaffed
or need to schedule additional personnel to cover the responsibility of those completing the
training program. Buy-in and encouragement from institutional leaders is essential to creating
strong participation, both in encouragement to attend training, but also to free up time and
resources as necessary.90
To conclude, OR team training programs, simulations, and frameworks such as CRM or Team-
STEPPS play an integral role in equipping surgeons, anesthesiologists, nurses, and other team
members with the skills necessary to provide high-quality, safe care for patients throughout the
perioperative period. These programs provide opportunities for team members to learn with and
from one another in a psychologically safe environment. Despite technological, administrative,
and financial challenges, such programs should be considered a cornerstone for development of
future programs to train such skills, especially NTS.
Fig. 2. Minimum components of the Joint Commission Universal Protocol Time-Out. Adapted from Norton 2007.99 .
Established cognitive aids for promoting teamwork
Structured communication tools, when properly implemented, are another piece of the arma-
mentarium that may be used to improve teamwork in the OR. Two such tools are team briefings
and safety checklists.
Team briefings
Team briefings in surgery evolved from the standard pre-procedure time-out.96-98 The time-
out, a simple checklist to promote adherence to basic safety practices, was introduced by the
Joint Commission on Accreditation of Health Care Organizations (now the Joint Commission) in
2004 as a universal precaution against wrong-site surgery (Fig 2).99 Preoperative time-outs be-
came effective communication tools as additional teamwork-focused items were added, includ-
ing team introductions and specific cues for input from all members of the surgical team.
The evidence for team briefings is primarily focused on OR safety and efficiency. By promot-
ing open discussion among team members, briefings have been shown to reduce communication
failures in the OR by nearly 2 thirds.100 , 101 Use of briefings is also associated with improved
safety culture102 , 103 and adherence to safety procedures.98 , 104 All this comes at minimal time
cost; trained teams can complete the briefing process in less than 2 minutes, with a signifi-
cant return on investment in terms of OR efficiency, including a 50% reduction in disruptions to
surgical flow and a 30% reduction in unexpected delays.101 , 105
Intraoperative safety checklists
Human factors engineers have long promoted checklists as a communication tool to improve
safety in a range of high-risk industries, most famously in aviation. Although several fields, in-
cluding anesthesiology, recognized the value of safety checklists early on,106 robust use of check-
lists as communications tools took hold in critical care before they arrived in the OR.107 This
changed in 2009 when a landmark study demonstrated that the World Health Organization Sur-
gical Safety Checklist, introduced as part of the 2006 Safe Surgery Saves Lives initiative, reduced
postoperative morbidity and mortality by more than a third across a range of contexts.108 Similar
results have been reported in a range of smaller studies and in several systematic reviews.109-114
Surgical Safety Checklist use is also associated with improved adherence to safety practices, OR
efficiency, and decreased cost.108 , 115 , 116
It is tempting to assume that the benefits of the Surgical Safety Checklist are mediated only
through reinforcement of standard safety practices, providing an additional layer in the “Swiss
cheese” model of errors.117 However, this is only one of several mechanisms by which the check-
list improves outcomes. There is strong evidence that the checklist improves teamwork and co-
operation in the OR118 and improvements in safety culture have been documented in several
studies.119 , 120 The impact of the checklist on these behavioral aspects of surgical performance
is arguably more important than the purely mechanical function of its tasks in ensuring the
delivery of safe care.
Some critics of the Surgical Safety Checklist point to inconsistent results in the literature
as evidence of its inefficacy.121 , 122 Achieving meaningful benefits from the checklist, as with
most safety initiatives, requires meaningful use. Consequently, checklist implementation requires
thoughtful, context-appropriate strategies to engage physicians, and surgeons in particular, as
leaders. Weak implementation strategies can cause confusion and fail to achieve meaningful,
sustainable use of the checklist.123 , 124 Principles of implementation to maximize checklist com-
pliance, sustainability, and impact include: (1) prospectively tailoring the checklist to the specific
context, (2) ensuring that checklist use is seamlessly integrated into the OR workflow, (3) ensur-
ing clinician participation and leadership in checklist development, implementation, and use,
and (4) utilizing context-appropriate behavior change and reinforcement mechanisms to pro-
mote adherence.125 Multifaceted approaches combining implementation strategies that operate
on multiple levels to engage clinicians and encourage checklist adoption have been shown to be
most effective.126 Incorporating cognitive aid use into simulation scenarios, particularly crisis-
related simulations, has proven to be especially impactful for affecting rates of utilization and
attitudes of team members.127
Non-technical training tools for the surgical team
NTS have been repeatedly demonstrated to impact patient outcomes across multiple medi-
cal specialties.37 , 128 The vast majority of adverse surgical events stem from system errors un-
related to technical skills with more than 40% resulting from breakdowns in communication
alone.37 Skill domains such as decision making, teamwork, leadership, and situational awareness
have specifically been shown to impact surgical success.128 Fortunately, these abilities can be
learned129-132 through a wide variety of methods: direct object use (eg, assessment checklist),
courses, teamwork training programs, or simulation.128 These tools are typically classified into 2
broad categories: (1) tools for the surgical team and (2) tools for the individual surgeon. Con-
sistent with adult learning principles, each tool primarily consists of an assessment scale with
behavioral anchors. Not only do the scales allow for objective evaluation of team or individ-
ual performance, but they also establish a framework for team members to self-monitor, obtain
feedback, and set personal development goals. Here, we provide an overview of the most studied
and implemented training tools first for teams, and then for individual surgeons.
Oxford nontechnical skills scale (NOTECHS)
The Oxford Nontechnical Skills Scale (NOTECHS) evaluation system was originally developed
in the aviation industry in response to teamwork and cognitive skills training requirements
aimed at promoting safety in the cockpit.133 Given the recognized need for performance-related
review in surgery, the scale’s behavior markers were adapted to form NOTECHS for surgical
teams. Working alongside human factors experts and aviation-crew resource-management train-
ers, a cohort of surgeons and anesthesiologists worked to modify the Oxford NOTECHS scale for
use in the OR.133 The tool relies on a trained observer to examine the behaviors of the nurs-
ing, anesthesiology, and surgery sub-teams along 4 dimensions: leadership and management,
teamwork and cooperation, problem-solving and decision-making, and situational awareness.133
These behaviors are scored to assess the NTS of each sub-team as well as the surgical team
collectively to ultimately highlight the strengths and weaknesses of the team’s performance.
When implemented alongside a team-training program, NOTECHS scores are often used to
evaluate whether training programs are effective at developing NTS and changing team perfor-
mance. Simultaneously, many have used the results to identify residual problems across the be-
havioral domains observed in an effort to select specific interventions to enhance future team
performance.134-136 Importantly, NOTECHS has achieved validity across multiple studies and has
excellent interrater reliability, ensuring that the tool is consistently and accurately measuring be-
haviors as designed.133 , 135 , 137 , 138 In regard to physicians in training, NOTECHS has been used to
assess surgical residents’ teamwork skills following the incorporation of simulation-based train-
ing in the residency program curriculum.139 Scores for communication, leadership, and team-
work significantly improved for more senior residents, but no change was observed for junior
residents over the course of the year. This finding may reflect the long learning curve for clinical
and technical skills that junior residents experience during residency training.
Observational teamwork assessment for surgery (OTAS)
Another tool built from teamwork assessments in aviation, military and naval settings, the
Observational Teamwork Assessment for Surgery (OTAS) was developed by a team of surgeons
for use in the OR.140 Similar to NOTECHS in structure, OTAS assesses team behaviors across
5 domains: leadership, communication, cooperation, coordination, and team monitoring. Mind-
ful of the fact that team performance evolves throughout the duration of a surgical case, OTAS
uniquely incorporates observations of the team across the entire arc of surgical care (ie, preop-
erative, intraoperative and postoperative stages).141
OTAS serves to benchmark teamwork skills and, like NOTECHS, can be used to objectively
track improvements in teams’ non-technical performance following targeted team training in-
terventions.141 A study that evaluated more than 50 general surgery procedures conducted by a
single surgical team found that, even though ratings of overall team performance were gener-
ally high, objective measures of intraoperative communication were often low across disciplines,
potentially representing areas of safety concern.140 Separately, ratings for surgeon communica-
tion skills were demonstrated to deteriorate over the course of a case as compared to other
members of the team, leading to an overall deterioration in team performance scores.140 , 142 In
one instance, an OTAS-based analysis identified that when senior surgeons routinely left the OR
upon completion of the critical steps of a procedure, team dynamics were altered so significantly
that the nursing staff was typically left with the sole responsibility of coordinating the remain-
der of the patient’s perioperative care. On more than one instance this resulted in a breakdown
in team communication as the case entered the postoperative phase. The information obtained
from OTAS scores was used to guide team debriefing and aid in the selection of targeted team
training interventions to improve subsequent team performance.
Operating room management attitudes questionnaire (ORMAQ)
The Cockpit and Flight Management Attitudes Questionnaires (CMAQ, FMAQ) have long been
used by the aviation industry to assess crew member perceptions of stress, teamwork, commu-
nication, and safety in the cockpit.143 In an effort to capture diagnostic information relating to
behavior and safety in surgical units, these questionnaires were reimagined for use in the op-
erative setting. Using items that are relevant to understanding error, predictive of performance,
and sensitive to training interventions, the Operating Room Management Attitudes Questionnaire
(ORMAQ) was created to measure the attitudes of OR staff toward stress, hierarchy, teamwork,
and error.143 Although ORMAQ has reliably demonstrated that surgical teams often are aware
of problematic leadership and breakdowns in teamwork, especially pertaining to patient safety,
significant differences in attitudes towards handling these issues exist between and across disci-
plines.144 Specifically, attending surgeons tend to perceive the quality of leadership and commu-
nication in the OR more favorably than trainees and nurses. Such differences in perceived psy-
chological safety and the ability to speak up have been observed across the hierarchy in other
medical settings as well,26 which has been attributed to the fact that it is typically safe for the
individual at the top of the hierarchy to speak her mind, but rarely so for the individual at the
bottom. Similarly, attending surgeons are less likely to perceive team debriefing following cases
as valuable,144 potentially because the benefits for safer, open communication are not accrued
by them as much as other team members. The fundamental value of ORMAQ lies in its ability to
evaluate and compare provider attitudes across disciplines to better guide future team training
interventions.144
Non-technical training tools for the individual surgeon
Nontechnical skills for surgeons (NOTSS)
The most widely used and extensively validated NTS training tool is the Royal College of
Surgeons of Edinburgh (RCSEd) Nontechnical Skills for Surgeons (NOTSS).143 , 145 , 146 NOTSS is a
behavioral marker system that relies on the structured observation and evaluation of NTS for an
individual surgeon in the OR.147 In an effort to identify the NTS required by surgeons for safe
surgical practice and the observable behaviors associated with those skills, the NOTSS develop-
ment team interviewed surgeons and OR personnel about safety, error, teamwork, and adverse
events.148 The resulting NOTSS tool was structured into 4 behavioral domains: communication
and teamwork, situation awareness, decision making, and leadership. Following didactic sessions,
video simulation, or live OR observations, trained observers numerically evaluate the intraopera-
tive behaviors of the trainee or attending surgeon.149-151 In order to better guide evaluation, be-
havioral anchors associated with varying levels of performance are provided to the observer.143
Surgeons from various subspecialties have been evaluated using the NOTSS tool to ensure mea-
surement concordance among users in practice and the tool has demonstrated high inter-rater
reliability across studies.148 , 150 , 152 Originally created by the Royal College of Surgeons in the UK,
NOTSS has been adapted for use in multiple countries, including Japan, Denmark, and the United
States.147
The richness of NOTSS lies in the depth of resources available for surgeons seeking to im-
prove their behavior. Traditionally, formal task debriefing has not been commonplace in surgery
at the conclusion of successful operations, despite the fact that 73% of trainee surgeons agreed
that regular debriefing in the OR was important for effective team coordination.144 Some studies
have attributed this to surgeons possessing a flawed opinion of their own capabilities, under-
lining the need for objective debriefing as opposed to self-reflection.153 Others have thought
that the overestimation of NTS performance may simply be more a lack of self-awareness that
is more pronounced among inexperienced surgeons. For example, in one study of 26 surgeons
completing a simulated laparoscopic cholecystectomy, each surgeon completed a self-assessment
following the simulation including both technical and NTS skills. Separately, trained observers
scored the simulated performances using identical scales tool. Although both junior and senior
surgeons accurately self-assessed their technical skill performance when compared with the ob-
server assessments, junior surgeons were found to overestimate significantly their non-technical
performance.154 These findings highlight the value of expert assessment and feedback in pro-
moting ongoing awareness and learning to enhance patient safety. To assist with this feedback,
the NOTSS taxonomy provides structured, standardized scoring with behavior anchors in order
to describe accurately performance and provide actionable instruction.143 , 146
Use of the NOTSS tool also provides an opportunity for trainees to gain insight into intraop-
erative achievements and errors, to ask questions, and to receive specific advice to guide future
performance in an effort to enhance patient safety.143 , 144 Surgical trainees, who typically receive
lower NOTSS scores than senior surgeons, have demonstrated improved NTS following the pro-
vision of feedback guided by the NOTSS tool both during and after surgical cases.155 By offering
a common language for surgeons of various training stages to discuss NTS, the NOTSS tool can
be used to guide comprehensive feedback sessions for learners that both attending surgeons and
surgical trainees find useful.156 Furthermore, surgical residents who have been observed using
the NOTSS tool and subsequently received feedback focused on identifying performance gaps
and targeting future behavior felt that formal and informal feedback on NTS should be inte-
grated into surgical residency training.157
Leadership in the operating room
A subset of many NTS taxonomies, including NOTSS, the domain of surgeon leadership has
arguably the most robust research literature regarding effects on performance as well as spe-
cific models for improvement. A systematic review in 2011 identified only a few studies focused
on intraoperative surgical leadership.158 Only 2 studies measured impact on team performance
or patient outcomes, but the review resulted in a preliminary set of core surgical leadership
behaviors, which were closely aligned with existing task and team leadership models. The same
authors followed up with a direct observation of surgeon performance and subsequent Surgeon’s
Leadership Inventory, providing a comprehensive model for evaluating surgeon leadership per-
formance within an intraoperative context.159 , 160 As was suggested regarding utilization of the
NOTSS taxonomy and concomitant behavioral anchors, surgeons seeking to improve their lead-
ership capacities within the OR might seek out feedback and conduct a self-assessment based
on these capacities. Further work has done more recently to map existing models of effective
leadership to the intraoperative context.161
Other global and skill-specific tools
Although NOTSS is the most commonly employed NTS training tool, the 360-degree evalu-
ation tool and the Metric for evaluating task execution in the operating room tool also serve
the same purpose.145 The 360-degree evaluation tool, implemented via the online PULSE 360
Program, relies on fellow perioperative team members to answer a series of questions regarding
the surgeon’s professionalism, communication skills, interpersonal style, leadership, and team-
work abilities outside of the OR. In one study, 360-degree evaluations of 385 surgeons across
several subspecialties resulted in beneficial behavioral change for more than 60% of participants
who reported making adjustments in their daily non-technical practice based on the feedback.162
Higher 360 feedback ratings have also been demonstrated to correlate with lower rates of mal-
practice litigation.163 Separately, the Metric for evaluating task execution in the operating room
tool has been used to identify gaps in non-technical ability by objectively measuring the com-
pletion of tasks pertaining to perioperative communication.128 The tool is useful in assessing
and providing feedback on an individual’s effect on a team’s performance.118 Thus far, studies
on both interventions have been limited, and no intraoperative 360 results are currently found
in the surgical literature.
Linking non-technical skill development to patient outcomes
Finally, regarding the evidence for the effectiveness of NTS on patient outcomes, there are
limited data to assess such a claim. Neily and colleagues found up to a 33% reduction in mor-
tality in patients cared for by teams who underwent NTS training as compared to patients who
were cared for by teams without such training.131 Similarly, Forse and colleagues found a compa-
rable reduction in mortality after training, but did not include a control group for comparison.65
When comparing complications, readmissions, and length of stay, a 2018 meta-analysis failed
to demonstrate statistically significant improvements for NTS training across a pooled group of
9 studies.164 Even so, given the heterogeneity of study design and small sample sizes, consid-
erable work remains to power studies to truly measure the impact of NTS training on patient
outcomes.
Mechanisms to enhance surgeon performance
This final section goes beyond the history of the development of enhanced team performance
practice within the health care industry to the practical improvement of operative teamwork in
day-to-day performance. How does one translate what is known to be good behavior and best
practice to achieve the “shared mental model” in everyday surgical practice? As discussed previ-
ously, Team STEPPS, NOTECHs, and NOTSS have, within structured research studies, in both the
simulated and real-life surgical environments, demonstrated that it is feasible for the surgeon,
through improvement of their own NTS, to enhance the performance of the whole team. Here,
we examine the practical acquisition of improved team performance based on the previous lit-
erature and research described both as a unit and as a practicing surgeon, and within simulated
and real environments.
Literature and online resources
NTS and enhanced team performance have been recognized as essential to the development
of the trainee surgeon in numerous training programs worldwide. As mentioned above, the
AHRQ in conjunction with the DoD has developed the TeamSTEPPS 2.0 resource. This curriculum
is freely available online through their website and consists of a self-paced course with presen-
tations and e-learning videos to enable health care professionals to access the fundamentals.165
The RCSEd have championed the development of NTS for surgeons and has worked in con-
junction with the NOTSS taxonomy creators to develop a wealth of online resources. The RCSEd
made several of these resources freely available, such as a downloadable copy of the NOTSS
Handbook and several introductory videos. This allows for understanding of the theory and
principles underlying the NOTSS taxonomy and provides introduction to the framework and its
practical application which can be undertaken at leisure.166 For trainees, a freely accessible e-
learning resource consisting of 6 different modules on NTS is provided to order to introduce
the behavioral aspects of surgical performance, risk identification and mitigation, human factors,
and the NTS for surgeons taxonomy.167 RCSEd members can also access “NOTSS in a Box,” which
is an online introduction for senior trainee or consultant surgeons to the key concepts and el-
ements of the NOTSS taxonomy and the assessment framework.168 Additional resources can be
found on the NOTSS.org website. This allows all practitioners involved in operative care to ac-
cess resources that introduce the concept of NTS and the NOTSS taxonomy. It also contains a free
video-based micro-series according to the 4 domains of the taxonomy: situational awareness, de-
cision making, communication and teamwork, and leadership. The simulated videos enable the
viewer to engage with and practice using the NOTSS rating system.169 , 170 Globally, other surgical
colleges such as the Royal Australasian College of Surgeons also have a wealth of NTS resources
available to their members including a collection of human factors e-learning modules which
focus on topics such as situational awareness and team dynamics.171 Lastly, there are numer-
ous recorded lectures podcast episodes available online which are dedicated to the discussion of
human factors, NTS, and team performance in the OR.
Courses
For those seeking formal coursework, the TeamSTEPPS curriculum developed by the DoD
with the ARHQ as well as the online materials have developed several official courses, beyond
the informal online curriculum, in order to “train the trainers” and facilitate implementation
at health systems across the globe. Courses are available through several organizations including
the American Hospital Association.165 , 172 Such courses aim to provide participants with a deeper
understanding of the principles underpinning the framework as well as the tools necessary to
implement the program within their own clinical environment.
The RCSEd has offered the NOTSS Masterclass since 2006.166 Aimed at senior trainees and
fully qualified surgeons, the course provides participants an introduction to human factors, risk
assessment, and NTS as well as practical implementation of the NOTSS taxonomy in their clinical
environment either with their own colleagues or using a surgical trainer. The course was suc-
cessfully transformed into an e-NOTSS program in 2020, potentially allowing for greater reach
and accessibility. Beyond this iteration, the NOTSS Masterclass has been adapted and delivered in
countries across the globe.166 In the United States several NOTSS courses have been run in con-
junction with the American College of Surgeons and early research has demonstrated that the
application of the taxonomy within American operative systems is feasible and yields promising
results.147
Notably, these training courses in NTS can be effective without being overly tedious.
Pradarelli and colleagues demonstrated that surgeons undergoing just 10 minutes of online
training could achieve significant rater agreement. Such resources may be beneficial for groups
of surgeon educators seeking to embed NTS training and assessment within their training
program and provide consistent evaluations across multiple cases by different attending sur-
geons.170 Finally, evaluation need not be limited to surgeon perspectives. Crossley and colleagues
demonstrated that a large group of minimally trained assessors consisting of anesthesiologists,
scrub technicians, and advanced practice practitioners produced both reliable and valid NOTSS
scores.173 For this reason, we encourage the introduction of NTS training and ratings to multidis-
ciplinary teams at academic and non-academic centers. Furthermore, the introduction of train-
ing and rating for surgeons might provide an opportunity to introduce similar concepts around
NTS training for other disciplines (eg, Anesthetists Non-Technical Skills, Scrub Practitioners’ List
of Intraoperative Non-Technical Skills), thus evaluating an entire operative team’s non-technical
performance in a single case.174 , 175 Furthermore, routine assessment could then be incorporated
into team development practice, with potentially a full team discussion or debrief focused upon
NTS with the taxonomies as the foundation as well as individual performance feedback.
Higher education
Further personal development opportunities are available in the form of postgraduate qual-
ifications and degrees in patient safety and human factors. Many institutions offer part-time,
long distance learning courses which contain elements of teamwork, enhanced performance,
and non-technical skills. Examples include the Master of Quality and Safety and Health Care
at Harvard Medical School, Master of Science in Patient Safety at Imperial College London, and
the Master of Science in Patient Safety and Clinical Human Factors at the University of Edin-
burgh, which has recently been accredited by the Chartered Institute of Ergonomics and Human
Factors.176 , 177
Coaching
Another method of enhancing NTS performance that has gained increasing traction recently is
surgical coaching. Conceptually, surgical coaching maintains that surgeons are high performance
professionals who work in a high risk, high stakes environment, where each decision, each
action, and each procedure matters.178 Therefore, surgeons should be supported and coached
throughout their careers to enhance, improve, and develop their practice.179 , 180 Utilizing coach-
ing techniques from other professions such as sports, education and corporate business, surgeon
peer-coaching frameworks have been conceptualized, piloted, and implemented state-wide level
in the United States with promising results.179 , 181 Novel successful surgical coaching programs
include the Wisconsin Surgical Coaching Program Rubric, Michigan Bariatric Surgery Collabora-
tive, and the Surgical Coaching for Operative Performance Enhancement (SCOPE) program from
Ariadne Labs and Brigham & Women’s Hospital in Boston, Massachusetts.182 , 183
Using the SCOPE coaching framework as an example (Fig 3), surgical coaching capitalizes on
a defined structure, which makes coaching fundamentally distinct from teaching or mentoring.
Coaches receive pre-program training in coaching concepts, methodology, framework, and imple-
mentation. Surgeons are typically paired together by a site champion, someone with insight into
peer coaching relationships which will produce results. Prior to the coaching session, coach and
Fig. 3. The surgical coaching impact cycle. (Color version of figure is available online.).
coachee meet to set intraoperative goals, and therefore, the lens through which they wish to re-
view the selected case. The coach then observes the coachee either in person or retrospectively
through video recording, often using standardized rating scales (NOTSS, Objective Structured As-
sessment of Technical Skills, etc.) to guide the performance review. The coachee’s performance
is then jointly reviewed by the 2 surgeons, and the coach provides feedback in structured fash-
ion, most of which is centered upon the previously agreed upon goal(s). Bidirectional discussion
between the coach and coachee facilitates action planning for further improvement, typically
identifying next steps towards improvement and enhancing performance before agreeing upon
the next case scheduled for review.183 This process may occur multiple times over a set dura-
tion or potentially over years. Coaches and coachees may benefit from switching roles as well.
Some literature suggests that broadly implemented and sustained coaching programs encourages
a collegial atmosphere and may even improve surgeon wellbeing.179-181 , 183
Coaching also has immense potential to enhance team performance and NTS. Several studies
have shown that the majority of coaching discussions do not center around technical perfor-
mance but instead, teaching and NTS. Indeed, the SCOPE program demonstrated that more than
60% of coaching discussions were focused on NTS.180 , 184 , 185 The potential for the surgeon to ap-
ply, record, and analyze their own performance within a team in conjunction with the NOTSS
taxonomy is powerful in this context. In the United States there are now established general
surgical coaching programs in several states through use of the Wisconsin Surgical Coaching
Rubric tool, which has produced positive results.179 , 182 Furthermore, surgical coaching is becom-
ing more accessible nationally through creation of organizations such as the Academy of Surgical
Coaching. The Academy aims to provide any surgeon with access to coaching as well as resources
such as video-based technology to remove geographical limitations.186 Excitingly, the Academy
is partnering with national surgical institutions such as the Society of Vascular Surgeons develop
their own coaching programs to support its members.187
Special considerations for teaching non-technical skills
Given the readily available resources for training and teaching NTS, many educators and sur-
geons desire to engage in the training and teaching processes. However, 2 major issues merit
mention regarding teaching and training of NTS in situ: the existing cognitive load that trainees
experience and interpersonal dynamics that may confound rating and interpretation of NTS be-
haviors.
Trainee cognitive load
The stressors of the OR on trainees are well documented by numerous methods considering
the cognitive load, or the amount of demand for performance on the limited capacity of the
brain. Astute teachers of NTS would do well to consider a resident’s psychological state prior to
providing any additional instruction in order to both maximize the value of such teaching and
avoid overshadowing the teaching of other critical aspects of surgery.188 This does not mean,
however, that only more advanced residents should receive guidance about NTS performance.
For example, even a surgical intern closing a laparotomy incision should be aware of the level of
analgesia a patient is receiving while emerging from general anesthesia (situational awareness)
and communicate appropriately with the anesthesia team regarding timing and patient move-
ment. That same intern, while assisting with gaining laparoscopic access to the abdomen via a
Hassan technique, may not be able to assess patient vital signs with insufflation given the sig-
nificant cognitive load demanded by learning to enter the peritoneum safely. However, a fellow
in a minimally invasive surgery program should be expected to have that level of awareness,
although considerations of cognitive load are important if the fellow is also guiding an intern
through the access technique. Regardless of surgical complexity, most trainees are able to incor-
porate at least some new component of the operation (an instrument, technique, etc.) without
significantly impacting their perceived cognitive load189 and, in a recent survey, approximately
half of trainees requested increased training of NTS while in the OR.190 Furthermore, expecta-
tions of case difficulty and attending teaching style play a significant role in perceived cognitive
load.191 Attending surgeons seeking to teach NTS should be explicit with trainees that this will
be part of the feedback that residents will receive during the case in order to properly set ex-
pectations and potentially create cognitive capacity for handling feedback in that domain.
Interpersonal dynamics
Additionally, teachers should be mindful of the trainee-teacher dynamics due to gender, race,
or culture. Although these differences may have an impact on the instruction of technical skills,
notions of leadership, communication, and other NTS are at greater risk for being impacted by
these interpersonal differences, often via microaggressions.192 For instance, white individuals are
more likely to be evaluated as more effective leaders and as having more leadership potential193
and significant evidence exists demonstrating that female surgical trainees experience gender
bias and report a culture of sexism that forces social adaptations to “fit into the role of sur-
geon.”194 For these reasons, the evaluation and instruction of NTS in trainees demands intro-
spection on the behalf of the evaluator or teacher to consider the potential role of bias. Formal
implicit bias training may assist with reducing the role of bias in evaluations and teaching. NTS
feedback is best received and enacted when a strong trainee-teacher relationship exists and the
teacher has a considerable knowledge of the resident’s leadership style, personality, and abili-
ties. Notably, increased focus on NTS evaluation and personal development stands to improve
self-regulation of specific behaviors that underlie many of the reported microaggressions in the
OR.192
Teaching and training in the clinical environment
There are numerous opportunities to guide trainees towards developing their NTS both in the
operating theater and beyond. As with becoming a teacher, individual time invested in under-
standing NTS via the NOTSS handbook, online video-based training, or other options will likely
position trainees to self-assess, understand NTS feedback, and set goals for personal growth.
In an operative context, NTS impact all phases of care. As part of the preoperative planning
process, many surgical trainees discuss operative approach, anatomical considerations, and set
goals for the case; these are all part of decision making. NTS could be included in this preop-
erative discussion as trainees actively plan how to set the tone for the OR, communicate during
crucial maneuvers, and plan for potential difficulties. Intraoperatively, NTS might be incorpo-
rated into the language of the surgical safety checklist as the team members establish a baseline
for communication and develop conditions for a shared situational awareness about the current
state of the patient. Discussion of NTS immediately before an operation can serve as a regular
reminder of the importance of NTS as well as the ideal behaviors and language used to describe
them. Attending surgeons should use the language of NTS to provide active feedback, just as they
would with technical or teaching skills throughout the course of the case. Postoperative debrief-
ings similarly might incorporate the domains of NTS into the discussion both as an evaluative
tool as well as planning for future improvements. Overall, the goal is to embed and normalize
the language of NTS into the feedback that team members are providing to one another on a
case-to-case basis.
Beyond the OR, the language of NTS provides valuable input into teamwork development
in other arenas of surgical care. Attending surgeons and senior-level trainees can incorporate
the language of NTS into their discussions of team performance, evaluative feedback, and peer
evaluation. Simulation sessions should capitalize on the psychological safety created by limiting
potential adverse patient-level consequences to train and test NTS domains in parallel to tech-
nical and cognitive skill development. Finally, quality and safety initiatives as well as morbidity
and mortality conferences should include discussions of the nature of NTS in both assessing out-
comes as well as developing novel initiatives to improve the care delivery process and culture
of the workplace. See Table 6 for a complete list of suggestions for incorporating NTS training
across the trajectory of surgical education.
Connected fields of research
The discussion of NTS of an individual or a team is connected to much of the additional

research regarding the psychology of surgery and high performing teams. Four notable areas
of overlap include the role of cognitive biases in surgical decision making, the nature of emo-
tional intelligence in a surgical context, the construct of psychological safety, and artificial intel-
ligence/virtual reality (VR). We present these areas given their rising prominence and relevance
to NTS as well as to simulate future research endeavors.
Cognitive bias
Separate from the work around implicit bias, which affects perception and interaction with
other individuals, cognitive bias research focuses on the way humans perceive and interpret
data, especially in high pressure situations.195 The complexity and volume of information as
well as time-sensitivity for many critical decisions in surgery make many of these decisions
prone to reliance on heuristics and thus the introduction of bias. Indeed, a prospective review
of 736 general surgical cases found that cognitive bias was present in 33% of cases with doc-
umented postoperative complications.196 Another study demonstrated that as many as 74% of
cases of diagnostic error involved at least one error attributable to cognitive bias.197 In surgery
Table 6
Recommendations for incorporation of non-technical skill training by postgraduate year of surgical training.
United States Suggestions for Integrating NOTSS

Postgraduate Level of
Training
PGY1 • Intern orientation programs

• “Leading up” training programs to equip junior residents with skills to provide
non-technical feedback to more senior residents in a non-confrontational manner
• Utilization of NOTSS as framework for providing feedback to senior residents regarding
impact of non-technical skills and development of team-based psychological safety (eg,
include interns in 360 feedback process with components of leadership,
communication, teamwork, and situational awareness included)
PGY2-3 • Increased awareness and iterative feedback about NTS performance

• Increased autonomy and decreased cognitive load of surgical care both in and out of
the operating room should allow for increased focus on NTS
PGY4-5 • Leadership curriculum emphasizing NTS

• Teaching NTS to junior residents (similar to how technical skills are passed along)
• Focus on operative decision making in preparation for independent practice
Fellow • Focus of fellowship training, especially those aspects of NTS that are most pertinent to
that specialty
• Multisource feedback (last chance for formative change in a training context)
• Formal and informal teaching and training of residents in non-technical skills through
interactions between fellow and more junior trainees
NTS, non-technical skills; PGY, postgraduate year.
specifically, biases due to anchoring, in which all subsequent reasoning is based on an initial
piece of information, or availability, in which reasoning is based on recent experiences, tend to
be most prevalent.43 Suggestions for improving cognitive biases within surgery are mostly de-
rived from successful measures found in other industries: structured case format presentations,
standardized questions, checklists, mindfulness training, and avoidance of emotion-evoking lan-
guage.43 Fighting these pitfalls of “fast thinking” in a field that often demands alacrity requires
the metacognition and regular reevaluation of any situation demanded by the NOTSS domains of
decision making and situational awareness, respectively. Future research may consider whether
the NOTSS taxonomy can discriminate between surgeon performance before and after cognitive
bias training as a potential measure of training effectiveness. Vice versa, courses focused on the
development of NTS may lead to reduced impact of prevalent cognitive biases and associated
errors.
Emotional intelligence
Another notable overlap with NTS in surgery involves emotional intelligence, or the ability to
recognize and understand one’s own emotions as well as the emotions of others and make deci-
sions based on such understanding.198 Popularized within the business literature, emotional in-
telligence was found to positively affect teamwork, communication, and leadership within health
care in a recent systematic review.199 More specifically, surgeons with higher emotional quo-
tient scores are more likely to have higher patient satisfaction scores200 and to perform self-
assessments consistent with independent raters.201 Some have even called for formal emotional
intelligence training, reporting that it functions as an underpinning for the core Accreditation
Council for Graduate Medical Education (ACGME) competencies of surgical training.202 A simi-
lar argument could be made for emotional intelligence serving as a fundamental component of
the domains in the NOTSS taxonomy. A surgeon’s ability to establish a shared understanding of
a situation, coordinate team activities, support others, cope under pressure, and maintain high
standards of personal behavior are all affected by the way one reads his or her own emotions as
well as those of the team. Furthermore, surgeons are not immune to unconscious bias and their
situational awareness and decision making may be influenced by interpretation of team mem-
ber tone, body language, and abilities. Given this overlap, we suggest that emotional intelligence
training might be measured for effectiveness using a NOTSS rating system. Similarly, behavioral
anchors of the NOTSS taxonomy may provide guidance for individuals seeking to target emo-
tional intelligence training at meaningful components of surgical behavior.
Psychological safety
Psychological safety, the concept that all team members feel supported and safe to take in-
terpersonal risks and voice appropriate concerns,26 has also exploded into the literature across
multiple fields, including health care, as a measure and mechanism for encouraging team perfor-
mance. Unfortunately, surgical teams have frequently been highlighted in this work for negative
reasons. For example, the inherent hierarchy of most ORs, with the surgeon often positioned as
the de facto leader can result in a “dangerous silence [from non-surgeon team members], which
may reflect an inappropriate safety climate.”25 , 26 This dominance of interpersonal dynamics un-
dermines effective collaboration and may repress an individual’s ability to express concern or
“speak up” about case or patient specifics until it is too late.203 Fortunately, structured teamwork
training programs for operative staff have demonstrated the potential to improve psychological
safety and, concomitantly, medical error.204 Although historically viewed as a solo practitioner’s
sport, surgery has increasingly become viewed as necessarily requiring a broad system of spe-
cialized individuals working in concert. Focusing on the relational coordination between these
members may have profound effects on how the team performs under pressure or in unpre-
dictable situations.205 Often the de facto leaders in the multiple contexts, surgeons would do
well to be acquainted with techniques to foster psychological safety on their teams both in and
out of the OR.
Human factors research
When contemplating the role of team-performance training within the working environment
it is useful to consider this within the context of human factors. Human factors are a scientific
discipline that focuses upon the interaction between humans and their working environment.
It utilizes disciplines such as physiology, informatics technology, engineering, psychology, and
sociology to analyze this interaction and focuses upon improving the axis between performance,
environment, and wellbeing to maximize outcomes.206 , 207 In the context of surgery, this does
not just mean patient outcomes and optimized systems performance, but also the wellbeing
of the professionals working within it. Patient safety initiatives, enhancing team performance,
and NTS can all be viewed as direct applications of human factors science.207 Through this lens
is it easier to unite the concepts such as decision making, communication and teamwork, and
leadership with the practicalities of the pre-theatre surgical checklist, scrub team counts, and
postoperative debriefs in unison as opposed to individual components.
Artificial intelligence for enhanced team performance analysis
Although in its infancy, the field of artificial intelligence within surgery has rapidly expanded
and continues to accelerate at an ever-increasing rate.208 In this context utilization and appli-
cation of computer vision, machine learning, and natural language processing have massive po-
tential to transform team development. Computer vision and machine learning techniques have
already been created to enable processing mapping and annotation of key elements of laparo-
scopic and robotic procedures with demonstrable success.209-211 This has enabled discussion sur-
rounding the benefits and application of such resources. The ability to predict the feasibility or
safety of key performative steps, indications for specific maneuvers, or even early detection of
adverse events such as hemorrhage, could be extended beyond the operative field into the wider
operative team.206 , 208 , 210 Such adjuncts to performance could improve patient outcomes through
real-time adaptation of team approach, re-alignment of the shared mental model, or the creation
of individual performance profiles for team members. Prospectively, such systems could be used
in team briefing to aid establishing roles and predicting critical points in the case ahead. Intraop-
eratively, automated alert of hemorrhage or indication of unanticipated difficulty could improve
a team’s situational awareness, allowing for real-time adjustments. Retrospectively, such tech-
nology could be used to analyze data to assess NTS from OR footage, individual performance,
and behavioral markers of stress or cognitive overload212 , 213 highlighted, allowing for training
and development.
There is increasing capability of technology to capture, record, store, and analyze “big data”
from OR performance. This has feasibly allowed for real-time, non-invasive analysis of any given
procedure at any time of day, regardless of case or personnel. One such example of machine
learning to the OR is the OR Black Box,206 , 207 loosely based upon multiple data capture systems
in aviation.214 The OR Black Box group has demonstrated that such a program, once deemed
institutionally acceptable, may contain transformational potential.207 , 215 By allowing for syn-
chronous patient, laparoscopic, and OR footage data to be recorded and processed, the OR Black
Box harnesses that information via machine learning and expert raters to provide focused, ret-
rospective post hoc reviews of operative team and individual performance with reported mean-
ingful outcomes and benefits.207 , 214 , 215
Other artificial intelligence applications may allow for large volumes of operative team per-
formance data that otherwise would take weeks, months, or years to be processed objectively,
and that for the most part currently go un-utilized.216 Although speculative, it is exciting to an-
ticipate the future of AI systems being incorporated into surgical coaching, team training, and
human factors analytics, especially if used in tandem with video-based monitoring and simula-
tion.
Virtual reality
VR platforms also demonstrate fantastic potential for enhancing team training and perfor-
mance. VR can be considered “an immersive, completely artificial computer-simulated image
and environment with real-time interaction.”217 Virtual reality technologies have already been
utilized within operative teaching and training through immersive viewing of procedures as
well as operative rehearsal for challenging training procedures within specialties such as neu-
rosurgery.218 This allows for high-risk segments of a procedure to be rehearsed with any mis-
steps or errors being identified and debriefed and new approaches trailed and adopted within
a safe space.219 , 220 VR training also allows for real time interaction between day-to-day team
members but also uniquely removes geographical limitations and could allow for subspecialist
involvement, guidance, or assessment to occur from elsewhere. Such approaches have already
been utilized in tele-surgery and tele-mentoring, but VR allows this to extend beyond current
digital limitations.221-223 VR simulation team training also allows for multiple virtual patients,
virtual pathology, and virtual team members enabling multiple scenarios of the same case to be
reviewed. Such practice and preparation may allow for preparedness evaluation, team cohesion
via shared mental models, and reduced stress and discordance. Early research has demonstrated
promising results as an adjunct to team training, especially those that incorporate 360-degree
immersive video technology, in both trauma and operative hospital environments with high sat-
isfaction scores and skills improvement.222 , 224 , 225 It is likely as VR technology advances and
becomes increasingly accessible, it will grow in its integration into team training environments.
Mixed Reality (MR) is the blending of physical and digital worlds, with these 2 realities defin-
ing the polar ends of a spectrum known as the mixed reality spectrum. Graphics overlaid on
video streams of the physical world is augmented reality (AR), compared with a fully digital VR
experiences detailed above. MR covers experiences between these 2 extremes. AR, MR, and VR
technologies are already being used for training in space exploration, industry, and health care
fields.
Surgical sabermetrics
Sabermetrics, popularized by the Michael Lewis novel Moneyball and film bearing the same
title, is a technique for the application of data science to monitor, analyze, and enhance the per-
formance of professionals.226 , 227 For professional baseball players, elements of sabermetrics for a
pitcher might include data points regarding arm angle, body mechanics, exit velocity, and pitch
rotational speed.226 , 228 Operations are similarly dynamic, high intensity, and requiring utmost
precision with taxing requirements on surgeon’s physical and cognitive capacities. This parallel
has led to the early application of performance data science to surgeon performance enhance-
ment. Regarding team performance, surgical sabermetrics facilitate analysis of both technical
performance elements (eg, needle angle, wrist supination, posture, instrument exchanges, effi-
ciency measures) and non-technical performance (eg, cognitive load, tone, interruptions), which
have previously been challenging to quantify. Current areas of surgical sabermetrics research also
overlap with many of the aforementioned topics, interweaving with components of audio-visual
artificial intelligence collected via OR Blackbox to provide detailed performance reports, assess-
ing learners in simulated and virtual reality-based training, or enhancing assessment and impact
of behavioral interventions such as coaching.229-231
Additional barriers
Currently, major barriers to NTS development and team training include the lack of unified
strategy, rhetoric, or funding on a national and institutional level. In both the UK and USA, lack
of access to simulation facilities, expertise, and a low clinician engagement have resulted in
mostly voluntary training for NTS, resulting in highly variable expertise and assessment abilities.
The above innovations and developments do not take into account the need for significant time
and financial investment, which may particularly disadvantage individuals practicing in resource
limited settings. Globally, there are numerous teams striving within inadequate systems without
routine access to electricity, internet, data storage, and video-based technology. The NOTSS tax-
onomy has been adjusted with the RCSEd to consider surgical practice in such variable resource
environments but continued innovation and implementation of structured team training needs
to be prioritized and developed.232
Conclusion
In summary, team performance can be enhanced through self-directed learning, formal

course-based skills acquisition, and deliberate practice by the surgeon and the members of the
surgical team. As described, there are a growing number of resources available to facilitate de-
velopment of such skills with considerable evidence to support their implantation and sustained
use. Due to the migration of such resources online, accessibility to team training improvement
resources has never been greater. However, the demands of current surgical training and as-
sociated financial costs continue to be challenging. Surgical teams, and health care staff more
broadly, have never needed greater support and training to enable them to perform at their
best. Training programs should aim to facilitate acquisition of individual and team-based NTS
through continued funding and promotion of the frameworks, programs, and courses. Improved
NTS of surgeons stand to benefit patients, coworkers, and the burdened health care systems
which attempt to improve surgical care while promoting provider well-being.
References
1. Jung CS, Lee SY. The Hawthorne studies revisited: evidence from the U.S. federal workforce. Adm Soc. 2015;47:507–
531. doi:10.1177/0095399712459731.
2. Jones SRG. Worker interdependence and output: the Hawthorne studies reevaluated. Am Soc Rev. 1990;55:176.
doi:10.2307/2095625.
3. Mathieu JE, Wolfson MA, Park S. The evolution of work team research since Hawthorne. Am Psychologist.
2018;73:308–321. doi:10.1037/AMP0 0 0 0255.
4. Goodwin GF, Blacksmith N, Coats MR. The science of teams in the military: contributions from over 60 years of
research. Am Psychologist. 2018;73:322–333. doi:10.1037/amp0 0 0 0259.
5. Sundstrom E, de Meuse KP, Futrell D. Work teams: applications and effectiveness. Am Psychologist. 1990;45:120–
133. doi:10.1037/0 0 03-066X.45.2.120.
6. Helmreich RL, Merritt AC, Wilhelm JA. The evolution of crew resource management training in commercial aviation.
Int J Aviat Psychol. 1999;9:19–32. doi:10.1207/S15327108IJAP0901_2.
7. Alonso A, Baker DP, Holtzman A, et al. Reducing medical error in the military health system: how can team training
help? Hum Resour Manag Rev. 2006;16:396–415. doi:10.1016/J.HRMR.20 06.05.0 06.
8. Lacerenza CN, Marlow SL, Tannenbaum SI, Salas E. Team development interventions: evidence-based approaches
for improving teamwork. Am Psychologist. 2018;73:517–531. doi:10.1037/AMP0 0 0 0295.
9. Shuffler ML, DiazGranados D, Salas E. There’s a science for that: team development interventions in organizations.
Curr Dir Psychol Sci. 2011;20:365–372. doi:10.1177/0963721411422054.
10. Salas E, Reyes DL, McDaniel SH. The science of teamwork: progress, reflections, and the road ahead. Am Psychologist.
2018;73:93–600. doi:10.1037/AMP0000334.
11. Salas E, Tannenbaum SI, Kraiger K, Smith-Jentsch KA. The science of training and development in organizations:
what matters in practice. Psychol Sci Publ Int Suppl. 2012;13:74–101. doi:10.1177/1529100612436661.
12. Brown KG, Sitzmann T. Training and employee development for improved performance. APA Handbook of Industrial
and Organizational Psychology. Vol 2: Selecting and developing members for the organization. Published online June
7, 2010:469-503. doi:10.1037/12170-016
13. Baldwin TT, Ford KJ. Transfer of training: a review and directions for future research. Pers Psychol. 1988;41:63–105.
doi:10.1111/J.1744-6570.1988.TB00632.X.
14. Blume BD, Ford JK, Baldwin TT, Huang JL. Transfer of training: a meta-analytic review. J Manag. 2009;36:1065–1105.
doi:10.1177/0149206309352880.
15. Brown J. Training needs assessment: a must for developing an effective training program. Public Pers Manag.
20 02;31:569–578. doi:10.1177/0 0910260 020310 0412.
16. Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis,
and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–284. doi:10.1037/0033-2909.119.2.254.
17. Keith N, Frese M. Effectiveness of error management training: a meta-analysis. J Appl Psychol. 2008;93:59–69.
doi:10.1037/0021-9010.93.1.59.
18. Kjellström S, Stålne K, Törnblom O. Six ways of understanding leadership development: an exploration of increasing
complexity. Leadership. 2020;16:434–460. doi:10.1177/1742715020926731.
19. Lacerenza CN, Reyes DL, Marlow SL, Joseph DL, Salas E. Leadership training design, delivery, and implementation:
a meta-analysis. J Appl Psychol. 2017;102:1686–1718. doi:10.1037/APL0 0 0 0241.
20. Mumford MD, Marks MA, Connelly MS, Zaccaro SJ, Reiter-Palmon R. Development of leadership skills: experience
and timing. Leadersh Q. 20 0 0;11:87–114.
21. Stewart J. Transformational leadership: an evolving concept examined through the works of burns, bass, Avolio, and
Leith wood | Canadian journal of educational administration and policy. Can J Educ Adm Policy. 2006;54. Accessed
April 12, 2022Available at:. https://journalhosting.ucalgary.ca/index.php/cjeap/article/view/42735 .
22. Judge TA, Piccolo RF, Ilies R. The forgotten ones? the validity of consideration and initiating structure in leadership
research. J Appl Psychol. 2004;89:36–51. doi:10.1037/0021-9010.89.1.36.
23. Klein C, DiazGranados D, Salas E, et al. Does team building work? Small Group Res. 2009;40:181–222. doi:10.1177/
1046496408328821.
24. Frazier ML, Fainshmidt S, Klinger RL, Pezeshkan A, Vracheva V. Psychological safety: a meta-analytic review and
extension. Pers Psychol. 2017;70:113–165. doi:10.1111/peps.12183.
25. Valentine MA, Nembhard IM, Edmondson AC. Measuring teamwork in health care settings. Med Care. 2015;53:e16–
e30. doi:10.1097/MLR.0b013e31827feef6.
26. Edmondson A. Psychological safety and learning behavior in work teams. Adm Sci Q. 2016;44:350–383. doi:10.2307/
2666999.
27. Cummings TG, Worley CG. Organization Development & Change. 9th ed. Cengage Learning; 2009. Available at: www.
ichapters.com, Accessed April 12, 2022.
28. Sawyer RK. Teaching creativity in art and design studio classes: a systematic literature review. Educ Res Rev.
2017;22:99–113. doi:10.1016/j.edurev.2017.07.002.
29. Klein HJ, Wesson MJ, Hollenbeck JR, Alge BJ. Goal commitment and the goal-setting process: conceptual clarification
and empirical synthesis. J Appl Psychol. 1999;84:885–896. doi:10.1037/0021-9010.84.6.885.
30. Stevens MJ, Campion MA. The knowledge, skill, and ability requirements for teamwork: implications for human
resource management. J Manag. 1994;20:503–530. Accessed April 12, 2022Available at:. https://www.krannert.
purdue.edu/faculty/campionm/Knowledge_Skill_Ability.pdf .
31. Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? a meta-analysis. Hum Factors.
2013;55:231–245. doi:10.1177/0018720812448394.
32. DeChurch LA, Mesmer-Magnus JR. The cognitive underpinnings of effective teamwork: a meta-analysis. J Appl Psy-
chol. 2010;95:32–53. doi:10.1037/A0017328.
33. Flanagan TA, Runde CE. Hidden potential: Embracing conflict can pay off for teams. Leadersh Action. 2008;28:8–12.
doi:10.1002/LIA.1241.
34. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc. 2007;2:115–125. doi:10.
1097/SIH.0B013E3180315539.
35. Smith-Jentsch KA. How to conduct a high impact team self-evaluation session. Organ Dyn. 2018;47:107–114. doi:10.
1016/J.ORGDYN.2017.11.002.
36. Ellis S, Davidi I. After-event reviews: drawing lessons from successful and failed experience. J Appl Psychol.
2005;90:857–871. doi:10.1037/0021-9010.90.5.857.
37. Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hos-
pitals. Surgery. 2003;133:614–621. doi:10.1067/msy.2003.169.
38. Mitchell PH, Wynia MK, Golden R, et al. Core principles & values of effective team-based health care. NAM Perspect.
2012;2. doi:10.31478/201210c.
39. Marlow SL, Hughes AM, Sonesh SC, et al. A systematic review of team training in health care: ten questions. Jt.
Comm. J. Qual. Patient Saf.. 2017;43:197–204. doi:10.1016/j.jcjq.2016.12.004.
40. Audet AM, Davis K, Schoenbaum SC. Adoption of patient-centered care practices by physicians: results from a na-
tional survey. Arch Intern Med. 2006;166:754–759. doi:10.1001/archinte.166.7.754.
41. Naylor M, Coburn K, Kurtzman E Inter-Professional Team-Based Primary Care for Chronically Ill Adults: State of the
Science. Vol 2. National Academy of Medicine; 2010.
42. Fulmer T, Hyer K, Flaherty E, et al. Geriatric interdisciplinary team training program: evaluation results. J Aging
Health. 2005;17:443–470. doi:10.1177/0898264305277962.
43. Hughes AM, Gregory ME, Joseph DL, et al. Saving lives: a meta-analysis of team training in healthcare. J Appl Psychol.
2016;101:1266–1304. doi:10.1037/apl0 0 0 0120.
44. Frich JC, Brewster AL, Cherlin EJ, Bradley EH. Leadership development programs for physicians: a systematic review.
J Gener Intern Med. 2015;30:656–674. doi:10.1007/s11606- 014- 3141- 1.
45. Blumenthal DM, Bernard K, Bohnen J, Bohmer R. Addressing the leadership gap in medicine: residents’
need for systematic leadership development training. Academic Medicine. 2012;87:513–522. doi:10.1097/ACM.
0b013e31824a0c47.
46. Curry LA, Brault MA, Linnander EL, et al. Influencing organisational culture to improve hospital performance in care
of patients with acute myocardial infarction: a mixed-methods intervention study. BMJ Qual Safety. 2018;27:207–
217. doi:10.1136/BMJQS-2017-006989.
47. Buljac-Samardzic M, Doekhie KD, van Wijngaarden JDH. Interventions to improve team effectiveness within health
care: a systematic review of the past decade. Human Resour Health. 2020;18. doi:10.1186/s12960- 019- 0411- 3.
48. Salas E, Rozell D, Mullen B, Driskell JE. The effect of team building on performance: an integration. Small Group Res.
2016;30:309–329. doi:10.1177/1046496499030 0 0303.
49. Kilpatrick K, Paquette L, Jabbour M, et al. Systematic review of the characteristics of brief team interventions to
clarify roles and improve functioning in healthcare teams. PLoS ONE. 2020;15. doi:10.1371/journal.pone.0234416.
50. Garden AL, le Fevre DM, Waddington HL, Weller JM. Debriefing after simulation-based non-technical skill train-
ing in healthcare: a systematic review of effective practice. Anaesth Intensive Care. 2015;43:300–308. doi:10.1177/
0310057x1504300303.
51. Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing with good judgment: combining rigorous feed-
back with genuine inquiry. Anesthesiol Clin. 2007;25:361–376. doi:10.1016/J.ANCLIN.20 07.03.0 07.
52. Rudolph JW, Simon R, Dufresne RL, Raemer DB. There’s no such thing as “nonjudgmental” debriefing: a theory and
method for debriefing with good judgment. Simul Healthc. 2006;1:49–55. doi:10.1097/01266021-20 060 0110-0 0 0 06.
53. LeSage P. Crew Resource Management : Principles and Practice. Jones & Bartlett Learning; 2011.
54. Diehl A. Crew resource management...it’s not just for fliers anymore. Flying safety. Published online 1994.
doi:10.2495/RISK140221
55. Catchpole KR, Dale TJ, Hirst DG, Smith JP, Giddings TAEB. A multicenter trial of aviation-style training for surgical
teams. J Patient Saf. 2010;6:180–186. doi:10.1097/PTS.0B013E3181F100EA.
56. Gore DC, Powell JM, Baer JG, et al. Crew resource management improved perception of patient safety in the oper-
ating room. Am J Med Qual. 2010;25:60–63. doi:10.1177/1062860609351236.
57. Wolf FA, Way LW, Stewart L. The efficacy of medical team training: improved team performance and de-
creased operating room delays: a detailed analysis of 4863 cases. Ann Surg. 2010;252:477–483. doi:10.1097/SLA.
0B013E3181F1C091.
58. Young-Xu Y, Neily J, Mills PD, et al. Association between implementation of a medical team training program and
surgical morbidity. Arch Surg. 2011;146:1368–1373. doi:10.1001/ARCHSURG.2011.762.
59. Wakeman D, Langham MR. Creating a safer operating room: groups, team dynamics and crew resource manage-
ment principles. Semin Pediatr Surg. 2018;27:107–113. doi:10.1053/J.SEMPEDSURG.2018.02.008.
60. Awad SS, Fagan SP, Bellows C, et al. Bridging the communication gap in the operating room with medical team
training. Am J Surg. 2005;190:770–774. doi:10.1016/J.AMJSURG.2005.07.018.
61. King HB, Battles J, Baker DP, et al. TeamSTEPPSTM : team strategies and tools to enhance performance and pa-
tient safety. Adv Patient Saf. 2008;3. Accessed April 12, 2022Available at:. https://www.ncbi.nlm.nih.gov/books/
NBK43686/ .
62. Capella J, Smith S, Philp A, et al. Teamwork training improves the clinical care of trauma patients. J Surg Educ.
2010;67:439–443. doi:10.1016/J.JSURG.2010.06.006.
63. Fischer MM, Tubb CC, Brennan JA, Soderdahl DW, Johnson AE. Implementation of TeamSTEPPS at a level-1 military
trauma center: the San Antonio military medical center experience. US Army Med Dep J. Published online October
2015:75-79. Accessed April 12, 2022. Available at: https://pubmed.ncbi.nlm.nih.gov/26606411/
64. Weld LR, Stringer MT, Ebertowski JS, et al. TeamSTEPPS improves operating room efficiency and patient safety. Am
J Med Qual. 2016;31:408–414. doi:10.1177/1062860615583671.
65. Forse RA, Bramble JD, McQuillan R. Team training can improve operating room performance. Surgery. 2011;150:771–
778. doi:10.1016/J.SURG.2011.07.076.
66. Rhee AJ, Valentin-Salgado Y, Eshak D, et al. Team training in the perioperative arena: a methodology for implemen-
tation and auditing behavior. Am J Med Qual. 2017;32:369–375. doi:10.1177/1062860616662703.
67. Kohn LT, Corrigan JM, Donaldson MS, eds. To err is human: building a safer health system.; 20 0 0. doi:10.17226/9728
68. Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf.
2014;23:359–372.
69. Noonan M, Olaussen A, Mathew J, Mitra B, Smit V, Fitzgerald M. What is the clinical evidence supporting trauma
team training (TTT): a systematic review and meta-analysis. Medicina (Kaunas). 2019;55(9):551.
70. McLaughlin C, Barry W, Barin E, et al. Multidisciplinary simulation-based team training for trauma resuscitation: a
scoping review. J Surg Educ. 2019;76:1669–1680.
71. Gjeraa K, Møller TP, Østergaard D. Efficacy of simulation-based trauma team training of non-technical skills. A
systematic review. Acta Anaesthesiol Scand. 2014;58:775–787.
72. Merién AER, van de Ven J, Mol BW, Houterman S, Oei SG. Multidisciplinary team training in a simulation setting
for acute obstetric emergencies a systematic review. Obstetr Gynecol. 2010;115:1021–1031.
73. Phipps Maureen G, Lindquist David G, McConaughey Edie, O’Brien James A, Raker Christina AS, Paglia Michael J.
Outcomes from a labor and delivery team training program with simulation component. Am J Obstet Gynecol.
2012;206:3–9.
74. Low XM, Horrigan D, Brewster DJ. The effects of team-training in intensive care medicine: a narrative review. J Crit
Care. 2018;48:283–289.
75. Cumin D, Boyd MJ, Webster CS, Weller JM. A systematic review of simulation for multidisciplinary team training in
operating rooms. Simul Healthc. 2013;8:171–179. doi:10.1097/SIH.0b013e31827e2f4c.
76. Tan SB, Pena G, Altree M, Maddern GJ. Multidisciplinary team simulation for the operating theatre: a review of the
literature. ANZ J Surg. 2014;84:515–522. doi:10.1111/ans.12478.
77. Institute of Medicine Committee on the Health Professions Education Summit. Health Professions Education: A Bridge
to Quality. National Academy Press; 2003.
78. Robertson JM, Dias RD, Yule S, Smink DS. Operating room team training with simulation: a systematic review. J
Laparoendosc Adv Surg Tech A. 2017;27:475–480. doi:10.1089/lap.2017.0043.
79. Stevens Louis-Mathieu MDP, Cooper Jeffrey BP, Raemer Daniel BP, et al. Educational program in crisis management
for cardiac surgery teams including high realism simulation. J Thorac Cardiovasc Surg. 2012;144:17–24.
80. Nicksa GA, Anderson C, Fidler R, Stewart L. Innovative approach using interprofessional simulation to educate sur-
gical residents in technical and nontechnical skills in high-risk clinical scenarios. JAMA Surg. 2015;150:201–207.
81. Stewart-Parker Emma M, Galloway R, Vig S. S-TEAMS: a truly multiprofessional course focusing on nontechnical
skills to improve patient safety in the operating theater. J Surg Educ. 2016;74:137–144.
82. Undre S, Koutantji M, Sevdalis N, et al. Multidisciplinary crisis simulations: the way forward for training surgical
teams. World J Surg. 2007;31:1843–1853.
83. Arriaga AF, Gawande AA, Raemer DB, et al. Pilot testing of a model for insurer-driven, large-scale multicenter sim-
ulation training for operating room teams. Ann Surg. 2014;259:403–410. doi:10.1097/SLA.0 0 0 0 0 0 0 0 0 0 0 0 0342.
84. Volk MS, Ward J, Irias N, Navedo A, Pollart J, Weinstock PH. Using medical simulation to teach crisis resource
management and decision-making skills to otolaryngology housestaff. Otolaryngol Head Neck Surg. 2011;145:35–42.
85. Phitayakorn Roy MDMF, Minehart Rebecca D, Hemingway Maureen W, Pian-Smith May CM, Petrusa E. The rela-
tionship between intraoperative teamwork and management skills in patient care. Surgery. 2015;158:1434–1440.
86. Paige John MD, Kozmenko Valeriy MD, Morgan Barbara MD, et al. From the flight deck to the operating room: an
initial pilot study of the feasibility and potential impact of true interdisciplinary team training using high-fidelity
simulation. J Surg Educ. 2007;64:369–377.
87. Kjellin A, Hedman L, Escher C, Felländer-Tsai L. Hybrid simulation: bringing motivation to the art of teamwork
training in the operating room. Scand J Surg. 2014;103:232–236.
88. Paige John T, Kozmenko V, Yang T, et al. High-fidelity, simulation-based, interdisciplinary operating room team
training at the point of care. Surgery. 2009;145:138–146.
89. Truong H, Sullivan AM, Abu-Nuwar MR, et al. Operating room team training using simulation: Hope or hype? Am J
Surg. 2021;222(6):1146–1153.
90. Garbee DD, Paige JT, Bonanno LS, Sonesh SC. Comprehensive Healthcare Simulation: InterProfessional Team Training
and Simulation. Springer; 2020.
91. Robertson JM, Klainer SB, Bradley DM, Yule S, Smink DS. Simulation-based training for interprofessional teams of
practicing clinicians. In: Comprehensive Healthcare Simulation: InterProfessional Team Training and Simulation. Com-
prehensive Healthcare Simulation. Springer International Publishing; 2020:211–223.
92. Forse R, Armour MDPF, Bramble JDP, McQuillan R. Team training can improve operating room performance. Surgery.
2011;150:771–778.
93. Young-Xu Y, Neily J, Mills PD, et al. Association between implementation of a medical team training program and
surgical morbidity. Arch Surg. 2011;146:1368–1373.
94. Neily J, Mills PD, Young-Xu Y, et al. Association between implementation of a medical team training program and
surgical mortality. JAMA. 2010;304:1693–1700. doi:10.1001/jama.2010.1506.
95. Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More than one way to debrief a critical review of health-
care simulation debriefing methods. Simul Healthc. 2016;11:209–217.
96. Altpeter T, Luckhardt K, Lewis JN, Harken AH, Polk HC. Expanded surgical time out: a key to real-time data collec-
tion and quality improvement. J Am Coll Surg. 2007;204:527–532. doi:10.1016/J.JAMCOLLSURG.2007.01.009.
97. Backster A, Teo A, Swift M, Polk HC, Harken AH. Transforming the surgical “time-out” into a comprehensive
“preparatory pause.”. J Card Surg. 2007;22:410–416. doi:10.1111/J.1540-8191.20 07.0 0435.X.
98. Lingard L, Regehr G, Cartmill C, et al. Evaluation of a preoperative team briefing: a new communication routine
results in improved clinical practice. BMJ Qual Saf. 2011;20:475–482. doi:10.1136/BMJQS.2009.032326.
99. Norton E. Implementing the universal protocol hospital-wide. AORN J. 2007;85:1187–1197. doi:10.1016/J.AORN.2007.
03.002.
100. Lingard L, Regehr G, Orser B, et al. Evaluation of a preoperative checklist and team briefing among surgeons, nurses,
and anesthesiologists to reduce failures in communication. Arch Surg. 2008;143:12–17. doi:10.10 01/ARCHSURG.20 07.
21.
101. Nundy S, Mukherjee A, Sexton JB, et al. Impact of preoperative briefings on operating room delays: a preliminary
report. Arch Surg. 2008;143:1068–1072. doi:10.1001/ARCHSURG.143.11.1068.
102. DeFontes J, Surbida S. Preoperative safety briefing project. Perm J. 2004;8:21. doi:10.7812/tpp/03-129.
103. Allard J, Bleakley A, Hobbs A, Coombes L. Pre-surgery briefings and safety climate in the operating theatre. BMJ
Qual Saf. 2011;20:711–717. doi:10.1136/BMJQS.2009.032672.
104. Paull DE, Mazzia LM, Wood SD, et al. Briefing guide study: preoperative briefing and postoperative debriefing
checklists in the Veterans Health Administration medical team training program. Am J Surg. 2010;200:620–623.
doi:10.1016/J.AMJSURG.2010.07.011.
105. Henrickson SE, Wadhera RK, ElBardissi AW, Wiegmann DA, Sundt TM. Development and pilot evaluation of
a preoperative briefing protocol for cardiovascular surgery. J Am Coll Surg. 2009;208:1115–1123. doi:10.1016/J.
JAMCOLLSURG.2009.01.037.
106. Charlton JE. Checklists and patient safety. Anaesthesia. 1990;45:425–426. doi:10.1111/J.1365-2044.1990.TB14325.X.
107. Piotrowski MM, Hinshaw DB. The safety checklist program: creating a culture of safety in intensive care units. Jt
Comm J Qual Improv. 2002;28:306–315. doi:10.1016/S1070- 3241(02)28030- X.
108. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global
population. N Engl J Med. 2009;360:491–499. doi:10.1056/NEJMSA0810119/SUPPL_FILE/NEJM_HAYNES_491SA1.PDF.
109. de Vries EN, Prins HA, Crolla RMPH, et al. Effect of a comprehensive surgical safety system on patient outcomes. N
Engl J Med. 2010;363:1928–1937. doi:10.1056/NEJMSA0911535/SUPPL_FILE/NEJMSA0911535_DISCLOSURES.PDF.
110. Askarian M, Kouchak F, Palenik CJ. Effect of surgical safety checklists on postoperative morbidity and mortal-
ity rates, Shiraz, Faghihy Hospital, a 1-year study. Qual Manag Health Care. 2011;20:293–297. doi:10.1097/QMH.
0B013E318231357C.
111. Haugen AS, Søfteland E, Almeland SK, et al. Effect of the World Health Organization checklist on patient
outcomes: a stepped wedge cluster randomized controlled trial. Ann Surg. 2015;261:821–828. doi:10.1097/SLA.
0 0 0 0 0 0 0 0 0 0 0 0 0716.
112. Bergs J, Hellings J, Cleemput I, et al. Systematic review and meta-analysis of the effect of the World Health Organi-
zation surgical safety checklist on postoperative complications. Br J Surg. 2014;101:150–158. doi:10.1002/BJS.9381.
113. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: a systematic review of impacts and implementation. BMJ Qual
Saf. 2014;23:299–318. doi:10.1136/BMJQS- 2012- 001797.
114. McDowell DS, McComb SA. Safety checklist briefings: a systematic review of the literature. AORN J. 2014;99. doi:10.
1016/J.AORN.2013.11.015.
115. Anderson KT, Bartz-Kurycki MA, Masada KM, et al. Decreasing intraoperative delays with meaningful use of the
surgical safety checklist. Surgery. 2018;163:259–263. doi:10.1016/J.SURG.2017.08.009.
116. Seme ME, Resch S, Haynes AB, et al. Adopting a surgical safety checklist could save money and improve the quality
of care in U.S. hospitals. Health Aff (Millwood). 2010;29:1593. doi:10.1377/HLTHAFF.2009.0709.
117. Reason J. Human error: models and management. Br Med J. 20 0 0;320:768. doi:10.1136/BMJ.320.7237.768.
118. Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and
communication in the operating room? a systematic review. Ann Surg. 2013;258:856–871. doi:10.1097/SLA.
0 0 0 0 0 0 0 0 0 0 0 0 0206.
119. Molina G, Jiang W, Edmondson L, et al. Implementation of the surgical safety checklist in south Carolina hospitals is
associated with improvement in perceived perioperative safety. J Am Coll Surg. 2016;222:725–736 e5. doi:10.1016/J.
JAMCOLLSURG.2015.12.052.
120. Haynes AB, Weiser TG, Berry WR, et al. Changes in safety attitude and relationship to decreased postoperative
morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf.
2011;20:102–107. doi:10.1136/BMJQS.20 09.040 022.
121. Urbach DR, Govindarajan A, Saskin R, Wilton AS, Baxter NN. Introduction of surgical safety checklists in
Ontario, Canada. N Engl J Med. 2014;370:1029–1038. doi:10.1056/NEJMSA1308261/SUPPL_FILE/NEJMSA1308261_
DISCLOSURES.PDF.
122. de Jager E, Gunnarsson R, Ho YH. Implementation of the World Health Organization surgical safety checklist cor-
relates with reduced surgical mortality and length of hospital admission in a high-income country. World J Surg.
2019;43:117–124. doi:10.10 07/S0 0268- 018- 4703- X.
123. Levy SM, Senter CE, Hawkins RB, et al. Implementing a surgical checklist: more than checking a box. Surgery.
2012;152:331–336. doi:10.1016/J.SURG.2012.05.034.
124. Haugen AS, Wæhle HV, Almeland SK, et al. Causal analysis of World Health Organization’s surgical safety check-
list implementation quality and impact on care processes and patient outcomes: secondary analysis from a
large stepped wedge cluster randomized controlled trial in Norway. Ann Surg. 2019;269:283–290. doi:10.1097/SLA.
0 0 0 0 0 0 0 0 0 0 0 02584.
125. Gillespie BM, Marshall A. Implementation of safety checklists in surgery: a realist synthesis of evidence. Implement
Sci. 2015;10. doi:10.1186/s13012-015-0319-9.
126. Putnam LR, Levy SM, Sajid M, et al. Multifaceted interventions improve adherence to the surgical checklist. Surgery.
2014;156:336–344. doi:10.1016/J.SURG.2014.03.032.
127. Arriaga AF, Bader AM, Wong JM, et al. Simulation-based trial of surgical-crisis checklists. N Engl J Med.
2013;368:246–253. doi:10.1056/NEJMSA1204720/SUPPL_FILE/NEJMSA1204720_DISCLOSURES.PDF.
128. Wood TC, Raison N, Haldar S, et al. Training tools for nontechnical skills for surgeons—a systematic review. J Surg
Educ. 2017;74:548–578. doi:10.1016/j.jsurg.2016.11.017.
129. Youngson GG. Teaching and assessing non-technical skills. Surgeon. 2011;9. doi:10.1016/j.surge.2010.11.004.
130. Mills P, Neily J, Dunn E. Teamwork and communication in surgical teams: implications for patient safety. J Am Coll
Surg. 2008;206:107–112. doi:10.1016/j.jamcollsurg.2007.06.281.
131. Neily J, Mills PD, Young-Xu Y, et al. Association between implementation of a medical team training program and
surgical mortality. JAMA. 2010;304:1693–1700. doi:10.1001/JAMA.2010.1506.
132. Gjeraa K, Spanager L, Konge L, Petersen RH, Østergaard D. Non-technical skills in minimally invasive surgery teams:
a systematic review. Surg Endoscopy. 2016;30:5185–5199. doi:10.10 07/s0 0464- 016- 4890- 1.
133. Mishra A, Catchpole K, Mcculloch P. The Oxford NOTECHS System: reliability and validity of a tool for measur-
ing teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009;18:104–108. doi:10.1136/QSHC.2007.
024760.
134. Morgan L, Pickering SP, Hadi M, et al. A combined teamwork training and work standardisation interven-
tion in operating theatres: controlled interrupted time series study. BMJ Qual Saf. 2015;24:111–119. doi:10.1136/
BMJQS- 2014- 003204.
135. Robertson ER, Hadi M, Morgan LJ, et al. Oxford NOTECHS II: a modified theatre team non-technical skills scoring
system. PLoS One. 2014;9. doi:10.1371/JOURNAL.PONE.0090320.
136. McCulloch P, Morgan L, New S, et al. Combining systems and teamwork approaches to enhance the effectiveness
of safety improvement interventions in surgery: the safer delivery of surgical services (S3) program. Ann Surg..
2017;265:90–96. doi:10.1097/SLA.0 0 0 0 0 0 0 0 0 0 0 01589.
137. Mishra A, Catchpole K, Dale T, McCulloch P. The influence of non-technical performance on technical outcome in
laparoscopic cholecystectomy. Surg Endosc. 2008;22:68–73. doi:10.10 07/S0 0464-0 07- 9346- 1.
138. Sharma B, Mishra A, Aggarwal R, Grantcharov TP. Non-technical skills assessment in surgery. Surg Oncol.
2011;20:169–177. doi:10.1016/J.SURONC.2010.10.001.
139. Nicksa GA, Anderson C, Fidler R, Stewart L. Innovative approach using interprofessional simulation to educate sur-
gical residents in technical and nontechnical skills in high-risk clinical scenarios. JAMA Surg. 2015;150:201–207.
doi:10.1001/jamasurg.2014.2235.
140. Undre S, Sevdalis N, Healey AN, Darzi A, Vincent CA. Teamwork in the operating theatre: cohesion or confusion? J
Eval Clin Pr. 2006;12:182–189. doi:10.1111/j.1365-2753.2006.00614.x.
141. Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N. Observational teamwork assessment for surgery: content valida-
tion and tool refinement. J Am Coll Surg. 2011;212. doi:10.1016/J.JAMCOLLSURG.2010.11.001.
142. Undre S, Koutantji M, Sevdalis N, et al. Multidisciplinary crisis simulations: the way forward for training surgical
teams. World J Surg. 2007;31:1843–1853. doi:10.1007/S00268-007- 9128- X.
143. Yule S, Flin R, Paterson-Brown S, Maran N. Non-technical skills for surgeons in the operating room: a review of the
literature. Surgery. 2006;139:140–149. doi:10.1016/J.SURG.2005.06.017.
144. Flin R, Yule S, McKenzie L, Paterson-Brown S, Maran N. Attitudes to teamwork and safety in the operating theatre.
Surgeon. 2006;4:145–151. doi:10.1016/S1479-666X(06)80084-3.
145. Wood TC, Raison N, Haldar S, et al. Training tools for nontechnical skills for surgeons—a systematic review. J Surg
Educ. 2017;74:548–578. doi:10.1016/j.jsurg.2016.11.017.
146. Yule S, Paterson-Brown S. Surgeons’ non-technical skills. Surg Clin North Am. 2012;92:37–50. doi:10.1016/J.SUC.2011.
11.004.
147. Pradarelli JC, Gupta A, Lipsitz S, et al. Assessment of the non-technical skills for surgeons (NOTSS) framework in
the USA. Br J Surg. 2020;107:1137–1144. doi:10.1002/bjs.11607.
148. Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons’ non-technical skills in the operat-
ing room: reliability testing of the NOTSS behavior rating system. World J Surg.. 2008;32:548–556. doi:10.1007/
S0 0268-0 07-9320-Z.
149. Jung JJ, Yule S, Boet S, Szasz P, Schulthess P, Grantcharov T. Nontechnical skill assessment of the collective surgical
team using the non-technical skills for surgeons (NOTSS) system. Ann Surg. 2020;272:1158–1163. doi:10.1097/SLA.
0 0 0 0 0 0 0 0 0 0 0 03250.
150. Jung JJ, Borkhoff CM, Jüni P, Grantcharov TP. Non-technical skills for surgeons (NOTSS): critical appraisal of its
measurement properties. Am J Surg. 2018;216:990–997. doi:10.1016/J.AMJSURG.2018.02.021.
151. Pradarelli JC, George E, Kavanagh J, Sonnay Y, Khoon TH, Havens JM. Training novice raters to assess nontechnical
skills of operating room teams. J Surg Educ. Published online 2020. doi:10.1016/j.jsurg.2020.07.042
152. Crossley J, Marriott J, Purdie H, Beard JD. Prospective observational study to evaluate NOTSS (Non-Technical Skills
for Surgeons) for assessing trainees’ non-technical performance in the operating theatre. Br J Surg. 2011;98:1010–
1020. doi:10.1002/bjs.7478.
153. Yule S, Flin R, Maran N, et al. Debriefing surgical trainees on non-technical skills (NOTSS). Cogni Technol Work.
2008;10:265–274. doi:10.1007/S10111- 007- 0085- 9.
154. Arora S, Miskovic D, Hull L, et al. Self vs expert assessment of technical and non-technical skills in high fidelity
simulation. Am J Surg. 2011;202:500–506. doi:10.1016/J.AMJSURG.2011.01.024.
155. Yamane M, Sugimoto S, Suzuki E, et al. Continuing surgical education of non-technical skills. Ann Med Surg.
2020;58:177. doi:10.1016/J.AMSU.2020.07.062.
156. Spanager L, Teglgaard Lyk-Jensen H, Dieckmann P, Wettergren A, Rosenberg J, Ostergaard D. Customization of a tool
to assess Danish surgeons’ non-technical skills in the operating room. Danish Med J. 2012;59. Accessed April 14,
2022Available at:. https://pubmed.ncbi.nlm.nih.gov/23171747/ .
157. Dedy NJ, Bonrath EM, Zevin B, Grantcharov TP. Teaching nontechnical skills in surgical residency: a systematic
review of current approaches and outcomes. Surgery. 2013;154:10 0 0–10 08. doi:10.1016/J.SURG.2013.04.034.
158. Parker SH, Yule S, Flin R, McKinley A. Towards a model of surgeons’ leadership in the operating room. BMJ Qual
Saf. 2011;20:570–579. doi:10.1136/BMJQS.2010.040295.
159. Parker SH, Flin R, McKinley A, Yule S. The Surgeons’ Leadership Inventory (SLI): a taxonomy and rating system for
surgeons’ intraoperative leadership skills. Am J Surg.. 2013;205:745–751. doi:10.1016/J.AMJSURG.2012.02.020.
160. Parker SH, Yule S, Flin R, McKinley A. Surgeons’ leadership in the operating room: an observational study. Am J
Surg. 2012;204:347–354. doi:10.1016/j.amjsurg.2011.03.009.
161. Arnold D, Fleshman JW. Leadership: leadership in the setting of the operating room surgical team. Clin Colon Rectal
Surg. 2020;33:191. doi:10.1055/S- 0040- 1709442.
162. Nurudeen SM, Kwakye G, Berry WR, et al. Can 360-degree reviews help surgeons? evaluation of multisource
feedback for surgeons in a multi-institutional quality improvement project. J Am Coll Surg. 2015;221:837–844.
doi:10.1016/j.jamcollsurg.2015.06.017.
163. Lagoo J, Berry W, Henrich N, Gawande A, Sato L, Haas S. Safely practicing in a new environment: a qualitative study
to inform physician onboarding practices. Jt. Comm. J. Qual. Patient Saf. 2020;46:314–320. doi:10.1016/j.jcjq.2020.03.
002.
164. Leuschner S, Leuschner M, Kropf S, Niederbichler AD. Non-technical skills training in the operating theatre: a meta-
analysis of patient outcomes. Surgeon. 2019;17:233–243. doi:10.1016/J.SURGE.2018.07.001.
165. TeamSTEPPS® | Agency for healthcare research and quality. Accessed April 16, 2022. Available at: https://www.ahrq.
gov/teamstepps/index.html
166. Non-technical skills for surgeons (NOTSS) | RCSEd. Accessed April 16, 2022. Available at: https://www.rcsed.ac.uk/
professional- support- development- resources/learning- resources/non- technical- skills- for- surgeons- notss.
167. NOTSS for Trainees | RCSEd. Accessed April 16, 2022. Available at: https://www.rcsed.ac.uk/
professional- support- development- resources/learning- resources/non- technical- skills- for- surgeons- notss/
notss-for-trainees.
168. NOTSS Training Project. Accessed April 16, 2022. Available at: https://notss.rcsed.ac.uk/.
169. Non-Technical Skills for Surgeons. Accessed April 16, 2022. Available at: https://www.notss.org/.
170. Pradarelli JC, Yule S, Smink DS. The eNOTSS platform for surgeons’ nontechnical skills performance improvement.
JAMA Surg. 2020;155:438–439. doi:10.1001/JAMASURG.2019.5880.
171. Human Factors | RACS. Accessed April 16, 2022. Available at: https://www.surgeons.org/Education/
skills- training- courses/training- in- professional- skills- tips/human- factors- modules.
172. Getting started with TeamSTEPPS | Team Training | Center | AHA. Accessed April 16, 2022. Available at: https:
//www.aha.org/center/team- training/getting- started- teamstepps.
173. Crossley J, Marriott J, Purdie H, Beard JD. Prospective observational study to evaluate NOTSS (Non-Technical Skills
for Surgeons) for assessing trainees’ non-technical performance in the operating theatre. Br J Surg. 2011;98:1010–
1020. doi:10.1002/BJS.7478.
174. Flin R, Patey R. Non-technical skills for anaesthetists: developing and applying ANTS. Best Pract Res Clin Anaesthesiol.
2011;25:215–227. doi:10.1016/J.BPA.2011.02.005.
175. Mitchell L, Flin R, Yule S, Mitchell J, Coutts K, Youngson G. Development of a behavioural marker system for scrub
practitioners’ non-technical skills (SPLINTS system). J Eval Clin Pract. 2013;19:317–323. doi:10.1111/J.1365-2753.2012.
01825.X.
176. MSc in Patient Safety | Faculty of Medicine | Imperial College London. Accessed April 16, 2022. Available at: http:
//www.imperial.ac.uk/patient-safety-translational-research-centre/education/msc-in-patient-safety/.
177. Masters in Patient Safety & Clinical Human Factors - MSc Online Course. Accessed April 16, 2022. https://www.
edinburghsurgeryonline.com/courses/msc- patient- safety- and- clinical- human- factors.
178. The coach in the operating room | The New Yorker. Accessed April 16, 2022. Available at: https://www.newyorker.
com/magazine/2011/10/03/personal-best.
179. Pradarelli JC, Hu YY, Dimick JB, Greenberg CC. The value of surgical coaching beyond training. Adv Surg. 2020;54:31–
47. doi:10.1016/J.YASU.2020.04.003.
180. Greenberg CC, Ghousseini HN, Quamme SRP, Beasley HL, Wiegmann DA. Surgical coaching for individual perfor-
mance improvement. Ann Surg. 2015;261:32–34. doi:10.1097/SLA.0 0 0 0 0 0 0 0 0 0 0 0 0776.
181. El-Gabri D, McDow AD, Quamme SP, Hooper-Lane C, Greenberg CC, Long KL. Surgical coaching for advancement of
global surgical skills and capacity: a systematic review. J Surg Res. 2020;246:499–505. doi:10.1016/J.JSS.2019.09.039.
182. vande Walle KA, Quamme SRP, Beasley HL, et al. Development and assessment of the Wisconsin surgical coaching
rubric. JAMA Surg. 2020;155:486–492. doi:10.1001/JAMASURG.2020.0424.
183. Pradarelli JC, Yule S, Panda N, et al. Optimizing the implementation of surgical coaching through feedback from
practicing surgeons. JAMA Surg. 2021;156:42–49. doi:10.1001/JAMASURG.2020.4581.
184. Greenberg CC, Byrnes ME, Engler TA, Quamme SP, Thumma JR, Dimick JB. Association of a statewide surgical coach-
ing program with clinical outcomes and surgeon perceptions. Ann Surg. 2021;273(6):1034–1039.
185. Beasley HL, Ghousseini HN, Wiegmann DA, Brys NA, Quamme SRP, Greenberg CC. Strategies for building peer sur-
gical coaching relationships. JAMA Surg. 2017;152. doi:10.1001/JAMASURG.2016.5540.
186. About - the academy for surgical coaching. Accessed April 16, 2022. Available at: https://surgicalcoaching.org/
about/.
187. Peer-to-peer coaching program created to strengthen wellness support | Society for Vascular Surgery.
Accessed April 16, 2022. Available at: https://vascular.org/news-advocacy/articles-press-releases/
peer-peer-coaching-program-created-strengthen-wellness.
188. Sewell JL, Young JQ, Boscardin CK, ten Cate O, O’Sullivan PS. Trainee perception of cognitive load during observed
faculty staff teaching of procedural skills. Med Educ. 2019;53:925–940. doi:10.1111/medu.13914.
189. Cavuoto LA, Hussein AA, Vasan V, et al. Improving teamwork: evaluating workload of surgical team during robot-
assisted surgery. Urology. 2017;107:120–125. doi:10.1016/j.urology.2017.05.012.
190. Al-Jundi W, Wild J, Ritchie J, Daniels S, Robertson E, Beard J. Assessing the nontechnical skills of surgical trainees:
views of the theater team. J Surg Educ. 2016;73:222–229. doi:10.1016/j.jsurg.2015.10.008.
191. Law KE, Lowndes BR, Kelley SR, et al. Surgeon workload in colorectal surgery: perceived drivers of procedural
difficulty. J Surg Res. 2020;245:57–63. doi:10.1016/j.jss.2019.06.084.
192. Sudol NT, Guaderrama NM, Honsberger P, Weiss J, Li Q, Whitcomb EL. Prevalence and nature of sexist and
racial/ethnic microaggressions against surgeons and anesthesiologists. JAMA Surg. Published online 2021:E1-E10.
doi:10.1001/jamasurg.2021.0265
193. Rosette AS, Leonardelli GJ, Phillips KW. The white standard: racial bias in leader categorization. J Appl Psychol.
2008;93:758–777. doi:10.1037/0021-9010.93.4.758.
194. Barnes KL, McGuire L, Dunivan G, Sussman AL, McKee R. Gender bias experiences of female surgical trainees. J Surg
Educ. 2019;76:e1–e14. doi:10.1016/j.jsurg.2019.07.024.
195. Balakrishnan K, Arjmand EM. The impact of cognitive and implicit bias on patient safety and quality. Otolaryngol
Clin North Am. 2019;52:35–46. doi:10.1016/J.OTC.2018.08.016.
196. Antonacci AC, Dechario SP, Antonacci C, et al. Cognitive bias impact on management of postoperative complications,
medical error, and standard of care. J Surg Res. 2021;258:47–53. doi:10.1016/J.JSS.2020.08.040.
197. Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005;165:1493–1499.
doi:10.1001/ARCHINTE.165.13.1493.
198. Placek SB, Franklin BR, Ritter EM. A cross-sectional study of emotional intelligence in military general surgery
residents. J Surg Educ. 2019;76:664–673. doi:10.1016/J.JSURG.2018.10.013.
199. Abi-Jaoudé JG, Kennedy-Metz LR, Dias RD, Yule SJ, Zenati MA. Measuring and improving emotional intelligence in
surgery: a systematic review. Ann Surg. 2022;275:E353–E360. doi:10.1097/SLA.0 0 0 0 0 0 0 0 0 0 0 05022.
200. Weng HC, Hung CM, Liu YT, et al. Associations between emotional intelligence and doctor burnout, job satisfaction
and patient satisfaction. Med Educ. 2011;45:835–842. doi:10.1111/J.1365-2923.2011.03985.X.
201. Nayar SK, Musto L, Fernandes R, Bharathan R. Emotional intelligence predicts accurate self-assessment of surgical
quality: a pilot study. J Surg Res. 2020;245:383–389. doi:10.1016/J.JSS.2019.07.051.
202. McKinley SK, Phitayakorn R. Emotional intelligence and simulation. Surg Clin North Am. 2015;95:855–867. doi:10.
1016/J.SUC.2015.03.003.
203. Leape LL. Make no little plans: The Lucian Leape Institute. Making Healthcare Safe. Published online 2021:371-400.
doi:10.1007/978-3-030-71123-8_22
204. Ridley CH, Al-Hammadi N, Maniar HS, et al. Building a collaborative culture: focus on psychological safety and error
reporting. Ann Thorac Surg. 2021;111:683–689. doi:10.1016/J.ATHORACSUR.2020.05.152.
205. Tørring B, Gittell JH, Laursen M, Rasmussen BS, Sørensen EE. Communication and relationship dynamics in surgical
teams in the operating room: an ethnographic study. BMC Health Serv Res. 2019;19. doi:10.1186/S12913-019- 4362- 0.
206. Jung JJ, Jüni P, Gee DW, et al. Development and evaluation of a novel instrument to measure severity of intraoper-
ative events using video data. Ann Surg. 2020;272:220–226. doi:10.1097/SLA.0 0 0 0 0 0 0 0 0 0 0 03897.
207. Jung JJ, Jüni P, Lebovic G, Grantcharov T. First-year analysis of the operating room black box study. Ann Surg.
2020;271:122–127. doi:10.1097/SLA.0 0 0 0 0 0 0 0 0 0 0 02863.
208. Hashimoto DA, Ward TM, Meireles OR. The role of artificial intelligence in surgery. Adv Surg. 2020;54:89–101.
doi:10.1016/J.YASU.2020.05.010.
209. Ryu J, Moon Y, Choi J, Kim HC. A kalman-filter-based common algorithm approach for object detection in surgery
scene to assist surgeon’s situation awareness in robot-assisted laparoscopic surgery. J Healthc Eng. 2018;2018.
doi:10.1155/2018/8079713.
210. Bonrath EM, Gordon LE, Grantcharov TP. Characterising “near miss” events in complex laparoscopic surgery through
video analysis. BMJ Qual Saf. 2015;24:516–521. doi:10.1136/BMJQS- 2014- 003816.
211. Ward TM, Mascagni P, Ban Y, et al. Computer vision in surgery. Surgery. 2021;169:1253–1256. doi:10.1016/J.SURG.
2020.10.039.
212. Dias RD, Zenati MA, Stevens R, Gabany JM, SJ Yule. Physiological synchronization and entropy as measures of team
cognitive load. J Biomed Inform. 2019;96:103250. doi:10.1016/J.JBI.2019.103250.
213. Dias RD, Ngo-Howard MC, Boskovski MT, Zenati MA, SJ Yule. Systematic review of measurement tools to assess
surgeons’ intraoperative cognitive workload. Br J Surg. 2018;105:491–501. doi:10.1002/BJS.10795.
214. Goldenberg MG, Jung J, Grantcharov TP. Using data to enhance performance and improve quality and safety in
surgery. JAMA Surg. 2017;152:972–973. doi:10.1001/JAMASURG.2017.2888.
215. Boet S, Etherington N, Lam S, et al. Implementation of the operating room black box research program at the
Ottawa hospital through patient, clinical, and organizational engagement: case study. J Med Internet Res. 2021;23.
doi:10.2196/15443.
216. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial intelligence in surgery: promises and perils. Ann Surg.
2018;268:70. doi:10.1097/SLA.0 0 0 0 0 0 0 0 0 0 0 02693.
217. Khor WS, Baker B, Amin K, Chan A, Patel K, Wong J. Augmented and virtual reality in surgery-the digital surgical
environment: applications, limitations and legal pitfalls. Ann Transl Med. 2016;4. doi:10.21037/ATM.2016.12.23.
218. Bernardo A. Virtual reality and simulation in neurosurgical training. World Neurosurg. 2017;106:1015–1029. doi:10.
1016/J.WNEU.2017.06.140.
219. Alzhrani G, Alotaibi F, Azarnoush H, et al. Proficiency performance benchmarks for removal of simulated brain
tumors using a virtual reality simulator NeuroTouch. J Surg Educ. 2015;72:685–696. doi:10.1016/J.JSURG.2014.12.014.
220. Sabbagh AJ, Bajunaid KM, Alarifi N, et al. Roadmap for developing complex virtual reality simulation scenarios:
subpial neurosurgical tumor resection model. World Neurosurg. 2020;139:e220–e229. doi:10.1016/J.WNEU.2020.03.
187.
221. Huang EY, Knight S, Guetter CR, et al. Telemedicine and telementoring in the surgical specialties: a narrative review.
Am J Surg. 2019;218:760–766. doi:10.1016/J.AMJSURG.2019.07.018.
222. Andersen DS, Cabrera ME, Rojas-Muñoz EJ, et al. Augmented reality future step visualization for robust surgical
telementoring. Simul Healthc. 2019;14:59–66. doi:10.1097/SIH.0 0 0 0 0 0 0 0 0 0 0 0 0334.
223. Nguyen NT, Okrainec A, Anvari M, et al. Sleeve gastrectomy telementoring: a SAGES multi-institutional quality im-
provement initiative. Surg Endosc. 2018;32:682–687. doi:10.10 07/S0 0464- 017- 5721- 8.
224. Couperus K, Young S, Walsh R, et al. Immersive virtual reality medical simulation: autonomous trauma training
simulator. Cureus. 2020;12. doi:10.7759/CUREUS.8062.
225. Chheang V, Fischer V, Buggenhagen H, et al. Toward interprofessional team training for surgeons and anesthesiol-
ogists using virtual reality. Int. J. Comput. Assist. Radiol. Surg. 2020;15:2109–2118. doi:10.1007/S11548- 020- 02276- Y/
TABLES/2.
226. Yule S, Janda A, Likosky DS. Surgical sabermetrics: Applying athletics data science to enhance operative perfor-
mance. Ann Surg. Published online 2021. doi:10.1097/AS9.0 0 0 0 0 0 0 0 0 0 0 0 0 054
227. Lewis M. Moneyball: The Art of Winning an Unfair Game. WW Norton; 2003 Accessed April 25, 2022Available at:
https:// www.worldcat.org/ title/ moneyball- the- art- of- winning- an- unfair- game/ oclc/ 51817522?page=citation .
228. Foley M. Adam Ottavino’s Harlem Hide-Out - The New York Times. The New York Times. https://www.nytimes.com/
2019/02/10/sports/baseball/adam-ottavino-yankees.html. Published February 10, 2019. Accessed April 25, 2022.
229. Kennedy-Metz LR, Dias RD, Stevens RH, Yule SJ, Zenati MA. Analysis of mirrored psychophysiological change of
cardiac surgery team members during open surgery. J Surg Educ. 2021;78:622–629. doi:10.1016/J.JSURG.2020.08.012.
230. Kennedy-Metz LR, Dias RD, Srey R, Rance GC, Furlanello C, Zenati MA. Sensors for continuous monitoring of sur-
geon’s cognitive workload in the cardiac operating room. Sensors. 2020;20:1–11. doi:10.3390/S20226616.
231. Davila VJ, Meltzer AJ, Fortune E, et al. Intraprocedural ergonomics of vascular surgeons. J Vasc Surg. 2021;73:301–
308. doi:10.1016/J.JVS.2020.04.523.
232. Scott JW, Lin Y, Ntakiyiruta G, et al. Identification of the critical nontechnical skills for surgeons needed for
high performance in a variable-resource context (NOTSS-VRC). Ann Surg. 2019;270:1070–1078. doi:10.1097/SLA.
0 0 0 0 0 0 0 0 0 0 0 02828.

Managing A Team in The Operating Room: The Science of Teamwork and Non-Technical Skills For Surgeons

Uploaded by

Copyright:

Available Formats

Managing A Team in The Operating Room: The Science of Teamwork and Non-Technical Skills For Surgeons

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Managing A Team in The Operating Room: The Science of Teamwork and Non-Technical Skills For Surgeons

Uploaded by

Copyright:

Available Formats

Current Problems in Surgery 59 (2022) 101172

Contents lists available at ScienceDirect

Current Problems in Surgery

journal homepage: www.elsevier.com/locate/cpsurg

Managing a team in the operating room: The

At 3:15 AM on an otherwise unremarkable Thursday morning, a routine appendectomy was

Origins of the science of teamwork

Fig. 1. Taxonomy of team development interventions. Adapted from Lacerenza et al.8 .

Team development interventions

Evidence-based Recommendations for Characteristics

Team development in health care

1. The team should have explicitly stated team goals

Deﬁning “team” within the context of health care delivery

Team development interventions (TDIs) within health care

Competency Exhibited Behaviors

Team training, team building, and team debrieﬁng

The application of teamwork science to surgery

Crew resource management

Categories Subcategories Description

employed aviation-based CRM experts to guide the implementation of a curriculum focused on

Simulation-based team training in surgery

Established cognitive aids for promoting teamwork

Intraoperative safety checklists

Non-technical training tools for the surgical team

Oxford nontechnical skills scale (NOTECHS)

Observational teamwork assessment for surgery (OTAS)

Operating room management attitudes questionnaire (ORMAQ)

Non-technical training tools for the individual surgeon

Nontechnical skills for surgeons (NOTSS)

Leadership in the operating room

Other global and skill-speciﬁc tools

Linking non-technical skill development to patient outcomes

Mechanisms to enhance surgeon performance

Literature and online resources

Special considerations for teaching non-technical skills

Trainee cognitive load

Teaching and training in the clinical environment

Connected ﬁelds of research

The discussion of NTS of an individual or a team is connected to much of the additional

United States Suggestions for Integrating NOTSS

PGY1 • Intern orientation programs

PGY2-3 • Increased awareness and iterative feedback about NTS performance

PGY4-5 • Leadership curriculum emphasizing NTS

NTS, non-technical skills; PGY, postgraduate year.

Human factors research

Artiﬁcial intelligence for enhanced team performance analysis

In summary, team performance can be enhanced through self-directed learning, formal

You might also like