EVIDENCE-BASED TRAINING

IMPLEMENTATION GUIDE

APPENDIX C
Threat and Error Management (TEM)
Introduction

Threat and error management (TEM) is a framework that practically integrates Human Factors into aviation
operations. It is not a revolutionary concept but it evolved gradually, as a consequence of the constant
drive to improve the margins of safety in aviation operations, through the practical integration of Human
Factors knowledge.

TEM developed as a product of the collective industry experience. This experience fostered the recognition
that past studies and operational consideration of human performance in aviation had largely overlooked
the most important factor influencing human performance in dynamic work environments. That is the
interaction between people and the operational context (i.e., organizational, regulatory and environmental
factors) within which people discharge their operational duties.

The recognition of the influence of the operational context in human performance further led to the
conclusion that study and consideration of human performance in aviation operations must not be an end in
itself. In regard to the improvement of margins of safety in aviation operations, the study and consideration
of human performance without context address only part of a larger issue. TEM therefore aims to provide a
principled approach to the broad examination of the dynamic and challenging complexities of the
operational context in human performance, for it is the influence of these complexities that generates
consequences directly affecting safety. TEM training is now embedded in ICAO SARPs as an intrinsic part
of Flight Crew Licensing requirements.

The TEM framework

TEM is a framework that assists in understanding, from an operational perspective, the inter-relationship
between safety and human performance in dynamic and challenging operational contexts.

The TEM framework focuses simultaneously on the operational context and the people undertaking
operational duties in such context. The framework is descriptive, practical and diagnostic of both human
and system performance. It is descriptive because it captures human and system performance in the
normal operational context, resulting in realistic descriptions. It is practical as pilots may use it intuitively
and is diagnostic because it allows quantifying the complexities of the operational context in relation to the
description of human performance in that context, and vice-versa.

70
 EVIDENCE-BASED TRAINING
IMPLEMENTATION GUIDE

Threat

Threat Management Threat Linked

Inconsequential
(Diagnosis / Recovery) Incident / Accident

Threat Linked Spontaneous

Crew Error Crew Error

Error Management Additional

Inconsequential
(Diagnosis / Recovery) Crew Error

Crew error linked

Undesired Aircraft State

Undesired Aircraft State

Inconsequential Management
(Diagnosis / Recovery)

Crew Error Linked

Incident / Accident

The TEM framework can be used in several ways. As a safety analysis tool, the framework can focus on a
single event, as is the case with accident/incident analysis; or it can be used to understand systemic
patterns within a large set of events, as is the case with operational audits. It can be used as a licensing
tool, helping clarify human performance needs, strengths and vulnerabilities, allowing the definition of
competencies from a broader safety management perspective. Importantly the TEM framework can be
used as a training and assessment tool, both at an individual and systemic level. It can help an
organization improve the effectiveness of its training interventions, and consequently of its
organizational safeguards.

The components of the TEM framework

There are three basic components in the TEM framework from the perspective of flight crews. These are
threats, errors and undesired aircraft states. The framework proposes that threats and errors are part of
everyday aviation operations that must be managed by flight crews, since both threats and errors carry the
potential to generate undesired aircraft states. Flight crews must also manage undesired aircraft states,
since they carry the potential for unsafe outcomes. Undesired state management is an essential
component of the TEM framework, as important as threat and error management. Undesired aircraft state
management largely represents the last opportunity to avoid an unsafe outcome and thus maintain safety
margins in flight operations.

71
 EVIDENCE-BASED TRAINING
IMPLEMENTATION GUIDE

a. Threats

Threats are defined as “events or errors that occur beyond the influence of the flight crew, increase
operational complexity, and which must be managed to maintain the margins of safety”. During typical
flight operations, flight crews have to manage various contextual complexities. Such complexities would
include, for example, dealing with adverse meteorological conditions, airports surrounded by high
mountains, congested airspace, aircraft malfunctions, errors committed by other people outside of the
cockpit, such as air traffic controllers, flight attendants or maintenance workers, and so forth. The TEM
framework considers these complexities as threats because they all have the potential to negatively
affect flight operations by reducing margins of safety.

Some threats can be anticipated, since they are expected or known to the flight crew. For example,
flight crews can anticipate the consequences of a thunderstorm by briefing their actions in advance, or
prepare for a congested airport by ensuring that they keep a watchful eye for other aircraft during the
approach. Some threats can occur unexpectedly, such as an in-flight aircraft malfunction that happens
suddenly and without warning. In this case, flight crews must apply skills and knowledge acquired
through training and operational experience. Lastly, some threats may not be directly obvious to, or
observable by, flight crews immersed in the operational context, although they may be uncovered by
safety analyses. These are considered latent threats. Examples of latent threats include equipment
design issues, optical illusions, or shortened turn-around schedules.

Regardless of whether threats are expected, unexpected, or latent, one measure of the effectiveness of
a flight crew‟s ability to manage threats is whether threats are detected with the necessary anticipation
to enable the flight crew to respond to them through deployment of appropriate countermeasures.

Threat management is a building block to error management and undesired aircraft state management.
The threat-error linkage is not necessarily straightforward and it is not possible to establish a linear
relationship, or one-to-one mapping between threats, errors and undesired states. However, archival
data demonstrates that mismanaged threats are normally linked to flight crew errors, which in turn are
oftentimes linked to undesired aircraft states. Threat management provides a highly proactive strategy
to maintain safety margins in flight operations by mitigating safety-compromising situations. As threat
managers, flight crews are the last line of defense to keep threats from impacting flight operations.

Table C-1 presents examples of threats, grouped under two basic categories derived from the TEM
framework. Environmental threats occur due to the environment in which flight operations take place.
Some environmental threats can be planned for and some will arise spontaneously, but they all have to
be managed by flight crews in real time. Organizational threats, on the other hand, can be controlled
(i.e., removed or, at least, minimized) at source by aviation organizations. Organizational threats are
usually latent in nature. Flight crews still remain the last line of defense, but there are earlier
opportunities for these threats to be mitigated by aviation organizations themselves.

72
 EVIDENCE-BASED TRAINING
IMPLEMENTATION GUIDE

Environmental threats Organizational threats

 Weather: thunderstorms, turbulence, icing,  Operational pressure: delays, late
wind shear, cross/tailwind, very low/high arrivals, equipment changes.
temperatures.
 Aircraft: aircraft malfunction, automation
 ATC: traffic congestion, TCAS RA/TA, ATC event/anomaly, MEL/CDL.
command, ATC error, ATC language difficulty,
ATC non-standard phraseology, ATC runway  Cabin: flight attendant error, cabin event
change, ATIS communication, units of distraction, interruption, cabin door
measurement (QFE/meters). security.

 Airport: contaminated/short runway;  Maintenance: maintenance event/error.

contaminated taxiway, lack of/confusing/ faded
signage/markings, birds, inoperative aids,  Ground: ground handling event, de-icing,
complex surface navigation procedures, airport and ground crew error.
constructions.
 Dispatch: dispatch paperwork event/error.
 Terrain: High ground, slope, lack of
references, “black hole”.  Documentation: manual error, chart error.

 Other: similar call-signs.  Other: crew scheduling event

Table C-1. Examples of threats

b. Errors
Errors are defined “actions or inactions by the flight crew that lead to deviations from organizational or
flight crew intentions or expectations”. Unmanaged and/or mismanaged errors frequently lead to
undesired aircraft states. Errors in the operational context reduce the margins of safety and increase
the probability of adverse events. They can be errors of commission or omission.

Errors can be linked to threats or be spontaneous (i.e., without direct linkage to specific, obvious
threats). They can also form part of an error chain. Examples of errors would include lapses in
handling, executing a wrong automation mode, failing to give a required callout, or misinterpreting an
ATC clearance. Regardless of the type of error, the effect on safety depends on whether the flight crew
detects and responds to the error before it leads to an undesired aircraft state and to a potential unsafe
outcome. This is why one of the objectives of TEM is to understand error management (i.e., detection
and response), rather than solely focusing on error causality (i.e., causation and commission). From
the safety perspective, operational errors that are detected, promptly responded to (i.e., properly
managed) and do not reduce margins of safety in flight operations become operationally
inconsequential. In addition to its safety value, proper error management represents an example of
successful human performance, presenting both learning and training value.

Capturing how errors are managed is more than capturing the prevalence of different types of error. It
is important to capture if and when errors are detected and by whom, the response(s) upon detecting
errors, and the outcome of errors. Some errors are quickly detected and resolved, thus becoming
operationally inconsequential, while others go undetected or are mismanaged. A mismanaged error is
defined as an error that is linked to or induces an additional error or undesired aircraft state.

73
 EVIDENCE-BASED TRAINING
IMPLEMENTATION GUIDE

Table C-2 presents examples of errors, grouped under three basic categories derived from the TEM
framework. In the TEM concept, errors have to be “observable” and therefore, the TEM framework
uses the “primary interaction” as the point of reference for defining the error categories.

The TEM framework classifies errors based upon the primary interaction of the pilot or flight crew at the
moment the error is committed. Thus, in order to be classified as aircraft handling error, the pilot or
flight crew member must be interacting with the aircraft (e.g. through its controls, automation or
systems). In order to be classified as procedural error, the pilot or flight crew member must be
interacting with a procedure (e.g. checklists, SOPs, etc.). In order to be classified as communication
error, the pilot or flight crew member must be interacting with people (air traffic controller, ground crew,
other crew members, etc.).

Aircraft handling errors, procedural errors and communication errors may be unintentional or involve
intentional non-compliance. Similarly, proficiency considerations (i.e., skill or knowledge deficiencies,
training system deficiencies) may underlie all three categories of error. In order to keep the approach
simple and avoid confusion, the TEM framework does not consider intentional non-compliance and
proficiency as separate categories of error, but rather as sub-sets of the three major categories of error.

 Manual handling/flight controls: vertical/lateral and/or speed

Aircraft handling errors
deviations, incorrect flaps/speedbrakes, thrust reverser or power
settings.
 Automation: incorrect altitude, speed, heading, autothrottle
settings, incorrect mode executed, or incorrect entries.
 Systems/radio/instruments: incorrect packs, incorrect anti-icing,
incorrect altimeter, incorrect fuel switches settings, incorrect speed
bug, incorrect radio frequency selected.
 Ground navigation: attempting to turn down wrong taxiway/runway,
taxi too fast, failure to hold short, missed taxiway/runway.
 SOPs: failure to cross-verify automation inputs.
Procedural errors
 Checklists: wrong challenge and response; items missed, checklist
performed late or at the wrong time.
 Callouts: omitted/incorrect callouts
 Briefings: omitted briefings; items missed.
 Documentation: wrong weight and balance, fuel information, ATIS,
or clearance information recorded, misinterpreted items on
paperwork; incorrect logbook entries, incorrect application of MEL
procedures.
 Crew to external: missed calls, misinterpretations of instructions,
Communication errors
incorrect read-back, wrong clearance, taxiway, gate or runway
communicated.
 Pilot to pilot: within crew miscommunication or misinterpretation

Table C-2. Examples of errors

74
 EVIDENCE-BASED TRAINING
IMPLEMENTATION GUIDE

c. Undesired Aircraft States

Undesired aircraft states are defined as „flight crew-induced aircraft position or speed deviations,
misapplication of flight controls, or incorrect systems configuration, associated with a reduction in
margins of safety”. Undesired aircraft states that result from ineffective threat and/or error management
may lead to compromising situations and reduce margins of safety in flight operations. Often
considered at the cusp of becoming an incident or accident, undesired aircraft states must be managed
by flight crews.

Examples of undesired aircraft states would include lining up for the incorrect runway during approach
to landing, exceeding ATC speed restrictions during an approach, or landing long on a short landing
distance limited runway. Events such as equipment malfunctions or ATC controller errors can also
reduce margins of safety in flight operations, but these would be considered threats.

Undesired states can be managed effectively, restoring margins of safety, or flight crew response(s)
can induce an additional error, incident, or accident.

Table C-3 presents examples of undesired aircraft states, grouped under three basic categories
derived from the TEM framework.

 Aircraft control (attitude).

Aircraft handling
 Vertical, lateral or speed deviations.
 Unnecessary weather penetration.
 Unauthorized airspace penetration.
 Operation outside aircraft limitations.
 Unstable approach.
 Continued landing after unstable approach.
 Long, floated, firm or off-centerline landing.
 Proceeding towards wrong taxiway/runway.
Ground navigation
 Wrong taxiway, ramp, gate or hold spot

 Incorrect systems configuration.

Incorrect aircraft configurations
 Incorrect flight controls configuration.
 Incorrect automation configuration.
 Incorrect engine configuration.
 Incorrect weight and balance configuration.

Table C-3. Examples of undesired aircraft states

75
 EVIDENCE-BASED TRAINING
IMPLEMENTATION GUIDE

An important training point for flight crews is the timely switching from error management to “undesired
aircraft state” management. An example would be as follows: a flight crew selects a wrong approach in
the Flight Management Computer (FMC). The flight crew subsequently identifies the error during a
crosscheck prior to the Final Approach Fix (FAF). However, instead of reverting to a basic mode (e.g.
heading) or manually flying the desired track, both flight crew become involved in attempting to
reprogram the correct approach prior to reaching the FAF. As a result, the aircraft flies through the
localizer, descends late, and the approach becomes unstable. This would be an example of the flight
crew getting “locked in” to error management, rather than switching to undesired aircraft state
management. The use of the TEM framework assists in educating flight crews that, when the aircraft is
in an undesired state, the basic task of the flight crew is undesired aircraft state management instead of
error management. It also illustrates how easy it is to get locked in to the error management phase.

Also from a training perspective, it is important to establish a clear differentiation between undesired
aircraft states and outcomes. Undesired aircraft states are transitional states between a normal
operational state (i.e., a stabilized approach) and an outcome. Outcomes, on the other hand, are end
states, most notably, reportable occurrences (i.e., incidents and accidents). An example would be as
follows: a stabilized approach (normal operational state) turns into an unstable approach (undesired
aircraft state) that results in a runway excursion (outcome).

The training and remedial implications of this differentiation are of significance. While at the undesired
aircraft state stage, the flight crew has the possibility, through appropriate TEM, of recovering the
situation, returning to a normal operational state, thus restoring margins of safety. Once the undesired
aircraft state becomes an outcome, recovery of the situation, return to a normal operational state, and
restoration of margins of safety is not possible.

Countermeasures

Flight crews must, as part of their normal operational duties, employ countermeasures to keep threats,
errors and undesired aircraft states from reducing margins of safety in flight operations. Examples of
countermeasures would include checklists, briefings, call-outs and SOPs, as well as personal strategies
and tactics. Flight crews dedicate significant amounts of time and energies to the application of
countermeasures to ensure margins of safety during flight operations. Empirical observations during
training and checking suggest that, as much as 70% of flight crew activities may be countermeasures-
related activities.

All countermeasures are necessarily flight crew actions. However, some countermeasures to threats, errors
and undesired aircraft states that flight crew‟s employ builds upon “hard” resources provided by the aviation
system. These resources are already in place in the system before flight crews report for duty, and are
therefore considered as systemic-based countermeasures. The following would be examples of “hard”
resources that flight crews employ as systemic-based countermeasures:

 Airborne Collision Avoidance System (ACAS);

 Ground Proximity Warning System (GPWS),

 Standard operation procedures (SOPs);

 Checklists;

 Briefings;

 Training.

76