Aeronautica, Issue 1, 2011

Line Operations Safety Audit (LOSA) for the management

of safety in single pilot operations (LOSA:SP) in Australia
and New Zealand
Laurie Earl, Patrick S. Murray, Paul R. Bates
Griffith University Aerospace Strategic Study Centre, Brisbane, QLD, Australia

Abstract
This paper investigates the feasibility, effectiveness and benefits of implementing a single pilot
operations variant of the multi-crew Line Operations Safety Audit (LOSA) methodology, in the
management of safety in single pilot operations. LOSA is designed to provide a proactive
snapshot of system safety and flight crew performance as a way of preventing incidents and
accidents (Klinect, 2006). The data indicators underlying this effort are based on a conceptual
framework known as Threat and Error Management (TEM) (Helmreich, R.L., Wilhelm, J.A.,
Klinect, J.R., & Merritt, A.C. 2001).
A number of incidents and accidents involving single pilot operations both in Australia and New
Zealand have given emphasis to the vulnerability of this group to safety issues and confirms
statistics that show this category of aviation has a higher incidence of accidents and incidents
than in other sectors of the industry (CASA, 2009, NZCAA, 2009). By adapting LOSA to single
pilot operations (LOSA:SP) the framework/methodology could provide a proactive method of
diagnosing operational safety performance strengths and weaknesses under normal operations
leading to the identification of additional training requirements and improved procedures without
relying on adverse safety events for such information.
A case study was undertaken at a mid-sized company operating exclusively single-pilot, twin
turbo-prop fixed wing aircraft to ascertain whether the methodology was viable in the single-pilot
environment. Observers rated pilot performance on a 4 point scale using four standard threat
and error counter-measure categories under 12 sub-headings.
Whilst the study achieved its objective of determining whether a single-pilot line operations
safety audit could be successfully developed, the data were indicative and insufficient for
statistical analysis. Larger samples are required from future studies for more definitive
conclusions and recommendations about threat and error management.

Introduction
Most of aviations understanding of safety performance is based on data concerning adverse safety
events, such as those collected from incident reporting and accident investigations (Maurino, 2001).
These can be seen as reactive measures of safety as they are dependant on negative flight outcomes
(Reason, 1997). Although these approaches will continue to be essential in guiding future aviation policy
and regulation, as well as informing aircraft and systems design (Applegate and Graeber, 2005), because
of their unreliability in preventing future incidents (Helmreich, 2006) they are increasingly being
supplemented by more proactive approaches.

Aeronautica, Issue 1, 2011

The Line Operations Safety Audit is a proactive measure that serves to fill this gap with its collection
of TEM data in normal flight operations. It can be seen as a proactive safety measure that complements
existing data sources such as line evaluations, quick access recorders, voluntary incident reports and
accident investigations.
The LOSA methodology, which is endorsed by the International Civil Aviation Organisation (ICAO)
(ICAO, 2002) is a formal process that requires expert and highly trained observers to occupy the jumpseat
during regularly scheduled flights in order to collect safety related data on environmental conditions,
operational complexity and flight crew performance. It provides a diagnostic snapshot of strengths and
weaknesses that an airline can use to bolster its safety margins and prevent their degradation.
(Helmreich, 2006). LOSA uses a targeted observation instrument based on the TEM framework.
LOSA has been developed and refined since 1996 with major international airlines becoming involved
forming a collaborative partnership with The University of Texas Human Factors Research Project
(UTHFRP) (Klinect, 2006). Captain Don Gunther, Manager of Human Factors Training at Continental
Airlines hailed LOSA as a success, saying that Continental Airlines provided the proof of concept for
LOSA that transformed it from a research tool to an industry-ready safety tool (Gunther, D. 2002).
In 2001, LOSA became a central focus of the Flight Safety and Human Factors Programme (Klinect et
al, 2003). ICAO has also introduced a standard making TEM training mandatory for airline flight crews
engaged in international operations (Merritt and Klinect, 2006), which must be delivered during initial as
well as recurrent training. TEM based LOSA is now considered to be best practice for normal operations
monitoring and aviation safety by ICAO, the Federal Aviation Authority (FAA) and the Civil Aviation
Authority (CAA).
More recently, regional airlines operating turboprop aircraft have seen the potential benefits of a Line
Operations Safety Audit with a plan by the LOSA collaborative to bring regional airlines into the LOSA
sphere (Rosenkrans, 2007). Air New Zealand regional carrier Mt Cook Airlines and Australias Regional
Express were amongst the worlds first regional operators to implement LOSA
The Guild of Air Pilots and Air Navigators (GAPAN) conducted a series of TEM courses in Australia
in 2008/9 for pilots involved in low capacity public transport and single - pilot operations. The ATSBs
safety report AR-2006-156(2) looked at the threats and errors that participants of these courses considered
were the five most common in their industry (ATSB, 2009).
This paper considers those smaller (single - pilot) operations and the actual threats and errors that occur
in flights.

Research Overview
The LOSA methodology provides the tools to record threats to safety that the pilot might encounter and
errors that are made by pilots and the response to those errors. An example of a threat would be adverse
weather conditions or an aircraft system malfunction. LOSA identifies the occurrence of these threats and
facilitates the analysis of the actions taken by pilots to manage them Thomas (2003). Errors are seen as an
unavoidable and ubiquitous aspect of normal operation (Thomas, 2003) but it is an important aspect of
pilot performance in how those errors are recognised, trapped and mitigated.
By using the threat and error management (TEM) conceptual framework, LOSA focuses
simultaneously on the operating environment and the humans working in that environment. As the TEM
taxonomy can also quantify specifics and effectiveness of performance, the results are also highly
diagnostic (Merritt and Klinect, 2006).
Traditionally the smaller operators have experienced a higher accident rate than larger carriers, both in
Australia, New Zealand and worldwide (ATSB, 2007). Occurrence figures show that smaller companies
with single pilot operation are experiencing an increase in accident statistics (CASA, 2009, CAA 2009).
Despite efforts in this area to increase safety with traditional methods (training, seminars, education,
regulation, inspection etc), there appears to have been less research in this segment of the industry when
compared with that conducted for and by major carriers and more recently regional carriers.
All LOSAs to date have been conducted in multi crew operations where the LOSA observer occupies a
jumpseat on the flight deck. However, it is proposed that, by applying the LOSA concept to single pilot
operations, with a carefully designed research methodology pertaining to this type of operation

Aeronautica, Issue 1, 2011

(LOSA:SP), such methodology could provide an opportunity to understand the operational context, pilot
processes and outcomes during single pilot routine flights. If successful, in a similar manner to LOSA,
LOSA : SP will give an insight into normal operations, through diagnosing strengths and weaknesses of
pilots without relying on accidents or incidents to gain that information. Helmreich compares CRM in
both multi-crew operations and single-pilot operations, identifying several points where CRM can be
adapted for single pilots (Flight Safety Australia, 2006). Introducing the LOSA methodology to improve
threat and error management in single pilot operation could therefore be achievable and successful in
reducing incidents and accidents. Thus safety could be enhanced in a field where other methods have
failed.

Objectives
McDonald and colleagues state that the results of a LOSA Like trial in passenger train operations,
where in cab observations were made on single driver operations, could ultimately highlight future
directions for training and awareness and make potential improvements to organisational systems and
processes (McDonald, A., Garrigan, B., Kanse, L. 2006).
By applying LOSA to single pilot operations (LOSA:SP) and using an appropriate data analysis
system, threats and errors could be decreased, awareness enhanced and training and education improved,
based on the results of the assessments.

Methodology
At the heart of LOSA are non-jeopardy observations, without which flight crew may be unwilling to
accept the presence of an observer in their domain. As LOSA has progressed, pilots and managers are
seeing the safety benefits of the system and as Airlines conduct future LOSA assessments the willingness
to contribute has increased.
Various methods were considered to collect data in single pilot operations, both where an observer
was possible but also in situations where an observer was not possible or not appropriate. Adaptations
were made to the LOSA methodology in order to facilitate the comprehensive and unique differences
applicable to the single pilot situation. This included, for example, revisions to some of the error
categories. However, LOSA data indicators based on the established TEM framework were retained due
to their proven nature. ICAO specifies ten characteristics that define LOSA (ICAO, 2002). These were
replicated for single pilot operations.
As in LOSA, LOSA:SP collected data on pilot demographics, threat occurrence and management,
error occurrence and management and CRM effectiveness through TEM - based behavioural markers.
Following initial meetings with management and crews at a mid-sized aviation company, a draft
proposal was sent outlining the research. Following agreement by the company and representatives from
the pilot group to participate in the research, an introductory newsletter was circulated, observer
expressions of interest were called for, and a presentation of LOSA to pilots and managers was conducted.
Subsequently a 4 day observer training course was conducted by a member of the LOSA Collaborative
and the two observers chosen for this study flew on 14 observation sectors covering a sample of the
companys route network and crew.
It was emphasised that one of the prime defining characteristics of LOSA is the de-identification and
confidentiality of all data which in this experiment would be collated and analysed by the research team at
Griffith University.
The aim of the research was to provide information on whether LOSA methodology could be usefully
deployed in single-pilot operations (LOSA:SP). The trial was developed to provide a useful short-term
safety focus for the company but on the understanding that the number of observations would not be
sufficient for statistical analysis.
The observations were supplemented by separate post-flight interviews with crew on safety concerns,
suggestions for safety improvements, automation and operational efficiency.

Aeronautica, Issue 1, 2011

Concerns
Concerns on observer reliability, establishing trust with those being observed, and an accurate
coding scheme are similar within most field observation methodologies. Possibly the most
important data quality issue is one of observation reactivity, which occurs when individuals may
alter their normal behaviours because of an observers presence in the cockpit.
Also, it is important to observe pilots who are both willing to participate in the study and who are
representative of the group. Amongst the considerations stated by pilots included balancing a desire to
participate in a study that may result in a safer operation versus potential monetary loss (loss of paid flight
hours). Also, for single pilot commercial passenger flight operations, the addition of an observer may
reduce the paying capacity of the spare seats.
Ethical clearance was obtained prior to commencement of the flights observations. This was
considered to be an integral part of establishing the feasibility of the study. The LOSA defining principles
emphasising complete de-identification of individuals together with strict confidentiality and no
jeopardy provisions (ICAO, 2002) assisted greatly.

Results
Results were illustrative rather than definitive and larger samples are required from future studies to draw
meaningful conclusions and recommendations regarding threat and error management. Furthermore, there
are no other similar audit results on single-pilot operations available as a basis for comparison.
However, some similarities with other LOSA audits were noted including comparable raw numbers of
threats and errors per flight, the type of threats (weather, Air Traffic Control (ATC), operational pressures,
airport conditions etc) and the distribution of error types. It was apparent that there were some operation
specific threats and induced errors.
Not all threats have equal impact. While all may have the potential to affect safety adversely, some
categories of threats are better managed than others by pilots. A LOSA establishes the rates of threats or
errors and highlights those that are mismanaged, i.e. those that are either not detected at all or those that
are detected but not managed adequately. The most worrying to safety are those threats or errors with a
high rate of occurrence and high rates of mismanagement; that is those that have increased risk potential.
As an example checklist errors alone may not seen as important, yet checklists are designed to trap
procedural errors. So, when checklists are missed on several occasions, in the event of an earlier
procedural error, the risk factor can increase dramatically and may be the final hole in Reasons Swiss
cheese model of accident causation (Reason, 1997)
The five flight phases in which a threat or error could occur were: pre-departure/taxi, take off/climb,
cruise, descent/approach/landing, and taxi-in. Generally, LOSAs show that the descent, approach and
landing phase has a proportionately larger number of errors due to the high workload at this time (Flight
Safety Foundation, 1998). Because of the nature of the operation, this case study indicated that threats
and errors occurred in approximately equal numbers in the pre-departure phase and the descent/approach
and landing phase. However, this observation may have been biased by the small number of observations.
In multi-pilot operations, there is invariably a requirement for pilots to verbalise their actions or
intentions. In single-pilot operations generally there is no such requirement, although anecdotally views
are split as to the value of this process. In the company involved in this experiment verbalisation of
procedures is optional and the pilot force is split between those who routinely verbalise and those who
dont. This case study compared the error rates between the two groups. Although there was relatively
little difference between the groups, the results suggested that pilots who verbalised their intentions were
more assiduous in cross-checking and had fewer mismanaged procedural errors. This was not a big
enough sample to draw firm conclusions, but this aspect merits further investigation.

Discussion
Whilst mechanisms such as incident reports and confidential reporting systems, together with line checks
are commonplace in organisations attempting to improve their safety performance, these may not provide

Aeronautica, Issue 1, 2011

sufficient information for an organisation to unmask hidden latent errors in the system (Thomas 2003).
LOSA acts as a pro-active evaluation tool and is potentially more powerful with respect to safety
measures.
Ashleigh Merritt, one of the original data analysts from early LOSAs comments that TEM based
LOSAs continue to provide valuable diagnostic information about an airlines strengths and vulnerabilities
and sees LOSA as best practice for normal operations monitoring and safety. (Merritt and Klinect, 2006)
Furthermore LOSA is now recognised and recommended to airlines as a best practice for normal
operations monitoring:
LOSA is proposed as a critical organisational strategy aimed at developing countermeasures to
operational errors. It is an organisational tool used to identify threats to aviation safety, minimise the risks
such threats may generate and implement measures to manage human error in organisational contexts.
(ICAO, 2002)
As LOSA developed as a powerful pro-active safety tool, the methodology was adapted to other areas
with equally positive results. Air Traffic Control (Normal Operations Safety Survey NOSS (Henry,
2007), the military, (Mission Operations Safety Audits MOSA (Burdekin, 2003)), and Queensland Rail
(Confidential Observations of Rail Safety CORS (McDonald et al, 2006)) all completed successful
LOSA Like audit activities.
Now that LOSA has been established as a successful and innovative tool, the adaptation to single pilot
commercial operations was a logical step. Within this case study, the analytical data achieved and the
subjective data obtained in interview indicated that the company had established a strong framework of
safety to support its flight operations.
The sample size in this study was selected to determine whether the LOSA methodology could be
adapted to single-pilot operations and obtain useful data. However, a sample size of around 60 flights has
been calculated as necessary to capture data in sufficient quantity to undertake valid quantitative data
analysis for definitive conclusions.
The objective of this study was achieved in that the LOSA methodology proved completely
transferrable so that all categories of the TEM model were observable with minimal adaptation of the
methodology for single-pilot operations
In this case study most of the adaptation (other than obvious multi-crew to single-crew adaptation
communication) was operator specific. This case study strongly indicated that while the methodology
could be used in single-pilot operations, further refinement may be necessary for differing operations.
E.g. rotary wing aircraft.
It is proposed that LOSA:SP model will provide a feedback mechanism to single pilot aircraft
operators which in turn will assist pilots to manage threats and errors more successfully in the future and
thereby increase their safety margins. Ultimately it is proposed that the data would form part of a
LOSA:SP archive which could be compared with the existing LOSA archive. Eventually it could be used
to inform training and regulatory interventions within the aviation industry, leading to safer outcomes in
single pilot operations..
References
Applegate, J.D., Graeber, R.C., 2005. Integrated safety systems design and human factors considerations
for jet transport aeroplanes. In D. Harris & H.C. Muir (eds), Contemporary issues in Human Factors
and Aviation Safety (pp. 3-23). Aldershot, UK, Ashgate
ATSB. (2007). Regional Airline Operations Safety Audit. Report B2004/0237. Canberra, Australia:
Australian Transport Safety Bureau
ATSB (2009) Perceived Threats, Errors and Safety in Aerial Work and Low Capacity Air Transport
Operations ATSB Transport Safety Report AR-2006-156(2)

Aeronautica, Issue 1, 2011

Burdekin, S. (2003) Mission Operations Safety Audits (MOSA): Measurements of behavioural
Performance or non Technical Skills from Military Aircrew. Aviation Safety spotlight 0403
Civil Aviation Authority (NZCAA).(2009) Annual Report, 2008 2009. Wellington, New Zealand:
New Zealand Government
Civil Aviation Authority (NZCAA). (2006). Advisory Circular 91, 11 Feb 2006. Wellington, New
Zealand: New Zealand Government.
Civil Aviation Safety Authority (CASA) (2009)
Australian Government

Annual Report, 2008-2009 Canberra, Australia

Fellows, P. (2005) NOSS Trial May/June 2005 presented at Airways New Zealand
Flight Safety Foundation, 1998. Killers in Aviation. Flight Safety Digest 17 (11-12), 12. Arlington, VA.
Gunther, D. (2002). Threat and error management training counters complacency, fosters operational
excellence. International Civil Aviation Organisation Journal, 57(4), 12 13
Helmreich, R.L. (2006). Red Alert. Flight Safety Australia. Oct 2006. pp 30 - 31
Helmreich, R.L., Wilhelm, J.A., Klinect, J.R., & Merritt, A.C. (2001). Culture, Error, and Crew Resource
Management. University of Texas at Austin Human Factors Research Project (UTHFRP) Pub 254
Henry, C., 2007. The Normal Operations Safety Survey (NOSS): Measuring System Performance in Air
Traffic Control. System Safety, 2007. 2nd Institution of Engineering and Technology International
Conference on Safety Management. University of Texas at Austin. International Civil Aviation
Organization, 2002a. Line Operations Safety Audit (LOSA). ICAO Document 9803, AN/761.
Montreal.
ICAO. (2002) Line Operations Safety Audit (LOSA).
International Civil Aviation Authority

Doc 9803. AN/761.

Montreal, Canada.

Klinect, J. R., 2006. Line operations Safety Audit: A cockpit observation methodology for monitoring
commercial airline safety performance. (Dissertation). The University of Texas. Austin, USA
Klinect, J.R., Murray, P., and Helmreich, R. (2003). Line Operations Safety Audit (LOSA): Definition
and operating characteristics. In Proceedings of the 12th International Symposium on Aviation
Psychology (p663 668). Dayton, OH: the Ohio State University
Maurino, D.E (2001). Proactive safety culture: Do we need human factors? In Klinect, Line operations
Safety Audit: A cockpit observation methodology for monitoring commercial airline safety
performance. (Dissertation). The University of Texas. Austin, USA.
McDonald, A., Garrigan, B., Kanse, L. (2006) Confidential Observations of Rail Safety (CORS): An
Adaptation of Line Operations Safety Audit. Proceedings of the Swinburne University Multimodal
Symposium on Safety Management and Human Factors, 9-10 Feb, 2006, Melbourne, Australia
Merritt, A, Klinect, J. (2006). Defensive Flying for Pilots: An Introduction to Threat and Error
Management. The University of Texas Human Factors Research Project: The LOSA Collaborative.
Austin, Texas

Aeronautica, Issue 1, 2011

Reason, J.T. (1997). Managing the risks of organisational accidents. Aldershot. Ashgate
Rosenkrans, W. (2007) Threat and Error Detectives. Flight Safety Foundation AerosafetyWorld March
2007
Thomas, M.J.W. (2003) Improving Organisational Safety Through the Integrated Evaluation of
Operational and Training Performance: an Adaptation of the Line Operations Safety Audit (LOSA)
Methodology. Human factors and Aerospace Safety 3(1) 25-45