Team Resource PDF

EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION
EUROCONTROL
This document is issued as EATMP Reference Material. The contents are not mandatory. They provide information and explanation or may indicate best practice.
Team Resource Management Test and Evaluation
HUM.ET1.ST10.2000-REP-01
Edition Edition Date Status Class
: : : :
1.0 30.11.1999 Released Issue EATMP
EUROPEAN AIR TRAFFIC MANAGEMENT PROGRAMME
DOCUMENT IDENTIFICATION SHEET
DOCUMENT DESCRIPTION
Document Title Team Resource Management Test and Evaluation
EWP DELIVERABLE REFERENCE NUMBER: HUM.ET1.ST10.2000-DEL02 PROGRAMME REFERENCE INDEX: HUM.ET1.ST10.2000-REP-01 EDITION: EDITION DATE: 1.0 30.11.1999
Abstract This report reflects the work done by the Team Resource Management Task Force II (TRMTFII). The Task Force introduced, tested and evaluated Team Resource Management (TRM) in a number of European States. The document aims to share the experiences of the Task Force and to advise the implementation and application of the TRM concept in the European Civil Aviation Conference area. The report includes a prototype course for TRM and the results of the evaluation of the test phase of the TRM project. Team Resource Management (TRM) Human Resources Team (HRT) CONTACT PERSON: Prototype Course Keywords Introduction Phase Test Phase Customisation Evaluation Phase M. WOLDRING TEL: 3566 DIVISION: DIS/HUM Air Traffic Control Safety Questionnaire (ATCSQ) TRM Facilitation
AUTHORS: Michiel WOLDRING and Anne ISAAC
DOCUMENT STATUS AND TYPE STATUS Working Draft Draft Proposed Issue Released Issue CATEGORY Executive Task Specialist Task Lower Layer Task CLASSIFICATION General Public EATMP Restricted
o o o
o o
o o
ELECTRONIC BACKUP INTERNAL REFERENCE NAME: HOST SYSTEM Microsoft Windows Type: Media Identification: G:Own_use:\Delvrabl\Released\trm\TRMTest.doc MEDIA SOFTWARE MicroSoft Office 97 (MS97)
DOCUMENT APPROVAL The following table identifies all management authorities who have successively approved the present issue of this document.
AUTHORITY Chairman Team Resource Management Task Force II (TRMTFII) Manager Human Resources Programme (HRS) Chairman Human Resources Team (HRT) Senior Director EATMP (SDE)
NAME AND SIGNATURE
DATE
11.11.1999 V. S. M. W0LDRING
11.11.1999 M. BARBARINO
12.11.1999 A. SKONIEZKI
16.11.1999 W. PHILIPP
Edition : 1.0
Released Issue
Page iii
DOCUMENT CHANGE RECORD The following table records the complete history of the successive editions of the present document.
EDITION 0.1 0.2 1.0
DATE 01.03.1999 10.09.1999 30.11.1999 Draft
REASON FOR CHANGE
SECTIONS PAGES AFFECTED All All All
Proposed Issue Released Issue
Page iv
Released Issue
Edition : 1.0
TABLE OF CONTENTS
DOCUMENT IDENTIFICATION SHEET................................................................................ ii DOCUMENT APPROVAL .................................................................................................... iii DOCUMENT CHANGE RECORD ........................................................................................ iv EXECUTIVE SUMMARY ....................................................................................................... 1 1. 1.1 1.2 1.3 1.4 2. INTRODUCTION ........................................................................................................ 3 Preface ....................................................................................................................... 3 Team Resource Management Task Force II ............................................................... 3 Purpose and Scope..................................................................................................... 4 Mandate...................................................................................................................... 4
INFORMATION MATERIAL........................................................................................ 7 2.1 Team Resource Management Video Trailer ................................................................ 7 2.2 Team Resource Management Presentation ................................................................ 7 NATIONAL WORKING GROUPS............................................................................... 9 TEAMWORK-RELATED INCIDENTS....................................................................... 11 TEAM RESOURCE MANAGEMENT PROTOTYPE COURSE ................................. 13 5.1 Copyright Statement ................................................................................................. 13 CUSTOMISATION OF THE PROTOTYPE COURSE ............................................... 15 TEAM RESOURCE MANAGEMENT TEST PHASE................................................. 17 7.1 Participating States ................................................................................................... 17 7.2 Approaches of the Different Participating Sites ......................................................... 17 A FOUR-STEP METHOD FOR CUSTOMISATION .................................................. 25 Rationale................................................................................................................... 25 Conditions for Application.......................................................................................... 25 The Four Steps ......................................................................................................... 26 Facilitation Training for Team Resource Management Facilitators ............................ 27 Ownership of the Customised Team Resource Management Course ....................... 27 TRAINING FOR TEAM RESOURCE MANAGEMENT FACILITATORS................... 29
3. 4. 5.
6. 7.
8. 8.1 8.2 8.3 8.4 8.5 9.
Edition : 1.0
Released Issue
Page v
10. TEAM RESOURCE MANAGEMENT EVALUATION PHASE ................................... 31 10.1 General Recommendations for Implementation..................................................... 31 10.2 Air Traffic Control Safety Questionnaire................................................................. 32 ANNEX: AIR TRAFFIC CONTROL SAFETY QUESTIONNAIRE..................................... 49
REFERENCES .................................................................................................................... 61 ABBREVIATIONS AND ACRONYMS................................................................................. 63 CONTRIBUTORS................................................................................................................ 65
Page vi
Released Issue
Edition : 1.0
EXECUTIVE SUMMARY This report reflects the work of the Team Resource Management Task Force II (TRMTFII). This Task Force prepared and coordinated the introduction, testing and evaluation phases of Team Resource Management (TRM) in a number of European States. The document aims to share the experiences of the Task Force and to advise the implementation and application of the TRM concept in the European Civil Aviation Community. Chapter 1 gives the preface, membership, working methods and mandate of the TRM Task Force and the purpose and scope of this document. Chapters 2 to 4 explain how the TRM information material was created, why TRM national working groups were established and how a database of teamwork-related incidents was created. Chapter 5 explains how a TRM prototype course was developed, including incident reports, video scenarios, exercises, a facilitator's handbook and a participant's handbook. Chapter 6 explains how the prototype course should serve as a basis for developing customised TRM courses for the different EUROCONTROL Member States. Chapter 7 describes how ten European sites have contributed to the test phase of TRM. Their experiences provide a large number of practical guidelines for implementation and application of the TRM concept. Chapter 8 gives a practical method for customisation of the prototype course, as developed and applied by EUROCONTROL. This chapter also explains how TRM facilitators were trained and how the method for customisation led to ownership of the end product. Chapter 9 provides the outline of a facilitator training for TRM as developed and conducted by the EUROCONTROL Institute of Air Navigation Services (IANS) in Luxembourg. Chapter 10 lists the learning points from the managerial aspects of the test phase and describes the results of the quantitative and qualitative evaluation of the test phase of the TRM project. References, membership of TRMTFII and a copy of the Air Traffic Control Safety Questionnaire (ATCSQ) are annexed. The complete TRM prototype course, including incident reports, video scenarios, exercises and the facilitator's and participant's handbooks, are available on CD-ROM and form an integral part of this report.
Edition : 1.0
Released Issue
Page 1
Page intentionally left blank
Page 2
Released Issue
Edition : 1.0
1. 1.1
INTRODUCTION Preface
In July 1994 the Human Resources Team (HRT) of the European Air Traffic Control Harmonisation and Integration Programme (EATCHIP), now the European Air Traffic Management Programme (EATMP), created a Study Group to investigate the possible benefits of, and the requirements for a TRM programme in the European Civil Aviation Conference (ECAC) area. Within its scope the TRM Study Group carried out a literature survey concerning relevant Crew Resource Management (CRM) / TRM publications, a teamworkrelated Air Traffic Control (ATC) incident survey, a questionnaire survey to determine the attitude of controllers to teamwork in ATC and a TRM training survey to identify current team training activities in and outside the ECAC area. The results of these studies clearly indicated that failures in teamwork function contribute to incidents and often have a negative effect on the performance of controllers. This exercise supported the need for a TRM programme and in 1995 the HRT agreed to establish a TRM Task Force for an initial twelve-month period. This Task Force included ECAC State representatives from Austria, France, Germany, Switzerland, the United Kingdom, the EATCHIP Development Directorate (DED; which no longer exists within the new EATMP structure) and the Institute of Air Navigation Services (IANS) of EUROCONTROL, and the International Federation of Air Traffic Controllers' Associations (IFATCA), a mixed team of active controllers, training staff and human factors experts. In 1996 the TRM Task Force completed its mandate and the HRT approved the guidelines for development and implementation of TRM. The guidelines for development dealt with concept, strategy, training courses and evolution. The guidelines for implementation gave direction for introduction, testing, evaluation and convergence.
1.2
Team Resource Management Task Force II

After the approval of the guidelines a second TRM Task Force received a new mandate to prepare and coordinate the introduction, testing and evaluation phases of TRM. TRMTFII was joined by more States and today consists of members from Austria, Denmark, France, Germany, Italy, Portugal, Romania, Switzerland, the United Kingdom, DIS and IANS. The Task Force first focused on the development of a TRM prototype course, with the support of an external consultant, then customised this product for eight test sites spread over Europe; Austria, Denmark, Germany, Ireland, Portugal, Romania, Switzerland and Maastricht Upper Airspace Control Centre (EUROCONTROL, MUAC) have used the TRM prototype course to create their customised courses. Some of these States were supported by). The United Kingdom and France have opted for independent development of their TRM courses; national external consultants supported them.
Edition : 1.0
Released Issue
Page 3
All test sites have contributed to the qualitative evaluation of the test phase. Five of these sites have contributed to the Air Traffic Control Safety Questionnaire (ATCSQ), a quantitative evaluation of changes in teamworkrelated attitudes. Team Resource Management (TRM) is clearly a safety-related issue. An important conclusion of the work done by the TRM Task Force is that TRM is also a very convenient way to introduce human factors related issues in Air Traffic Management (ATM). This would support changes in attitudes and behaviours in the longer term, reducing errors and contributing to better safety cultures.
1.3
Purpose and Scope

This report will present the work of the TRMTFII, i.e. the experiences and conclusions of the introduction, customisation, test and evaluation phases of the TRM project and its prototype course in various States of the ECAC area. The main purpose of this report is to provide practical guidelines for customisation and implementation of the TRM concept. Important lessons were learned and these are the basis for the advice to roll out TRM in the whole ECAC area.
1.4
Mandate
The Task Force started its work in the beginning of 1997 and was made up of representatives from Austria, France, Germany, Italy, Romania, Switzerland, the United Kingdom, DED and IANS. During 1998 Denmark, Portugal and MUAC joined the customisation, test and evaluation phases. The mandate for TRMTFII included the following tasks:

producing information material; creating national TRM working groups; creating a database for teamwork-related incidents; designing a generic prototype course and evaluation material; customising the prototype course; training facilitators and choosing course participants; running and evaluating test courses.
The work in 1997 concentrated on development of information material, installation of national TRM working groups, the selection of team-related incidents and development of the TRM prototype course. The customisation of the prototype course to national requirements, the training of the TRM facilitators and the test courses took place in 1998. Austria, Denmark, France, Germany and the United Kingdom evaluated their test courses on a national basis. Romania, Ireland, Portugal, Switzerland and MUAC contributed to the ATCSQ. The evaluation of the ATCSQ was conducted by the DED5 Human
Page 4
Released Issue
Edition : 1.0
Resources Bureau (now the ATM Human Resources Unit or DIS/HUM in short) at the beginning of 1999.
Edition : 1.0
Released Issue
Page 5
Page 6
Released Issue
Edition : 1.0
2. 2.1
INFORMATION MATERIAL Team Resource Management Video Trailer

For introduction purposes a so-called video trailer has been developed by the video unit of IANS. The trailer shows examples of teamwork in sport, industry and aviation, and aims to introduce the main objectives of TRM. The production is in VHS PAL COLOUR and the duration is four minutes.
2.2
Team Resource Management Presentation

An example presentation was developed for introduction of the test and evaluation phases to national management. The presentation shows the principles of TRM, an overview of the prototype course and the expected further developments. It also includes a framework for an action plan for the local test and evaluation phases. The presentation is available in MicroSoft PowerPoint and in hard copy. It can easily be adapted to the implementation and national or local requirements.
Edition : 1.0
Released Issue
Page 7
Page 8
Released Issue
Edition : 1.0
3.
NATIONAL WORKING GROUPS

The States that participated in the test and evaluation phases each formed TRM national working groups. The national working groups act as local steering committees and ideally consist of an operational manager, a training manager, an incident/accident investigator and operational controllers (the TRM facilitators). Tasks performed by these TRM working groups included: acting as a national or local TRM focus point and a reference to TRM Task Force; organisation of the local introduction of TRM; creation of local introduction material; decision on local introduction methods (newsletters, articles, posters, briefings, etc.); recruitment and selection of TRM facilitators; selection of participants; organisation of the training for facilitators; organisation of the customisation of the TRM prototype course; choice of (local) incidents for the customisation; identification of cultural issues; organisation of a pilot-course (participants, place, time, personal invitations, etc.); evaluation of the pilot-course and management of eventual amendments; definition of feedback to participants and Operations (OPS); communication on progress to OPS and other stakeholders.
Edition : 1.0
Released Issue
Page 9
Page 10
Released Issue
Edition : 1.0
4.
TEAMWORK-RELATED INCIDENTS
The Task Force tried to create a database of real teamwork-related ATC incidents for exercises and case studies in the prototype course. For reasons of confidentiality some were made anonymous. The difficulties - internationally - in the release of confidential information in incident reports meant, however, that most of the incidents were taken from the public domain. Different experiences during the test phase showed that the use of local occurrences generated better in-depth discussions, and the learning experiences from local incidents and case studies were more powerful. An active policy in the international exchange of teamwork-related incidents will positively influence the successful implementation of TRM in Europe.
Edition : 1.0
Released Issue
Page 11
Page 12
Released Issue
Edition : 1.0
5.
TEAM RESOURCE MANAGEMENT PROTOTYPE COURSE

A generic prototype course has been developed with the support of knowledge and experience in Human Factors and operational ATC from several ECAC Member States together with that of an experienced CRM developer in order to gain from the experience of airlines. The prototype course lasts three days and has been prepared in eight separate modules: Introduction, Teamwork, Team Roles, Communication, Situational Awareness, Decision-making, Stress and a Conclusion. It includes evaluation material on both the course and the cultural differences between the States involved (in this context a mixture of national, organisational and safety culture). Incident reports have been placed in several modules. These are all real ATC-related incidents from various countries. Videos and video scenarios have been supplied within some of the modules. The videos have been carefully chosen to illustrate specific points within the modules whereas the video scenarios have been written to enable each country to create their own videos with local conditions and environmental framework. Several activities and exercises have been developed for each module. There are two handbooks accompanying the TRM prototype course: the facilitator's and participant's handbooks. The facilitator's handbook contains all the teaching materials, including information regarding support for the facilitators, a summary of the slides found in each module, instructional notes and pages for the facilitator to take support notes and copies of all activities and exercises. The participant's handbook contains information which includes a selection of the most important slides, summaries with the main safety points regarding each module and pages for the participants to take notes throughout the course. The prototype course, including incident reports, video scenarios, exercises, facilitator's and participant's handbooks are annexed to this report on a CD-ROM.
5.1
Copyright Statement
The TRM prototype course has been developed by an external company, DEDALE (France), under the auspices of the EATCHIP TRMTFII and with the support of a wide range of air traffic personnel from several European States. Copyright and any other right of ownership in respect of the deliverables or parts thereof belong to DEDALE, as a continuation of copyrights and intellectual property rights already existing. The right to use, translate, adapt, modify, print and publish all the deliverables or parts thereof in any manner whatsoever, has been granted by DEDALE to the EUROCONTROL Agency, which may transfer all or part of this grant to ECAC Member States or national ATM organisations on their own terms.
Edition : 1.0
Released Issue
Page 13
ECAC Member States and national organisations may use the material for their own use but may not commercialise the product in any way. Before any ECAC Member State or national ATM organisation uses these deliverables they must accept the above terms.
Page 14
Released Issue
Edition : 1.0
6.
CUSTOMISATION OF THE PROTOTYPE COURSE

The prototype course should serve as a basis for developing customised TRM courses. The Task Force aimed at harmonisation by the application of the 80:20 rule, as predicted in the paper on Development of TRM in European ATC (Isaac & Barbarino, 1998). 80% of the TRM prototype course contains common material and instructions for facilitators and the remaining 20% provides sufficient scope for the States to adapt the course to their needs and to include national examples and cultural influences.
K K
National examples Cultural aspects
20% 80%
K K
Common lesson material Instructions for facilitators
Figure 1: The 80/20 rule for harmonisation Most of the participating States required external support for their customisation, some contracted companies with experience in CRM, while others were supported by EUROCONTROL. A four-step method was developed to guarantee efficient customisation and local ownership of the TRM concept and materials (Woldring & Amat, 1998). This method has been integrally adopted in the training for TRM facilitators at the IANS in Luxembourg. In Chapter 8 we describe the method that EUROCONTROL developed and applied, and some lessons learnt while facilitating the TRM customisation and facilitator training in Austria, Portugal, Ireland, Denmark and MUAC in the Netherlands.
Edition : 1.0
Released Issue
Page 15
Page 16
Released Issue
Edition : 1.0
7. 7.1
TEAM RESOURCE MANAGEMENT TEST PHASE Participating States

Ten European States have contributed to the test phase of the TRM project. The United Kingdom and France have opted for independent development of their TRM courses; they were supported by national external consultants with expertise in airline CRM. Austria, Denmark, Germany, Ireland, Portugal, Romania, Switzerland and EUROCONTROL Upper Airspace Control Centre (MUAC) in the Netherlands have used the TRM prototype course to create their customised courses. Some of these States were supported by DED for training of their facilitators and the actual customisation.
7.2
Approaches of the Different Participating Sites

During the regular Task Force meetings we exchanged experiences and conclusions of the different test-sites. These are reflected in the following chapters in order to provide practical guidelines for customisation and implementation of the TRM concept to those who plan for a TRM roll-out. The reports contain information on TRM promotion, the people involved and their expertise, the balance between practice and theory, methods for customisation, positive experiences and concerns, selection and training of facilitators, impact for customisation and roll-out and the methods of evaluation. During its last meeting The Task Force developed the following table of approaches and experiences (see Table 1) to facilitate the comparison of the experiences in the different States.
Edition : 1.0
Released Issue
Page 17
Table 1: TRM test phase comparative statement UK 1. How many people were involved? 2. National Working Groups? 9 10 Switzerland 4 Germany 10 France 10 Portugal 4 Denmark 5 Romania
12
5 responsible for continuation training of 5 different ATM units 5 facilitators
Head of Qualification Programmes 1 instructor 2 ATCOs c
Project Leader (HF expert) 7 ATCOs (OPS) Inc/Acc investig. Reporting to OPS & HUM Mgrs b DEDALE 8 ATCOs 2 HF experts (future facilitators) - 9 months
Head of training Inc/Acc investig.
Head of continuation training 3 OPS with training experience
ATC Experts 1 psycho
3. Customisation: with DED5 (a)? ext. supp. (b)? self- (c)?
b 3 to 6
4. How many participants ( May 1999)? 5. How many courses/ participants per course? In/outdoor? 6. Any translation?
16
300
21
33
40
255
12/12/12
2/8 NATS conference centre
30 / 10 Outdoor
2 / 12-9 Academy
3 (national) / 7-3 < all 5 ACCs ENAC
7/6
21 / 10-15 (max.) International Hotel
3 (ROMATSA)
No
Yes (F + D)
Yes
Yes
Yes (see 12.
Yes
Yes
Page 18
Released Issue
Edition : 1.0
Table 1: TRM test phase comparative statement (continued) UK 7. Any facilitator (OPS, training, psycho)? Any co-facilitator? OPS/psycho Switzerland OPS Germany 2 trainers 1 OPS supervisor always co-facilitation France 8 people: OPS 1 Inc/Acc investig. Always co-facilitation (novice and experienced) 10 days facilitation technique (they knew the concepts because in development) 50% (different structure) Portugal - 9 facilitators: 1 trainer 6 ATCOs (OPS) 2 HF experts - co-facilitation Denmark 2 OPS 2 trainees (former ATCOs) Romania 4 OPS
8. Training of facilitator (on content TRM or facilitation technique)? 9. Percentage of application of the DEDALE prototype? 10. Duration and structure?
Course adaptation combined with facilitation training 19 days Less than 10%
Content TRM\ Facilitation technique
24 man days (i.e. 3 facilitators for 8 days with IANS) 50%
1 week with DED5 2 facilitators in IANS
1 week with DED5 Customisation done the same week 90% of our course is taken from the prototype and only represent 50% of it 2 days: 09:00-18:00 08:00-16:00
3 with IANSRANA 1 with IANS
50%
50% of the total DEDALE content 85% of the total Portuguese content
100% because was the prototype
3 days including participation of one pilot
3 days
3 days / 8 modules
3 days / 8 modules
5 mornings: 09:0013:00 Unusual situations simulation PM
3 days: 08:00-17:00 st for the 1 5 days for nd rd the 2 & 3 No: all together
11. Split into TWR, APP or en-route controllers?
LHR Terminal control
5 ATM units 5 adapted courses
Always mixed
Only ACC
Yes
Always mixed as much as possible
Edition : 1.0
Released Issue
Page 19
Table 1: TRM test phase comparative statement (continued) UK 12. Top 3 learning experiences? Decisionmaking
Communication
Switzerland Communication (humanoriented) Exchange of experience Tool
Germany Translation is a must Theory/exercises mixed Do not use: 'you should' or 'you must'
France Very enthusiastic facilitators A place where you share experience and taboos Quite safetyoriented Powerful videos
Portugal Any translation should be very accurate Enthusiasm is a major criteria for choosing facilitators Units should be mixed TRM is a very important tool to ATCOs
Denmark Theory & exercise well mixed Facilitators role essential; stay away from content ATCOs are ready to talk about 'soft' issues 'Follow up' 'Very good course' 'Naming things we do' 'So now we are aware and can react'
Romania Careful translation is a must
Stress
13. Participants reactions?
V. High 85% +
'To be communicated annually'
'I hope every ATCO benefits from TRM' 'The winner is the team' 'TRM will only be accepted if you present it the way you did' 'I am glad our company has got TRM'
I am now 'thinking safety' Before: I will be very critic' After: 'Thank you'
All decided that 'TRM was a very important course'. It is therefore necessary to carry on.
14. Evaluation? How?
ATCSQ Feedback
Own course evaluation form 2 ATC units ATCSQ
ATCSQ national basis Action research for course feedback Transfer forms for every module for the participants to fill in most important aspects resp. their plans / intentions, based on what they learnt in this course
Course evaluation (questionnaire) No attitude evaluation
Our own, via a questionnaire
'Action research' for facilitators Evaluation form for the school administration mgrs, etc.
For the 1 :
st
ATCO by
EUROCONTROL
For the 2 and rd 3 : by our system
nd
Page 20
Released Issue
Edition : 1.0
7.2.1
Austria Customisation was performed in December 1997 by a team of five operational controllers facilitated by the Human Resources Bureau - DED5. The team felt under-confident and it was decided to seek external support for the actual running of TRM. Austria started with a one-day seminar covering stress and the TRM Module Teamwork in December 1998. There were about twenty participants, representing most of the units. The TRM Stress Module was facilitated by a psychologist. TRM was promoted by a letter to all controllers. It is planned to select TRM facilitators from future TRM courses. A questionnaire for evaluation is under development. Evaluation will be anonymous and no data from the evaluation will be shared with other parties.
7.2.2
Denmark Customisation was performed in Week 25 of 1998 by a team of four operational controllers/facilitators and two human resources experts, and was facilitated by the DED5 Human Resources Bureau. At the start there was some under-confidence amongst the new facilitators but, as things progressed, their confidence grew. The total effort of forty man-days was spread over fifteen working days. Due to a short course time (two days per course), the Stress Module has been delayed until further notice. Twenty-one two-day courses were held during winter 98/99, making a total of 280 participants. TRM promotion is done through an article in a local information bulletin. Information has been provided at several continuation training seminars.
7.2.3
EUROCONTROL Maastricht Upper Airspace Control Centre Their first TRM course was held in Week 10 of 1998. Customisation was performed in Week 5 by a team of four operational controllers facilitated by DED5. First results of the evaluation were very positive. MUAC contributed to the ATCSQ evaluation. A second course took place in March 1999. Participants in these first courses will be invited to take on the role of TRM facilitator for future roll-out of the programme. Separate training in facilitation techniques for these volunteers will be organised.
7.2.4
Ireland Two operational controllers/facilitators from the national working group of the Irish Aviation Authority customised the Irish TRM course together with the Portuguese national working group in Week 6 of 1998. Facilitator training and customisation were facilitated by DED5. The first Irish TRM course was facilitated in Week 19. DED5 was asked to support the facilitation of the 'Stress' Module. Ireland contributed to the ATCSQ evaluation.
7.2.5
France The customisation team consists of the TRM Task Force member and eight Air Traffic Controllers (ATCOs) from three Approach Control Centres (ACCs),
Edition : 1.0
Released Issue
Page 21
who have all been working on the project for approximately nine months, spread over eight two-three days seminars, in total twenty working days per person. Customisation has been facilitated by the CRM consultant who also supported the Task Force in the making of the prototype course. The prototype course has been translated into French. Selection of relevant material from the prototype has been done by answering the question: 'What are the most important factors for safety?'. A lot has been taken from the prototype course, but much of it has been structured differently. Video-material has been developed on the subjects sector-splitting and taboos. Error management, situational awareness, teamwork and team roles are the most relevant parts of the course. Facilitators were trained between December 1998 and February 1999 (ten days in total). Three courses were organised from March to May 1999, with thirty-three participants from five French ACCs. The first course has been developed for en-route control. After the first experiences a similar iterative process will start in 1999 for the design of a course for approach control. 7.2.6 Germany In Germany two TRM courses were held successfully at their training academy in January and March 1999. TRM has been promoted in the Deutsche Flugsicherung (DFS, German ATC Corporation) by an article in the DFS magazine, a series of posters at their training academy and presentations on TRM during the course of 'team days' at operational units. The prototype material was completely translated into German by the members of the national working group because it was considered very important to discuss Human Factors with the course participants in their mother tongue. The working group consisted of two active controllers and two trainers. Two of them facilitated the prototype courses while the two others were watching and looking for things that needed improvement. The course content was found relevant by the course participants and beneficial to them. Germany has only used about 50% of the original prototype material because of time constraints (the course takes three intensive days). Nevertheless, the course contains all eight modules and consists of about 35% theory input and another 65% of participant activity. The TRM Module entitled Team Roles was found most difficult to customize although it was considered very important for the course. Facilitator training took place at the IANS and will probably be organized within DFS in future. After the official start of the programme the facilitators will be controllers from operational units. They will be trained after they have attended a regular TRM course. The estimated effort to train approximately 1500 controllers is 5300 man-days, including training for another twelve facilitators. Germany uses, on a national basis, the ATCSQ for the evaluation of the results. 7.2.7 Portugal Team Resource Management (TRM) was introduced through a presentation by DED5, after which a poster with the 'TRM Rules' was sent to all ATC units. A TRM article was published and participants in the test-courses received a TRM T-shirt. The first three Portuguese TRM courses were held in Weeks 11, 12 and 13 of 1998, in combination with contingency training. Facilitator training
Page 22
Released Issue
Edition : 1.0
and customisation were facilitated by DED5, together with colleagues of the national working group of Ireland. The Portuguese national working group consists of the TRM Task Force member (national coordinator), four ATCOs and a member of the Incident Investigation Bureau. They were all involved in the customisation of the prototype course. Course material was translated into Portuguese. The Stress Module has been developed and facilitated by a psychologist. First evaluation of the course was very positive and included an encouraging note: 'This was, in terms of training, the most important thing that happened in my 29 years as an ATCO'. Two On-the-Job Training (OJT) instructors assisted at the course and one TRM facilitator attended the simulation session. After the regular technical debriefing a TRM-related debriefing was performed. There was no assessment of TRM skills. Portugal contributed to the ATCSQ evaluation. 7.2.8 Romania Customisation was done by the national working group during January and February 1998. Facilitators were trained at IANS. A first three-day course was held in Week 11 of 1998. Participants were chosen from six geographical locations. The feedback from participants was positive; it was suggested to extend the course into a five-day course and that the Stress Module should be facilitated by a psychologist. A second course was scheduled in October 1998. Romania contributed to the ATCSQ evaluation. 7.2.9 Switzerland Swisscontrol introduced a yearly four to five day Human Factors workshop, currently dealing with TRM. The aim of the workshop is to reach and to maintain the same level of human factors knowledge in all Swiss ATM units, adapted to local cultures. Participants are those responsible for continuation training and facilitators from the Geneva Area Control Centre (ACC), Aerodrome Control Tower (TWR) and Flight Data Assistance (FDA) and from the Zurich ACC, TWR, FDA and Aeronautical Information Services (AIS). The Modules Introduction, Communication, Teamwork and Team Roles were presented in November 1997, supported by Swissair. Two blocks were presented in November 1998: Decision-making and Situational Awareness, supported by Swissair, and Errors and Safety Culture, supported by the Universities of Bern and Freiburg. Human factors specific courses for instructors and OJT trainers in Geneva highlight the relationship between coach and trainee by applying the TRM topics Teamwork, Team Roles, Situational Awareness and Communication. TRM has also been introduced to all Ab Initio controllers. Switzerland contributed to the ATCSQ evaluation. 7.2.10 United Kingdom The first UK TRM course was held in Week 6 of 1998. All modules were found relevant and the length - three days - appropriate. The TRM Stress Module was facilitated by a psychologist. The two keys to success were the
Edition : 1.0
Released Issue
Page 23
involvement of the facilitators from the beginning of the customisation and the fact that facilitators must believe in the material. UK contracted British Airways for facilitator training and customisation. Customisation was found to be more difficult than facilitation training; only a small percentage of the prototype was used. The Leadership Module proposed in the prototype was not found suitable to the British culture. It was found to be important to use recent and local case studies. Evaluation was performed using both the National Air Traffic Services (NATS) course evaluation questionnaire (88.6% satisfaction) and the ATCSQ. A first analysis of the ATCSQ showed that the relationship of TRM to safety, the peer pressure and junior versus senior pressure were the most relevant issues.
Page 24
Released Issue
Edition : 1.0
8.
A FOUR-STEP METHOD FOR CUSTOMISATION

EUROCONTROL developed and applied a four-step method for the customisation of the prototype course in the different test-sites for TRM (Woldring & Amat, 1998). Important lessons were learnt while facilitating the TRM customisation and facilitator training in Austria, Portugal, Ireland, Denmark and MUAC.
8.1
Rationale
The subject Human Factors is independent of culture - often seen as a rather 'fuzzy' subject. We realised that a structured approach would certainly help to overcome any reluctance. There was also a benefit in explicitly applying a facilitation technique: the technique would work as a role model to show that facilitation of complex discussions is quite possible (discussions were needed to help the participants understand the relevance of the different topics). A standard method would also enable us to compare the customisations in the different States. And, last but not least, we would gain time. Initial customisation of all eight modules took only approximately four days, after which participants needed one more week to finalise the product (translation, creation of incident reports, exercise preparation, duplication, rehearsal).
8.2
Conditions for Application

The method has always been applied at least one month after an initial presentation of the TRM concept and the outline of the prototype for management and operational controllers. In these presentations the importance of the formation of a national working group was underlined and a copy of the prototype course was handed over to the group. The national working group would act as steering group and ideally existed of an operational manager, a training manager, an incident/accident investigator and operational controllers (the TRM facilitators). The incident/accident investigators in Portugal and in MUAC actively contributed to the customisation. In the other States we had to build on the experience of the operational controllers for the creation of local incident examples. The controllers/facilitators were selected on their active interest in human factors. We trained them in basic facilitation techniques, we repeated the overview of the TRM concept and the prototype and we explained the four-step method before we started the actual customisation.
Edition : 1.0
Released Issue
Page 25
8.3
8.3.1
The Four Steps

Step 1 Step 1 was given the slogan overview. Main objectives of step 1 are to introduce the issues and the scope of the expected discussions and to emphasise the structure of each module. This structure is obviously not compulsory and can be changed, but the predefined structures were successfully used in steps 1, 2 and even in 3. It was very important for the participants to keep enough distance from the material during steps 2 and 3 so that they could put things into context. We used one Overhead Projector (OHP) slide per module to explain the most important messages. This slide corresponds to a table of content but is formulated in questions. For each question we briefly explained the underlying human factors issues and the different exercises and activities.
OVERVIEW LOOK THROUGH RELATE TO YOU JUST FOR YOU

Figure 2: The four steps to customise the TRM prototype course 8.3.2 Step 2 Step 2 was given the slogan look through. Objectives are to understand the aims of each module (content) and to understand the suggested techniques of facilitation or instruction (process). In this step we presented the whole module by leading the participants through the course material. We ran - where necessary - parts of the course, showed different games, looked at available videos, performed exercises and debated and debriefed case studies and incident reports. 8.3.3 Step 3 Step 3 was given the slogan relate to you. The main objective of step 3 is to assess the relevance of each part of the prototype course to the participants culture - both in terms of content (is the message relevant for us?) and in terms of process (does this technique suit our culture?). In step 3 we discussed in detail the content and the relevance of the different topics and techniques, suggestions for alternative messages, questions,
Page 26
Released Issue
Edition : 1.0
exercises, examples, pictures etc. In this phase we applied explicitly the different facilitation techniques that were taught at the beginning of the week. 8.3.4 Step 4 The slogan for step 4 was just for you. The main objectives of this step are to screen the prototype material, and to modify the selected material that requires customisation. Next to that a realistic schedule per module had to be decided. The modifications and the order in the prototype material were made immediately. Items were put on a 'to-do' list when the customisation required much local information (local incidents, new developed exercises, relevant statistics).
8.4
Facilitation Training for Team Resource Management Facilitators

At the beginning of each customisation we trained the future TRM facilitators (operational controllers) in a selection of facilitation techniques. Recurrent fears were expressed by the participants: how do I get a group to talk and, when they talk, how do I stop or steer them, what to do with unexpected conclusions, how to get rid of my tendency to teach and how to predict the outcome of a discussion? TRM facilitation is based on techniques for selfpresentation, mini-lessons, open and closed questions and introducing and summarising discussions in using flip charts. Already during the facilitation training we used examples and exercises from the TRM prototype course (during the customisation we could repetitively refer to the exercises in which the TRM facilitators performed themselves). During the actual customisation we explicitly applied the different facilitation techniques; this functioned as a role model for the inexperienced future controllers/facilitators. Especially during step 4 of the customisation we encouraged the participants to act as facilitators. Fears gradually disappeared. The different facilitation techniques that we applied and taught were highly appreciated by all participants in all States; everybody participated actively.
8.5
Ownership of the Customised Team Resource Management Course

The interactivity between facilitators and participants changed during the four steps in the customisations. Step 1 was very much one-way communication, the trainers/facilitators explaining and informing the participants. In step two the participants became active through questions for clarification and background information and in the exercises and case studies the participants were fully involved. Step 3 was characterised by discussions amongst the participants and exchange of ideas with the trainers/facilitators for alternatives for the prototype materials. In step 4 the participants were in charge, they created the final product, with only little process-oriented interference from the trainers/facilitators.
Edition : 1.0
Released Issue
Page 27
Facilitator
Step 1 Step 2
Step 3 Step 4
Participants
Figure 3: Activity swap during the four steps This swap in activity during the process of customisation led to ownership of the end product by the participants - they considered the outcome as 'their' course.
Page 28
Released Issue
Edition : 1.0
9.
TRAINING FOR TEAM RESOURCE MANAGEMENT FACILITATORS

A first training in facilitation techniques was organised by the Task Force and EUROCONTROL in November 1997. The members of the TRM Task Force and an international selection of trainers and operational controllers who volunteered for the TRM facilitators role were invited to attend. Conduct of the training was outsourced to RANA International Inc., through Cranfield University, UK. The RANA techniques were incorporated in the facilitation techniques as applied during the test and evaluation phases of the TRM project. The Institute of Air Navigation Services (IANS) has now developed a training course for future TRM facilitators. The principle objectives of the training are to develop positive attitudes and behaviour towards teamwork skills and human performance in ATC and to reduce or minimise the impact of teamwork-related errors within the ATM system. The aim of the course is to provide participants with a clear understanding of TRM concepts and to practice and choose appropriate facilitation techniques. The training course includes basics of presentation and teaching techniques, questioning techniques, a demonstration of the complete TRM prototype course, practical facilitation exercises and an introduction to the principles of customisation. The course lasts eight working days.
Edition : 1.0
Released Issue
Page 29
Page 30
Released Issue
Edition : 1.0
10. 10.1
TEAM RESOURCE MANAGEMENT EVALUATION PHASE General Recommendations for Implementation

The Task Force faced several managerial aspects for the testing and evaluation phases of TRM, such as the promotion of the TRM concept in the different test sites, the selection and training of facilitators and the adaptation of existing training activities to the TRM concept. Some of the managerial experiences may serve as valuable lessons for the implementation and further guidance of the roll-out of TRM: All sites where English is not the mother tongue have translated the TRM programme into their own language. All TRM courses should run with at least two facilitators. Some environments have mixed functional units and may want to split the course and customise the programme to suit these different units. There has generally been a positive experience when outside professionals have been used to help support the customisation process. TRM training for facilitators throughout Europe should continue with the IANS and the EUROCONTROL ATM Human Resources Unit (or DIS/HUM), with States requesting help when needed.
10.1.1
Team Resource Management Promotion Active management support and a carefully prepared information campaign exert critical influence over the attitudes towards the TRM concept. Active promotion should convince management, operations and (future) TRM facilitators. Also the involvement of incident/accident investigators in the customisation was of high value in the choice of local examples for the different case studies and exercises. The Task Force has also successfully experimented with aide-memoires like T-shirts with TRM logo, TRM posters, and TRM credit cards with TRM logo and key message. Other ideas are TRM ties or shawls, TRM coffee or tea cups (a well-known attribute in any 24-hour operation) and TRM bonbons. A personal invitation to the different TRM events works better than using pamphlets. An article in the national or in-house magazine with a clear link to aviation safety - as communicated in the Introduction' Module of the prototype course - has also proven to be a good promoter.
Edition : 1.0
Released Issue
Page 31
10.1.2
Team Resource Management Platform Amongst all volunteering or selected TRM facilitators a clear need developed for national and international exchange of experiences. A TRM platform should initiate several actions, like: create, update and exchange a list of people currently active in TRM facilitation and/or development; provide a regular newsletter for TRM facilitators; open a web site on the Internet for information; open a mailing list on the Internet for exchange of experiences and ideas; initiate facilitator visits and/or meetings; national and ECAC-wide; organise regular workshops for facilitators with the aim of exchanging experience and materials such as books, case studies and exercises, to increase self-confidence and comfort in the concept and to raise international awareness.
10.2
Air Traffic Control Safety Questionnaire

The objective of TRM is the use of all available resources - information, equipment and people - in order to achieve the safe and efficient movement of air traffic. This objective is obviously ambitious and as with the history of its counterpart on the flight deck - CRM - there will be difficulties in the measurement of its effectiveness. However, to ignore the challenge of this evaluation would be foolish. To this end the measurement and evaluation of the TRM programme was undertaken. The programme evaluation should not only provide information on the effects of the training, but it should also provide direction for continued training. The most basic type of information comes from participant evaluations, usually collected by questionnaire at the end of the training course (see Annex). Positive reactions to the training provide necessary, but not sufficient evidence of impact. That is, while a positive reaction to the training is not sufficient in itself to indicate a positive shift in behaviour, by the same token a negative reaction to the training is an almost certain indication that positive behaviour change is not going to occur. A second source of information comes from the use of an evaluative questionnaire concerned with attitudes and behaviours, which can be administered before and after the training sessions (see Annex). Often a more robust method of evaluating the changes of these attitudes and behaviours can be captured by administrating a third identical questionnaire some four to six months after the training course.
Page 32
Released Issue
Edition : 1.0
A third and more rigorous evaluation comes from the correlation of these attitudinal changes with observation or interview of the same personnel to gauge meaningful behavioural changes. From this methodology, measurable positive changes in interaction should be present following the training. Lastly, the ultimate validation can be found in the correlation of the training programme and a reduction in the frequency of incidents within the system. The latter two methodologies are highly complex and take considerable time to achieve. It is for this reason that the test and evaluation phase of the TRM programme used only the course evaluation and the monitoring of attitudinal and behavioural changes as an assessment of its effectiveness. 10.2.1 Development of the Air Traffic Control Safety Questionnaire The development of the Air Traffic Control Safety Questionnaire (ATCSQ) was based on the work undertaken in flight crew resource management (FMAQ, Helmreich, Merritt, Sherman, Gregovich & Weiner, 1993) and operation room management (ORMAQ, Helmreich, Schaefer, Hines & Sexton, 1996). The TRM programme clearly defines seven areas of concern in its training and these can be grouped in the following way: Human Error and Safety, Teamwork, Team Roles, Communication, Situational Awareness, Decision-making, Stress Management.
A matrix of issues within these seven areas were identified as the basis of questions to be evaluated with the ATCSQ. These can be seen in Figure 4.
Edition : 1.0
Released Issue
Page 33
Human Error and Safety

reporting occurrences confidence in own ability confidence in team performance in ALL situations reporting safety issues to management concern for impaired colleagues management concerns for safety
Communication
communication with pilots communication of decisions to team members explicit encouragement of participation critique self and other team members when appropriate use of correct phraseology
Situational Awareness
workload issues monitoring all relevant operational factors retaining the picture in all situations discussing the common picture with other team members
Teamwork and Team Roles

use of appropriate techniques to manage interpersonal and operational conflicts adaptation to interpersonal differences issues of supervisory position control in training situations styles of leadership the outcome of automation on teamwork
Decision-making
involvement of whole team in decision-making making decisions in normal and emergency situations decision-making when fatigued decision-making by adjacent sectors
Stress Management
coping effectively with stress avoiding conflict managing time for accomplishing tasks satisfaction with job sharing stress problems with colleagues understanding own stress
Figure 4: Areas in TRM The questionnaire itself was designed in such a way that it could be divided into four main sections. Section 1 contained a number of questions regarding the perceived satisfaction of basic skills, handling of normal and emergency traffic, training instruction, shift arrangements, leave and quality of manuals. The responses
Page 34
Released Issue
Edition : 1.0
were given on a five-point scale (A-E) from Very Unsatisfactory to Very Satisfactory. Section 4, at the end, covered questions regarding demographic data such as gender, years of service, years in functional positions and present position. Sections 2 and 3 contained all those questions associated with attitudes and behaviours towards ATCO work which were summarised in Figure 4. Both sections were deliberately designed to ascertain two different concepts. One is the attitudes of the controllers towards various issues, i.e. Q11: asking for assistance makes one appear incompetent, and the other is whether the controllers actually showed these attitudes or behaviours in their working environment, i.e. Q46: When my workload is high I ask for assistance. To this end, although all the questions in each section had responses based on a fivepoint scale (from 1 to 5), they necessarily had slightly different response structures. All seven areas found in TRM appeared in the questions in both sections. These individual areas were randomly allocated but balanced across both sections. The responses in Section 2 were on a five-point scale from Strongly Agree to Strongly Disagree. In Section 3 the responses were associated, although they reflected the five-point divisions, with the frequency with which the controller agreed with the statements (0-100%). 10.2.2 Stability and Reliability The ATCSQ was subject to both test-retest and Cronbach Alpha reliability tests. The test-retest reliability was undertaken with 25 subjects from States other than those involved in the test and evaluation phase of the TRM programme. The test-retest reliability was r=.67 which was considered reliable for this type of questionnaire, a subjective and self-report format. A t test was also performed on this data and revealed no significant differences between the first and second responses. This shows that the control group did not change their responses indicating the stability of this questionnaire. The reliability of the questionnaire items was subject to a Cronbach Alpha test with 147 subjects1 from all sites involved in the test and evaluation phase. The result for this reliability was r=.71, when considering the main 74 question items, which was highly reliable considering the nature of the questionnaire. When considering the reliability of the first 38 question items the Cronbach Alpha result was r=.64. However, there were several questions which were clearly causing confusion. These were analysed further and it was realised that: either there was a natural reversal in their meaning, i.e. Q 20 Controllers do not use their strips to help maintain a mental picture;
It should be noted that although the total number of cases in this evaluation was 147, in some analyses only some of these cases could be considered because of subjects who did not respond to all questions.
Edition : 1.0
Released Issue
Page 35
or the language was perhaps too complex, i.e. Q 7- I am reluctant to disagree with my supervisors
These issues should be addressed in the further development and use of the ATCSQ. The ATCSQ has proven to be a stable and reliable instrument for the purpose of electing responses in the seven domains for which it was designed. Several questions have been identified as needing alteration, particularly in multilingual and multicultural environments. 10.2.3 Demographic Statistics Tables 2 and 3 detail firstly the overall and then the country breakdown concerning factors such as gender, average years in ATC and the average years in each functional position. Table 2: Breakdown of demographic data Total Number Gender 147 Male Female No response Total Tower Approach ACC 120 16 11 16.3 12.9 12.3 9.0
Average years in ATC
Table 3: Breakdown of demographic data by site Site/Items Gender M F Average years in ATC Average years: in Tower in Approach in ACC 10.2.4 Descriptive Statistics The following graphs (see Figures 5 to 17) indicate the variables associated with the first section of the questionnaire, those questions on the perceived satisfaction of basic skill, handling of normal and emergency situations, training, instruction, shift arrangements, leave and the quality of manuals in the operational units from all test sites. 1 84 12 14.6 12.2 11.3 5.5 2 12 11.4 9.6 9.6 7.8 3 17 21.8 18.3 16.1 16.2 4 8 3 20.5 1.7 9.0 17.7 5 9 1 26.2 11.3 11.4 15.6
Page 36
Released Issue
Edition : 1.0
40
35
30
25
20
15
10
0 V. Unsatisfactory Unsatisfactory Neutral Categories Satisfactory V.Satisfactory
Figure 5: Q1 - your own basic ATC training With regard to their own basic ATC training the majority of responses indicated a satisfactory (38%) or very satisfactory (11%) response.
40
35
30
25
20
15
10
Figure 6: Q2 - Your own basic ATC instructor training In response to the satisfaction with their own ATC instructor training the majority indicated either satisfactory (38%) or neutral (13%).
Edition : 1.0
Released Issue
Page 37
30
25
20
15
10
Figure 7: Q3 - Your own validation or recurrent training With regard to satisfaction with their own validation or recurrent training responses varied between very satisfactory (7%), satisfactory (28%), neutral (12%), unsatisfactory (13%) and very unsatisfactory (2%).
30
25
20
15
10
Figure 8: Q4 - Your own OJT instructor skills The responses to their own OJT instructor skills again varied. 12% responded as unsatisfactory, 11% as neutral, whereas the highest responses came from the categories of satisfactory (28%) and very satisfactory (7%).
Page 38
Released Issue
Edition : 1.0
30
25
20
15
10
Figure 9: Q5 - Simulator training The majority of responses indicated that attitudes towards simulator training were satisfactory (29%). However, 14% felt this training was unsatisfactory and 11% indicated a neutral response.
25
20
15
10
Figure 10: Q6 - Operations Manuals Approximately 23% responded that they were satisfied with the quality of operations manuals and 7% felt very satisfied. 15% considered they were unsatisfactory and 2% felt they were very unsatisfactory. 13% indicated a neutral response.
Edition : 1.0
Released Issue
Page 39
25
20
15
10
Figure 11: Q7 - Safety Manuals There was rather a mixed response with regard to the quality of safety manuals. The majority felt these were unsatisfactory (20.5%), although 17% found them satisfactory. Nearly 15% gave a neutral response.
35
30
25
20
15
10
Figure 12: Q8 - Shift cycle The majority of responses with regard to shift cycles were satisfactory (24%) or very satisfactory (13%). 9% indicated a neutral response.
Page 40
Released Issue
Edition : 1.0
35
30
25
20
15
10
Figure 13: Q9 - Shift schedule A similar pattern with regard to shift schedules indicated that the majority were satisfied (23%) or very satisfied (16%) with their present arrangements.
35
30
25
20
15
10
Figure 14: Q10 - Length of leave Again, the majority (23%) indicated that they were satisfied or very satisfied (9%) with their length of leave. However, 13% indicated a neutral response to this question.
Edition : 1.0
Released Issue
Page 41
40
35
30
25
20
15
10
Figure 15: Q11 - My skills in handling normal operations 27% of those who responded considered that their skills in handling normal traffic were satisfactory or very satisfactory (21%).
30
25
20
15
10
Figure 16: Q12 - My skills in handling emergencies Responses to skills in handling emergency traffic were a little more varied. 28% considered their skills were satisfactory whilst 7% felt they were very satisfactory. 7% felt they were unsatisfactory and 17% indicated a neutral response.
Page 42
Released Issue
Edition : 1.0
30
25
20
15
10
Figure 17: Q13 - Feedback on my daily operational performance 30% of those who responded felt that feedback on their daily operational performance was satisfactory and 7% felt it was unsatisfactory. 5% felt that it was very satisfactory whilst 2% felt it was very unsatisfactory. 16% indicated a neutral response.
These responses indicate generally a high level of satisfaction for issues involving training, although there was some concern shown in OJTI and simulator issues. Shift cycles, shift schedules and length of leave were considered satisfactory. Concern was shown for the status of the operations and safety manuals and the issues involved when handling emergency traffic and feedback in the operational environment.
10.2.5
Statistical Analyses T test analysis was undertaken on the first 38 items of the questionnaire (Section 2) from the first administration before the course, and to the second administration after the course. For the largest complete sample from four test sites (126 subjects) the result was t=6.55 (d.f.125) p<.000, which is a highly significant result indicating there had been a change in the attitude of subjects. As this sample could not be used as a whole for further ad hoc analysis, little can be deduced in terms of actual areas of change. However, a second sample of 29 subjects from three test sites was more thoroughly analysed. This t test analysis revealed a result of t=3.89 (d.f.28) p<.001, again a highly significant result indicating there had been a change in the attitude of subjects. Further investigation indicated that 17 questions, within the 38 items, showed a significant change. Table 4 indicates these results in more detail.
Edition : 1.0
Released Issue
Page 43
Table 4: t test results from the ATCSQ Question Number 1. Question Text Controllers leave personal problems behind when operating the position. Automation reduces the requirement for team members to monitor the traffic situation closely. I am less effective when stressed or fatigued. Effective team coordination requires that controllers take the personalities of other controllers into account. I am reluctant to disagree with my supervisors. Controllers should be aware and sensitive to the personal problems of other controllers. To resolve conflicts controllers should openly discuss their strategies with each other. Trainees should not question senior team members decisions. Controllers visibly impaired by alcohol or drugs should be kept from going on duty. Good communication is as important as technical proficiency in the controlling environment. I like my job. My unit would be capable of handling the situation if there was a system breakdown. t value -2.42 d. f. 28 p value <.022
2.
2.71
28
<.011
3. 6.
3.11 3.13
28 28
<.004 <.004
7. 8.
-3.15 3.68
28 28
<.004 <.001
13.
4.46
28
<.000
16. 25.
2.21 2.35
28 28
<.035 <.026
26.
4.74
28
<.000
27. 29.
5.52 4.91
28 28
<.000 <.000
Page 44
Released Issue
Edition : 1.0
Table 4: t test results from the ATCSQ (continued) Question Number 30. Question Text I should maintain the traffic picture of the controllers I work with. The regulatory organisation rules should not be broken. Supervisors who encourage suggestions from team members are ineffective. I should inform those controllers who are affected by my plans and control actions and ask for their acknowledgement. I perform as well with other units as with my own. t value 3.54 d. f. 28 p value <.001
32. 34.
2.56 4.52
28 28
<.016 <.000
35.
2.99
28
<.006
38.
3.46
28
<.002
The changes found in the response to each question varied from either disagree to agree or from agree to disagree. The strength of change in all cases, however, was significant. Examples of clear changes in attitudes can be seen in the following highly significant responses. Q1. In their first response, before the course, subjects slightly agreed that they left their personal problems behind when operating the position. However, their second response, after the course, indicated that they slightly disagreed with this statement. Q2. Subjects slightly agreed before the course that automation reduced the requirement for team members to monitor the traffic situation closely. After the course they slightly disagreed with this statement. Q3. Subjects in their first response indicated that they agreed that automation would reduce the requirement for team members to monitor the traffic situation. In their second response they disagreed with this statement. Q6. In their first response before the course, subjects slightly agreed that effective team coordination requires controllers to take into account the personalities of other controllers. In their second response, after the course, they agreed more strongly with this statement. Q7. In their first response subjects slightly agreed with the statement am reluctant to disagree with my superiors. In their second response they chose a neutral category.
Edition : 1.0
Released Issue
Page 45
Q8. Subjects in their first response indicated that they disagreed that they should be aware and sensitive to the personal problems of other controllers. In their second response they agreed with this statement. Q13. Subjects in their first response slightly agreed that to resolve conflicts, controllers should openly discuss their strategies with each other. In their second response they slightly disagreed with this statement. Q16. Subjects before the course slightly agreed that trainees should not question senior team members decisions, but after the course they were more neutral. Q25. Subjects changed from strongly agree to slightly agree when asked whether controllers visibly impaired by alcohol and drugs should be kept from going on duty. Q26. In their first response subjects agreed more strongly than their second response that good communication is as important as technical proficiency in the controlling environment. Q27. Subjects changed from strongly agree to slightly agree when asked whether they liked their job. Q30. Subjects changed during the course from disagree to agree when asked about maintaining the traffic picture of the controllers they work with. Q32. Firstly subjects slightly agreed that regulatory organisation rules should not be broken. In the second response they were more neutral. Q34. When considering the question supervisors who encourage suggestions from team members are ineffective, the subjects responded before the course that they strongly agreed and after the course in a more neutral way. Q35. In their first response, subjects slightly agreed that they should inform those controllers who are affected by their plans and control action and ask for their acknowledgement. In their second response they were more neutral. Q38. Subjects changed during the course from 'slightly agreeing' to the statement: 'I perform as well with other units as with my own' to 'slightly disagreeing' with this statement.
The ATCSQ has been found to be a useful indicator of attitude change within certain domains. Clearly with such a small number of responses, little can be deduced with any certainty, but the statistical analysis does indicate that there were strong changes between the first and last responses, particularly with respect to teamwork and team roles.
Page 46
Released Issue
Edition : 1.0
10.2.6
Summary and Conclusions The Air Traffic Control Safety Questionnaire (ATCSQ) was developed to enable the evaluation of the Team Resource Management (TRM) programme. Several other questionnaires which have been developed for similar purposes were reviewed and the architecture of the ATCSQ reflected these developments within the flight deck and operating room environments. The questionnaire consists of four main sections. The first section concerns attitudes towards the quality of training, working conditions and documentation. The last section concerns demographic information: the second and third sections contain the main evaluative information, the second being concerned with attitudes and the third with those responses associated with behaviour. The results of the evaluation of this questionnaire clearly indicate the ATCSQ to be a robust, reliable instrument and informative for the purpose for which it was designed. A few questions indicated some confusion and should therefore be re-worded for future use. In terms of the results concerning the attitudes towards the professional training and working environment, the questionnaire clearly indicated acceptable satisfaction within most of the areas of training, but some aspects with respect to operation and safety manuals, handling of emergency traffic and feedback in daily operations were not as positive. The results associated with the attitudes of the subjects were based on a small response sample 29 subjects. This could, to some extent, give a rather biased picture of the outcome of the course itself and must be considered when discussing the conclusions. The results, however, illustrated a significant change within some of the subject domains. Most of these changes were in the hypothesised direction, i.e. the course had changed the participants' attitudes in favour of better and more cooperative teamwork and more sympathetic team roles. However, there were a few responses which were difficult to explain; for instance, question 27 which changed from strongly agreeing that they enjoyed their job to slightly agreeing to this statement. Some of these responses may be the result of a strong bias within the subject sample which typically skews the results in a small number of responses. Overall the results indicate a well-balanced and informative questionnaire which can be considered robust and reliable. Responses concerning the change in attitudes between the two courses are a little more difficult to determine. However, the results indicate that the questionnaire is sensitive to changes in attitude, and with a larger sample and strict adherence to data gathering a more meaningful database can be considered in the future. This will not only strengthen the use of such a questionnaire but will also help individual States to customise their individual needs in the their TRM programmes.
Edition : 1.0
Released Issue
Page 47
Page 48
Released Issue
Edition : 1.0
ANNEX:
AIR TRAFFIC CONTROL SAFETY QUESTIONNAIRE
ATCSQ
This questionnaire is part of a study aiming at understanding ATC operational safety. You will greatly assist our research by completing this survey. All data are strictly confidential. Results will be presented only at the group level. No individual feedback will be given to management, so please be honest in your responses.
Please indicate how satisfied you are with each of the following aspects of ATC operations. Please answer by writing beside each item the letter from the scale below.
Very Unsatisfactory
Unsatisfactory
Neutral
Satisfactory
Very Satisfactory
1. 2. 3. 4. 5. 6.
Your own basic ATC training Your own basic ATC instructor skills Your own validation or recurrent training Your own OJT instructor skills Simulator training (if relevant) Operational Manuals (including Standard Procedures)
7. 8. 9.
Safety Manuals Shift cycle Shift schedule
10. Length of leave 11. My skills in handling normal operations 12. My skills in handling emergencies 13. Feedback on my daily operational performance
Edition : 1.0
Released Issue
Page 49
Please answer the following questions by ticking the box which best describes your opinion. 1. Controllers leave personal problems behind when operating the position.
Strongly Disagree Slightly Disagree Neutral Slightly Agree Strongly Agree
2.
Automation reduces the requirement for team members to monitor the traffic situation closely.
3.
I am less effective when stressed or fatigued.

4.
It is not my place to give pilots advice other than airways information and clearance details.
5.
Team members share responsibility for prioritising activities in high workload situations.
6.
Effective team coordination requires that controllers take the personalities of other controllers into account.
Page 50
Released Issue
Edition : 1.0
Please answer the following questions by ticking the box which best describes your opinion. 7. I am reluctant to disagree with my superiors.
8.
Controllers should be aware of, and sensitive to, the personal problems of other controllers.
9.
I work in an environment where the groups achievements are valued over my individual success.
10. It is easier to make decisions when you first take over on an operating position.
11. Asking for assistance makes one appear incompetent.

12. My suggestions about safety will be acted upon if I express them to management.
Edition : 1.0
Released Issue
Page 51
Please answer the following questions by ticking the box which best describes your opinion. 13. To resolve conflicts controllers should openly discuss their strategies with each other.
14. It is easier to communicate with my own team than other teams and units.
15. I make better decisions at my workstation when I am given more time to think.
16. Trainees should not question senior team members' decisions.

17. If I perceive a problem with operations, I would speak up, regardless of who may be affected.
Page 52
Released Issue
Edition : 1.0
Please answer the following questions by ticking the box which best describes your opinion. 18. During periods of low work activity I would rather relax than keep busy with small tasks.
19. Flight crews never demand too much.

20. Controllers do not use their strips to help maintain a mental picture.
21. Casual, social conversation in the operating environment during periods of low workload can improve team coordination.
22. My decision-making skill is as good in critical situations as in routine situations.

23. It is important to avoid negative comments about the procedures and techniques of other controllers.
Edition : 1.0
Released Issue
Page 53
Please answer the following questions by ticking the box which best describes your opinion. 24. Discussing the traffic picture with other controllers helps to keep your own picture clearer.
25. Controllers visibly impaired by alcohol or drugs should be kept from going on duty.
26. Good communication is as important as technical proficiency in the controlling environment.

27. I like my job.

28. Only controllers on position should make decisions about opening or collapsing sectors.
29. My unit would be capable of handling the situation if there was a system breakdown.
Page 54
Released Issue
Edition : 1.0
Please answer the following questions by ticking the box which best describes your opinion. 30. I should maintain the traffic picture of the controllers I work with.
31. Controllers should feel obligated to mention their own psychological stress or physical problems to their co-workers before or during a shift.
32. The regulatory organisation rules should not be broken.

33. Our training has prepared us to work as a well coordinated team in an emergency.
34. Supervisors who encourage suggestions from team members are ineffective.
35. I should inform those controllers who are affected by my plans and control actions, and ask for their acknowledgement.
Edition : 1.0
Released Issue
Page 55
Please answer the following questions by ticking the box which best describes your opinion. 36. Increased automation reduces the need for team communication.
37. It is better to agree with other team members than to voice a different opinion.
38. I perform as well with other units as with my own.

39. Leadership of the team comes from the sector supervisor.

40. I always follow correct phraseology when controlling.

Page 56
Released Issue
Edition : 1.0
Please answer the following questions by marking on the scale ( X ) the point which best represents the frequency with which you would agree with the statements in the last six months.
______________________________________________________________________________ 0% 50% 100% NEVER ALWAYS 41. In abnormal situations, I rely on my superiors to tell me what to do. I maintain good interpersonal relationships with fellow controllers. I make poor decisions when I am fatigued. Never 44. 45. 46. 47. I retain a mental picture of the pilot's situation. I expect to be consulted on matters that affect the performance of my duties. When my workload is high I ask for assistance. The gender of the pilot affects my communication ability. I critique other controllers' techniques. Never 49. 50. Incidents, near-misses, etc., are reported according to the regulations. I perform effectively during critical operations even when fatigued. I make judgement errors in emergencies. Never 52. 53. I work with people who cooperate well with one another. I am distracted from the traffic situation by my thoughts. Never Never Always Always Always Never Never Always Always Always Never Never Never Never Always Always Always Always Always
Never Never
Always Always
42. 43.
48.
51.
Edition : 1.0
Released Issue
Page 57
54.
My colleagues are adequately trained in emergency procedures. The OJT instructor should take control of the position in critical and non-standard situations. We should trust all decisions made by adjacent sectors. Pilots make more call sign errors than controllers. I am encouraged by management to report any unsafe conditions I observe. My performance is not adversely affected by working with an inexperienced or less capable controller. Controllers encourage questions by team members during normal operations and critical operations. Morale in this unit is good.
Never Never
Always Always
55.
56. 57. 58.
Never Never Never
Always Always Always
59.
Never
Always
60.
Never
Always
61.
Never 62. Uncertain situations require quick decision-making. Controllers make decisions on their own. Never 64. I let other controllers know when my workload is becoming (or about to become) too high. Communication between controllers is reduced when the traffic is busy. I find it more difficult to maintain the traffic picture when I am tired. The executive controller should always take control in an emergency. I know if pilots are under-confident when they are on the radio. Never
Always Always
63.
Always
Never
Always
65. 66. 67. 68.
Never Never Never Never
Always Always Always Always
Page 58
Released Issue
Edition : 1.0
69.
I feel nervous or tense at work. Never Always Always Always
70.
I become impatient with flight crews who expect a lot. Each controller should monitor other controllers for signs of stress or fatigue. Personal problems adversely affect my performance. Handovers are provided improperly.
Never Never
71. 72.
Never
Always
73. 74.
Never Controllers should question the decisions or actions of other controllers. Management compromises safety for profitability. Controllers in my work environment are afraid to express disagreement with their supervisors. Never Never
Always Always Always
75.
76.
Never
Always
Edition : 1.0
Released Issue
Page 59
The next section describes briefly styles of leadership. Please read the descriptions, thinking of your ideas as well as your typical supervisor.
Style 1. Usually makes his/her own decisions promptly and communicates them to his/her subordinates clearly and firmly. Expects them to carry out the decisions loyally and without raising difficulties. Style 2. Usually makes his/her decisions promptly, but, before going ahead, tries to explain them fully to his/her subordinates. Gives them the reasons for the decisions and answers whatever questions they may have. Style 3. Usually consults with his/her subordinates before he/she reaches his/her decisions. Listens to their advice, considers it, then announces his/her decision. He/she then expects all to work loyally to implement it whether or not it is in accordance with the advice they gave. Style 4. Usually calls a meeting of his/her subordinates when there is an important decision to be made. Puts the problem before the group and invites discussion. Accepts the majority viewpoint as the decision.
77.
Which of the above styles of leadership would you most prefer to work under? (Circle one answer only.) Style 1 Style 2 Style 3 Style 4
78.
In your organisation, which one of the above styles do you find yourself most often working under? (Circle one answer only.) Style 1 Style 2 Style 3 Style 4
Years in ATC:
Years in TWR: Male (M) Years in APP: Female (F)
Years in this ATC unit:
Years in ACC:
(circle as appropriate)
What is your present position in your ATC unit? (You may tick more than one.): Tower controller Approach controller Area controller Student Where is your present unit?: Sector chief OJT-instructor Supervisor Other. Please specify:
Page 60
Released Issue
Edition : 1.0
REFERENCES
Barbarino, M. (1997). Team Resource Management in European Air Traffic Services. Paper given at the 9th International Symposium on Aviation Psychology, Ohio State University, Columbus, Ohio, USA. EATCHIP Human Resources Team (1996). Guidelines for Developing and Implementing Team Resource Management (HUM.ET1.ST10.1000GUI-01). Brussels: EUROCONTROL. Helmreich, R.L., Merritt, A.C., Sherman, P.J., Gregorich, S.E. and Wiener, E.L. (1993). The Flight Management Attitudes Questionnaire (FMAQ) (Nasa/UT/FAA Tech.Rep.No.93-4). Austin: the University of Texas, USA. Helmreich, R.L., Schaefer, H.G., Hines, W. and Sexton, J.B. (1996). The Operation Room Management Attitudes Questionnaire (ORMAQ): Cross-cultural data. Isaac, A.R. and Barbarino, M. (1998). Development of Team Resource Management in European Air Traffic Control. Paper given at the 10th International Symposium on Aviation Psychology, Auckland, New Zealand. Woldring, V.S.M. and Amat, A.-L. (1998). Team Resource Management in Air Traffic Control: Customisation of a Prototype Course. Paper given at the 23rd Conference of the European Association for Aviation Psychology, Vienna, Austria.
Edition : 1.0
Released Issue
Page 61
Page 62
Released Issue
Edition : 1.0
ABBREVIATIONS AND ACRONYMS

For the purposes of this document the following abbreviations and acronyms shall apply: ACC AIS APP ATC ATCO ATCSQ ATM CRM DED DED5 DFS DIS DIS/HUM EATCHIP EATMP ECAC ENAC ET EWPD FAA FDA Area Control Centre Aeronautical Information Services Approach Control Centre Air Traffic Control Air Traffic Control Officer / Air Traffic Controller (UK/US) Air Traffic Control Safety Questionnaire Air Traffic Management Crew Resource Management EATCHIP Development Director(ate) (no longer exists within the new EATMP structure) Human Resources Bureau (now HUM Unit or DIS/HUM) German ATC Corporation Deutsche Flugsicherung (Germany) Director(ate) Infrastructure, ATC Systems & Support ATM Human Resources Unit (also known as HUM Unit; formerly DED5) European Air Traffic Control Harmonisation and Integration Programme (now EATMP) European Air Traffic Management Programme (formerly EATCHIP) European Civil Aviation Conference Ecole Nationale dAviation Civile (F) Executive Task EATCHIP/EATMP Work Programme Document Federal Aviation Administration (USA) Flight Data Assistant/ce
Edition : 1.0
Released Issue
Page 63
FMAQ HFSG HRT HUM HUM Unit IANS IFATCA LHR MUAC NASA NATS OHP OJT OPS ORMAQ Q REP ROMATSA SDE SDOE ST TRM TRMTFII TWR
Flight Management Attitudes Questionnaire Human Factors Sub-Group Human Resources Team Human Resources (Domain) ATM Human Resources Unit (also known as DIS/HUM; formerly DED5) Institute of Air Navigation Services (EUROCONTROL) International Federation of Air Traffic Controllers' Associations London Heathrow (UK) Maastricht Upper Airspace Control Centre (EUROCONTROL) National Aeronautics and Space Administration (USA) National Air Traffic Services Ltd. (UK) OverHead Projector On-the-Job Training Operations Operation Room Management Attitudes Questionnaire Question Report Romanian Air Traffic Services Administration Senior Director(ate) EATMP (formerly SDOE) Senior Director(ate) Operations and EATCHIP (now SDE) Specialist Task Team Resource Management Team Resource Management Task Force II aerodrome control ToWeR
Page 64
Released Issue
Edition : 1.0
CONTRIBUTORS
MEMBERS OF THE HRT TEAM RESOURCE MANAGEMENT TASK FORCE II (TRMTFII) CHAIRMAN Mr. Michiel WOLDRING EUROCONTROL DIS/HUM 96, rue de la Fuse B - 1130 BRUSSELS AUSTRIA Ms. Andrea PECHHACKER AUSTRO CONTROL Schnirchgasse 11 A - 1030 VIENNA DENMARK Mr. Per HENRIKSEN CAA Civil Aviation Authority Maglebylille, Block 3 DK - 2770 KASTRUP FRANCE Ms. Sylvie FIGAROL CENA Toulouse 7, Avenue Edouard Belin F - 31055 TOULOUSE GERMANY Mr. Axel MUENSTER DFS - Deutsche Flugsicherung Flugsicherungsakademie Postfach 1243 D - 63202 LANGEN IRELAND Messrs. Philip McDONNELL & Jim HALPIN IAA Irish Aviation Authority Scotch House Hawkins Street IRL - DUBLIN 2 ITALY Dr. Maurizio SCHOLTZE ENAV Ente Nazionale di Aviazione Civile Ufficio Affari Internazionali Via Salaria 716 I - 00138 ROME
Edition : 1.0
Released Issue
Page 65
PORTUGAL Mr. Antnio ABREU GUERRA NAV E.P. (DENAL) Aeroporto de Lisboa Rua C - Edificio 118 P - 1700 LISBON ROMANIA Mr. Bogdan BRAGUTA Romanian CAA SOS. Bucuresti - Ploiesti km 16.5 Bucuresti Otopeni Airport RO - BUCHAREST SWITZERLAND Mr. Bernard KOCHER Swisscontrol Case Postale 796 CH - 1215 GENEVA 15 UNITED KINGDOM Mr. Phil SIMMONS NATS National Air Traffic Services Ltd. CAA House 45-59 Kingsway GB - LONDON WC2B 6TE EUROCONTROL Headquarters Ms. Anne-Laure AMAT DED5 Dr. Anne ISAAC DIS/HUM EUROCONTROL 96, Rue de la Fuse B-1130 BRUSSELS Institute of Air Navigation Services (IANS) Mr. Eoin McINERNEY EUROCONTROL Institute of Air Navigation Services 12, rue Antoine de Saint-Exupry L-1432 LUXEMBOURG
Document Configuration Management (DCM) Assistance Ms. Carine HELLINCKX EUROCONTROL, DIS/HUM
Page 66
Released Issue
Edition : 1.0

Team Resource PDF

Uploaded by

Copyright:

Available Formats

Team Resource PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Team Resource PDF

Uploaded by

Copyright:

Available Formats

EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION

Team Resource Management Test and Evaluation

Edition Edition Date Status Class

1.0 30.11.1999 Released Issue EATMP

EUROPEAN AIR TRAFFIC MANAGEMENT PROGRAMME

DOCUMENT IDENTIFICATION SHEET

AUTHORS: Michiel WOLDRING and Anne ISAAC

Team Resource Management Test and Evaluation

NAME AND SIGNATURE

Team Resource Management Test and Evaluation

EDITION 0.1 0.2 1.0

DATE 01.03.1999 10.09.1999 30.11.1999 Draft

REASON FOR CHANGE

SECTIONS PAGES AFFECTED All All All

Proposed Issue Released Issue

Team Resource Management Test and Evaluation

8. 8.1 8.2 8.3 8.4 8.5 9.

Team Resource Management Test and Evaluation

Team Resource Management Test and Evaluation

Team Resource Management Test and Evaluation

Page intentionally left blank

Team Resource Management Test and Evaluation

Team Resource Management Task Force II

Team Resource Management Test and Evaluation

Purpose and Scope

Team Resource Management Test and Evaluation

Team Resource Management Test and Evaluation

Page intentionally left blank

Team Resource Management Test and Evaluation

INFORMATION MATERIAL Team Resource Management Video Trailer

Team Resource Management Presentation

Team Resource Management Test and Evaluation

Page intentionally left blank

Team Resource Management Test and Evaluation

NATIONAL WORKING GROUPS

Team Resource Management Test and Evaluation

Page intentionally left blank

Team Resource Management Test and Evaluation

Team Resource Management Test and Evaluation

Page intentionally left blank

Team Resource Management Test and Evaluation

TEAM RESOURCE MANAGEMENT PROTOTYPE COURSE

Team Resource Management Test and Evaluation

Team Resource Management Test and Evaluation

CUSTOMISATION OF THE PROTOTYPE COURSE

National examples Cultural aspects

Common lesson material Instructions for facilitators

Team Resource Management Test and Evaluation

Page intentionally left blank

Team Resource Management Test and Evaluation

TEAM RESOURCE MANAGEMENT TEST PHASE Participating States

Approaches of the Different Participating Sites

Team Resource Management Test and Evaluation

5 responsible for continuation training of 5 different ATM units 5 facilitators

Head of Qualification Programmes 1 instructor 2 ATCOs c

Head of training Inc/Acc investig.

Head of continuation training 3 OPS with training experience

ATC Experts 1 psycho

3. Customisation: with DED5 (a)? ext. supp. (b)? self- (c)?