Draft CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes

1
Draft CAR/SAM Regional Guidance Material on Air
Traffic Services Quality Assurance Programmes

• Quality for Tomorrow Begins Today
• NAM/CAR/SAM Air Traffic Services Quality
  Assurance Programme Seminar
• Mr. Mark Rios
• Regional Officer, Air Traffic Management
• ICAO NACC Regional Office
• (16 to 20 October 2000, Mexico City, Mexico)

2

• INTERNATIONAL CIVIL AVIATION ORGANIZATION
•  
• NORTH AMERICAN, CENTRAL AMERICAN AND CARIBBEAN
  OFFICE
•  
• CAR/SAM REGIONAL GUIDANCE MATERIAL ON
•  
• AIR TRAFFIC SERVICES
•  
• QUALITY ASSURANCE PROGRAMMES
•  
• DRAFT
•  
• Version 0.1
•  
• October 2000

3

• THIS MATERIAL IS UNDER DEVELOPMENT AND IS
  SUBJECT TO CHANGE BEFORE ADOPTION BY GREPECAS AS
  REGIONAL GUIDANCE MATERIAL

4
Forward

• CAR/SAM/3 RAN (October 1999)
• ATS Quality Assurance Programme
• Purpose
• ATS Incidents
• Quality of Services
• CAR/SAM/3 Recommendation 5/36
• ICAO develop guidance material on ATS quality
  assurance programmes for worldwide use
• CAR/SAM Regional Guidance Material on ATS QA
  Programmes
• first phase

5
Contents

• Chapter 1. Definitions
• Chapter 2. Background
• Chapter 3. ATS Quality Assurance Programmes
• Chapter 4. Controller Proficiency Checks
• Chapter 5. Air Traffic Incident Reporting,
  Investigations, and
  Investigation Teams
• Chapter 6. Air Traffic Incident Prevention
  Programmes
• Chapter 7. Air Traffic Services Evaluation
  Programme
• Chapter 8. Quality of Services Improvement
  Programmes
• Chapter 9. Proficiency Training Programmes
• Chapter 10. ATS Safety Management
• Chapter 11. Human Factors

6
Chapter 1. Definitions
New Definitions

• ATC proficiency training
• ATS operational duties
• Desk audit
• Follow-up ATS evaluation
• Full-unit ATS evaluation
• In-flight/preflight evaluations
• Operational deviation
• Operational error

• Pilot deviation
• Refresher training
• Runway incursion
• Simulation training
• Skill enhancement training
• Special evaluations
• Supplemental training
• Three-step closure process

7
Chapter 2. Background

• Important to prepare ATS QA programme guidelines
  for States
• Heightened awareness of ATS incidents and
  recommendations made by
• CAR/SAM/2 RAN (Santiago, 1989)
• Recommendation 6/11 Investigation of ATS
  Incidents
• Recommendation 6/12 Reporting of ATS Incidents
• GREPECAS 8 (Dominican Republic, 1998)
• Conclusion 8/7 Actions to be taken by States to
  reduce ATS incidents in the CAR/SAM Regions
• Conclusion 8/8 Actions to be taken by the users
  to reduce ATS incidents in the CAR/SAM Regions
• Conclusion 8/9 Actions to be taken by the ICAO
  Regional Offices to reduce ATS incidents in the
  CAR/SAM Regions

8
Chapter 2. Background

• Informal working groups
• International organizations
• Airlines
• Reports received by ICAO Regional Offices
• Increase of ATS incidents by 43 per cent between
  1996 and 1998
• Justified the need for introduction of ATS QA
  programme for the CAR/SAM Regions
• ICAO NACC/SAM Regional Offices began work
  programme

9
Chapter 3. ATS Quality Assurance Programmes

• Introduction
• ATS QA programmes should focus on Identifying and
  correcting deficiencies before they result in an
  ATS incident or accident and
• Should continually work towards improving the
  overall quality of air traffic services
• Chapter contains proactive quality assurance
  strategies

10
Chapter 3. ATS Quality Assurance Programmes

• Scope and purpose
• Specific guidance on reporting, investigating and
  resolving various types of ATS incidents that
  impact the quality of ATS
• Designed to work in conjunction with
• ICAO SARPs
• States regulations
• Objectives
• Prevent ATS incidents
• Continually improve overall quality of air
  traffic services

11
Chapter 3. ATS Quality Assurance Programmes

• Structure
• Depends on size and composition of ATS provider
• Acceptable and productive programme structure may
  involve
• ATS chief designating/selecting experienced ATC
  expert to be ATS unit quality assurance
  officer/specialist
• ATS unit QA specialist assumes duties and
  responsibilities of QA for ATS unit
• Larger ATS units could develop ATS QA
  unit/department
• Several ATS QA specialists
• Assistant chief with extensive ATC experience

12
Chapter 3. ATS Quality Assurance Programmes

• Implementation
• CAR/SAM/3 Recommendation 5/38 Implementation of
  an ATS quality assurance programme
• That,
• a) States/ATS service providers within the
  CAR/SAM Regions implement an ATS quality
  Assurance programme with associated supporting
  documents. The programme should outline the
  purpose, goals, objectives and responsibilities
  of the programme within the State/ATS service
  provider and
• b) Each ATS unit should establish such a
  programme

13
Chapter 3. ATS Quality Assurance Programmes

• Responsibilities
• ICAO NACC and SAM Regional Offices
• provide guidance and assistance to States/ATS
  service providers in the development of ATS
  quality assurance programmes
• States/ATS service providers
• Should implement an ATS quality assurance
  programme with associated supporting documents
• Programme should outline the purpose, goals,
  objectives and responsibilities within the
  State/ATS service provider and each ATS unit
  should establish a programme
• National ATS QA programmes are maintained and
  evaluated for effectiveness

14
Chapter 3. ATS Quality Assurance Programmes

• ATS unit chiefs
• Maintain a level of awareness and involvement in
  their ATS units operations/programmes so as to
  ensure maximum quality and efficiency
• All ATS personnel
• Responsible for maintaining the highest level of
  quality performance

15
Chapter 3. ATS Quality Assurance Programmes

• Programme content
• ATS QA programmes should establish methods to
  identify and correct shortcomings and recognize
  successes in the following areas
• ATS incident prevention
• Teamwork
• Communications
• Customer service/feedback

16
Chapter 3. ATS Quality Assurance Programmes
ATS incident prevention

• ATS incident prevention
• ATS refresher training
• Aeronautical phraseology improvement
• English language proficiency
• Hearback/readback
• ATS surface incident
• Incentive/recognition
• List of good operating practices
• Recordings monitor evaluation
• Simulation training

• Personal accounts of lessons learned
• Periodic quality assurance briefings in ATS units
  covering trends, customers input, evaluations,
  etc.
• Aggressive resolution of problems identified
• Incorporation of past ATS incidents scenarios
  into training
• Internal, national, and regional ATS evaluation

17
Chapter 3. ATS Quality Assurance Programmes
Teamwork

• Air traffic service teamwork training
• Teamwork incentive/recognition programmes
• Roles of different positions/jobs

• ATS Operational Supervisor training course
• Team meetings/briefings
• Expectations of all employees clearly communicated

18
Chapter 3. ATS Quality Assurance Programmes
Communications

• Electronic bulletin board system
• Internet/intranet access to data
• National database containing national and local
  ATS QA data
• Newsletters

• ATS QA seminars, conferences and workshops
• All staff meetings
• Reports from International organizations, i.e.
  IFATCA, IATA, IFALPA, PAAST, etc., and other
  industry safety reports

19
Chapter 3. ATS Quality Assurance Programmes
Customer service/feedback

• Pilot education programmes
• Surveys of internal and external customers
• Interaction with other aviation organizations
• Air traffic controller/Operational Supervisor
  evaluation of shift performance

• All staff meetings
• Familiarization flights
• Contacts with customer organizations (i.e. local
  flight schools, airlines, aviation organizations,
  etc.)
• Pilot safety seminars and airport management
  workgroups

20
Chapter 4. Controller Proficiency Checks

• Introduction
• To provide continuous enhancement of technical
  proficiency in the ATS environment,
  individualized training requirements for
  technical performance should be identified and
  accomplished
• Controller proficiency checks are intended to
  provide feedback by ATS supervisors/ATS QA
  officers/specialists to controllers and
  supervisors regarding their proficiency, and to
  develop plans to enhance their proficiency

21
Chapter 4. Controller Proficiency Checks

• Responsibilities
• ATS unit chiefs to establish and maintain ATS
  unit proficiency standards
• ATS authority should develop guidelines
  specifying required level of knowledge
  (theoretical and practical)
• ATS operational personnel should be required to
  periodically demonstrate that their performance
  meets proficiency standards
• Controller proficiency checks should be
  accomplished on each controller and supervisor

22
Chapter 4. Controller Proficiency Checks

• ATS unit chief, supervisor or quality assurance
  specialist should conduct task of controller
  proficiency checks
• Develop proficiency check rosters to ensure all
  ATS operational personnel are checked biannually
• ATS operational supervisors and QA specialist
  should continuously assess controllers
  performance through direct or indirect methods
• Indirect remote monitoring, voice recordings,
  observations by other supervisors or ATS unit QA
  specialists

23
Chapter 4. Controller Proficiency Checks

• What if results of controller proficiency check
  indicates controller would benefit from
  proficiency training?
• Use references to determine type of training
  needed
• ICAO Doc 9426, Air Traffic Services Planning
  Manual, Chapter 3 Training and Proficiency
  Requirements
• Draft CAR/SAM Regional Guidance Material for ATS
  QA Programmes, Chapter 9. Proficiency Training
• Assignment of proficiency training may not
  necessarily mean a deficiency was noted
• A controller may demonstrate an acceptable level
  of technical performance, but might still benefit
  from proficiency training in a particular skill
  or task

24
Chapter 4. Controller Proficiency Checks

• ATS supervisor or QA specialist should discuss
  the completed controller proficiency check with
  the controller
• Proficiency checks are not intended to be
  pass/fail or satisfactory/unsatisfactory
• What if a controllers performance is found to be
  unsatisfactory?
• Rating should be suspended and appropriate
  proficiency training, followed by a
  re-certification process should be completed

25
Chapter 4. Controller Proficiency Checks

• ATS units should conduct an annual review of all
  controller proficiency checks with a view to
  identify recurring and significant proficiency
  needs
• Documentation
• Discuss proficiency check results with the
  controller and document in the controllers
  training record

26
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Introduction
• Air traffic incidents
• AIRPROX
• Operational error
• Operational deviation
• Runway incursion
• Pilot deviation
• Casual factor
• Procedural
• Ground facilities (equipment)
• Human

27
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• General reporting procedures
• Reporting by pilots
• Reporting by ATS
• ATS incident report form for use by pilots and
  controllers
• Investigation and documentation
• Analysis of ATS incident
• Release of information
• In the interest of accident/incident prevention
• Publish report as soon as possible

28
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Air traffic incident investigation process
• Step-by-step process
• Suspected ATS incident event
• Identify and report ATS deficiencies
• That problems could be corrected
• Reporting of ATS incidents
• Requirement within States
• Responsibility of all ATS personnel
• Immediately report suspected ATS incidents to ATS
  supervisor or appropriate ATS administration
  official
• Without fear of reprisal

29
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Preliminary ATS incident investigation
• Fact finding
• Determine what occurred in the ATS system
• Determine validity of the suspected ATS incident
• Confirmation of ATS incident
• Take appropriate action
• Advise other ATS unit if involved
• Relieve controller(s) may be involved
• Gather pertinent information (e.g. flight
  progress strips, etc.)
• Review voice recordings and prepare working copy

30
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Review computer and radar data
• Conduct preliminary interviews
• Notify ATS management
• Complete preliminary ATS Incident Investigation
  Worksheet
• Follow-up investigation
• ATS unit chief should
• Ensure ATS incident investigations are conducted
• Coordinate with other ATS unit chief if involved
• Designate ATS incident Investigator-In-Charge
  (ATS-IIIC)
• Assemble ATS incident investigation team
• Ensure complete, thorough, objective and timely
  investigation

31
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• ATS-IIIC
• Conduct complete investigation
• Forward findings and recommendations to ATS unit
  chief
• Determine scope of other ATS unit investigation
  if involved and exchange information
• Assign duties to team members
• Conduct interviews in accordance with national
  and local requirements
• ATS incident investigation team
• Assist ATS-IIIC
• Remain under guidance of ATS-IIIC during
  investigation process

32
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Investigation process
• Fact finding
• In-depth inquiry into all causal factors

• Procedures
• Training
• Supervision
• Equipment
• Control room environment
• External factors
• Controller actions
• Airspace configuration
• Traffic flows
• Pilot actions

• Consequence of ACAS/TCAS event
• Meteorological conditions
• Control position configuration
• Coordination procedures
• Aerodrome environment
• Human factors
• Accuracy of automated radar systems
• Radar data

33
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Investigation process (cont..)
• Interviews
• Conduct interviews with all possible personnel
  involved or who may have knowledgeable of the
  incident
• Conduct interview in atmosphere of shared concern
• Consider national and/or local interview
  procedures
• Allow written comments and recommendations from
  interviewee
• ATS-IIIC or investigation team should conduct
  interviews

34
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Investigation process (cont.)
• Voice recordings
• Review voice recording
• Make certified copy of original recording
• 5 minutes before to 5 minutes after last contact
• Include track time if available
• Written transcript of communications only when
  needed
• Radar data
• Radar data may be used to validate occurrence of
  ATS incident
• Written procedures should be developed explaining
  process
• Consider automation capabilities and accuracy of
  data

35
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Reclassification of ATS incident
• What if during the investigation data indicates a
  reclassification is warranted?
• The incident may be reclassified as
• Pilot deviation
• Other ATS unit deviation
• From operational error to operational deviation
• From operational deviation to operational error
• No occurrence

36
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Return to operational duties
• ATS chiefs should remain involved in the post ATS
  incident process to ensure complete and
  consistent handling
• Reasons for suspension of controllers rating and
  assignment of proficiency training
• Assist and enhance controllers performance
• Best of his/her abilities
• Meet minimum quality standards
• Improve overall performance and quality of ATS

37
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Return to operational duties (cont.)
• Consider the following when returning a
  controller to operational duties
• Events leading up to and surrounding ATS incident
• Controllers statement
• Procedure or separation minimum involved
• Voice recordings
• Controllers training record
• Controller proficiency checks
• Proficiency training
• Controller currency on the control position
• Computer data
• Involvement in previous ATS incidents
• Results of interviews and recommendations made by
  ATS-IIIC and investigation team

38
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Return to operational duties (cont.)
• Deficiencies in controllers performance
• Determine whether to suspend controllers rating
• Suspension of rating not based solely on
  involvement in ATS incident but based on
  performance
• Determine appropriate actions and refresher
  training
• Complete refresher training
• Re-certify controller through controller
  proficiency check

39
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Final ATS incident reports
• ATS unit chief
• Analyze data submitted by ATS-IIIC and
  investigation team in Final ATS Incident
  Investigation Report to determine and/or endorse
  classification of incident and
• Categorization of ATS incident (i.e, procedural,
  facility, human factor, etc.)
• Causal factors
• Recommendation and corrective actions
• Complete Final ATS Incident Report within 30 days

40
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• ATS incident tracking process
• Annual ATS incidents tracking process
• Will assist CAAs with follow-up investigations
  and trend analysis
• Sample
• ATS Incident Report XXXX-E-00-001
• ATS Incident Report XXXX-T-00-002
• ATS Incident Report XXXX-F-00-003
• Legend
• XXXX ATS unit identification
• E, T, or F Type of ATS unit (E Enroute
  (ACC) T Approach (APP) and
• Aerodrome Control Tower F FIC, AFIS, etc.)
• 00 Last two digits of the year
• 001 ATS incident number in sequence by year
  for the unit

41
Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams

• Training and proficiency records
• Document in training and proficiency records
• ATS incident records retention
• Determine retention period
• 2 ½ years is recommended
• National ATS authority responsibilities
• Maintain central source of ATS incident data
• Review final ATS incident investigation reports
• Develop and distribute annual ATS incident
  analysis report
• Determine effectiveness and efficiency of
  programme
• Maintain liaison with ATS unit chiefs
• Maintain oversight of ATS incident reports

42
Chapter 6. Air Traffic Incident Prevention
Programmes

• Introduction
• Establish policies and programmes that result in
  quality incident free ATS
• Requires commitment of all ATS personnel
• Consistently perform duties and responsibilities
  in accordance with ICAO SARPs, CAA and ATS unit
  directives and procedures
• Accept responsibility for maintaining the safest
  and highest levels of operational performance
• Goals/objectives
• ATS incident free environment
• Safe, orderly, and expeditious ATS to users
• Everyone is responsible for ATS incident
  prevention
• Know, apply, and adhere to requirements in
  performance of operational duties and
  responsibilities

43
Chapter 6. Air Traffic Incident Prevention
Programmes

• ATS incident prevention
• Many initiatives can be pursued
• Five primary areas
• Communications
• Phraseology
• Supervision
• Teamwork
• ATC proficiency

44
Chapter 6. Air Traffic Incident Prevention
Programmes
Prevention objectives

• Identify activities/efforts which enhance
  individual, ATS unit, and system performance
• Recognize commendable activities/efforts
• Identify deficiencies (individual, equipment,
  procedural)
• Promptly correct deficiencies
• Ensure actions taken corrected the deficiencies
  or problems
• Initiatives to ensure effective controller
  performance, teamwork, communications
• Communicate performance expectations

• Maintain summary of ATS incidents and incorporate
  into training
• Brief ATS personnel on ATS incidents
• Monitor/evaluate voice recordings
• Review local/national/regional ATS incidents
• Annual QA review
• ATS unit staff meetings
• Improve communications among all ATS personnel to
  create atmosphere conducive to sharing
  information
• Attendance to local, national, regional QA
  seminars, conferences, etc.

45
Chapter 6. Air Traffic Incident Prevention
Programmes
ATS supervisors

• Communicate performance expectations with
  controllers
• Take prompt action when performance does not meet
  expectations (on-the-spot corrections, etc.)
• Address individual/team accountability
• Efficient/consistent oversight of ATS unit
  operation
• Ensure distractions and noise levels in ATS unit
  are kept at a minimum

• Require personnel to maintain high degree of
  professionalism, teamwork, control position
  discipline, awareness
• Promote open flow of communications
• Conduct ATS unit team meetings
• Place emphasis on hearback/readback
• Keep ATS unit chief/support staff informed
  (operations concerns, etc.)

46
Chapter 6. Air Traffic Incident Prevention
Programmes
ATS quality assurance officers/specialist

• Random voice recording reviews
• Brief ATS unit on trends and causal factors
  related to ATS incidents, random recording
  reviews, evaluations, etc.
• Maintain awareness of ATS operational environment
• Provide staff expertise and support
• Assist in ensuring QA goals and objectives are
  being met

• Ensure scenarios involving hearback/readback
  errors are included in training
• Prepare/distribute ATS incident summaries to all
  ATS personnel
• Ensure ATS unit maintains summary of ATS units,
  causal factors, trends, etc.
• Incorporate into classroom and annual proficiency
  training

47
Chapter 6. Air Traffic Incident Prevention
Programmes
ATC personnel

• Keep ATS supervisors advised of traffic problems
  and equipment limitations
• Suggest improvements to ATS unit and/or incident
  prevention
• Maintain situational awareness
• Assist busier control positions

• Assist busier control positions
• Review own operating techniques and ATS unit
  procedures to effect highest quality of
  performance
• Promptly report ATS incidents
• Utilize memory aids

48
Chapter 6. Air Traffic Incident Prevention
Programmes

• Voice recording monitor evaluations
• To ensure proper phraseology, good operating
  practices, adherence to ICAO standards,
  national/local directives and practices
• Guidelines
• Conduct semi-annually on all ATS operational
  personnel
• Review of voice recording by ATS supervisor/ATS
  QA specialist
• Document comments
• Develop action plan to address performance
  deficiencies
• ATS supervisor/QA specialist review and discuss
  voice recording with controller

49
Chapter 6. Air Traffic Incident Prevention
Programmes

• Review of ATC procedures and practices that may
  contribute to ATS incidents
• Constant review of ATC procedures and practices
• Identify, report, recommend, and implement
  amendments
• Ensure safety of aviation and prevention of ATS
  incidents

50
Chapter 6. Air Traffic Incident Prevention
Programmes

• ATS incident review groups
• Implement ATS incident review groups
• National/local levels
• Purpose
• Periodic reviews of prior ATS incidents
• Identify other potential problem areas requiring
  attention
• Effort to prevent ATS incidents
• Composition
• ATS unit chief
• Controllers
• Other ATS staff
• ATS unit QA specialist
• Pilots/airlines when appropriate

51
Chapter 6. Air Traffic Incident Prevention
Programmes

• Safety information sharing networks and
  international organizations
• Several safety information networks available
• Facilitate free exchange of information of actual
  and potential safety deficiencies
• Expanding focus to include air traffic control
  services
• Goals
• Collect data
• Share information
• Assist with identification and prevention of
  incidents and accidents

52
Chapter 6. Air Traffic Incident Prevention
Programmes

• Safety information sharing networks and
  international organizations (cont.)
• ADREP
• ICAO Accident/Incident Reporting Manual, Doc 9156
• Global Aviation Safety Plan (GASP)
• Endorsed by 32nd Session of the ICAO Assembly
• Objectives
• Safety initiatives that offer best safety
  dividend
• Annual reviews of accident statistics
• Decrease in worldwide accident rate
• Enhance identification of short comings and
  deficiencies
• Increase and improve ICAOs own capability to
  compile, assess and disseminate safety-related
  information
• ICAO Website www.icao.int

53
Chapter 6. Air Traffic Incident Prevention
Programmes

• Global Aviation Information Network
• To encourage the sharing of aviation safety data
  worldwide in an effort to further reduce
  accidents
• Plans to expand programme to include air traffic
  control
• Website www.gainweb.org
• Aviation Safety Reporting Programme (ASRP)
• Reporting programme designed to identify unsafe
  operating conditions
• Deficiencies and discrepancies
• Limited immunity
• Voluntary and confidential

54
Chapter 6. Air Traffic Incident Prevention
Programmes

• PAN American Aviation Safety Team (PAAST)
• Multi-organizational team
• Pool essential resources
• Addresses priority safety areas
• Deliver practical products to aviation community
  in Latin American and Caribbean Regions
• Participation in international organization study
  groups to analyze and recommend solutions to ATS
  incidents is encouraged

55
Chapter 6. Air Traffic Incident Prevention
Programmes

• Runway incursion prevention plan
• Increase in runway incursions
• Aerodrome operations increasing throughout the
  world
• Potential for runway incursions increase
• Need for prevention plan which specifically
  addresses runway incursions
• Strategies
• Position relief briefings
• Position relief checklist
• Proficiency training Prevention of runway
  incursions
• Feedback

56
Chapter 6. Air Traffic Incident Prevention
Programmes

• Runway incursion prevention plan (cont.)
• Memory aids
• Use of flight progress strips/boards
• Runway in use sign
• Review usage of memory aids on semi-annual basis
• Procedures
• Review aerodrome movement operating practices
• Designate runway crossing point(s)
• Place special emphasis on safe runway crossings
• Use of proper procedures in runway crossings
  including interphone procedures, runway in use
  sign by ATS supervisors and controllers

57
Chapter 6. Air Traffic Incident Prevention
Programmes

• Procedures (cont.)
• Avoid use of conditional clearances except as
  authorized by ICAO SARPs
• Certain conditional clearances are permitted
  however, they should be kept to a minimum, the
  exception rather than the norm should be
  practiced, and under extreme caution
• Include runway check procedures in letter of
  agreement
• Define vehicular movement procedures/areas in
  letter of agreement
• Vehicular movements should be restricted to
  perimeter roads, vehicle access roads and other
  non-movement areas
• Describe closed runway procedures

58

• END OF PART ONE

59
Chapter 7. ATS Evaluation

• Introduction
• Chapter deals with need for constant and
  continuous evaluation of individual ATS units and
  of the overall ATS system
• Purpose and scope
• Ensure that provision of service is maintained at
  the highest standard
• Units/personnel apply polices, standards, rules,
  procedures and separation minimum in approved
  manner

60
Chapter 7. ATS Evaluation

• ATS evaluation activities include
• Assessing service provided to users
• Ensuring operating procedures conform with
  standards
• Assessing operational requirements
• Identifying potential unsafe procedures or
  operating practices
• Detecting problem areas or deficiencies
• Examining effectiveness of intra/inter-unit
  communications and coordination
• Examining personnel utilization, position
  workload and establishments to ensure
  compatibility
• Recommending corrective actions

61
Chapter 7. ATS Evaluation

• Conduct of evaluation
• Designate team of experienced personnel to
  conduct ATS evaluations
• At least once every two years
• Notify ATS unit chief prior to conducting
  evaluation
• Meeting at completion of evaluation with ATS unit
  chief to discuss significant findings and
  recommendations
• Prepare written report ATS evaluation team
• Prepare action plan ATS unit

62
Chapter 7. ATS Evaluation

• ATS evaluation procedures
• This section provides standardized procedures for
  evaluating compliance with ICAO SARPs and
  specified directives and procedures at the
  national and local ATS unit level within a State
• Evaluation process
• Full unit ATS evaluation
• Preparation and notification
• Use national checklist (to be added to material
  at later date)
• Notify ATS unit chief 30 days prior to evaluation
• Request data from ATS unit for pre-evaluation
  review
• Solicit special interest items that ATS unit
  would like assessed

63
Chapter 7. ATS Evaluation

• Evaluation process (cont.)
• In-briefing
• Introduce team members
• Evaluation schedule and activities
• ATS unit chief staff
• Conducting the evaluation
• Direct observation
• Control position and/or voice recording or data
  monitor
• Attendance at staff meetings
• Observation of training
• Review of administrative records
• Interviews/discussions
• Review of in-flight/preflight evaluation reports

64
Chapter 7. ATS Evaluation

• Evaluation process (cont.)
• Daily briefings
• Progress of evaluation
• Provided by lead ATS unit evaluator to ATS unit
  chief
• Out-briefing
• Briefing of ATS evaluator/teams findings
• Attendance
• ATS unit chief
• ATS unit personnel encouraged
• Present draft copy of ATS evaluation report
• Evaluation critique
• Critique form to be completed by ATS unit chief

65
Chapter 7. ATS Evaluation

• Follow-up evaluations
• Preparation and notification
• Unannounced or minimum notification
• On-site evaluation
• Desk audit
• Combination of both
• Conduct no earlier than 6 months after full unit
  ATS evaluation
• May request data for pre-evaluation review
• Basically follow same process as full unit ATS
  evaluation

66
Chapter 7. ATS Evaluation

• In-flight/preflight evaluations
• Conduct at least 2 per year on each ATS unit
• Conducted by ATS evaluators and other authorized
  ATS personnel
• Use airlines or other operators in which an
  agreement to conduct such observations is in
  place
• The following air traffic services may be
  assessed
• Air traffic services reporting office
• Automatic terminal information service
• Aerodrome control service
• Approach control service
• Air traffic advisory service
• Other services deemed appropriate

67
Chapter 7. ATS Evaluation

• Special evaluations
• May be conducted when deemed necessary by the ATS
  authority or if requested by the ATS unit
• Evaluation reports
• Document results of all ATS evaluations
• Complete and distribute final ATS evaluation
  reports within 30 days of the out-briefing
• Contents of final ATS evaluation reports
• Results of evaluation pertaining to areas
  assessed
• Describe reportable items
• Assign tracking control numbers to all items
  identified

68
Chapter 7. ATS Evaluation

• Responses to ATS unit evaluations
• Required on all items rated less than
  satisfactory
• Three-step closure process
• Action plan
• Develop and forward action plans to appropriate
  ATS authority within 30 days of receiving final
  ATS unit evaluation report
• First response
• Completed and distributed to ATS authority with
  60 days of ATS unit out-briefing
• Second response
• Completed and distributed with 180 days after
  date of the ATS unit evaluation out-briefing and
  every 180 days thereafter until all items are
  closed

69
Chapter 8. Quality of Services Improvement
Programmes

• Introduction
• Initiatives that can improve the overall quality
  of air traffic services being provided
• Random voice recording reviews
• ATS unit quality assurance officers/specialists
  should complete several random voice recording
  reviews on a monthly basis
• Ensure overall quality of ATS is maintained

70
Chapter 8. Quality of Services Improvement
Programmes

• ICAO phraseology improvement programme
• Implement ATS unit phraseology improvement
  programme
• Benefit to pilots and controllers
• Results can improve quality of service and
  prevention of ATS incidents
• Method
• Random voice recording reviews
• Voice recording monitor evaluations
• Direct observations
• Recognition of ATS personnel who demonstrate
  outstanding phraseology or significant improvement

71
Chapter 8. Quality of Services Improvement
Programmes

• ATS customer service/feedback
• Important to establish good communications
  between ATS authorities/units and users
• Users can offer valuable feedback
• Feedback from others organizations/departments/int
  ernally
• Obtain feedback through surveys
• ATS quality assurance survey
• Conduct annually
• External
• Internal
• Analyzed and validate data collected
• Share results with ATS personnel
• Issues affecting quality of service should be
  addressed

72
Chapter 8. Quality of Services Improvement
Programmes

• Pilot/controller user forums
• Conduct annually
• Can produce good relations and enhance
  communications between ATS authorities, pilots
  and controllers
• Main objective
• Bring together people in the ATS system that work
  together on a daily basis
• Pilot on the flight deck
• Controller in the control room/tower
• Better understanding of each others
  responsibilities and duties
• Avoid conducting in a meeting type format
• Can also be used to present and explain
  information on the local or national ATS system
  and procedures

73
Chapter 8. Quality of Services Improvement
Programmes

• Participation in pilot safety seminars
• Participation of ATS authorities in pilot safety
  seminars
• Present information regarding the ATS system,
  safety and quality assurance
• Pilot visits to ATS units
• Encourage pilot visits to ATS units
• ATS system familiarization
• ATS system familiarization/education for pilots
• Consider developing an ATS system education
  programme for pilots
• Education pilots in how to best utilize the ATS
  system
• Functions
• Responsibilities
• Benefits
• Services

74
Chapter 8. Quality of Services Improvement
Programmes

• Familiarization training flights for ATS
  personnel
• Establish programmes with airlines
• ATS supervisors and controllers should be
  encourage to participate
• Allows ATS unit personnel to experience first
  hand the activities on the flight deck
• View programme as proficiency training

75
Chapter 8. Quality of Services Improvement
Programmes

• Recognition of quality performance
• Recognition of positive quality performance is as
  important as identifying deficiencies
• Recognition as individuals or as a team
• Providing a high standard of performance and
  quality of service
• ATS performance measurements
• TO BE DEVELOPED AT A LATER DATE

76
Chapter 9. Proficiency Training Programmes

• Introduction
• Proficiency training is necessary to maintain and
  update the knowledge and skills necessary to
  apply air traffic control procedures in a safe an
  efficient manner
• Proficiency training includes
• Refresher training
• Supplemental training
• Skill enhancement
• Remedial training

77
Chapter 9. Proficiency Training Programmes

• May be accomplished in many different ways
• External
• Internal (local/in-house-in proficiency training)
• Most practical and efficient
• Train staff as instructors (may involve external
  training)
• After returning from training they would train
  others to become instructors (train-the-trainer)
• Proficiency training may include
• Training videos - Separation minima
• Discussions/briefings - ATS incidents
• Operational procedures - Contingency procedures
• Emergency procedures - Effects of volcanic ash
  to aviation
• Coordination procedures

78
Chapter 9. Proficiency Training Programmes

• Proficiency training
• Is required for
• All ATS operational personnel (supervisors and
  controllers)
• Support personnel who maintain operational
  currency
• Purpose
• Maintain and upgrade knowledge and skills
  necessary to apply air traffic control procedures
  in safe and efficient manner
• Training needs will vary between different ATS
  units
• Describe proficiency training programme in ATS
  unit directive
• Annual schedule of proficiency training
  requirements
• Document training provided in controllers
  training record

79
Chapter 9. Proficiency Training Programmes

• Types of proficiency training
• Refresher training
• Recurring training conducted to maintain and
  update previously learned knowledge and skills
• Supplemental training
• Training administered to ATS operational
  personnel prior to implementation of new/revised
  procedures, regulations, equipment

80
Chapter 9. Proficiency Training Programmes

• Types of proficiency training (cont.)
• Skill enhancement training
• Training assigned, normally by the supervisor,
  when it is determined that a need exists to
  increase the skill(s) of a controller in a
  position on which he/she is certified
• Remedial training
• Training assigned to correct specific performance
  deficiencies

81
Chapter 9. Proficiency Training Programmes
Refresher training topics

• Unusual situations
• Infrequently used procedures
• Safety alerts/traffic advisories
• Wake turbulence
• Line up and wait procedures
• Deicing procedures
• Bird activity information
• Strayed or unidentified aircraft orientation
• Interception of civil aircraft
• Meteorology procedures for ATC
• Radar backup systems
• ATS contingency plan procedures
• Effects of volcanic ash to aviation
• Coordination procedures
• Civil/military coordination/joint use airspace
• Separation minima

• Radar vectoring techniques
• Speed control techniques
• Situational awareness
• ATS incident reduction
• Aircraft performance and characteristics
• ATC communications
• Prevention of runway incursions
• Special VFR operations
• Level assignment
• Local manual of operations
• Letters of agreement
• Arrival/departure procedures
• Weather
• Noise abatement
• ATS unit file/life safety procedures

82
Chapter 10. ATS Safety Management

• This material is extracted from an amendment
  proposal to Annex 11 and is subject to change
  before adoption by the ICAO Council

83
Chapter 10. ATS Safety Management

• Introduction
• States implement systematic and appropriate ATS
  safety management programmes to ensure that
  safety is maintained in the provision of ATS
  within airspaces and at aerodromes
• ATS safety management programmes
• Provide for regular monitoring and assessment of
  safety level achieved
• Identify potential hazards and determine the need
  for remedial action
• Ensure that remedial action necessary to maintain
  an acceptable level of safety is implemented

84
Chapter 10. ATS Safety Management

• Objectives of ATS safety and capacity management
• Ensure established ATS level of safety is met
  within an airspace or aerodrome
• Safety-related enhancements are implemented
• ATS capacities conforming to actual needs as
  far as practicable and subject to safety
  considerations
• Facilitate efficiency and economy in aircraft
  operations through use of flexible airspace

85
Chapter 10. ATS Safety Management

• ATS safety management activites
• Monitoring overall safety levels and detection of
  any adverse trend
• Safety reviews of ATS units
• Safety assessments
• Airspace re-organizations
• New or changed ATS procedures
• Establish need for safety enhancing measures
• Review ATS capacities
• Traffic demands, balancing capacity and traffic
  demands
• Flexible use of airspace
• Gain capacity and efficiency

86
Chapter 10. ATS Safety Management

• Monitoring of safety levels
• Periodically collect and review safety related
  data to evaluate level of safety achieved in ATS
  and to detect any adverse trends
• Safety reviews
• Conduct safety reviews of ATS units on a regular
  basis
• Safety review topics include
• Regulatory issues
• Operational and technical issues
• Licensing and training issues

87
Chapter 10. ATS Safety Management

• Airspace re-organization and new or changed ATS
  procedures
• Need for safety assessments
• Acceptable level of safety
• Safety-significant factors
• Need for safety-enhancing measures
• Reviews of airspace and ATC capacity
• Capacity assessments
• Regulation of ATC capacity and traffic volumes
• Enhancements to ATC capacity
• Flexible use of airspace

88
Chapter 11. Human Factors

• Chapter designed in progress
• To be completed at a later date
• Proposed contents of chapter
• Information useful to air traffic control
  personnel
• Controller/pilot voice communications
• Memory
• Fatigue
• Effects of stress on information processing
• Techniques
• Help reduce probability of voice communications
  errors
• Remembering specific information
• Identifying stress that could affect performance
• Reduce fatigue

89
Next steps

• Draft regional guidance material to be presented
  to
• GREPECAS ATM/CNS Subgroup and then to GREPRECAS
• ICAO Headquarters for consideration as worldwide
  guidance material - 2002
• NAM/CAR/SAM ATS QA Programme Workshops
• 2001 (English)
• 2002 (Spanish)
• ATS QA Programme Training Course
• Should be developed under TRAINAIR programme
• 2002/2003

90
Conclusions

• ATS quality assurance programmes would place
  safety at the forefront while expediting and
  maintaining an orderly flow of air traffic
• Quality assurance is a dynamic process used to
  continually improve an ATS system

91
Conclusions

• Quality will continue to be measured against
  historical data
• Number of ATS incidents
• Delays
• Employee and customer feedback
• Factors that cannot be measured readily must also
  be recognized
• Willingness to function
• Team
• Training
• Actions taken to support zero ATS incidents all
  factor into quality assurance

92
Conclusions

• The success of the quality assurance effort is
  dependent on the recognition that all ATS
  providers, independently and collectively, must
  strive to provide the best service possible

93
Conclusions

• Write what you do
• Do what you write
• Check what you do

• Escriba lo que hace
• Haga lo que escribe
• Verifique lo que hace

International Standardization Organization/Organiz
acion Internacional de Normalizacion Technical
Committee 176 - ISO 9000 Basic Requirements/
Comite Tecnico 176 - Requisitos Basicos ISO 9000
94

• The End
• Fin
• Thank you
• Gracias