A Review of CAT II/III LAAS Integrity Requirements and their Antecedents

1
A Review of CAT II/III LAAS Integrity
Requirements and their Antecedents
Sam Pullen Stanford University (with lots of help
from Tim Murphy of Boeing)
Stanford GPS Laboratory Group Meeting 4 August
2006
2
English Word of the Day

• Antecedent (Webster online dictionary)
• 1 a substantive word, phrase, or clause whose
  denotation is referred to by a pronoun (as John
  in "Mary saw John and called to him") broadly
  a word or phrase replaced by a substitute ?
  grammar only
• 2 the conditional element in a proposition (as
  if A in "if A, then B") ? grammar only
• 3 the first term of a mathematical ratio ?
  rarely used
• 4 a a preceding event, condition, or cause b
  plural the significant events, conditions, and
  traits of one's earlier life ? very general
• 5 a PREDECESSOR especially a model or
  stimulus for later developments b plural
  ANCESTORS, PARENTS

3
Presentation Outline

• Review of LAAS Precision Approach Requirements
• Antecedents of these requirements
• ICAO Annex 10 Requirements for ILS
• FAA AC 25.1309 and AC 120-28D wording
• FAA Hazard Risk Index table
• Total Aircraft Safety sub-allocation
• What should the real be, and how should it be
  derived?
• Some initial thoughts

4
Precision Approach Requirements in Updated LAAS
MASPS(RTCA DO-245A, December 2004)
5
GBAS Service Level (GSL) Definitions
Table 1-1 (Section 1.5.1) of DO-245A
GSL Typical Operation(s) which may be Supported by this Level of Service
A Approach operations with vertical guidance (performance of APV-I designation)
B Approach operations with vertical guidance (performance of APV-II designation)
C Precision approach to lowest Category I minima
D Precision approach to lowest Category IIIb minima, when augmented with other airborne equipment
E Precision approach to lowest Category II/IIIa minima
F Precision approach to lowest Category IIIb minima
6
GSL Requirements Table
Table 2-1 (Section 2.3.1) of DO-245A
GSL Accuracy Accuracy Integrity Integrity Integrity Integrity Continuity
GSL 95 Lat. NSE 95 Vert. NSE Pr(Loss of Integrity) Time to Alert LAL VAL Pr(Loss of Continuity)
A 16 m 20 m 2 10-7 / 150 sec 6 sec 40 m 50 m 8 10-6 / 15 sec
B 16 m 8 m 2 10-7 / 150 sec 6 sec 40 m 20 m 8 10-6 / 15 sec
C 16 m 4 m 2 10-7 / 150 sec 6 sec 40 m 10 m 8 10-6 / 15 sec
D 5 m 2.9 m 10-9 / 15 s (vert.) 30 s (lat.) 2 sec 17 m 10 m 8 10-6 / 15 sec
E 5 m 2.9 m 10-9 / 15 s (vert.) 30 s (lat.) 2 sec 17 m 10 m 4 10-6 / 15 sec
F 5 m 2.9 m 10-9 / 15 s (vert.) 30 s (lat.) 2 sec 17 m 10 m 2 10-6 / 15 s (vert.) 30 s (lat.)
7
Antecedents of Precision Approach
Requirements1 FAA Hazard Risk Index
Useful reference Ch. 3 of FAA System Safety
Handbook (12/30/00) http//www.faa.gov/library/man
uals/aviation/risk_management/ss_handbook/media/Ch
ap3_1200.PDF
8
FAA Risk Severity Classifications

• Minor failure condition which would not
  significantly reduce
• airplane safety, and which involve crew
  actions that are well within
• their capabilities
• Major failure condition which would
  significantly
• (a) Reduce safety margins or functional
  capabilities of airplane
• (b) Increase crew workload or conditions
  impairing crew efficiency
• (c) Some discomfort to occupants
• Severe Major (Hazardous in ATA, JAA)
  failure condition resulting
• in more severe consequences than Major
• (a) Larger reduction in safety margins or
  functional airplane capabilities
• (b) Higher workload or physical distress such
  that the crew could
• not be relied upon to perform its tasks
  accurately or completely
• (c) Adverse effects on occupants
• Catastrophic failure conditions which would
  prevent continued safe
• flight and landing (with probability --gt 1)

Cat I
Cat III
Taken from AC No. 25.1309-1A, AMJ 25.1309, SAE
ARP4761 (JHUAPL summary)
9
FAA Hazard Risk Index (HRI) Table

• Several versions exist, all with essentially the
  same meaning
• Source of this version 1999 Johns Hopkins
  Applied Physics Laboratory GPS Risk Assessment
  Study final report http//www.faa.gov/asd/intern
  ational/GUIDANCE_MATL/Jhopkins.pdf

Cat. I ILS case
Cat. III ILS case
10
Antecedents of Precision Approach
Requirements2 FAA Advisory Circulars Defining
Certification and Airworthiness Criteria

• For AC 25.1309-1A, System Design and Analysis,
  6/21/88
• http//www.airweb.faa.gov/Regulatory_and_Guidance
  _Library5CrgAdvisoryCircular.nsf/0/50BFE03B65AF9E
  A3862569D100733174?OpenDocument
• For AC 120-28D, Criteria for Approval of
  Category III Weather Minima for Takeoff, Landing,
  and Rollout, 7/13/99
• http//www.airweb.faa.gov/Regulatory_and_Guidance
  _Library5CrgAdvisoryCircular.nsf/0/BBADA17DA0D0BB
  D1862569BA006F64D0?OpenDocument

11
Key Elements of AC 25.1309-1A

• AC 25.1309-1A is the primary basis for safety
  certification within the FAA
• AC 25.1309-1A specifies a fail-safe policy
  (quote)
• In any system or subsystem, the failure of any
  single element, component, or connection during
  any one flight (e.g., brake release through
  ground deceleration to stop) should be assumed,
  regardless of its probability. Such single
  failures should not prevent continued safe flight
  and landing, or significantly reduce the
  capability of the airplane or the ability of the
  crew to cope with the resulting failure
  conditions.
• Subsequent failures during the same flight,
  whether detected or latent, and combinations
  thereof, should also be assumed, unless their
  joint probability with the first failure is shown
  to be extremely improbable.
• AC 25.1309-1A defines the likelihood and severity
  terms found in the Hazard Risk Index
• Provides guidance as to what factors can be taken
  credit for in probability assessments and how
  this should be done
• Refers to RTCA DO-178 for software safety
  assurance guidance
• More recent SAE standards (ARP 4754 and 4761)
  provide much more detailed guidance on FAA
  safety-assurance methods

12
Summary of CAT III Airworthiness Requirements
(Table from Tim Murphy of Boeing)
Condition Airworthiness Requirements Model Related Success Criteria
1 AC 120-28D Nominal Performance App. 3, Section 6.3.1 Demonstrate equivalent or better performance under nominal conditions. (All variables varying across entire range). Meet 10-6 box
2 AC 120-28D Performance with Malfunction App. 3, Section 6.4.1 For all failures with probability gt 10-9 demonstrate safe landing -gt Land in box (with probability 1) given environment and other variables nominal.
3 JAR AWO Subpart 1 Performance Demonstration Limit case conditions Demonstrate performance when one of the variables is at its most critical value while the others vary in their expected manner Land in defined box with 10-5 -gt Conditional probability approach
Tim Murphys presentation is inside RTCA SC-159
WG-4 Archive File http//sc159.tc.faa.gov/wg4/060
706/Jun072006.htm
13
CAT III Touchdown Zone (or Box)
Figure from Figure 3 of Tim Murphys requirements
report to FAA Boeing Doc. D6-83447-4,
10/19/05 Numbers taken from App. 3, Section 6 of
FAA AC 120-28D
Additional bank angle hazard requirement
limits probability of any part of wing or engine
touching ground to 10-7 or less
14
Translation of Touchdown Zone into Landing System
Requirements

• Provided in ICAO Annex 10 for ILS (April 1985)
• not available online
• Annex 10 was amended for MLS and is being amended
  for GBAS ? Amendment 79 is latest (?)
• Annex 10 specifies 95 accuracy limits and
  monitor limits in terms of ILS measurements (DDM)
• Translation to LAAS required knowledge or
  assumption of several non-obvious intermediate
  parameters
• In my understanding, ILS requirements in Annex 10
  were designed around already-fielded ILS systems
  that were already deemed to be safe
• CAT III guidance requirements were not much more
  strict ? main difference was tighter,
  higher-reliability monitoring needed

15
Antecedents of Precision Approach
Requirements3 Example Risk Allocations
Source R.J. Kelly, J.M. Davis, Required
Navigation Performance (RNP) for Precision
Approach and Landing with GNSS Application,
Navigation, Vol. 41, No. 1, Spring 1994, pp. 1
30. http//www.ion.org/search/view_abstract.cfm?jp
jidno106
16
Breakdown of Worldwide Accident Causes 1959 -
1990 (from ICAO Oct. 1990 Study)

• Total hull loss probability per flight
  (mission) as of 1990 1.87 10-6
• Current probability per commercial departure in
  U.S. 2.2 10-7 (3-year rolling average last
  updated in March 2006)
• http//faa.gov/about/plans_reports/Performance/per
  formancetargets/details/2041183F53565DDF.html

17
U.S. Accident Breakdown by Cause (2000-01)
2001
2000
From NSTB Annual Review of Aircraft Accident
Data, 2000 and 2001 ARC 04/01 06/01
http//www.ntsb.gov/publictn/A_Stat.htm
18
Semi-unofficial Serious Accident Risk
Allocation (proposed in 1983 SAE paper)
Numbers based on approximations of observed
accident history.
Total Serious Accident Risk
10-6 per flight hour
90
10
9 10-7 p. f. hr.
1 10-7 p. f. hr.
All Other Causes (human error, weather, etc.)
Aircraft System Failures (engines, control,
avionics, etc.)
Assume 100 sepa-rate aircraft systems
Not subject to certification thus not broken
down in detail here.
Each individual system is allocated 1 10-9 p.
f. hr. (or per flight).
D.L. Gilles, The Effect of Regulation 25.1309
on Aircraft Design and Maintenance, SAE Paper
No. 831406, 1983.
19
How should the real CAT II/III requirements
(and other aviation safety requirements) be
determined (work in progress ?)?
20
Weaknesses in Current Safety Approach

• No clear means to adapt safety requirements to
  continued improvement in overall aircraft safety
• 10-9 requirement per individual aircraft system
  appears to be out-of-date given that current
  overall serious accident risk is approaching 10-7
  per flight
• 10-6 probability for landing in CAT III touchdown
  zone seems dated
• No clear means to appropriately balance
  rare-event probabilities
• 10-9 qualifies as extremely improbable, but 5
  10-9 only qualifies as improbable and must be
  treated as latent with probability 1 according
  to strict reading of AC 25.1309-1A
• No means to trade off safety benefit vs. safety
  risk for new systems that, when working properly,
  reduce the risk of accidents caused by
  pilot/weather/ATC/etc.
• Most new systems, including SBAS and GBAS, likely
  retire more pilot/weather/ATC risk than they
  introduce due to the possibility of their own
  failure

21
FAA Safety Engineering Tries to Adapt

• FAA shows no interest in fundamentally changing
  current certification standards
• Instead, FAA reacts to accidents on a
  case-by-case basis and tries to change individual
  rules interpretations subtly and quietly
• New interpretations also apply to new systems,
  such as SBAS and GBAS
• Example 1 aircraft rolling out long and off
  runway (recent SWA 737 accident at Midway)
• FAA now promulgating requirements clarification
  mandating a specific 15 runway margin see
  http//aviationnow.com/avnow/news/channel_busav_st
  ory.jsp?idnews/FAA06196.xml

22
FAA Safety Engineering Tries to Adapt (2)

• Example 2 TWA 800 (July 1996) 747 explosion
  most likely caused by ignition of center fuel
  tank
• NTSB accident report (August 2000)
  http//www.ntsb.gov/publictn/2000/AAR0003.pdf
• Many small fuel-tank risk- reduction steps
  implemented under SFAR 88 beginning in 2001
• Major ignition-suppression retrofit proposed in
  Notice of Proposed Rule Making (NPRM Nov. 2005)
• http//dmses.dot.gov/docimages/pdf94/373450_web.pd
  f
• Lengthy technical and cost-benefit debate on this
  NPRM continues to this day see
• http//dmses.dot.gov/docimages/pdf94/373645_web.pd
  f
• http//dmses.dot.gov/docimages/pdf95/389033_web.pd
  f

23
FAA Safety Engineering Tries to Adapt
(3)(Continuation of Example 2 TWA 800 Accident)

• Previous certification of fuel tank safety relied
  on need for multiple triggering events to occur ?
  joint probability was below 10-9 per flight
• However, initiating event could lie undiscovered
  for many flights prior to being detected by
  periodic maintenance
• New FAA specific risk concept requires that
  knowable latent defects be treated as present
  with probability 1
• Thus, 10-9 mitigation argument no longer holds in
  this case
• Also, undetected latent failure could leave
  aircraft only one failure away from
  catastrophic incident
• FAA and manufacturers have been debating this
  application of specific risk since 2002 see
• https//www.faa.gov/regulations_policies/rulemakin
  g/committees/arac/minutes/media/TAE_OCT_05.pdf
• http//edocket.access.gpo.gov/2006/pdf/E6-4024.pdf

24
Summary

• A complex set of requirements and guidance
  documents links todays CAT II/III landing
  requirements to overall FAA safety objectives
• As CAT II/III requirements are refined to be more
  GBAS-specific, re-thinking of the intent of the
  antecedents of these requirements is important
• FAA safety requirements evolution is limited in
  scope and is limited to new systems like SBAS
  and GBAS and response to external events, e.g.,
  accidents
• Further changes to better reflect improved
  overall aircraft safety and safety contribution
  of newer systems would be desirable

25
Backup Slides Follow
26
Integrity Requirement Definitions

• Integrity relates to the trust that can be placed
  in the information provided by the navigation
  system
• Misleading Information (MI) occurs when the true
  navigation error exceeds the appropriate alert
  limit (an unsafe condition) without annunciation
• Time-to-alert is the time from when an unsafe
  condition occurs to when the alarm message
  reaches the pilot (guidance system)
• A Loss of Integrity (LOI) event occurs when an
  unsafe condition occurs without annunciation for
  a time longer than the time-to-alert limit, given
  that the system predicts it is available

27
Notes to GSL Requirements Table
Section 2.3.1 of DO-245A
1. The values given for GNSS accuracy and alert
limits are those required for the intended
operation at the lowest height above threshold
(HAT) where the GNSS guidance is relied upon. 2.
The definition of the integrity requirement
includes an alert limit and a time to alert,
against which the requirement can be assessed. 3.
The accuracy requirements include the nominal
performance of a fault-free airborne
subsystem. 4. The integrity requirements are
specified in terms of a probability to be
evaluated over a specified period. The duration
of this period is intended to correspond to the
most critical portion of an approach landing
for the operations the GSL is intended to
support. Integrity risk includes the probability
of latent failures, and the exposure time to
these types of failures may exceed the specified
period, therefore the requirement must apply
during any period. Note that if the integrity
requirements for GSL D-F are met, the integrity
requirements for GSL A-C are also automatically
met. 5. For these GSLs (D, E, and F), the
combined lateral and vertical risk shall not
exceed 1 10-9, where the risk for vertical
applies over any 15 sec, and the risk for lateral
applies over any 30 sec. The lateral period is
longer because these GSLs are intended to support
operations that require LAAS guidance during
roll-out. 6. The time-to-alert (TTA) is the
maximum time between the onset of a failure
condition that affects the integrity of any
information that could be applied by the airborne
subsystem and the time that the alert indication
is available at the output of the airborne
subsystem, where the airborne subsystem is
assumed to have zero latency. Compliance with the
TTA requirement must include consideration of the
probability of missed VDB messages by a
fault-free airborne subsystem.
28
Actual Hull Loss Probability Breakdown (from
October 1990 ICAO Study Data)

• Total final approach and landing risk (as of
  1990) 7.8 10-7 per flight ( 42 of total
  risk!)
• Target level of safety (via tunnel concept) for
  final approach and landing 0.2 10-7 per
  flight ( 13 of total risk)
• Hazard due to loss of navigation system integrity
  is only a small part of the total final approach
  and landing risk