CDM Data Quality Update

1
CDM Data Quality Update

• AD Meeting
• 18 April 2001
• Mike Golibersuch (Volpe/EGG)

2
Overview

• Analysis of OAG-Only Time Out Cancellations (CDM
  Meeting Action Item)
• Status of Volpe Data Quality Web Site
• Proposed CDM Data Quality Metrics

3
Analysis of OAG-Only Timeout Cancellations

• Request for this analysis arose during discussion
  of Daily Download at CDM Meeting.
• The Problem some flights are created in ETMS
  from OAG data, but no other messages are received
  and the flight does not operate. Eventually the
  flight Time Out Cancels.
• Check and Correct could reduce problem by
  detecting incorrect OAG-scheduled flights in ETMS
  database.
• Airlines asked that Volpe quantify the problem,
  to provide an indicator of one type of benefit
  obtainable with Check and Correct.
• OAG-Only TOCs were analyzed for the 60-day period
  from 2/5/01 4/5/01

4
OAG-Only TOCs Per Week By Airline(2/5/01
4/5/01)
ACA 3xxx ? CDN (removed rule -- 4/11)
Strike 3/27
USA 49xx ? PDC (added rule -- 4/11)
5
Data Quality Web Site OAG-TOC Report
6
Frequency Distribution of OAG-TOC Flights
7
Distribution of OAG-Only TOCs
8
Conclusions and Implications

• Problems with OAG data are episodic. Continuous
  monitoring is needed.
• Check and Correct capability could improve
  things, but benefit may be moderate.
• Other potential benefits of Check and Correct
  are less easily quantified.

9
Volpe Data Quality Web Site

• Provides access to detailed message history by
  flight.
• Two products available now to help find data
  problems
• OAG Time-outs highlights OAG schedule errors.
• GDP Time-outs daily report of any flight in a
  GDP that time-out cancelled.
• Next Steps
• Performance Issues and TOC Issues Are Being
  Addressed
• Data Quality Metrics Will Be Added
• Access to the Site
• Web Site Address is 152.122.152.338000
• Those who want access must provide client IP
  address to Volpe
• Volpe will provide User Name and Password
• Each Airline May View
• Summary Data for All Other Airlines
• Detailed Flight Data Only for Its Own Flights

10
Data Quality Web Site -- GDP Time-outs
11
Data Quality Web Site -- Flight History
12
ETMS/CDM Data Quality Metrics

• Several candidate CDM data quality metrics were
  discussed at the last AD Meeting
• Based on general consensus, 3 Metrics are
  proposed for initial use as measurements of CDM
  data quality
• Each metric will be tracked and reported on the
  Volpe Data Quality Web Site
• After the measures have been tracked for a period
  of time, standard benchmarks will be defined
• Several other indicators will be tracked for
  analysis purposes, but at least initially will
  not be treated as formal data quality measures
• Each airline will have access to summary data
  (averages, trends) for all airlines, but will
  have access to detailed data only for its own
  flights

13
Data Quality Metrics

• Timeout-Cancelled Flights
• As percent of
• All flights planned to operate (FS, FC, or FM)
• All cancelled flights
• Airline-Cancelled Flights (FX, SI) Flights that
  Operate
• As percent of
• All flights planned to operate
• Airline-cancelled flights
• Reinstated flights (FX followed by FC) are
  OK, and are not counted
• Unscheduled Flights (no OAG data) that Operate
  without a CDM Message
• As Percent of
• All flights that operate
• Unscheduled flights that operate

14
Metric 3. Unscheduled Flights that Operate
without CDM Message
15
How Will the Data Be Displayed

• Bar Charts comparing average values by airline
  over a user-specified time interval
• Trend Line showing weekly value over 4-month
  timespan for a selected airline
• Airline may request a list of its flights that
  match the criteria during a specified time
  interval
• Airline may select a flight from the list to
  display flight details and message history

16
Other Indicators

• Scheduled flights that operate without a CDM
  message
• As Percent of all scheduled flight that operate
• Useful for detecting problems with daily download
  
• Flights that depart at least 15, 30, 45, or 60
  minutes after EDT
• As Percent of
• All flights that operate
• All delayed flights (I.e., actual departure after
  OAG schedule)
• Controlled flights that depart at least 15,
  30,45, or 60 minutes after EDT
• As Percent of controlled flights that operate
• For measurement purposes, EDT is determined as
  follows
• Use ETMS EDT 10 minutes prior to actual
  departure. I.e., for measuring prediction
  accuracy, disregard last minute updates
• From this EDT, remove any Time Out Delay that was
  automatically added by ETMS