PGE Delivery and Data Processing Plans for the Version 4.0 Software Release

1
PGE Delivery and Data Processing Plansfor
theVersion 4.0 Software Release

• Steven Friedman
• AIRS Science Processing
• March 30, 2004

2
Topics

• Version 4.0 Schedule
• Version 4.0 Data Processing Plans
• Routine Data Processing at GSFC DAAC
• Data Processing at AIRS TLSCF
• Version 4.0 Reprocessing Considerations

3
Strange Archeological Sites in the LA Area
4
Version 4.0 Schedule Key Objectives

• Version 4.0 to include enhanced validation
  coverage area
• Extends coverage to land areas
• Expands latitude range beyond ?40
• Feature Set Enhancements to include
• Level 1
• AIRS IR Calibration enhancements - New gains,
  limits, etc. Geolocation, Potential
  algorithmic upgrades (?)
• Microwave Calibration enhancements,
  Algorithmic upgrades (?)
• Level 2 the key ingredient to V4.0
• Further improvement beyond V3.0 and V3.5
  baselines
• Incorporates many performance enhancements
• Includes better retrieval algorithm, handling of
  outliers, QA
• Final feature set must be solidified by early
  summer!

5
V4.0 Timeline
6
Version 4.0 Milestones

• V4.0 Next Release to DAAC NOAA
• Prototyping Ends 6/01/04
• Final Inputs to CCB 6/01/04 Last day to propose
  updates
• Development Period Ends 7/05/04
• Initial Build V4.0 7/06/04 Anomalies / small
  changes
• V4.0 Operational Begin Validation
  DP 7/21/04 Critical anomalies ONLY!
• Delivery to DAAC NOAA 8/31/04
• V4.0 Operational Begin Processing 9/30/04
• V3.5 Interim Release to NOAA 4/16/04 Improved
  L2, QA
• V3.7 Internal JPL Benchmark 5/03/04 L3 Test,
  L2 In-process

• Please keep in mind
• All scheduled milestones are established to
  ensure that we deliver a quality V4.0 product on
  schedule.
• There will always be new and better improvements,
  and there are other releases planned for the
  future.

7
Routine Data Processing at the GSFC DAACwith V4.0

• With V4.0 the GSFC DAAC will process
• Level 0 through Level 2 PGEs Browse PGEs
• Level 1B calibrated radiances, QA subsets,
  daily browse
• Level 2 retrieval standard,, retrieval
  support, CC radiances daily browse (retrieval
  and CC)
• Level 3 PGE 3 new standard products for V4.0
• Daily, 8-day, and monthly additional products
  planned for later releases
• Subsetted data sets
• Considering developing new subsetted radiance
  data sets

8
Routine Data Processing at the GSFC DAACwith
V4.0 (contd.)

• With V4.0 the GSFC DAAC will also process
• Match-Up PGE (products delivered to JPL ONLY!)
• Radiosonde Match-Up supports trending analysis
• Fixed-location Match-Ups
• Dedicated Validation Sites
• 5 special validation sets ACAR, ACFT, Dobson,
  Dobson_P, Yoe
• No Dynamic Match-Ups will be performed at the
  GSFC DAAC a new fixed-location Match-Up is
  being developed fixed buoys

9
Data Processing at the AIRS TLSCFwith V4.0

• Beginning with V4.0 the JPL TLSCF will
• Stop baseline processing Level 0 through
  Level 1B Match-Up
• Continue to process Focus Days (nominally, 48-day
  intervals) starts with Level 0 producing the
  following products
• Level 1A all AIRS, AIRS VIS, and AMSU products
• Level 1B calibrated radiances, QA subsets, daily
  browse
• Level 2 retrieval standard, CC radiances,
  retrieval support, daily browse (retrieval
  and CC)
• Level 3 daily product
• Continue to process other data as necessary
• Support continued software development test
• Support validation campaigns
• We will order additional Level 0 and/or Level 1B
  as needed

10
Data Ingest at the AIRS TLSCFwith V4.0

• Beginning with V4.0 the JPL TLSCF will
• Stop ingesting Level 0 from the GSFC DAAC on a
  daily basis
• Continue ingesting Level 0 from the GSFC DAAC for
• All Focus Days
• Special campaigns
• Continue ingesting products from the GSFC DAAC
• All Level 2 standard retrieval products
• All Daily Browse Products Level 1B and Level 2
• Radiosonde Match-Up Products
• Occasionally ingest 20 granules of all Levels 0,1
  and 2 products(support spot checking)
• Begin ingesting Level 3 and fixed-site Match-Up
  Products
• Continue ingesting selected granules from NOAA
  NRT process

11
Processing and Reprocessing at the GSFC DAAC
with V4.0

• GSFC DAAC will begin processing with V4.0 when
  the release becomes operational
• All forward processing will be performed with
  V4.0
• Reprocessing will commence providing coverage
  back toAugust 31, 2002 and can be accomplished
  by
• Marching backwards towards August 31, 2002? or
  
• Marching forwards from August 31, 2002?
• Reprocessing includes rerunning all data from
  Level 0 Performs Level 1A 1B, Level 2,
  Match-Up and Browse PGEs
• For AIRS, allocated processing resources are 3x
• 1x dedicated to forward processing
• 2x dedicated to reprocessing
• It will take approximately 1¼ years to complete
  reprocessing(may be less time, given Level 2
  performance improvements)

12
Reprocessing Considerations after V4.0

• After V4.0, reprocessing will not keep up with
  annual upgrades
• Continuous improvement of PGEs, delivered to DAAC
  on an incremental basis deliveries scheduled
  annually
• Our desire to produce the best possible product
  versus the DAACs desire to produce complete
  collections
• Something has to give and processor resources
  are fixed
• Either we hold back deliveries to pace
  reprocessing or we consider a hybrid
  approach after V4.0
• Could we consider stabilizing Level 1A(?) and
  Level 1B(?) with V4.0 until the last reprocessing
  cycle at end of mission?
• Allow table upgrades, but delay science algorithm
  changes
• Would dedicate processor resources to Level 2
  reprocessing

13
V3.0 Reprocessing Status at the GSFC DAAC

• Processing status as of March 22, 2004

2004
2003
Calendar Day of Year
Solid bars indicate unprocessed data.
14
(No Transcript)