Ocean PEATE - PowerPoint PPT Presentation

Loading...

PPT – Ocean PEATE PowerPoint presentation | free to download - id: 297a9c-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Ocean PEATE

Description:

Performance of new components is closely evaluated following installation in ... Level-3 processing has been demonstrated using sample IDPS Build 1.4 OCC and SST ... – PowerPoint PPT presentation

Number of Views:35
Avg rating:3.0/5.0
Slides: 64
Provided by: fred
Category:
Tags: peate | image | library | ocean

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Ocean PEATE


1
Ocean PEATE
  • Fred Patt
  • NPP SDS Peer MOR
  • Building 8 Auditorium
  • August 27, 2008

2
Ocean PEATE Agenda
  • Ocean PEATE Overview
  • Implementation Status
  • Documentation
  • Support Staff
  • EDR Assessment
  • Issues/Concerns

3
Ocean PEATE Overview
  • The NPP Ocean PEATE will be implemented within
    the framework and facilities of the current NASA
    Ocean Data Processing System (ODPS)
  • This system has been successfully supporting
    operational, satellite-based remote-sensing
    missions since 1996, and its capabilities
    continue to evolve and expand to meet the demands
    and challenges of future missions.

4
Ocean PEATE Functions
  • Acquire VIIRS RDRs, SDRs, and Ocean EDRs from the
    SD3E and ADS/CLASS
  • Assess the quality of the NPP Ocean EDRs for
    accomplishing NASAs climate research
    requirements
  • Provide suggested algorithm improvements to the
    IDPS via the Project Science Office Element
    (PSOE)
  • Process selected data subsets in support of
    Evaluation and Validation activities

5
Ocean PEATE Interface Diagram
Analysis Results, Proposed Algorithm Updates
xDRs, IPs, Ancillary Data
Management Direction
xDRs, IPs, Ancillary Data (if unavailable from
SD3E)
Ocean PEATE
Pre-flight Algorithms, Data, Info
Software, Data
Alternate Ancillary Data
xDR Eval. Results, Algorithm Updates
Calibration Updates and Evaluations
Interaction
In Situ Data
In Situ Data
Algorithm Updates, Test Requests Results
Matchups
6
Ocean PEATE External Interfaces (1 of 2)
  • SDS Science Data Distribution and Depository
    Element (SD3E)
  • Provides NRT access to raw data
  • Primary source of RDRs
  • Provides selected SDRs and EDRs
  • SDS Integration and Test System Element (ITSE)
  • Build and test updates to operational code in
    mini-IDPS
  • Run tests on selected data per request of PEATE
  • Archive Distribution Segment (ADS)
  • Primary source for archived data
  • xDRs, IPs, Ancillary Data, Operational
    Algorithm/Source Code and Calibration Products
  • Ancillary Data Providers (ADP)
  • Provides alternate ancillary data sets (e.g.,
    ozone, meteorological data sets)
  • CasaNOSA
  • Serves as the NPP pre-flight repository of
    Government held data for distribution to
    Government user teams
  • Place to acquire pre-launch NPP algorithms and
    supported data files

7
Ocean PEATE External Interfaces (2 of 2)
  • NASA VIIRS Ocean Science Team (VOST)
  • Coordinate activities with PEATEs and PSOE on xDR
    and recommended algorithm improvements. Supports
    Independent Calibration Validation Activities
  • NPP Instrument Calibration Support Element
    (NICSE)
  • Provides alternative calibration LUTs and
    recommended improvements to calibration
    algorithms
  • PEATE provides results of LUT and algorithm tests
  • Project Science Office Element (PSOE)
  • Provides management direction
  • Accepts algorithm update recommendations
  • SeaBASS/ODPS
  • Provides Ocean Color in situ data
  • RSMAS/U. Miami
  • Provides SST in situ locations
  • PEATE provides SST EDR matchups
  • Ocean Science Community
  • Relies on Ocean PEATE to provide evaluation
    products and results

8
Roles Responsibilities of Science Team / PEATE
PEATE Science Team
Acquire, catalog, and distribute xDRs Perform automated matchups with SeaBASS data Subscribe, browse, download xDRs Evaluate Ocean EDRs using matchups
Generate Level-3 products from Ocean EDRs Compare VIIRS Ocean Level-3 files with heritage sensors
Acquire test LUTS from NICSE Evaluate VIIRS calibration using test LUTS
Deliver/install recommended software updates at ITSE Generate recommended algorithm changes
Upload RDRs to ITSE, input test run requests and download SDRs and/or EDRs Develop algorithm test cases and scenarios and evaluates test products
Interface with SDS, ITSE, ADS, Provide feedback to Project Scientist
9
Ocean PEATE Agenda
  • Ocean PEATE Overview
  • Implementation Status
  • Documentation
  • Support Staff
  • EDR Assessment
  • Issues/Concerns

10
ODPS Design Overview
  • Fully automated, distributed data system for
    acquiring, processing, archiving, and
    distributing scientific data
  • Highly scalable
  • Easily adaptable to support multiple concurrent
    missions
  • Graphical user interfaces for controlling and
    monitoring system functions and activity
  • Non-platform specific

11
ODPS COTS and Freeware
  • Linux OS (CentOS 4.x)
  • Solaris OS
  • Sybase RDBMS
  • Subversion (source code management)
  • Pro-active DBA
  • Interactive Data Language (IDL)
  • Generic Mapping Tool (GMT)
  • Netpbm (graphic image toolkit)
  • HDF5 Library
  • Languages C, PERL, SQL

12
ODPS Architecture Hardware
  • Processing Servers
  • Intel-based dual Xeon / AMD-based dual Opteron
  • 8 GB RAM
  • Five 72 GB SCSI drives
  • Storage Servers
  • Intel-based P4 / AMD-based single Opteron
  • 1 GB / 2 GB RAM
  • 1.5 TB IDE RAID 5 (3ware) / 9.6 TB SATA RAID 6
    (Areca)
  • 2 hot spare drives per RAID5
  • Database Server
  • Sun V880
  • 8-16 GB RAM
  • 6-12 70 GB SCSI HDD

13
ODPS Current Components
14
Building 28 Room W220 Computing Facility
15
Reliability and Redundancy
  • Critical components (database server, network
    systems) have full maintenance contracts to
    ensure rapid response to problems
  • Multiple-server components (ingest, processing,
    storage, distribution) have substantial
    redundancy to maintain full capability spares
    maintained for rapid replacement.
  • Testing nodes are separated from mainstream
    production components.

16
Technology Refresh
  • Hardware technology advances (CPU, storage) are
    continuously monitored to select new components
    for evaluation.
  • Typical upgrade threshold is a doubling in
    capacity (18 months).
  • Two generations of hardware are generally in use.
  • Candidate components are procured, installed in
    testing cluster and rigorously evaluated in a
    production-like environment.
  • Following successful evaluation, multiple copies
    are procured, installed, tested and swapped in
    for older components.
  • ODPS design allows new components to be rapidly
    added to resource tables without interrupting
    system operations.
  • Performance of new components is closely
    evaluated following installation in operations
    environment.
  • Critical components are run in parallel with
    existing system to ensure reliability under
    production loading.

17
Existing Software Reuse
  • ODPS Components
  • Database
  • VDC/Scheduler
  • Data Acquisition and Ingest
  • Level-3 Scheduler
  • File migration and management
  • Archive Device Manager
  • Data distribution
  • Level-2 multi-mission software (vicarious
    calibration)
  • Level-3 multi-mission software (long-term trends
    and comparisons
  • Level-2 to Level-3 comparison software (residual
    sensor errors)
  • SeaBASS (in situ data management)
  • Matchup/extraction software

18
Ocean PEATE New Capabilities (1 of 2)
  • Acquire, ingest and catalog NPP VIIRS data
    products RDRs, SDRs and Ocean EDRs (Data
    Acquisition Ingest, Device Manager and File
    Migration and Management).
  • Status Initial interface test run with SD3E.
  • Process selected Ocean EDRs (SST and OCC) to
    Level-3 to support data product and algorithm
    evaluations (Level-3 Scheduler, VDC and Level-3
    binner).
  • Status Prototype Level-3 processing has been
    demonstrated using sample IDPS Build 1.4 OCC and
    SST EDRs.
  • Perform VIIRS OCC EDR matchups with SeaBASS Ocean
    Color in situ data (extract code).
  • Status Pending re-test of acquisition and ingest
    capabilities with latest IDPS build.

19
Ocean PEATE New Capabilities (2 of 2)
  • Incorporate VIIRS SDR processing for vicarious
    calibration analysis.
  • Status Pending re-test of acquisition and ingest
    capabilities with latest IDPS build.
  • Produce VIIRS proxy data using VOST-developed
    software (VDC/Scheduler).
  • Status Under development
  • Acquire SST in situ data from RSMAS and perform
    matchups with SST EDRs
  • Status SST acquisition in place for MODIS
    matchups pending re-test of acquisition and
    ingest capabilities with latest IDPS build.
  • Support browse and distribution of data products
    for team members (Data Distribution).
  • Status Pending re-test of acquisition and ingest
    capabilities with latest IDPS build.

20
Ocean PEATE Data Storage Estimate
Data Type Daily 1 Year 5 Years
RDR 150 GB 53.5 TB 268 TB
SDR (M-band) 242 GB(2) 8.6 TB(1,2) 43 TB(1,2)
OCC EDR 84 GB 3 TB(1) 15 TB(1)
SST EDR 19 GB 0.7 TB(1) 3.4 TB(1)
Inter. Products 70 GB N/A N/A
Ancillary Data 0.1 GB .04 TB .2 TB
Total 565 GB 66 TB 330 TB
  • Assumptions
  • Long-term storage is sized for 100 of RDRs and
    10 of SDRs and EDRs
  • SDR volume includes geolocation

21
New Hardware for the Ocean PEATE
  • 7 Storage Servers _at_ 9.6 TB first-year VIIRS
    data storage
  • Additional servers acquired post-launch to handle
    years 2 5
  • No new processing or network capacity required
    technology refresh cycle to be continued within
    the ODPS as described.

22
Ocean PEATE Build Schedule
  • Build 1 (L-18 months)
  • All interfaces fully implemented and tested
  • Verify initial versions of operational code
    ported and running in ITSE
  • L-3 product code developed and tested
  • Prelaunch VIIRS test data storage and SDS
    interface testing support with existing ODPS
    storage capacity
  • Initial test products generated for review by
    VIIRS Ocean Science Team
  • Build 2 (L-12 months)
  • Routine exercise of interfaces to acquire proxy,
    surrogate (Aqua?) and/or simulated data
  • Verify pre-launch version of operational code
    running in ITSE
  • Browse and distribution capability developed and
    tested
  • Test products routinely acquired as available and
    posted for access by VIIRS Ocean Science Team
  • Data storage for one year

23
Ocean PEATE Testing Status (1 of 6)
Req. Requirement Support Verification Status
3.4.1.1 PEATEs xDR Ingest from the SD3E New Data Acquisition and Ingest Interface Test X
3.4.1.2 PEATEs IP Ingest from the SD3E New Data Acquisition and Ingest Interface Test
3.4.1.3 PEATEs Official Ancillary Data Ingest from the SD3E New Data Acquisition and Ingest Interface Test
3.4.1.4 PEATEs Calibration Product Ingest from the SD3E New Data Acquisition and Ingest Interface Test
3.4.1.5 PEATEs Request Listings New Data Acquisition and Ingest Interface Test X
3.4.1.6 PEATEs Product Subscriptions from the ADS PEATE Personnel Interface Test
3.4.1.7 PEATEs Ad-hoc Requests to the ADS PEATE Personnel Interface Test
3.4.1.8 PEATEs xDR Ingest from the ADS New Data Acquisition and Ingest Interface Test
3.4.1.9 PEATEs Official Ancillary Data Ingest from ADS New Data Acquisition and Ingest Interface Test
3.4.1.10 PEATEs Calibration Product Ingest from ADS New Data Acquisition and Ingest Interface Test
24
Ocean PEATE Testing Status (2 of 6)
Req. Requirement Support Verification Status
3.4.1.11 PEATEs IP Ingest from the ADS New Data Acquisition and Ingest Interface Test
3.4.1.12 PEATEs ADS Query and Request Status Ingest PEATE Personnel Interface Test
3.4.1.13 PEATEs Alternate Ancillary Data Ingest Existing ODPS Capability Inspection X
3.4.1.14 PEATEs Pre-launch Data Ingest Existing ODPS Capability Inspection X
3.4.1.15 PEATEs Engineering Service Ingest PEATE Personnel Interface Test
3.4.2.1 Store/Catalog Data New Data Management ITSE Functional Test X
3.4.3.1 Manage Software Configuration Existing Subversion Repository Inspection X
3.4.4.1 Operational Algorithms Ingest from the SD3E ITSE Functional Test
3.4.4.2 PEATEs Management Direction Ingest PEATE Personnel Interface Test
3.4.4.3 PEATEs Instrument Service Report Ingest PEATE Personnel Interface Test
25
Ocean PEATE Testing Status (3 of 6)
Req. Requirement Support Verification Status
3.4.4.4 PEATEs NICSE Cal LUT Ingest ITSE Interface Test
3.4.4.5 PEATEs ITSE Algorithm Status PEATE Personnel Interface Test
3.4.4.6 Pre-Launch Assessment New PEATE Tool Development Functional Test
3.4.4.7 xDR Implementation Assessment ITSE Functional Test
3.4.4.8 EDR Algorithms ITSE Functional Test
3.4.4.9 SDR and EDR Testing Environment ITSE Functional Test
3.4.4.10 Comparison Tools Development Existing ODPS Capability Inspection X
3.4.4.11 Comparing against Other Data New Level-3 Processing Functional Test
3.4.4.12 Comparing against Other Products New Level-3 Processing Functional Test
3.4.4.13 Analyzing Algorithm Improvement/ Enhancement ITSE/Ingest Interface/ Functional Test
26
Ocean PEATE Testing Status (4 of 6)
Req. Requirement Support Verification Status
3.4.4.14 Generating SDR ITSE Functional Test
3.4.4.15 Generating EDR ITSE Functional Test
3.4.4.16 Test Data VOST Simulation Software / ODPS New Data Processing Design
3.4.4.17 Developing SDR Algorithm Improvement/Enhancement ITSE Functional Test
3.4.4.18 Developing EDR Algorithm Improvements/Enhancements ITSE Functional Test
3.4.4.19 Testing Algorithm ITSE/Acquisition and Ingest Functional/ Interface Test
3.4.4.20 Documentation ODPS Web Site Inspection X
3.4.4.21 PEATEs Management Report PEATE Personnel Interface Test
3.4.4.22 PEATEs Instrument Service Request PEATE Personnel Interface Test
3.4.4.23 PEATEs Quality Control Response ITSE Functional Test
27
Ocean PEATE Testing Status (5 of 6)
Req. Requirement Support Verification Status
3.4.4.24 PEATEs xDR Ingest from the SD3E New Data Acquisition and Ingest Interface Test
3.4.5.1 PEATEs Science Data Export New Product Distribution Interface Test
3.4.5.2 PEATEs Data Compression Existing ODPS Capability Inspection X
3.4.8.1 Ocean Product Assessment New and Existing ODPS Capabilities Functional Test
3.4.8.2 Ocean EDR Evaluation New and Existing ODPS Capabilities Functional Test
3.7.1 Mission Support All New Capabilities Aggregate of All Testing
3.7.3 Availability Existing ODPS Capabilty Inspection X
3.7.4 System Failure Existing ODPS Capabilty Inspection X
3.7.5 Operational Duty Cycle Existing ODPS Capabilty Inspection X
3.7.6 Design Life New Data Product Storage Inspection
28
Ocean PEATE Testing Status (6 of 6)
Req. Requirement Support Verification Status
3.7.7 Maintainability Existing ODPS Capability Inspection X
3.7.8.1 Unclassified Security Restriction Existing ODPS Facility Inspection X
3.7.8.1 Complying with NASA Security of Information. Technology1 Code 600 Security Plan Inspection X
3.7.8.1 Operators Access and Privilege Existing ODPS Capability Inspection X
3.7.8.1 SDS Data Verification New Data Acquisition and Ingest Interface Test
3.7.9 Data Integrity New Data Acquisition and Ingest Interface Test




29
Ocean PEATE Requirements Implementation
  • 98 of requirements (55) implemented by Build 2
    add additional storage capacity before launch

30
Sustaining Operations and Maintenance
  • ODPS runs 24x7 with on-site support 8x5.
  • Support is shared across all projects.
  • Staffing needs are covered by existing OBPG
    support personnel.

31
Ocean PEATE Agenda
  • Ocean PEATE Overview
  • Implementation Status
  • Documentation
  • Support Staff
  • EDR Assessment
  • Issues/Concerns

32
Documentation
  • Ocean PEATE Level 4 Requirements and Operations
    Concept
  • ICDs
  • SD3D to PEATEs
  • MOU with RSMAS
  • ODPS Project Data Management Plan (review)
  • ODPS Software Management Plan
  • Network IT Security Plan

33
Ocean PEATE Agenda
  • Ocean PEATE Overview
  • Implementation Status
  • Documentation
  • Support Staff
  • EDR Assessment
  • Issues/Concerns

34
Ocean PEATE Team
  • Ocean PEATE Management
  • Gene Feldman
  • Fred Patt
  • Validation Support
  • Sean Bailey
  • Jeremy Werdell
  • Software Support
  • Bryan Franz
  • Joel Gales
  • Data System
  • John Wilding
  • Dan Knowles
  • Muriel Taylor
  • Systems Management
  • Paul Smith
  • Joe McSorley
  • Chris Moellers

35
Ocean PEATE Staffing Levels
  • All staffing needs will be met with existing OBPG
    staff
  • Increases will be accommodated as other projects
    (e.g., MODIS Aqua) wind down.

36
Ocean PEATE Agenda
  • Ocean PEATE Overview
  • Implementation Status
  • Documentation
  • Support Staff
  • EDR Assessment
  • Issues/Concerns

37
VIIRS Ocean Products for Evaluation
  • All VIIRS Ocean products to be evaluated will be
    acquired from the IDPS, ADS/CLASS, or the ITSE.
  • Testing and evaluation of algorithm changes will
    require regenerating product time series in the
    ITSE, to be downloaded to the Ocean PEATE.
  • The Ocean PEATE will design changes to the code
    in the ITSE for the purpose of algorithm
    improvement or problem resolution, develop
    appropriate test cases and request runs to verify
    and evaluate the changes.

38
IDPS VIIRS Ocean EDR Data Flow
Processing Module
VIIRS Product
VIIRS RDR
Dynamic Ancillary Data
Previous VIIRS Gridded Products
Static Ancillary Data
Previous VIIRS Gridded Products
NAAPS TOD
MODIS Land/Water Mask
NCEP Geopotential Height
Ancillary Files
Ancillary Files
DEM
NDT
39
IDPS VIIRS Ocean EDR Data Flow
VIIRS Geolocation
VIIRS SDR
Processing Module
VIIRS_SDR_01 RDR Decompression
VIIRS_GEO_01 Geolocation
VIIRS_SDR.IM 375m SDR
VIIRS_SDR.MOD 750m SDR
VIIRS SDR
VIIRS Product
VIIRS RDR
Dynamic Ancillary Data
Previous VIIRS Gridded Products
Static Ancillary Data
Previous VIIRS Gridded Products
NAAPS TOD
MODIS Land/Water Mask
NCEP Geopotential Height
Ancillary Files
Ancillary Files
DEM
NDT
40
IDPS VIIRS Ocean EDR Data Flow
VIIRS Geolocation
VIIRS SDR
Processing Module
VIIRS_SDR_01 RDR Decompression
VIIRS_GEO_01 Geolocation
VIIRS_SDR.IM 375m SDR
VIIRS_SDR.MOD 750m SDR
VIIRS SDR
VIIRS Product
VIIRS RDR
Dynamic Ancillary Data
Previous VIIRS Gridded Products
Static Ancillary Data
Previous VIIRS Gridded Products
VIIRS_GD_08 750m Granulation
NAAPS TOD
VIIRS_GD_25 NAAPS Granulation
MODIS Land/Water Mask
VIIRS_GD_27 L/W Mask Granulation
NCEP Geopotential Height
Ancillary Files
Ancillary Files
VIIRS_GD_11 Ancillary Profile
VIIRS_GD_09 GFS Granulation
ALL_GD_01 Time Interpolation
VIIRS_GD_28 Surface Pressure Adjustment
VIIRS_GD_12 Bathymetry Granulation
DEM
VIIRS_GD_13 Temperature Granulation
NDT
41
IDPS VIIRS Ocean EDR Data Flow
VIIRS Geolocation
VIIRS SDR
Processing Module
VIIRS_SDR_01 RDR Decompression
VIIRS_GEO_01 Geolocation
VIIRS_SDR.IM 375m SDR
VIIRS_SDR.MOD 750m SDR
VIIRS SDR
VIIRS Product
VIIRS RDR
Dynamic Ancillary Data
Previous VIIRS Gridded Products
Static Ancillary Data
Previous VIIRS Gridded Products
VIIRS_GD_08 750m Granulation
NAAPS TOD
VIIRS_GD_25 NAAPS Granulation
MODIS Land/Water Mask
VIIRS_LN_06 Active Fires
VIIRS_CM_01 Cloud Mask
VIIRS_GD_27 L/W Mask Granulation
NCEP Geopotential Height
Ancillary Files
Ancillary Files
VIIRS_GD_11 Ancillary Profile
VIIRS_GD_09 GFS Granulation
ALL_GD_01 Time Interpolation
VIIRS_GD_28 Surface Pressure Adjustment
VIIRS_GD_12 Bathymetry Granulation
DEM
VIIRS_GD_13 Temperature Granulation
NDT
42
IDPS VIIRS Ocean EDR Data Flow
VIIRS Geolocation
VIIRS SDR
Processing Module
VIIRS_SDR_01 RDR Decompression
VIIRS_GEO_01 Geolocation
VIIRS_SDR.IM 375m SDR
VIIRS_SDR.MOD 750m SDR
VIIRS SDR
VIIRS Product
VIIRS_SN_02 Ice Quality
VIIRS RDR
Dynamic Ancillary Data
Previous VIIRS Gridded Products
Static Ancillary Data
VIIRS_CL_01 Cloud Optical Properties
Previous VIIRS Gridded Products
VIIRS_ST_02 Surface Temp
VIIRS_GD_08 750m Granulation
NAAPS TOD
VIIRS_SN_03 Ice Concentration
VIIRS_GD_25 NAAPS Granulation
MODIS Land/Water Mask
VIIRS_LN_06 Active Fires
VIIRS_CM_01 Cloud Mask
VIIRS_GD_27 L/W Mask Granulation
NCEP Geopotential Height
Ancillary Files
Ancillary Files
VIIRS_ST_01 Sea Surface Temperature
SST EDR
VIIRS_GD_11 Ancillary Profile
VIIRS_GD_09 GFS Granulation
VIIRS_AR_01 Aerosol Type
ALL_GD_01 Time Interpolation
VIIRS_GD_28 Surface Pressure Adjustment
VIIRS_OC_01 Ocean Color / Chlorophyll
VIIRS_GD_12 Bathymetry Granulation
OCC EDR
DEM
VIIRS_GD_13 Temperature Granulation
NDT
43
ODPS MODIS Ocean Product Data Flow
Processing Module
MOD_PR01 Level-0 to 1A
MODIS Product
Dynamic Ancillary Data
MODIS Level-0
MODIS Level-1A
Static Ancillary Data
MODIS 1 km Level-1B
MOD_PR03 Geolocation
MOD_PR02 Level-1A to 1B
MODIS SST
Platform ATTEPH Data
MSl12 Level-1B to 2
MODIS Geolocation
MODIS Ocean Color
MET Ancillary Data
Ozone Ancillary Files
Land/Water Mask
44
Ocean Team Algorithm Change Process
N
ST member develops evaluation strategy
Products improved?
PEATE staff request ITSE granule-level test
ST analyzes products and test results
Y
PEATE staff upload test input data and
request evaluation test run
ST member identifies algorithm change
ITSE performs test run
ST provides algorithm change recommendation to
PSOE
ST member and PEATE staff develop code change
PEATE retrieves test products
ITSE performs test run
PEATE and ITSE staff install and build code change
ST member verifies test products
PEATE retrieves test products
N
Expected results?
Code correct?
Y
N
Y
ST member evaluates test products
PEATE and ST member check code change
45
Evaluation vs. Product Level
  • Granule-based Evaluations
  • Level-1 (SDR) Evaluations
  • Onboard calibration analyses
  • Vicarious calibration
  • Level-2 (EDR) Evaluations
  • Matchup analyses
  • Flag analyses
  • Time Series Evaluations
  • Level-2 (EDR) Evaluations
  • Residual detector (striping) and scan (RVS)
    dependence
  • Level-3 Product Evaluations
  • Sensor and algorithm comparisons
  • Temporal anomaly evaluations

46
Vicarious Calibration
  • Vicarious calibration using surface measurements
    (e.g., MOBY) gives constant gain correction to
    TOA radiances over the mission
  • Analyses of calibration results provide insight
    into temporal and geometric sources of variation.

47
Vicarious Calibration
  • Strengths
  • Fine-tuning of sensor absolute radiometric
    response allows for correction of prelaunch
    radiometric calibration
  • Limitations
  • System-level calibration that incorporates
    atmospheric correction into measurements,
    convolving algorithm and sensor effects.
  • One-time gain correction per band for the entire
    sensor lifetime assumes sensor stability has
    already been established through other means not
    a substitute for lack of instrument
    characterization.
  • Observation limitations and rigorous data quality
    control severely restrict the number of useful
    measurements (typically 1 or 2 per month per
    site).
  • Specific to ocean algorithms, not applicable to
    other surface types.
  • Availability of MOBY or equivalent is unknown for
    VIIRS.

48
Matchup Analysis
  • Ocean data granules in ODPS catalog are
    automatically matched with in situ data
  • SeaWiFS Bio-optical Archive and Storage System
    (SeaBASS) stores and manages in situ holdings
    from field programs and supported investigators.
  • Ocean staff acquire, QC and analyze new data
    samples.
  • Over 60,000 in situ profiles stored covering
    SeaWiFS/ MODIS time frame.

49
Matchup Process Flow Chart
50
Matchup Analysis
  • Strengths
  • The only truly independent validation of the
    science data products using ground truth
    measurements.
  • Limitations
  • Quality of in situ data is highly variable and
    difficult to assess.
  • Coverage for OC in situ data is limited, both
    geographically and temporally.
  • Assumes that highly localized (meters)
    measurements are representative of pixel (km)
    area.
  • In situ measurements require discipline expertise
    to analyze and compare with satellite values
  • Generally useful only for assessing static biases
    in final products.
  • Availability of in situ data in VIIRS timeframe
    is unknown.

51
Flag Analyses
  • Determine loss of coverage resulting from each
    flag
  • Verify effectiveness of flags at removing invalid
    or questionable data

50-meter depth
Sensor zenith
52
Residual Detector Dependence
  • Level-3 multi-day products (same or different
    sensor) are resampled to generate values at
    Level-2 resolution.
  • Comparisons are performed by detector and mirror
    side to evaluate residual detector response
    errors

53
Residual Scan Dependence
  • Level-3 multi-day products (same or different
    sensor) are resampled to generate values at
    Level-2 resolution.
  • Comparisons are performed by pixel number and
    mirror side to evaluate residual response-vs-scan
    (RVS) errors

54
Residual Detector/Scan Dependence
  • Strengths
  • Provides evaluation of response vs. detector,
    scan angle and mirror side independent of onboard
    calibration.
  • Can also be used to evaluate errors in other
    sensor characteristics
  • Limitations
  • Works well mainly in temporally stable areas.
  • Noise levels may be large compared with residual
    errors.

55
Level-3 Product Generation
  • Sensor, algorithm and interannual comparisons
    require Level-3 binned (equal-area) products.
  • The Ocean PEATE is implementing software to
    process VIIRS EDRs to Level 3 binned products in
    current ODPS (SeaWiFS-like) format.
  • Use current binning code with new input functions
    to read EDRs
  • This will automatically provide the additional
    capabilities to produce multi-temporal
    composites, standard mapped image (SMI) products
    and Level-3 browse files.
  • Time-series analyses of recommended algorithm
    changes will require reprocessing EDRs in the
    ITSE.

56
Sensor and Algorithm Comparisons
  • Level-3 parameters (e.g., nLw) compared for
    common spectral bands
  • Common bins extracted and compared over the
    period of overlap between the sensors
  • Comparisons are performed globally (deep water,
    oligotrophic, mesotrophic, eutrophic), zonally
    and for specified regions.

57
Zonal Cross-Comparisons
58
Chlorophyll Image Comparisons
SeaWiFS Deep-Water Chlorophyll 2002 193
MODIS Deep-Water Chlorophyll 2002 193
SeaWiFS Deep-Water Chlorophyll 2006 365
MODIS Deep-Water Chlorophyll 2006 365
59
Level-3 Comparisons
  • Strengths
  • Sensitive to small differences in products from
    different sensors or algorithms.
  • Excellent coverage available, both temporal and
    geographic.
  • Can assess continuity among data sets (Climate
    Data Records)
  • Limitations
  • No obvious truth in comparisons.
  • Sensitive to band-pass differences.
  • May be affected by time-of-observation
    differences.
  • Availability of other sensors for comparison
    (SeaWiFS or MODIS) is unknown climatology is
    available as an alternative

60
Temporal Anomaly Evaluations
  • Level-3 global averages for the entire mission
    are fit to a periodic function to remove natural
    annual variability.
  • The differences between the global averages
    and the annual cycle are then plotted over the
    mission.
  • The results show both geophysical variations
    and any unexpected changes due to uncharacterized
    instrument effects or algorithm artifacts.

61
Temporal Anomaly Example
62
Temporal Anomaly Evaluations
  • Strengths
  • Very sensitive to small changes in sensor
    performance
  • Limitations
  • Difficult to distinguish sensor from real
    geophysical changes.
  • Can be affected by sampling variations.

63
Issues/Challenges/Concerns
Issue/Challenge/Concern Action/Mitigation
Limits of available test data Ability to adequately test data flows across interfaces Verification of EDR evaluation processes VIIRS Ocean Science Team will develop an independent simulation of the VIIRS ocean bands
Extent of prelaunch end-to-end (observatory ground system) data flows Develop/maintain system readiness for testing opportunities
ADS to PEATE interface and test planning Communication initiated, TIM planned
Complexity of IDPS internal processes and data flows impacts ability to diagnose software and algorithm problems in the ITSE Exercise in the ITSE as possible
Version synchronization of operational products with IDPS software in the ITSE Coordinate software updates with ITSE support staff
SD3E to Ocean PEATE data transfers Performance improvements being worked on both sides
IDPS file naming convention Attempt to get patch for ITSE develop workarounds for testing with SD3E
About PowerShow.com