Briefing on the Integrated Ocean Observing System IOOS and the Data Management and Communication Sys

1
Briefing on the Integrated Ocean Observing System
(IOOS) and the Data Management and Communication
System

• Steven Worley
• NCAR
• 19 August 2005
• Objectives
• What is IOOS?
• What is DMAC?
• How might OHH participate?

2
The Integrated Ocean Observing System
(IOOS)Oceans Coasts Component of the U.S.
Integrated Earth Observation System (IEOS) the
International GEOSS
An Interagency Collaborationfor the Public Good
NOAA Navy NASA NSF USACE
USGS MMS EPA USCG DOE
http//www.ocean.us
3
IOOS GoalsIntegration for Broad Benefit

• Detect and Predict Change
• Mitigate effects of natural hazards
• Improve safety and efficiency of marine ops
• Improve national security
• Reduce public health risks
• Protect and restore marine ecosystems
• Sustain marine resources

Society oriented goals
4
E.G. Florida Fish and Wildlife Research
InstituteRed tides off southwest Florida, Aug. 17

• Reported system effects
• Expanding area since last week
• Benthic mortalities
• Fish kills
• Discolored water
• Turtle mortalities
• Human respiratory irritation

http//www.floridamarine.org/
5
E.G. Florida Fish and Wildlife Research
InstituteRed tides off southwest Florida, Aug. 17

• Actions
• Airplane over flights not good weather more
  flights needed
• Ocean sampling cruise 10-12 August

6
E.G. Florida Fish and Wildlife Research
InstituteRed tides off southwest Florida, Aug. 17

• Observation and what ifs?
• Respiratory irritation greater with westerly wind
• What if Hi-res wind predictions were available?
• What if satellite sensed data, not impacted by
  weather, was easily used?
• What if real-time ocean circulation models were
  run to analyze and predict?
• What if ocean models with T, O2, nutrients, red
  tide samples, were easy to run?

Contention We have the ability to do these
estimates IOOS strives to enable them and address
the goals previously shown
7
The Political Environment

• Commission on Ocean Policy
• Report
• Executive Order
• Ocean Action Plan
• Legislation
• Senate (S. 361 - recently passed)
• House (H.R. 1489, 1584)

8
IOOSTwo Interdependent Components
Global Ocean Component
Coastal Ocean Component
Regional Coastal Observing Systems
National Backbone
9
IOOS Coastal Ocean Component

• National Backbone
• Largely federal
• Satellite remote sensing
• Reference, fixed-site stations
• Links to global component
• Fewer variables

• Regional Coastal Systems
• Largely non-federal (may be
• federally funded)
• Regional priorities needs
• Greater resolution
• More variables

Both use IOOS data standards and exchange
protocols
10
The Global ComponentAn International
Collaboration
IOOS is the U.S. contribution to international
GOOSand the Oceans Coasts component of IEOS
GEOSS
11
IOOS Three Subsystems

• Measurements
• Remote and in situ observations
• Telemetry
• Data Management and Communications (DMAC)
• Integration
• Data analysis and modeling
• Data synthesis, evaluation
• Change detection and prediction

12
IOOS Education Component

• Ocean.US Education Workshop, 22-24 March 2004,
  Charleston, SC
• Result Workshop Report
• Promoting Lifelong Ocean Education Using the
  Integrated Ocean Observing System (IOOS) to Shape
  Tomorrows Earth Stewards and Science and
  Technology Work Force.
• Muriel Cole Ocean.US liaison to OHH
• Hear more at IOOS Human Health Workshop, January
  2006

13
DMAC is the component that enables the diverse
sources of data to be accessed in a consistent
manner.
DMAC is composed of data standards,
communications protocols, software tools,
archiving and community agreements.
14
Why so much focus on DMAC?

• DMAC is the I in IOOS

Why is DMAC the top priority?

• The weak link in the chain
• Huge payoff from investment

15
DMAC Three Guiding Principles
Existing Protocol
DMAC

• Do no harm

• Expand access

Data Suppliers

• Increase efficiency

16
The DMAC Plan is a framework for interoperability
among independent, heterogeneous systems, large
and small

• Organizations will manage their data
  independently
• Standards enable interoperability
• Data discovery and delivery to applications
• New - Web services standards
• communication between applications
• E.g. Browsers that initiate Google searches

17
The heart of integration is StandardsExample
Surface Marine Buoys

• Work horse for coastal and open ocean
  observations
• International, National and Regional use
• Surface , near-surface and subsurface
  observations

18
What Simple Standards buy you today

• Meeting present standards
• A ticket onto GTS
• Real-time provides utility for national forecasts
  warnings operations
• Accepted QA /QC
• Immediate payoffs
• Large user base
• Improved local storm detection, intensity and
  track forecasts
• Beach rip tide, boating safety and trip planning,
  others

19
But todays standards are too narrow. ?
Inefficiencies and lost opportunities

• E.g. Delayed mode not standardized
• Data not uniformly
• archived
• QCed
• Data not uniformly available for
• climate assessment
• commercial applications
• research (esp. interdisciplinary)
• Commonalities are not exploited (e.g. a time
  series)
• Variables winds, waves, temperature, salinity,
  biota, chemicals, fish landings
• Platforms surface marine buoys, terrestrial
  stream gauges, tide gauges
• Products forecasts, satellite products, coral
  bleaching (paleo), El Nino index

20
IOOS Data Management andCommunications Subsystem
Metadata, Data Discoveryand Data
TransportStandards and Protocols
21
IOOS Data Management andCommunications Subsystem
Ships
Hand Measurements
Satellites
Floats
Primary DataAssembly QC
Moorings
RegionalData ManagementSystems
Products
Users
InternationalData ManagementSystems
Maps
Forecasts
Terrestrial and AtmosphericData
ManagementSystems
Metadata, Data Discoveryand Data
TransportStandards and Protocols
On-line Browse
Archive Centers
Modeling
22
The DMAC Plan a community effort

• Brought together
• 6 Federal agencies
• 6 Universities
• 4 Regional/State agencies
• 3 Private sector orgs.
• Steering Committee,2 Outreach Teams, 4 Expert
  Teams
• Data Facilities Outreach
• User Outreach
• Data Discovery Metadata
• Data Transport
• Data Archive Access
• Applications Products

23
The DMAC Plan a community effort

• Four levels of review
• Selected specialists
• Public workshop
• Two public reviews
• 150 reviewers
• 6 Federal Agencies
• 22 Universities Institutes
• 13 Private Sector
• 13 Regional/State agencies
• 5 International
• Presented at numerous meetings
• ASLO
• AGU
• AMS
• JCOMM, WMO
• IODE

24
The DMAC Plan What does it contain?

• Articulates the vision, requirements,and
  technological approaches.
• Recommends next steps, estimates costs and
  timeframes
• A roadmap, not a specification. No attempt to
  map tasks to specific organizations

25
The DMAC Plan first order guidance

• COOS in RAs advise about IOOS NOW
• DMAC Steering Committee designated initial
  guidance
• Report standards and standards processes
  presently used to Ocean.US
• Use FGDC (ISO 19115) metadata standards if
  starting out
• Use community accepted standards otherwise
• Create metadata in XML-schema with a style sheet
• Gridded data install server and provide access
  through OPeNDAP
• Use RDBMS
• OPeNDAP enabled
• Enterprise GIS, OGC compliant
• Data providers ensure irreplaceable data are
  permanently archived
• Archive Centers structure archives to be
  accessible and DMAC compliant
• Security until IOOS has data and network
  security guidelines use those prescribed by like
  federal centers
• Use NDBC services for mooring data
• Many more, and will be updated, see
  http//www.ocean.us

26
Next Steps Work on DMAC Plan Recommendations

• New DMAC-Steering Team
• Formed in April 2005
• Open, fair, balance representation
• About 25 members
• Objective Minimally functioning DMAC framework
  in the near-term

27
Advancing the DMAC PlanWorking Structure under
the Steering Team

• Expert Teams
• Metadata Discovery
• Standards Process
• Archive
• Transport Access
• Caucuses
• Private Sector
• International
• Modeling
• Working Groups
• IT Security
• On-Line Browse
• Systems Engineering
• Status All but two WGs have SOWs and ETs are
  being formed.
• Possible areas of OHH Participation

28
More on participation

• Expert Teams
• Metadata Discovery
• Standards Process
• Archive
• Transport Access
• How to participate
• Contact, rosalind.e.cohen_at_noaa.gov
• Fit into an ET
• Review SOW, online soon

29
E.G. SOW for the Archive Expert Team

• Statement of need brief,
• Forge cooperation between multiple centers to
  insure
• Data flow integrity
• Publish data for discovery and access
• Stewardship and long-term curation
• DMAC metadata and data transport methods are
  critical cross communication with other ETs
• Preserve data through life cycles of storage
  systems

30
SOW for Archive ET, continued

• Membership
• All US agencies with IOOS data responsibilities
• International counterparts
• Representative from National Archives (NARA)
• Long term strategies and preservation policies
• Members must have knowledge and have authority to
  advance ET goals at the organization level
• Approx. 10 members

31
SOW for Archive ET, continued

• Short term Tasks (2005-2006)
• Review and refine DMAC Plan with report to DMAC
  ST
• Define archive critical metadata provide to
  Metadata ET
• Assess, inventory, and report on IOOS relevant
  archives
• Access status
• Metadata status
• Identify preservation weakness
• Identify gaps in irreplaceable data archives

32
SOW for Archive ET, continued

• Long term Tasks (2007-2008)
• Demonstrate, test bed project, merging metadata
  from multiple ACs into DMAC data discovery portal
• Use DMAC meta data standards and transport
  protocols for two-way data flow
• Receipt and delivery of data
• Establish comprehensive metrics system wide
• Composite data polices from all agencies
• Participate with governing committees to form a
  IOOS data policy
• Many other goals outlined in DMAC plan!

33
SOW for Archive ET, continued

• Schedule
• Form, to extent possible the Archive ET, October
  2005. Current members
• Bob Keeley (MEDS/Canada)
• John Lever (Navy/CNMOC)
• Landry Bernard (NOAA/NDBC)
• Steven Worley (NCAR/NSF)
• Validate membership before DMAC ST, Nov. 2005
• Hold first AET meeting, early 2006
• Use, web forums, email, and twice annual meetings
  to address Short Term tasks

34

• Conclusions
• IOOS and DMAC have begun
• Much growing to do
• Must keep up with rapid developments at COOS and
  RAs
• Fostering broad participation is the challenge
• Technology is tracking the need well
• Our success will be measured by how well we
  fulfill the societal needs

35
Questions? http//www.ocean.us