Data Management and Client/Server Interactive System for Access Ocean and Climate Data Developed at PMEL

1
Data Management and Client/Server Interactive
System for Access Ocean and Climate Data
Developed at PMEL

• Willa Zhu - ? ?
• PMEL/NOAA -- JISAO/University of Washington
  Seattle, Washington
• Aug 17. 2005

2
Outline

• Part I. Organization ????
• Part II. Data Management ????
• Part III. Client/Server Interactive
  System??/????????

3
Part I Organization

• ????

4
National Oceanic Atmospheric Administration
5
OAR

6
(No Transcript)
7
PMEL Research Activities

• Ocean Climate Research
• Argo Profiling Floats (real-time ocean
  temperature and salinity for climate, fisheries,
  and more)
• Atmospheric Chemistry
• Carbon Dioxide
• Chlorofluorocarbons
• TAO (buoy array)
• TMAP (equatorial ocean modeling)

• Ocean Environment Research
• Tsunami (hazard mitigation)
• VENTS (hydrothermal studies)
• FOCI (fisheries oceanography)
• SEBSCC (ecosystem studies)

8
PMEL Software Development Teams

• EPIC Team develop software mainly supporting
  in-situ data management, analyses, display, and
  distribution

• TMAP Team develop software mainly supporting
  gridded data and model data access and display

• Other teams and individual developers

9
PMEL External Support

• PMEL software development teams have been working
  with other NOAA inter-agencies and universities
  on many projects.

• National Oceanographic Data Center (NODC), NOAA
• International Pacific Research Center (IPRC),
  US-Japan
• National Severe Storm Laboratory (NSSL), NOAA
• Pacific Hydrographic Branch, NOAA
• Etc.

10
Part II Data Management

• ????

11
PMEL Data Resource

• Collected by local PI from different project
  (e.g. TAO, Argos floats)
• Some archived data from distributed source (e.g.
  World Ocean Database from NODC)
• Datasets are provided by other institutes and
  research labs in order to use PMELs data
  service through joint project (e.g. OSCAR)

12
In-situ Data Type

• CTD
• XBT
• Bottle
• ADCP
• Current mooring
• Drifting buoy data

• Atmosphere data
• Ocean surface velocity fields
• Climate data
• Sea level data
• Time series

13
Data Format

• netCDF
• Matlab
• ASCII
• Database (mySQL, postgreSQL)

14
Data Process

• Raw data collected and QC done by individual PI
  (his/her programmer or research assistant).
• Software or tools developed by PMEL software
  development team are used (e.g. EPIC, Ferret)
• Final version of In-situ data will be loaded into
  database and made available via web

15
EPIC Data Management

• EPIC provides utilities for database (mySQL and
  PostgreSQL) support
• Metadata loaders
• Data selection programs
• netCDF data file
• Searchable metadata includes
• Geo-location, time, and depth range
• cruise identifiers, and mooring name
• instrument name and type

16
EPIC Data Analysis Display

• EPIC analysis tools can work with
• profile data (CTD casts, XBT, bottle)
• time series (moored instruments)
• ADCP (acoustic doppler current profiler)
• drifting buoy data
• Tools can calculate wide range of oceanographic
  parameters including dynamic height, geostrophic
  velocity, and potential density.

17
Part III Client/Server Interactive System

• ??/???
• ?????

18
Client/Server Architecture
Desktop application client
Web client
Web Service Application



Data Server


19
Web Interface for Data Access

• EPIC Web Browser for Profile and Time Series data
• Argo Profiling CTD Floats Data
• TAO Data
• Bering Climate Data
• Real-time ocean surface currents
• Arctic time series Data Collection
• Historic Weather Data Archives

20
EPIC Web Browser Architecture
Data Selection
Return results (list of data files)
CGI, PHPEPIC programsPPLUS, etc.
read
EPIC Web Browser for Profile and Time Series data
21
Epic Web Browser
http//www.epic.noaa.gov/epic/ewb

• Data Selection
• Profile
• Time Series

22
Profile Data Selection

• Choose Data Collection
• Show information of selected data collections
• total number of files
• overall time coverage
• location range on map

23
View data Inventory

• description of data collection
• total number of files
• time coverage
• location range
• depth range

24
Specify search constrains

• Location range (draw box on map or text entry)
• Depth range
• Time range
• Other parameters

25
Plot Options and Results
Temp. vs. Sal.
(T, Sal, Sigma-t, Ox) vs. Depth
26
Single Data Listing
List Meta-Data/Attributes
List data in ASCII format
27
Multiple Data Selection Refinement
28
Access multiple data files for data display
Vertical section plot parameters

• Variable to grid
• Horizontal axis type
• Number of grid nodes
• Depth axis range
• Area or line
• Color map
• Contour levels
• Plot data points
• Label profiles with cast number
• Other parameters

29
Access multiple data files for data
listing/downloading

• list data
• view attributes
• download data

30
Access multiple data files for attribute listing
View Attributes (in HTML table or ASCII format)
31
Access multiple data files for data listing
Access multiple data files for data listing
List data (in HTML table or ASCII format)
32
Time Series Stacked Plot
33
Time Series Histogram Plot
34
Web Interface for Data Access

• EPIC Web Browser for Profile and Time Series data
• Argo Profiling CTD Floats Data
• TAO Data
• Bering Climate Data
• OSCAR (realtime ocean surface currents)
• Unaami Data Collection (Arctic time series)
• Historic Weather Data Archives

35
(No Transcript)
36
Argo Web Graphics
37
Argo CTD Floats Web Listing
38
Web Interface for Data Access

• EPIC Web Browser for Profile and Time Series data
• Argo Profiling CTD Floats Data
• TAO Data
• Bering Climate Data
• OSCAR (real-time ocean surface currents)
• Unaami Data Collection (Arctic time series)
• Historic Weather Data Archives

39
TAO Data Display On Web
40
Bering Climate Data
41
(No Transcript)
42
Atmospheric Data Web Access
43
Atmospheric Data Web GraphicsCustomized displays
Upper Air Data Profile Plot
Surface Data Time Series Plot
44
Client/Server Architecture
Desktop application client
Web client
Web Service Application



Data Server


45
Desktop Application (support in-situ Data)

• EPIC Software for in-situ data - Data management,
  display and analysis of observational in situ
  data
• Java OceanAtlas - Java application for
  viewing/manipulation oceanographic profile data
• ncBrowse - Interactive graphical display of data
  and attributes from a wide range of netCDF data
  files

46
Client/Server Architecture
Desktop application client
Web client
Web Service Application



Data Server


47
Data Server

• Climate Data Portal - links geographically
  distributed data servers into a network to
  provide a portal to in-situ ocean observations
  data in a common data format.
• Dapper
• community-standard OPeNDAP in-situ data server
• OPeNDAP has also been known as DODS.
• consistent with the consensus vision that has
  evolved for "An Integrated Ocean Observing System
  for the United States
• consistent with the OPeNDAP-standard sequence
  structures utilized by the Grads Data Server (the
  GDS server)
• Dapper also serves gridded datasets.

48
What is OPeNDAP?http//opendap.org

• OPeNDAP Open-source Project for a Network Data
  Access Protocol
• OPeNDAP is a framework that simplifies all
  aspects of scientific data networking.
• OPeNDAP provides software which makes local data
  accessible to remote locations regardless of
  local storage format.
• OPeNDAP also provides tools for transforming
  existing applications into OPeNDAP clients (i.e.,
  enabling them to remotely access OPeNDAP served
  data).
• OPeNDAP software is freely available.

49
What OPeNDAP can do for you?http//opendap.org/ge
tStarted/whatDODS.html

• OPeNDAP allows you to make your data available
  remotely.
• Your data can be in a variety of data format,
  including netCDF and HDF
• see our complete list of OPeNDAP servers

• OPeNDAP allows you to access remote data over the
  internet  
• using familiar data analysis and visualization
  packages like
• Matlab
• Ferret
• IDL
• see our complete list of OPeNDAP clients
• without worrying about data storage formats
• accessing only the data of interest

50
Dapper architecture diagram
Dapper HTTP Server
OPeNDAP client

CORBA (IIOP)

Climate Data Portal


Java netCDF library
MySQL

51
Dapper clients from PMEL

• ncBrowse

52
Dapper clients from PMEL

• Java OceanAtlas

53
Summary

• PMEL carries out interdisciplinary scientific
  investigations in oceanography and atmospheric
  science.
• Software development at PMEL provides efficient
  and cost effective data management and access
  system
• Client/Server interactive interface allows user
  to access both local and remote datasets.