Establishing a UserDriven, WorldClass Oceanographic Data Center by the Right People, in the Right Pl

1
Establishing a User-Driven, World-Class
Oceanographic Data Center by the Right People, in
the Right Place , and at the Right Time

• L. Charles Sun

National Center for Ocean Research 20-24 June,
2005, Taipei, Taiwan
2
Outline

• Time, Place, and People
• Steps in Establishing an NODC
• Mission and Role of an NODC
• QC and QA
• Products and Services
• Information Technology
• Organizational Considerations and Chart
• Collaboratory
• IDARS, Argo GTSPP Three examples of
  Collaboratories
• Data Portal Gateway to Ocean Data
• Climate Data Portal The Proven Prototype
• Other Technologies for the Collaboratory
• The Future

3
Time, Place, and People

• Time Since 1975
• Place The Center of the world
• People We are the right people

4
Steps in Establishing an NODC - I

• Recruit a team of interested parties to propose a
  mission and organizational model for the center.
• Construct a draft mission.
• Conduct negotiations with the potential partners.

5
Steps in Establishing an NODC - II

• Prepare a draft administrative organization.
• Prepare a final version of the mission and
  information on partnerships for final approval.

6
Organization Chart
7
Mission of an NODC

• To safeguard versions of oceanographic data and
  information.
• To provide high quality data to a wide variety of
  users in a timely and useful manner.

8
Roles of an NODC

• Conventional role as a minimum
• Contemporary role in response to advances in
  data collection and information technology

9
Conventional Role - I

• Receive data, perform quality control, archive
  and disseminate it on request.
• Keep copies of all or part of its data holdings
  in the format in which the data were received.
• Developing and protecting national archives of
  oceanographic data

10
Conventional Role - II

• Produce and provide inventories of its holdings
  on request.
• Referral of the users to sources of additional
  data and information not stored in the NODC.
• Participate in international oceanographic data
  and information exchange.

11
Contemporary Role - I

• Receive data via electronic networks on a daily
  basis, process the data immediately, and provide
  outputs to the user or to the data collectors for
  data in question.
• Report the results of quality control directly to
  data collectors as part of the quality assurance
  module for the system.

12
Contemporary Role - II

• Process and publish data on the Internet and on
  CD/DVD-ROMs.
• Publish statistical studies and atlases of
  oceanographic variables.
• Performing a level of quality control on its data
  holdings

13
Quality Control and Assurance

• Data can be detected easily by a data center
• Obvious errors such as an impossible date and
  time and location
• Data cannot usually be detected by a data center
• Subtle errors such as an instrument may be off
  calibration

14
Information Technologies - I

• Data Storage/Archive
• Data Processing
• Local Area Networking
• Wide Area Networking the Internet (and the GTS)

15
Information Technologies - II

• Publishing DVD/CDROMs
• Graphics Capability (Graphical Information
  System)
• Software Development Implementation
• Hardware procurement Maintenance

16
Products Development - I

• Work with the client to determine what the real
  need. Examples of data products include atlases,
  datasets of ocean observations filtered by area,
  time and variables observed

17
Products Development - II

• Review the world wide web sites of existing NODCs
  for ideas and examples of data and Information
  products.

18
Services

• Providing directory and inventory information
• Acting as a referral center
• Receiving data for specific processing followed
  by delivery of the processed data

19
Organizational Considerations

• A centralized data center
• A distributed data center
• Centers of Data Data Portals or Virtual
  Collaboratories

20
What is a Collaboratory?
The fusion of computers and electronic
communications has the potential to dramatically
enhance the output and productivity of
researchers. A major step toward realizing that
potential can come from combining the interests
of the scientific community at large with those
of the computer science and engineering community
to create integrated, tool-oriented computing and
communication systems to support scientific
collaboration. Such systems can be called
"collaboratories." From "National
Collaboratories - Applying Information Technology
for Scientific Research," Committee on a National
Collaboratory, National Research Council.
National Academy Press, Washington, D. C., 1993.
21
Acknowledgement
Soreide, N. N. and L. C. Sun, 1999 Virtual
Collaboratory How Climate Research can be done
Collaboratively using the Internet. U.S. China
Symposium and Workshop on Climate variability,
September 21-24, 1999, Beijing, China Presented
by Len Pietrafesa, North Carolina State
University.
22
Collaboratory Infrastructure

• Data Portal
• Computer and networking hardware and software
• Increased network bandwidth/speed
• Next Generation Internet (NGI) connection
• Visualization
• Interactive Java graphics
• 3D, Virtual Reality, collaborative virtual
  environments
• immersion technology CAVE, ImmersaDesk...
• Relationships
• Observing System Project Offices
• Research community, Academia...
• Other Collaboratory nodes
• Steering Committee

23
Structure of the Collaboratory for Ocean Research
International Steering Committee
Collaboratory Partner
Collaboratory Partner
Collaboratory Partner
Collaboratory Partners Customers Providers of
Data Information Users of Data Information
Observations Satellite Groups
Modeling Forecasting Groups
Research Groups
New Users Educational Administrators
General Public
24
IDARS as an example...

• Real-Time Coastal Water Temperature Data
• Real-Time Argo Profile Data
• Real-Time Global Temperature and Salinity
  Profile Data
• Time Series Data
• NOAA CoastWatch AVHRR SST Images

http//www.nodc.noaa.gov/idars/
Interactive Data Access and Retrieval System
25
Argo as an example...
26
GTSPP as an example...
Global Temperature-Salinity Profile Program
27
Argo and GTSPP

• Argo and GTSPP set a standard in the
  international ocean data management community
• Data dissemination in near-real time
• Researcher involvement has assured data quality
• Benefits of data dissemination
• Wide use of Argo and GTSPP data
• Traditional research, modeling, forecasting
  groups
• Related disciplines, educational, administrative,
  public
• With recent advances in technology, we can do
  much more...

28
Distributed Object Technology

• Data servers and datasets are objects software
  packages of procedures and data that contain
  their own context
• Solid, commercial underpinning for distributed
  object technology in the ocean sciences

29
Adaptability and Scalability of distributed
object systems

• Distributed object systems in the commercial
  arena
• Are robust, reducing system maintenance and
  upkeep costs
• Supported by Object Management Group (OMB)
• standards body for Internet Inter-ORB Protocol
  (IIOP) distributed object protocols
• CORBA/IIOP and Java RMI/IIOP
• consortium of large (Fortune 500) companies
• Cross platform independence, compliance with
  standards

30
The Data Portal a gateway to ocean data

• Why do we need a Data Portal?
• Each center of data provides a highly customized
  Web sites for their data
• but different datasets have different navigation
  and interface characteristics
• so the user faces a bewildering spectrum of data
  access interfaces and locations
• Data Portal is single, uniform, consistent
  gateway to ocean data in a common format
• User goes to a single location and sees a
  consistent interface
• Complements the customized data access

31
Data Portal/Visualization/Collaboration
Distributed data Observed data Satellite
data Data and information products
Model outputs Visualization
Data Information Users
Traditional users Modelers Forecasters Researcher
s New users Educators Students General Public
Uniform network access
32
Data Portal
Data Server
One or more Web Servers
User
Observing System Server
CORBA
TAO data support
Data
Web Browser
Java Servlet
Client Support
Network
Network
Graphics
CORBA
Java Application
CORBA
Common Object Request Broker Architecture (CORBA)
is an industry standard Middleware. CORBA is
used in the NOAAServer software from which this
effort will leverage. Based on performance
indicators, Java Remote Method Invocation (RMI),
an alternative middleware, could easily be
substituted for CORBA.
Data
33
Data Portal
Data Servers
One or more Web Servers
User
Observing System Servers
CORBA
TAO data support
Data
Web Browser
Java Servlet
CORBA
Client Support
Network
Network
Drifter Data support
Data
Graphics
CORBA
Java Application
CORBA
Common Object Request Broker Architecture (CORBA)
is an industry standard Middleware. CORBA is
used in the NOAAServer software from which this
effort will leverage. Based on performance
indicators, Java Remote Method Invocation (RMI),
an alternative middleware, could easily be
substituted for CORBA.
Data
34
Data Portal
Data Servers
One or more Web Servers
User
Observing System Servers
CORBA
TAO data support
Data
Web Browser
Java Servlet
CORBA
Client Support
Network
Network
Drifter Data support
Data
Graphics
CORBA
In-Situ/Satellite Data Servers
CORBA
Java Application
In-Situ/Satellite data support
Data
Model Output Servers
CORBA
CORBA
Model data support
Data
Gridded Data Servers
CORBA
Common Object Request Broker Architecture (CORBA)
is an industry standard Middleware. CORBA is
used in the NOAAServer software from which this
effort will leverage. Based on performance
indicators, Java Remote Method Invocation (RMI),
an alternative middleware, could easily be
substituted for CORBA.
Gridded data support
Data
Data
35
How do we build a Data Portal?

• Build on a proven prototype
• connects 5 geographically distributed data
  servers in Silver Spring, Boulder, Seattle
• CORBA for network connections
• unified interactive Java graphics
• data from distributed servers are co-plotted
  together on the same axis on the users desktop
• http//www.pmel.noaa.gov/nns/noaaserver/nodc-coad
  s-tao.html
• http//www.pmel.noaa.gov/nns/noaaserver/coads-tao
  -raster.html

36
Prototype Data Portal CDP
Silver Spring MD
Climate Data Portal
37
Climate Data Portal Sample Plots
38
Data Selection Web Interface

• Utilizes CORBA for network connections.
• Utilizes EPIC Web Technology
• Java Applets
• JavaScript
• Java Servlets
• Searches data by keywords, location and time
  ranges.

39
Web Interface screen Shots
40
Other Technologies for the Collaboratory

• Networks (100 Megabits/sec today, 10 Gigabits/sec
  in future)
• Next Generation Internet (NGI) and Internet 2
• Visualization
• Interactive Java graphics
• 3D, Virtual reality
• Immersion technology
• Collaboration tools
• high-speed telecommunications systems for
  advanced collaboration applications
• tele-immersion systems allow individuals at
  different locations to share a single virtual
  environment
• Use networks not airplanes for collaboration

41
Virtual Reality

• Virtual Reality lets the scientist touch the
  data, move into it, and see it from different
  viewpoints
• The realism of virtual reality enables the
  scientist and the lay person to understand
  complex ideas more easily
• Scientists using virtual reality affirm this new
  technology discloses features of their data and
  model outputs which were undiscovered with
  standard visualization techniques
• Virtual reality can be approachable and
  affordable
• Widens audience for scientific data and
  information
• Government administrators and decision makers
• Educators and students
• General public
• Some examples follow

Courtesy of Nancy N. Soreides, PMEL
42
Why use Virtual Reality?
El Nino
La Nina
Virtual reality modeling language (VRML)
rendering of temperatures and sea surface
topography along the equator in the tropical
Pacific, viewed from South America, showing the
dynamics of El Nino and La Nina. Using an
inexpensive PC and a web browser with a free
plug-in, the images can be rotated, animated, and
zoomed. Changes in the equatorial Pacific during
El Nino and La Nina are clearly understood by
scientist and layman. http//www.pmel.noaa.gov/to
ga-tao/vis/vrml/ or http//www.pmel.noaa.gov/vrml
Courtesy of Nancy N. Soreides, PMEL
43
Stereographic Virtual Reality
3D, interactive virtual reality visualizations
are not difficult for a scientist to create or to
view, from the web or from the desktop, and the
effect can be enhanced dramatically by including
the capability of stereographic viewing. With a
PC and a 99-cent pair of red/green sci-fi
glasses, the spheres and vectors will pop out of
the page in stereo, revealing the true 3D
location of the fish, the steep slopes of the
bathymetry, and the vertical motions near the
submarine canyon. The images can be rotated,
animated and zoomed. http//www.pmel.noaa.gov/her
mann/vrml/stereo.html
Stereo
Fish larvae and velocity vectors in a submarine
canyon, from a circulation model of Pribolof
Canyon in the Bering Sea. Use red/green glasses
to see images on the right in stereo.
Stereo
Courtesy of Nancy N. Soreides, PMEL
44
Immersive Virtual Reality

• Immersive devices provide the graphical illusion
  of being in a three-dimensional space by
  displaying visual output in 3D and stereo, and by
  allowing navigation through the space.
• Navigating through our virtual environments and
  viewing the data from different vantage points
  greatly increases our ability to perform analysis
  of scientific data.
• The impact of such visualizations in person is
  stunning, and must be experienced by the
  scientist to be fully comprehended .
• Users of these advanced immersion technologies
  affirm that no other techniques provide a similar
  sense of presence and insight into their
  datasets.

Courtesy of Nancy N. Soreides, PMEL
45
The CAVE
View of the CAVE
The CAVE is a multi-person, high resolution, 3D
graphics video and audio virtual environment. The
size of a small room (10x10x10 foot), it consists
of rear-projected screen walls and a
front-projected floor. Using special
"stereoscopic" glasses inside a CAVE, scientists
are fully immersed in their data. Images appear
to float in space, with the user free to "walk"
around them, yet maintain a proper
perspective. The CAVE was the first virtual
reality technology to allow multiple users to
immerse themselves fully in the same virtual
environment at the same time.
Scientist inside the CAVE
CAVES have been deployed in academia, government,
and industry, including NASA, NCAR, NCSA, Argon
National Laboratory, Caterpillar Corp., General
Motors, among others. http//www.pyramidsystems.co
m/CAVE.html
Courtesy of Nancy N. Soreides, PMEL
46
The ImmersaDesk
Courtesy of Nancy N. Soreides, PMEL
47
The Future
The development of scientific data manipulation
and visualization capabilities requires an
integrated systems approach including the
end-to-end flow of data from generation to
storage to interactive visualization, and must
support data retrieval, data mining, and
sophisticated interactive presentation and
navigation capabilities. Data Exploration of
petabyte databases will required both technology
development and altered work patterns for
research scientists and engineers.
Data and Visualization Corridors, Report on the
1998 DVC Workshop Series, Edited by Paul H.
Smith and John van Rosendale, Sponsored by the
Department of Energy and the National Science
Foundation, 1998.
Courtesy of Nancy N. Soreides, PMEL
48
THANK YOU ALL!
Charles.Sun_at_noaa.gov