Distributed Archives Interoperability

1
Distributed Archives Interoperability

• Cynthia Y. Cheung
• NASA Goddard Space Flight Center
• Virtual Observatory of the Future Conference
• Pasadena, California
• June 15, 2000

2
Goal of Virtual Observatory

• One global virtual database to enable search for
  virtual "objects" on a large scale?

3
Current Status

• Global Astrophysics Data Resources Loosely
  Connected By the Internet
• Observational data archives or repositories
• Derived data products (astronomical catalogs,
  browse images, video)
• Data analysis packages
• Visualization/presentation packages
• Special services (bibliography,
  discipline-specific knowledge bases, directories)
• Requires both Vertical and Horizontal Integration

4
Path to the Future
User

• Current (connections via hyperlinks) one to one
• Near Future (connections to multiple DBs all at
  once, via middleware) one to many
• Long Term (multiple inter-connectivity, federated
  databases) many to many
• Distributed Autonomous data centers
• Intelligent Agents
• User-defined Profiles and Preferences
• Access via Multiple Interfaces

Resources
User
Middleware
Resources
User
Resources
5
How Do We Define Interoperability?

• Integrated search and discovery
• URL Registry (e.g., Yellow Pages, AstroBrowse)
• Query processor (e.g., AMASE, ISAIA)
• Browsing/visualization to support selection (ADC
  Data Viewer, AEQ)
• Batch queries (Feed output stream of one data
  service to another)
• Tools to support integration of results
• Data and software exchange
• FTP of data and software updates (pull)
• Download of Browser Plug-in (pull)
• Automated Updates (HST DB replication) (push)
• Hybrid Techniques (with data cache or aircache)
  (push pull)
• Packaging of software with data (see XDF poster)

6
Interoperability (contd)

• Integrated Analysis Environment (IRAF, AIPS,
  etc.)
• Standard Interfaces for Software Modules
• Data Format Standards (FITS, ASCII, IEEE Binary,
  Astronomical XML)
• Semantic Interoperability
• Data Understanding (Domain Context)
• Human Interface, S/W Mapping, Object-oriented
  Mapping
• Data Annotation (Metadata)
• Data Organization
• Indexing for Fast Access
• Query Optimization across Multiple Sites
• Query Plan Execution
• Cross-correlation on a Large Scale

7
Technical Issues and Challenges

• Example Positional correlation of objects in a
  region of the sky across multiple wavelengths
  (Radio, IR, Optical, UV, X-rays, Gamma Rays)
• Data volume and network bandwidth
• Cache of pre-computed results (e.g., astronomical
  catalogs)
• Data filtering at data site, ship results only
  (see poster)
• Deployment of user code (platform independent
  S/W) or RPI
• Data visualization for exploration and selection
• Registration, Sensitivity, Positional Accuracy
• Coordinate transformation (on a large scale)
• Calibration and normalization
• Guide stars for VOF registration?

8
Technical Issues and Challenges (contd)

• Semantic Standards
• Data Format and Usage Convention (e.g., FITS)
• Extensible Standards to Encompass Different
  Disciplines
• Metadata Standards (DTD, XML)
• Astronomical Nomenclature
• Tools for Schema Mapping
• Track Changes and Updates at Other Sites
• Tools for Data Integration and Fusion
• Interface with User Preferences
• Intelligent Software Agents