UK E-Science Initiative and its Application to SDO - PowerPoint PPT Presentation

1 / 25
About This Presentation
Title:

UK E-Science Initiative and its Application to SDO

Description:

... at 10 Gbps covering UK, France, Germany,Sweden, Switzerland with 2.5 Gbps ... Material needed for more detailed searches should be described by metadata in ... – PowerPoint PPT presentation

Number of Views:24
Avg rating:3.0/5.0
Slides: 26
Provided by: jleonard
Learn more at: http://hmi.stanford.edu
Category:

less

Transcript and Presenter's Notes

Title: UK E-Science Initiative and its Application to SDO


1
UK E-Science Initiativeand itsApplication to SDO
  • J.L. Culhane
  • MSSL

2
SUMMARY
  • The UK Astrogrid
  • Dealing with SDO Data Volumes
  • The PPARC E-Science AO
  • HMI Data Products and Pipeline

3
What is the Grid?
  • Ian Foster, Argonne National Lab University of
    Chicago
  • A Grid is a system that
  • Coordinates resources that are not subject to
    centralized control.
  • Uses standard, open, general-purpose protocols
    and interfaces.
  • Delivers nontrivial qualities of service.
  • - Ian Foster, What is the Grid? A Three Point
    Checklist

Network
PC
Laptop
Mainframe
Phone / PDA
Space Missions
GRID
Printer
4
UK Astrogrid
  • Astrogrid is one of three major world-wide
    projects (along with European AVO and US-VO
    projects) which aim to create an astronomical
    Virtual Observatory
  • Astrogrid has a significant Solar Physics
    component
  • The Virtual Observatory will be a set of
    co-operating and interoperable software systems
    that
  • allow users to interrogate multiple data centres
    in a seamless and transparent way
  • provide powerful new analysis and visualisation
    tools
  • give data centres a standard framework for
    publishing and delivering services using their
    data.

5
How does Astrogrid work?
Web Service A web service is any piece of
software that makes itself available over the
Internet and uses a standardized XML messaging
system. - Ethan Cerami, Top Ten FAQs for Web
Services, The OReilly Network
Web Service
Data Archive
User
Web Interface
RESOURCES
Web Service
Web Service
Data Storage
Web Service
Distributed Network of Registries
Data Transformation Processing
6
Astrogrid Registry
Registry Dynamic database of metadata
describing a set of Internet-available resources.
A registry is used to identify and locate
resources satisfying user-specified criteria, and
to direct more detailed information requests to
the relevant services. Robert Hanisch, STSCI
  • METADATA
  • Basic ID, title, service type
  • Curation Location, contact, publisher, creator,
    etc.
  • Metadata Allowed methods, input / output
    variables, etc.
  • Metadata Format Wavelength, coordinates,
    instrument coverage

Registries contain information about resources
7
Solar Interior to Outer Atmosphere
  • Science goal Connect observations of the
    interior to fluctuations in the solar atmosphere
  • Data Required Helioseismology observations
    connected with solar atmosphere observations
  • Current difficulties Being able to search
    efficiently for solar atmospheric events that may
    be responding to an excitation source in the
    interior
  • Grid future Ability to
  • Search easily for events e.g. flux emergence, AR
    evolution, flares, coronal mass ejections, over
    specific time periods
  • Extract parameters over the cycle from the
    atmosphere and interior in order to compare their
    evolution
  • Crucial for SDO to relate convection zone
    observations to magnetic field data for
    Photosphere and above

8
SDO HMI Archiving and Processing
  • SDO instruments generate raw data ( 2 Tbyte/day)
    along with derived products
  • Derived products result from pipeline processing
    that must keep up with the flow of incoming data
  • GRID or Virtual Observatory approach could allow
  • Distributed data holding
  • Distributed processing capability
  • Network bandwidths and processing power at single
    sites set limits
  • Available network bandwidths for users could
    limit data transfer from/between multiple
    archives
  • All data at one site implies considerable
    processing power accessible by many distributed
    users

9
Distributed Archive Approach
  • Multiple copies of the data desirable
  • Needs a minimum of two geographically separated
    sources with the advantages
  • Greater resilience in ability to supply users
  • Load sharing between different providers
    (network and processing)
  • Avoids need for single site to provide excessive
    processing power

10
Single Archive Approach
  • Solar data normally stored in a raw form and need
    to be processed before use
  • Processing involves extraction and calibration of
    selected observations.
  • For data (e.g. helioseismology data) involving
    extended time intervals, processing data at
    source is desirable
  • Advantages that result
  • Reduced amount of information to be returned to
    user
  • Affords the instrument teams more control over
    the processing and quality of their data products
  • but
  • Heavy loading of processors at single archive
    site unless requests are for high-level
    lower-volume data products

11
Network Issues
  • UK has SuperJanet backbone currently at 10 Gbps
  • Local access points operate at 2.5 Gbps (e.g. UCL
    interconnect rate to backbone)
  • Europe has Geant backbone at 10 Gbps covering
    UK, France, Germany,Sweden, Switzerland with 2.5
    Gbps local interconnects
  • Transatlanic connection to Geant currently 2.5
    Gbps with upgrade to 10 Gbps planned for 2004
  • Discussion of Global 1 Tbps network by 2006??
  • Geant driven in part by needs of HEP community
    for LHC hence SDO may not have a problem in
    moving data between sites

12
PPARC E-Science AO
  • Proposals due by 31st May, 2003
  • Existence of first level Astrogrid infrastructure
    assumed
  • Proposals should
  • Be for the application of infrastructure and
    related techniques to real data sets
  • Underpin science but close connection between
    projects and the science programme is essential
  • Demonstrate an enabling role for eventual science
    exploitation
  • Ensure development of standards and deployment of
    Grid infrastructure
  • SDO bid is now anticipated by PPARC

13
HMI Data Analysis Pipeline
Net Access/ Mirror
Enabling Code/ Algorithms
Data Product
HMI SRR/SCR Presentation April 8-10
14
HMI Science Data Analysis Plan
Science Exploitation
HMI SRR/SCR Presentation April 8-10
15
HMI Data Volumes
Net Access
HMI SRR/SCR Presentation April 8-10
16
  • END OF TALK

17
What is Astrogrid?
Astrogrid is a 5 M data grid project that will
link data archives, resources, and disciplines
from UK space institutions into a virtual
observatory.
  • Data Archives
  • Mullard Space Science Laboratory
  • Rutherford Appleton Laboratory
  • University of Cambridge
  • University of Leicester
  • Royal Observatory Edinburgh
  • Queens University Belfast
  • Jodrell Bank Observatory

18
GRID/Virtual Observatory
  • Within a virtual observatory
  • Not required for all datasets to be stored at a
    single site
  • Metadata and registries allow system to handle a
    distributed archive.
  • Different organisations or countries could host
    the different datasets or different parts of the
    datasets (e.g. split by time).
  • Complete catalogues relating to particular
    datasets should be held wherever the data are
    held.
  • Distributed data holding reduces the pressure on
  • Network connection to an archive
  • Processing capabilities needed at the archive
    site
  • Most accessed data could be selectively copied to
    distributed archives e.g. EGSO, Astrogrid
  • Derived data products should be held at
    distributed sites
  • Material needed for more detailed searches should
    be described by metadata in appropriate
    registries.

19
Example Solar / Stellar Flares
Science Problem A solar physicist studying the
flare mechanism would like to gather data on both
solar and stellar flares. Data Required X-ray
datasets lightcurves, spectra, and redshift /
blueshift information from SOHO, Yokhoh, EXOSAT,
ROSAT, XMM, Chandra, etc. Current Issues No
stellar flare catalogue (at time of science
problem writing), datasets provided by several
different archives with no common interface.
NEW Stellar Flare Catalogue
Merged Solar Flare List
User
Web Interface
20
HMI Data Archive
21
HMI Data Flow
22
HMI Dataflow Concept
HMI SRR/SCR Presentation April 8-10
23
HMI Standard Data Products
24
UK Astrogrid Scientific Aims
  • Improve the quality, efficiency, ease, speed, and
    cost-effectiveness of on-line astronomical
    research
  • Make comparison and integration of data from
    diverse sources seamless and transparent
  • Remove data analysis barriers to
    interdisciplinary research
  • Make science involving manipulation of large
    datasets as easy and as powerful as possible.

25
UK Astrogrid Practical Goals
  • Develop, with our IVOA partners (including
    European Grid of Solar Observations/EGSO),
    internationally agreed standards for data,
    metadata, data exchange and provenance
  • Develop a software infrastructure for data
    services
  • Establish a physical grid of resources shared by
    AstroGrid and key data centres
  • Construct and maintain an AstroGrid Service and
    Resource Registry
  • Implement a working Virtual Observatory system
    based around key UK databases and of real
    scientific use to astronomers
  • Provide a user interface to that VO system
  • Provide, either by construction or by adaptation,
    a set of science user tools to work with that VO
    system
  • Establish a leading position for the UK in VO
    work
Write a Comment
User Comments (0)
About PowerShow.com