eSDO Algorithms, Visualization and Data Access

1
eSDO Algorithms, Visualization and Data Access

• Elizabeth Auden
• 21 September 2006
• AHM 2006
• Nottingham

2
What is eSDO?

• eSDO is an e-science project funded by PPARC to
  make SDO data and algorithms available to the UK
  solar community using the virtual observatory.
• Algorithms
• Code feature recognition, local helioseismology,
  global helioseismology
• Grid integration AstroGrid and JSOC
• Date centres
• AstroGrid interface with US JSOC data centre
• UK data centre
• Visualization tools
• Catalogues
• SDO Streaming tool
• On-the-fly image galleries and movies

3
eSDO Schedule

• RESEARCH - Phase A 1 October 2004 30 September
  2005
• IMPLEMENTATION - Phase B 1 October 2005 30
  September 2007
• Overview of Phase A activities Phase A Summary
  Report
• available on eSDO project wiki
  http//www.mssl.ucl.ac.uk/twiki/bin/pdf/SDO/PhaseA
  SummaryReport
• Detailed report including 6 deliverable
  documents Phase A Report
• available on eSDO project wiki
  http//www.mssl.ucl.ac.uk/twiki/bin/pdf/SDO/PhaseR
  eport

4
eSDO Project Team

• MSSL
• Elizabeth Auden, project manager
• Len Culhane, PI
• Lidia van Driel-Gesztelyi
• Mike Smith
• RAL
• Richard Harrison
• Andrzej Fludra
• Vito Graffagnino
• Sheffield
• Mike Thompson
• Sergei Zharkov
• Birmingham
• Yvonne Elsworth
• Bill Chaplin
• Thierry Toutain

US Links JSOC, Joint Operations Centre Stanford
University Lockheed Martin
5
AlgorithmsCode

• Feature Recognition / Event Analysis
• Coronal loop detection, MSSL
• Non-linear magnetic field extrapolation, MSSL
• Helicity computation, MSSL
• Small event detection, RAL
• CME dimming region recognition, RAL
• Local Helioseismology
• Perturbation map generation, Sheffield
• Subsurface flow analysis, Sheffield
• Local helioseismology inversions, Sheffield
• Global Helioseismology
• Mode parameters analysis, Birmingham
• Mode asymmetry analysis, Birmingham

6
AlgorithmsGrid Integration

• JSOC
• SDO pipeline at Stanford University / Lockheed
  Martin
• eSDO algorithms deployed as pipeline modules
  (automated or user-run)
• Data will be obtained locally from JSOC archive
• AstroGrid CEA
• UK hosted algorithms hosted as web services
• Accessible through AstroGrid workflows
• Data transferred from MySpace, remote data centre
  or users local storage
• SolarSoft
• Algorithm C modules wrapped in IDL
• Distribution through MSSL SolarSoft gateway
• Data transferred to algorithm from users local
  storage

7
AlgorithmsAstroGrid Deployment

• 9 algorithms have been deployed as CEA
  applications

8
Data CentresUS Data Centre

• Primary US Data Store
• Files stored in SUMS (Storage Unit Management
  System) at Stanford University with metadata in
  distributed DRMS (Data Resource Management
  System)
• DSA / DRMS integration for grid data access
  investigations
• VSO (Virtual Solar Observatory) primary US access
• AstroGrid / VSO Interface
• VSO queries in AstroGrid workflows through STAP
  (Simple Time Access Protocol) web service
• US / UK Network Latency Tests
• Test expediency of file transfer mechanisms
• Test between Stanford, MSSL, UCL, RAL

9
Data CentresPossible UK Data Centre

• Storage
• 30 TB disc cache
• 60 day rolling cache HMI/AIA
• True cache for HMI / AIA
• DRMS / SUMS
• Transfer
• Export formats from JSOC
• Data search DSA
• Data transfer - CEA
• Test installation MSSL

Investigation of DRMS / SUMS
10
Data CentresPossible European Data Centre

• HELAS European Helio and AstroSeismology Network
• EU Framework 6 support for European
  helioseismology data and algorithms
• Possible full mirror of SDO data in Germany
• Discussion of possible collaboration between eSDO
  and HELAS in late September 2006 at HELAS
  workshop in Nice
• Use UK-held DSA / DRMS interface for AstroGrid
  access to SDO data in Germany instead of UK?

11
Data Centres eSDO / Solar-B Prototype system
perspective
UKSSDCADS file system
Note AstroGrid components (JES, DSA, MySpace,
Workbenchl) may be located anywhere.
12
Data Centres User access through AstroGrid
AstroGrid workbench STAP service access through
HelioScope, DSA access through the Task Launcher.
Task Launcher locate DataSet Access components,
submit ADQL queries
STAP service will point to URLs in UK, US or
Germany. DSA will integrated with DRMS (Data
Resource Management System) in UK or Germany.
HelioScope solar data results from Simple Time
Access Protocol services.
13
Visualization ToolsCatalogues

• Small Event / CME Dimming catalogue
• Generated from continuously running of small
  event detection / CME dimming region recognition
  algorithms
• Tables of event / CME statistical information
• Helioseismology Mode Parameters catalogue
• GONG month of data generated from mode parameters
  analysis algorithm
• Generated in US
• Thumbnail catalogue
• Basic metadata for generated thumbnail images

Catalogues can be accessed through ADQL query to
DSA.
14
Visualization ToolsSDO streaming tool

• User Facilities
• Pan in space P
• Zoom in space P
• Pan in time
• Zoom in time
• Data Products
• LoS Magnetogram
• HMI continuum map
• AIA images
• Future Work
• FFT / wavelet analysis for coronal wave
  investigations

15
Visualization ToolsOn-the-fly Image Galleries
and Movies

• Thumbnail Extraction
• Extract thumbnails in US from HMI and AIA science
  products
• Image Gallery Generation
• User specifies start time, stop time, cadence,
  data product
• Image gallery generated on the fly in web browser
• Movie Generation
• User specifies start time, stop time, cadence,
  data product
• MPEG movie generated on the fly

Find data with AstroGrid HelioScope. Use PLASTIC
to load data into visualization tools.
16
Future Work

• Phase B implementation development now in
  progress
• Summer / Autumn 2006
• Unit and science testing for algorithms
• Add STAP service client to visualization tools
• Add movie functionality to streaming tool
• Investigate DRMS integration with DSA
• 2007
• completed algorithms
• algorithm integration with AstroGrid, JSOC and
  SolarSoft
• Coronal wave analysis in SDO streaming tool
• Work with HELAS to explore European SDO data
  centre
• Catalogues