Astro-WISE paradigm for Citizen eScience

1
Astro-WISE paradigm for Citizen eScience

• Gijs Verdoes Kleijn
• Edwin Valentijn
• Marjolein Cuppen
• for the Astro-WISE consortium

2
Outline

• What is Astro-WISE?
• An information accumulator for eScience
• Massive datasets
• Data centric
• Federated
• Paradigm for Citizen eScience?
• Astronomers and public collaborating in one system

3
Story-line

• Goals
• Where it started data avalanches from Wide-field
  imaging surveys OmegaCAM/VST
• Needles in haystacks
• Haystacks
• Share federate distributed resources (hardware
  science teams)
• All-in-one information system publishing,
  archiving, calibration, post-calibration analysis
• Concepts the paradigm
• Data model in object model
• Extreme backwards-chaining
• All All I/O through database VirtualTelescopePlaa
  t met vereenvoudiging
• Implementation
• Python as glue
• Try yourself

4
ESO Public Imaging Surveys
KIDS500 nights 1500 sq.deg. u g r i
VIKING250nights z, Y, J, H, K
UKIDSS workshop 2007
5
Paranal Monthly Data Rates 2007 statistics

• All Current Paranal
• Instruments 4 (433.2 GB)

• OmegaCAM 24 (2500 GB)

• VIRCAM 72
• (7500 GB)

• Courtesy Mark Neeser

6
Science drivers

• Needles in haystacks
• high-z QSOs (6.4ltzlt7.2)
• Extreme Galactic White Dwarfs
• (Rare) AGNs
• The unanticipated
• Haystacks
• Dark matter distribution weak lensing
• Dark energy
• Evolution of galaxy clusters
• Unexpected
• Needles are easier than haystacks

7
Optimize scientific return Wide-field Imaging

• Facilitate (un)foreseen uses of calibrated and
  raw Tera/Petabyte data sets by global pools of
  collaborators and resources
• This requires enabling changes at any time in
• Physical parameters (e.g., instrument/atmosphere)
• Improved algorithms
• Improved insight in 1. or 2.
• Requires a single integrated information system

Integrated Information system
8
Astro-WISE (/eScience) boundary condition

• Pool via federation into a single system
• Distributed science teams
• Distributed storage, compute and database
  resources
• Peer-to-peer network

9
TARGET diagram
The philosophy that drives how we store
information

• Data model in object model
• OOP (python), inheritance, attribute persistency
• Full backwards-chaining of dependencies

10
Extreme data lineage
11
Astro-WISE information system data centric
architecture

• All data beyond pixel data is Metadata
• all pixel data ltgtdata servers
• all Metadata ltgt database
• all I/O to database
• Compute clusters / GRIDs
• Graphical User Interfaces (Web-services)
• All components scalable

12
Astro-WISE - Virtual Survey Telescope
Pipeline Methods/Algorithms In on-line
repository Python (wrapped)
CPUs

• DATABASE
• LINKS
• METADATA images
• CATALOGS
• Contains ALL input/output

• DATA SERVERS
• Images
• Calibration science
• Raw final

OmegaCEN review 2008
13
www.astro-wise.org
14
www.astro-wise.org
XLDB -Lyon Aug 2009
15

MegaCAM, LBC, ISAAC, LOFAR, SDSS DR7, 2MASS,
USNO
16
Web servicesTarget processor
17
Quality view
18
Benefits from integrated dynamic database
relevant for Citizen eScience
(
)

• Both final and raw products accessible
• GUI can be fully web services no local
  installation needed to use it
• can be installed everywhere with off-the-shelf
  hardware
• Objects are linked, all bits are traced
• on-the fly re-processing
• Annotation record building
• Builtin workflow
• Easy publishing via Virtual Observatory and
  webservices
• Own compute GRID operational
• A federated integrated-system
• Naturally serves global collaborative projects
• Scalable image server/database/compute GRID
  resources can grow via federation
• Mydb environment operational for individual
  research
• Quick scripting many tasks are 5 Line Scripts
• Fully user tunable own provided
  algorithms/scripts allowed
• Python existing software can be wrapped
  (implemented for Sextractor, swarp, being
  implemented for GALPHOT, GALFIT)

19
(Citizen) eScience

• Discovery of the exotic
• gravitational lenses, mergers,
• SNe,
• transients,
• Satellites
• asteroids
• Survey Quality Control
• -visual inspection KIDS 3yr manpower. -object
  masking-astrometry

20
(Citizen) eScience

• Open-up massive datasets to the public
• Playing -gt exploring -gt science
• A single environment for public and scientist
• Feeling at home in the Universe on spaceship Earth

21
(Citizen) eScience

• Open-up massive datasets to the public
• Continum playing -gt exploring -gt science
• A single environment for public and scientist
• Feeling at home in the Universe on spaceship Earth

www.astro-wise.org www.rug.nl/TarGet
22
(No Transcript)
23
www.rug.nl/TarGet
TASK24
2009-13 32M