Title: APAC Initiatives for Large-Scale Data Sets and Grid Computing
1APAC InitiativesforLarge-Scale Data Sets and
Grid Computing
- Robin Stanton
- Bernard Pailthorpe
- Australian Partnership for Advanced Computing
Presentation to NeSC 28 May 2003
2Topics
- Infrastructure for eResearch
- Australian Partnership for Advanced Computing
- APAC
- GrangeNet
- Grid projects at ANU supported by APAC
- APAC Initiatives
3Changing How Science is Done
- Collect data from digital libraries, laboratories
and observation - Analyze data with models run on the Grid
- Visualize and share data over the Web
- Publish results in a Digital Library
From Sid Karin, SDSC/NPACI
4Grid Services for eResearch
User Communities
Bio-informatics
Astronomy
- - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - -
Physics
Environment
Collaborative Visualisation
Distributed Computing
Information Access
Cooperative Environments
On-Line Instruments
Web Services Advanced Communications Services
5APAC Achievements
- The APAC partnership formed June 2000
- A partner in each State as well as ANU and CSIRO
- The APAC National Facility operational April
2001. - APAC and partner facilities serviced over 1,100
users. - Over 110 projects supporting users and developing
expertise in 13 computational science and
engineering themes. - Over 50 university courses prepared and delivered
in computational science and engineering.
6APAC National Facility
- Computing Systems
- HP AlphaServer SC ES45 (127 nodes)
- ranked number 63 in latest TOP500 list
- Dell Linux cluster, HP Marvel
- Mass Storage
- Storagetek (robotic silo) tape library
- Capable of a petabyte (1015 bytes) of storage
- Visualisation
- visualisation virtual reality systems
- Staff
- staff at the Australian National University (ANU)
http//nf.apac.edu.au
7GrangeNetA GRid And Next GEneration
Networkwww.grangenet.net
Supported by the Federal Governments BITS
Advanced Networks Program
8APAC Partners and Backbone Networks
Darwin
Brisbane QPSF
USA
Canberra ANU
Perth IVEC CSIRO
Sydney ac3
Adelaide SAPAC
APAC National Facility
Melbourne VPAC CSIRO
GrangeNet Backbone AARNet Links
Hobart TPAC CSIRO
9Gravitational Wave Astronomy
- GWA involves exchange and simultaneous data
processing between multiple detectors - Gravity wave detectors environmental monitoring
- ACIGA Australia
- LIGO USA VIRGO,GEO Europe TAMA Japan
- Technical collaborations with GriPhyN and iVDGL
- Operational data-pipeline centred on APAC MDSS
- Upgrading to Lightweight Data Replicator (LDR)
Australian Consortium for Interferometric
Gravitational Astronomy
10ACIGA Data Grid
Australian Consortium for Interferometric
Gravitational Astronomy
Rsync/LDR
GridFTP
ACIGA APAC resources
Environmental Monitors
11High-Energy Particle Physics
- Belle Physics Collaboration
- K.E.K. B-factory detector, Tsukuba, Japan
- Matter/Anti-matter investigations, Atlas
test-run - 45 Institutions, 400 users worldwide
- 10 TB data currently
- Universities of Sydney and Melbourne active
participants - Australian collaborators leading Grid adoption
- Australian Data-grid centred on APAC MDSS
- Exploiting Globus 2.x, Gfarm
- Atlas Experiment
- Large Hadron Collider (LHC) at CERN
- Collaboration 2000 people, 150 institutes
internationally, 34 countries - 3.5 PB data per year
- operational in 2007
12Virtual Observatories
- MACHO Project Data
- Largest online astrophysical data set in
Australia - 10TB Data collected over 10 years
- Hosted on APAC MDSS
- Web interface at wwwmacho.anu.edu.au
- Currently using Z39.50 metadata standard
- Mapping metadata to VOTable 1.0 standard
- Emerging IVO metadata standard
- International Virtual Observatory
- MACHO data being incorporated into SDSC SRB
system - www.ivoa.net
- Australian Virtual Observatory
- IAU demo of Data Grid and Visualisation testbed
- Distributed data-sets, Tomcat rendering software
- www.atnf.csiro.au/projects/avo/
13Bioinformatics
- Many initiatives to support bio-community
- Bio-mirror supported by AARNet and ANU
- www.bio-mirror.net
- Bio-database search by VPAC and Ausbiotech
- www.ausbioinfo.com
- Australian National Genomic Information Service
(ANGIS) - www.angis.org
- ARC Centre for Bioinformatics (M Ragan)
(www.imb.uq.edu.au) - Plan to coordinate infrastructure
- Replicate access mechanisms to data sets
- Provide common Web interfaces to applications
- Provide specialised systems (Gaussian, Blast..)
14Earth Observation
- GADS Grid Access Data Service
- for oceanographic and climate data
- World Ocean Circulation Experiment (WOCE)
- project funded by APAC
- TPAC (Uni of Tasmania..), Bureau of Meteorology,
University of Reading - Grid access via Web services
- Interface to DODS/OPeNDAP
- Used by Earth Systems Grid and NERC DataGrid
15Cultural Language Archives
- PARADISEC
- Pacific and Regional Archive for Digital Sources
in Endangered Culture - Uni of Sydney, Uni of Melbourne, ANU
- Digitised oral and music recordings from
Asia-Pacific region - Integrate with sociological data sets
- International archival standard for digital audio
- 24bit 96KHz Stereo metadata
- APAC MDSS to host 10,000 hours
- 2GB/Hr gt 20TB total
16APAC Initiatives
- Provide more support for data-intensive
computing - APAC support for large-scale data sets
- ask research organisations for proposals to have
large-scale data sets managed by APAC and its
partners - concentrate on national and international data
access - develop plans for managing these data sets
- Install and operate an APAC Grid
- Consider a pilot project for eResearch
- support a research community through data
management