Parallel Programming on Computational Grids - PowerPoint PPT Presentation

Loading...

PPT – Parallel Programming on Computational Grids PowerPoint presentation | free to download - id: 10483f-MDhiM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Parallel Programming on Computational Grids

Description:

Seamless integration of geographically distributed computers, ... Gridlab Application Toolkit (Java GAT) Ibis is a Java-centric grid programming system ... – PowerPoint PPT presentation

Number of Views:91
Avg rating:3.0/5.0
Slides: 51
Provided by: csVu
Learn more at: http://www.cs.vu.nl
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Parallel Programming on Computational Grids


1
Parallel Programming on Computational Grids
2
Outline
  • Grids
  • Application-level tools for grids
  • Parallel programming on grids
  • Case study Ibis

3
Grids
  • Seamless integration of geographically
    distributed computers,databases, instruments
  • The name is an analogy with power grids
  • Highly active research area
  • Global Grid Forum
  • Globus middleware
  • Many European projects
  • e.g. Gridlab Grid Application Toolkit and
    Testbed
  • VL-e (Virtual laboratory for e-Science) project
  • .

4
Why Grids?
  • New distributed applications that use data or
    instruments across multiple administrative
    domains and that need much CPU power
  • Computer-enhanced instruments
  • Collaborative engineering
  • Browsing of remote datasets
  • Use of remote software
  • Data-intensive computing
  • Very large-scale simulation
  • Large-scale parameter studies

5
Web, Grids and e-Science
  • Web is about exchanging information
  • Grid is about sharing resources
  • Computers, data bases, instruments
  • e-Science supports experimental science by
    providing avirtual laboratory on top of Grids
  • Support for visualization, workflows, data
    management,security, authentication,
    high-performance computing

6
The big picture
Application
Application
Application
Potential Generic part
Potential Generic part
Potential Generic part
Management of comm. computing
Virtual Laboratory Application oriented services
Management of comm. computing
Management of comm. computing
7
Applications
  • e-Science experiments generate much data, that
    often is distributed and that need much
    (parallel) processing
  • high-resolution imaging 1 GByte per
    measurement
  • Bio-informatics queries 500 GByte per
    database
  • Satellite world imagery 5 TByte/year
  • Current particle physics 1 PByte per year
  • LHC physics (2007) 10-30 PByte per year

8
Grid programming
  • The goal of a grid is to make resource sharing
    very easy (transparent)
  • Practice grid programming is very difficult
  • Finding resources, running applications, dealing
    with heterogeneity and security, etc.
  • Grid middleware (Globus Toolkit) makes this
    somewhat easier, but is still low-level and
    changes frequently
  • Need application-level tools

9
Application-level tools
  • Builds on grid software infrastructure
  • Isolates users from dynamics of the grid hardware
    infrastructure
  • Generic (broad classes of applications)
  • Easy-to-use

10
Taxonomy of existing application-level tools
  • Grid programming models
  • RPC
  • Task parallelism
  • Message passing
  • Java programming
  • Grid application execution environments
  • Parameter sweeps
  • Workflow
  • Portals

11
Remote Procedure Call (RPC)
  • GridRPC specialize RPC-style (client/server)
    programming for grids
  • Allows coarse-grain task parallelism remote
    access
  • Extended with resource discovery, scheduling,
    etc.
  • Example NetSolve
  • Solves numerical problems on a grid
  • Current development use web technology (WSDL,
    SOAP) for grid services
  • Web and grid technology are merging

12
Task parallelism
  • Many systems for task parallelism (master-worker,
    replicated workers) exist for the grid
  • Examples
  • MW (master-worker)
  • Satin divideconquer (hierarchical master-worker)

13
Message passing
  • Several MPI implementations exist for the grid
  • PACX MPI (Stutgart)
  • Runs on heterogeneous systems
  • MagPIe (Thilo Kielmann)
  • Optimizes MPIs collective communicationfor
    hierarchical wide-area systems
  • MPICH-G2
  • Similar to PACX and MagPIe, implemented on Globus

14
Java programming
  • Java uses bytecode and is very portable
  • Write once, run anywhere
  • Can be used to handle heterogeneity
  • Many systems now have Java interfaces
  • Globus (Globus Java Commodity Grid)
  • MPI (MPIJava, MPJ, )
  • Gridlab Application Toolkit (Java GAT)
  • Ibis is a Java-centric grid programming system

15
Parameter sweep applications
  • Computations what are mostly independent
  • E.g. run same simulation many times with
    different parameters
  • Can tolerate high network latencies, can easily
    be made fault-tolerant
  • Many systems use this type of trivial parallelism
    to harness idle desktops
  • APST, SETI_at_home, Entropia, XtremWeb

16
Workflow applications
  • Link and compose diverse software tools and data
    formats
  • Connect modules and data-filters
  • Results in coarse-grain, dataflow-like
    parallelism thatcan be run efficiently on a grid
  • Several workflow management systems exist
  • E.g. Virtual Lab Amsterdam (predecessor VL-e)

17
Portals
  • Graphical interfaces to the grid
  • Often application-specific
  • Also portals for resource brokering, file
    transfers, etc.

18
Outline
  • Grids
  • Application-level tools for grids
  • Parallel programming on grids
  • Case study Ibis

19
Distributed supercomputing
  • Parallel processing on geographically distributed
    computing systems (grids)
  • Examples
  • SETI_at_home ( ), RSA-155, Entropia, Cactus
  • Currently limited to trivially parallel
    applications
  • Questions
  • Can we generalize this to more HPC applications?
  • What high-level programming support is needed?

20
Grids versus supercomputers
  • Performance/scalability
  • Speedups on geographically distributed systems?
  • Heterogeneity
  • Different types of processors, operating systems,
    etc.
  • Different networks (Ethernet, Myrinet, WANs)
  • General grid issues
  • Resource management, co-allocation, firewalls,
    security, authorization, accounting, .

21
Approaches
  • Performance/scalability
  • Exploit hierarchical structure of grids(previous
    project Albatross)
  • Heterogeneity
  • Use Java JVM (Java Virtual Machine) technology
  • General grid issues
  • Studied in many grid projects VL-e, GridLab, GGF

22
Speedups on a grid?
  • Grids usually are hierarchical
  • Collections of clusters, supercomputers
  • Fast local links, slow wide-area links
  • Can optimize algorithms to exploit this hierarchy
  • Minimize wide-area communication
  • Successful for many applications
  • Did many experiments on a homogeneous wide-area
    test bed (DAS)

23
Example N-body simulation
  • Much wide-area communication
  • Each node needs info about remote bodies

CPU 1
CPU 1
CPU 2
CPU 2
Amsterdam
Delft
24
Trivial optimization
CPU 1
CPU 1
CPU 2
CPU 2
Amsterdam
Delft
25
Wide-area optimizations
  • Message combining on wide-area links
  • Latency hiding on wide-area links
  • Collective operations for wide-area systems
  • Broadcast, reduction, all-to-all exchange
  • Load balancing
  • Conclusions
  • Many applications can be optimized to run
    efficiently on a hierarchical wide-area system
  • Need better programming support

26
The Ibis system
  • High-level efficient programming support for
    distributed supercomputing on heterogeneous grids
  • Use Java-centric approach JVM technology
  • Inherently more portable than native compilation
  • Write once, run anywhere
  • Requires entire system to be written in pure Java
  • Optimized special-case solutions with native code
  • E.g. native communication libraries

27
Ibis programming support
  • Ibis provides
  • Remote Method Invocation (RMI)
  • Replicated objects (RepMI) -
    as in Orca
  • Group/collective communication (GMI) - as in MPI
  • Divide conquer (Satin) -
    as in Cilk
  • All integrated in a clean, object-oriented way
    into Java, using special marker interfaces
  • Invoking native library (e.g. MPI) would give
    upJavas run anywhere portability

28
Compiling/optimizing programs
JVM
source
bytecode
Javacompiler
bytecoderewriter
bytecode
source
JVM
JVM
  • Optimizations are done by bytecode rewriting
  • E.g. compiler-generated serialization (as in
    Manta)

29
Satin a parallel divide-and-conquer system on
top of Ibis
  • Divide-and-conquer is inherently hierarchical
  • More general than master/worker
  • Satin Cilk-like primitives (spawn/sync) in Java
  • New load balancing algorithm (CRS)
  • Cluster-aware random work stealing

30
Example
interface FibInter public int fib(long
n) class Fib implements FibInter int fib
(int n) if (n lt 2) return n return fib(n-1)
fib(n-2)
Single-threaded Java
31
Example
interface FibInter extends ibis.satin.Spawnable
public int fib(long n) class Fib
extends ibis.satin.SatinObject implements
FibInter public int fib (int n) if (n lt
2) return n int x fib (n - 1) int y fib
(n - 2) sync() return x y
Java divideconquer
32
Ibis implementation
  • Want to exploit Javas run everywhere property,
    but
  • That requires 100 pure Java implementation,no
    single line of native code
  • Hard to use native communication (e.g. Myrinet)
    or native compiler/runtime system
  • Ibis approach
  • Reasonably efficient pure Java solution (for any
    JVM)
  • Optimized solutions with native code for special
    cases

33
Ibis design
34
Status and current research
  • Programming models
  • ProActive (mobility)
  • Satin (divideconquer)
  • Applications
  • Jem3D (ProActive)
  • Barnes-Hut, satisfiability solver (Satin)
  • Spectrometry, sequence alignment (RMI)
  • Communication
  • WAN-communication, performance-awareness

35
Challenges
  • How to make the system flexible enough
  • Run seamlessly on different hardware / protocols
  • Make the pure-Java solution efficient enough
  • Need fast local communication evenfor grid
    applications
  • Special-case optimizations

36
Fast communication in pure Java
  • Manta system ACM TOPLAS Nov. 2001
  • RMI at RPC speed, but using native compiler RTS
  • Ibis does similar optimizations, but in pure Java
  • Compiler-generated serialization at bytecode
    level
  • 5-9x faster than using runtime type inspection
  • Reduce copying overhead
  • Zero-copy native implementation for primitive
    arrays
  • Pure-Java requires type-conversion (copy) to
    bytes

37
Java/Ibis vs. C/MPI on Pentium-3 cluster (using
SOR)
38
Grid experiences with Ibis
  • Using Satin divide-and-conquer system
  • Implemented with Ibis in pure Java, using TCP/IP
  • Application measurements on
  • DAS-2 (homogeneous)
  • Testbed from EC GridLab project (heterogeneous)

39
Distributed ASCI Supercomputer (DAS) 2
40
Satin on wide-area DAS-2
41
Satin on GridLab
  • Heterogeneous European grid testbed
  • Implemented Satin/Ibis on GridLab, using TCP
  • Source van Nieuwpoort et al., AGRIDM03
    (Workshop on Adaptive Grid Middleware, New
    Orleans, Sept. 2003)

42
GridLab
  • Latencies
  • 9-200 ms (daytime),9-66 ms (night)
  • Bandwidths
  • 9-4000 KB/s

43
Configuration
44
Experiences
  • No support for co-allocation yet (done manually)
  • Firewall problems everywhere
  • Currently use a range of site-specific open
    ports
  • Can be solved with TCP splicing
  • Java indeed runs anywhere
  • modulo bugs in (old) JVMs
  • Need clever load balancing mechanism (CRS)

45
Cluster-aware Random Stealing
  • Use Cilks Random Stealing (RS) inside cluster
  • When idle
  • Send asynchronous wide-area steal message
  • Do random steals locally, and execute stolen jobs
  • Only 1 wide-area steal attempt in progress at a
    time
  • Prefetching adapts
  • More idle nodes ? more prefetching
  • Source van Nieuwpoort et al., ACM PPoPP01

46
Performance on GridLab
  • Problem how to define efficiency on a grid?
  • Our approach
  • Benchmark each CPU with Raytracer on small input
  • Normalize CPU speeds (relative to a DAS-2 node)
  • Our case 40 CPUs, equivalent to 24.7 DAS-2 nodes
  • Define
  • T_perfect sequential time / 24.7
  • efficiency T_perfect / actual runtime
  • Also compare against single 25-node DAS-2 cluster

47
Results for Raytracer
RS Random Stealing, CRS Cluster-aware RS
48
Some statistics
  • Variations in execution times
  • RS _at_ day 0.5 - 1 hour
  • CRS _at_ day less than 20 secs variation
  • Internet communication (total)
  • RS 11,000 (night) - 150,000 (day) messages
    137 (night) - 154 (day) MB
  • CRS 10,000 - 11,000 messages 82 - 86 MB

49
Summary
  • Parallel computing on Grids (distributed
    supercomputing) is a challenging and promising
    research area
  • Many grid programming environmenents exist
  • Ibis a Java-centric Grid programming environment
  • Portable (run anywhere) and efficient
  • Future work Virtual Laboratory for e-Science
    (VL-e)
  • Government-funded (20 M.Euro) Dutch project

50
VL-e program
About PowerShow.com