New Development in the AppLeS Project or User-Level Middleware for the Grid

1
New Development in the AppLeS ProjectorUser-Lev
el Middleware for the Grid

• Francine Berman
• University of California, San Diego

2
The Evolving Grid
In the beginning, there were applications and
resources, and it took ninja programmers andmany
months to implement the applications on the Grid
Applications
Resources
3
The Evolving Grid
And behold, there were services, and programmers
saw that it was good (even though their
performancewas still often less than desirable)
Applications
Grid Middleware
Applications
Resources
Resources
4
The Evolving Grid
and it came to pass that user-level middleware
was promised to promote the performance of Grid
applications, and the users rejoiced
Applications
Applications
User-Level Middleware
Grid Middleware
Applications
Grid Middleware
Resources
Resources
Resources
5
The Middleware Promise

• Grid Middleware
• Provides infrastructure/services to enable
  usability of the Grid
• Promotes portability and retargetability
• User-level Middleware
• Hides the complexity of the Grid for the end-user
• Adapts to dynamic resource performance variations
• Promotes application performance

6
How Do Applications Achieve Performance Now?

• AppLeS Application-Level Scheduler
• Joint project with R. Wolski
• AppLeS application self-scheduling Grid
  application
• AppLeS-enabled applications adapt to dynamic
  performance variations in Grid Resources

7
AppLeS Architecture
AppLeS-enabledapplications
Grid Middleware
Resources
8
From AppLeS-enabled applications to User-Level
Middleware
9
AppLeS User-Level Middleware

• Focus is development of templates which
• target structurally similar classes of
  applications
• can be instantiated in a user-friendly timeframe
• provide good application performance

AppLeS Template Architecture
Application Module
Scheduling Module
Deployment Module
Grid Middleware and Resources
10
APST AppLeS Parameter Sweep Template

• Parameter Sweeps class of applications which
  are structured as multiple instances of an
  experiment with distinct parameter sets
• Joint work with Henri Casanova
• First AppLeS Middleware package to be distributed
  to users
• Parameter Sweeps are common application structure
  used in various fields of science and engineering
• Most notably Simulations (Monte Carlo, etc.)
• Large number of tasks, no task precedences in the
  general case ? easy scheduling ?
• I/O constraints
• Need for meaningful partial results
• multiple stages of post-processing

11
APST Scheduling Issues

• Large shared files, if any, must be stored
  strategically
• Post-processing must minimize file transfers
• Adaptive scheduling necessary to account for
  changing environment

12
Scheduling Approach

• Contingency Scheduling Allocation developed by
  dynamically generating a Gantt chart for
  scheduling unassigned tasks between scheduling
  events
• Basic skeleton
• Compute the next scheduling event
• Create a Gantt Chart G
• For each computation and file transfer currently
  underway, compute an estimate of its completion
  time and fill in the corresponding slots in G
• Select a subset T of the tasks that have not
  started execution
• Until each host has been assigned enough work,
  heuristically assign tasks to hosts, filling in
  slots in G
• Implement schedule

Network links
Hosts(Cluster 1)
Hosts(Cluster 2)
Resources
1 2 1 2
1 2
Scheduling event
Time
Scheduling event
G
13
Scheduling Heuristics
Scheduling Algorithms for PS Applications

• Self-scheduling Algorithms
• workqueue
• workqueue w/ work stealing
• workqueue w/ work duplication
• ...

• Gantt chart heuristics
• MinMin, MaxMin
• Sufferage, XSufferage
• ...

Easy to implement and quick No need for
performance predictions Insensitive to data
placement
More difficult to implement Needs performance
predictions Sensitive to data placement

• Simulation results (HCW 00 paper) show that
• heuristics are worth it
• Xsufferage is good heuristic even when
  predictions are bad
• complex environments require better planning
  (Gantt chart)

14
APST Architecture
Command-line client
APST Client
Controller
interacts
triggers
Scheduler
APST Daemon
Actuator
Metadata Bookkeeper
store
Grid Resourcesand Middleware
15
APST

• APST being used for
• INS2D (NASA Fluid Dynamics application)
• MCell (Salk, Molecular modeling for Biology)
• Tphot (SDSC, Proton Transport application)
• NeuralObjects (NSI, Neural network simulations)
• CS simulation Applications for our own research
  (Model validation, long-range forecasting
  validation)
• Actuators APIs are interchangeable and mixable
• (NetSolveIBP) (GRAMGASS) (GRAMNFS)
• Scheduler API allows for dynamic adaptation
• No Grid software is required
• However lack of it (NWS, GASS, IBP) may lead to
  poorer performance
• More details in SC00 paper

16
APST Validation Experiments
Tokyo Institute of Technology
NetSolve IBP
APST Daemon APST Client
NetSolve NFS
17
APST Test Application MCell

• MCell General simulator for cellular
  microphysiology
• Uses Monte Carlo diffusion and chemical reaction
  algorithm in 3D to simulate complex
  biochemical interactions of molecules
• Focus of new multi-disciplinary ITR project
• Will focus on large-scale execution-time
  computational steering , data analysis and
  visualization

18
Experimental Results

• Experimental Setting
• Mcell simulation with 1,200 tasks
• composed of 6 Monte-Carlo simulations
• input files 1, 1, 20, 20, 100, and 100 MB
• 4 scenarios
• Initially
• (a) all input files are only in Japan
• (b) 100MB files replicated in California
• (c) in addition, one 100MB file
• replicated in Tennessee
• (d) all input files replicated everywhere

19
New Directions Mega-programming

• Grid programs
• Can reasonably obtain some information about
  environment (NWS predictions, MDS, HBM, )
• Can assume that login, authentication,
  monitoring, etc. available on target execution
  machines
• Can assume that programs run to completion on
  execution platform

• Mega-programs
• Cannot assume any information about target
  environment
• Must be structured to treat target device as
  unfriendly host (cannot assume ambient services)
• Must be structured for throwaway end devices
• Must be structured to run continuously

20
Success with Mega-programming

• Seti_at_home
• Over 2 million users
• Sustains teraflop computing
• Can we run non-embarrassingly parallel codes
  successfully at this scale?
• Computational Biology, Genomics
• Genome_at_home

21
Genome_at_home

• Joint work with Derrick Kondo
• Application template for peer-to-peer platforms
• First algorithm (Needleman-Wunsch Global
  Alignment) uses dynamic programming
• Plan is to use template with additional genomics
  applications
• Being developed for web rather than Grid
  environment

G T A A G
A 0 0 1 1 0
T 0 1 0 1 1
A 0 0 2 2 1
C 0 0 1 2 2
C 0 0 1 2 2
G 1 0 1 2 3
Optimal alignments determined by traceback
22
Mega-programs

• Provide the algorithmic counterpart for very
  large scale platforms
• peer-to-peer platforms, Entropia, etc.
• Condor flocks
• Large free agent environments
• Globus
• New platforms networks of low-level devices,
  etc.
• Different computing paradigm than MPP, Grid

Genome_at_home

DNAAlignment
Legion
Entropia
Condor

Globus
free agents
23

• Coming soon to a computer near you
• Release of APST v0.1by SC00
• Release of AMWAT (AppLeS Master/ Worker
  Application Template) v0.1 by Jan 01
• First prototype of genome_at_home 2001
• AppLeS software and papers http//apples.ucsd.edu

• Thanks!
• NSF, NPACI, NASA
• Grid Computing Lab
• Fran Berman (berman_at_cs.ucsd.edu)
• Henri Casanova
• Walfredo Cirne
• Holly Dail
• Marcio Faerman
• Jim Hayes
• Derrick Kondo
• Graziano Obertelli
• Gary Shao
• Otto Sievert
• Shava Smallen
• Alan Su
• Renata Teixeira
• Nadya Williams
• Eric Wing
• Qiao Xin

24
Scheduling Results

• 1 Heuristics for Scheduling Parameter Sweep
  Applications in Grid Environments
• H. Casanova, A. Legrand, D. Dzagorodnov, F.
  Berman (HCW00)

Scheduling Algorithms for PS Applications ?
Easy to implement and quick No need for
performance predictions Extremely adaptive No
planning (resource selection, I/O, )

• Simulation results in 1 show that
• heuristics are worth it
• Xsufferage is good heuristic even when
  predictions are bad
• complex environments require better planning
  (Gantt chart)

25
APST Architecture
Command-line client
APST Client
Controller
interacts
triggers
store
NWS
26
Scheduling Results

• Heuristics for Scheduling Parameter Sweep
  Applications in Grid Environments
• H. Casanova, A. Legrand, D. Dzagorodnov, F.
  Berman (HCW00)
• Simulation results show that
• Heuristics are worth it
• Xsufferage is good heuristic even when
  predictions are bad
• Complex environments require better planning

• Data transfer lt 40 X task computation time

Scheduling event every 250 seconds