Heuristics for Scheduling Parameter Sweep Application in Grid environments

1
Heuristics for Scheduling Parameter Sweep
Application in Grid environments

• M1
• Kenji KANEDA (SIG-PDS)

2
Outline of Presentation

• Background Overview of AppLeS
• (Task, Network) Model
• Scheduling
• Simulation Experiments
• Related Work
• Summary

3
Background (?????)

• ?????????????????????????????????
• ?????????????????
• ????Firewall?Private Network??????????????????????
  ?
• ??????????????????????????????????????

4
Background

• ??????????????????????
• Software DSM
• MPI
• parameter sweep application
• embarrassing parallel
• ??????????????????????

5
AppLeS (Application Level Scheduling)

• parameter sweep application?????????
• ??
• ???parameter spaces
• ????????
• ????????????????
• ????????????

6
AppLeS?????????
?? Server
Information Service
output
check
input
input
User
7
Outline of Presentation

• Background Overview of AppLeS
• (Task, Network) Model
• Scheduling
• Simulation Experiments
• Related Work
• Summary Future Work

8
Network Model (1/2)

• network topology
• k???????????????????
• ????
• ????????????????????
• ??????????????
• ??????????????????????????
• ???????????????????(????????)

9
Network Model (2/2)
10
Task Model

• ?????n?????
• ???????????
• ??????????????????
• ??????????????????????????????????
• ?????????
• ????????????????????????
• ????????????
• MCell (micro-physiology application)
• 3-D Monte-Calro simulation???
• ?????????MBytes

11
Outline of Presentation

• Background Overview of AppLeS
• (Task, Network) Model
• Scheduling
• Simulation Experiments
• Related Work
• Summary

12
Scheduling

• ??
• makespan??????????????????
• makespan ???????????????????????????????????????
  ???????????
• adaptive?????????
• ?????????????
• ??
• ????schedule()????????

13
schedule()

1. ??schedule()???????????
2. ???????????????????????
3. ???????????????T???
4. T??????Heuristic??????

Gantt Chart??????????????????????????
14
??schedule()????????

• ??schedule()???????????
• ????????????????????
• ??????????????????
• ?????????
• ???
• ?????
• ????????????????????????????????????

15
????????? (1/2)

• ??
• ?????????????????
• Gantt Chart??????????

16
????????? (2/2)

• ?????????
• NWS???????????????????Heuristic???
• ?)???????????????
• ??????????????????
• ??????????????????????CPU load
• ????????????????????CPU load
• ??????????????

17
???????????????

• schedule()???????????
• ???schedule()???????????????
• ?????????????????????????????
• ????
• random
• maximizing data re-use
• ???????????????????????????

18
????????????

• ????????Heuristic??????????
• Gantt Chart??????????????

Networks Links
Cluster 1
Cluster 2
Host 1.1
Host 1.2
Host 2.1
Host 2.2
Host 2.3
Link 1
Link 2
Link 1
input
output
19
Heuristics Algorithm (1/5) ?????

• Hjkj th??????k th???
• C(Ti,Hjk)???Ti????Hjk??????

??f(x)?????????x??
argmax???
20
Heuristics Algorithm (2/5) Min-min
Ti?????????????Hjk???
T??????????????Ts???
21
Heuristics Algorithm (3/5) Max-min
Ti?????????????Hjk???
T????????????????Ts???
22
Heuristics Algorithm (4/5) Sufferage
2????????????????????????Ti???
??????????????????????????
23
Heuristics Algorithm (5/5) XSufferage
???????????2???????????????????????Ti???
???????????????????????????
24
Simulation Environments

• task
• 1600 tasks (?????)
• input unshared 10K file shared file
• output 10K file
• network topology
• 5 clusters (6, 6, 8, 20, 20)
• network bandwidth 6 600Kbytes/sec
• interval of scheduling event 500sec

25
Simulation Result (1/2)
26
Simulation Result (2/2)
QoI performance estimation accuracy
27
Outline of Presentation

• Background Overview of AppLeS
• (Task, Network) Model
• Scheduling
• Simulation Experiments
• Related Work
• Summary

28
Related Work Condor(1/2)

• ??
• _at_University of Wisconsin-Madison
• High Throughput Computing????
• ???????
• ??
• ClassAd
• ???????????????????????
• OS, Memory, etc.
• ???????????????????????????????

29
Related Work Condor (2/2)

• ??
• Remote System Call
• remote host??local host???????????????
• open()??????????????
• checkpoint
• ??????????????????????
• fault tolerant
• preemption
• stack, process???????????????????
• Fork(), kernel thread????

30
Related Work Nimrod

• ??
• _at_Monash University
• Parameter sweep application?????
• ??
• Deadline scheduling
• ????????????????????????????????????????????????
• GUI interface ???

31
Related Work Bricks

• ??
• _at_Tokyo titech University
• ????????????????????????
• ??
• Deadline Schedule?????????
• Load Correction??????
• Fallback??????

32
Related Work Scheduling on Heterogeneous
Environments

• ???????????
• regular graph
• irregular graph
• ??????? (???)
• ????????
• ??????????
• ???????DAG
• Task migration
• ???????????
• Throughput
• round trip time

33
Outline of Presentation

• Background Overview of AppLeS
• (Task, Network) Model
• Scheduling
• Simulation Experiments
• Related Work
• Summary

34
Summary

• AppLeS
• parameter sweep application??????????
• ?????Heuristics Algorithm???
• ??????????????
• XSufferage??????????????
• ????????????????????
• QoI??????????

35
References

• Heuristics for Scheduling Parameter Sweep
  applications in Grid environments
• Henri Casanova, Arnaud Legrand, Dmitrii
  Zagorodnov and Francine Berman
• (HCW'2000)
• Using Simulation to Evaluate Scheduling
  Heuristics for a Class of Applications in Grid
  Environments
• Francine Berman, Henri Casanova, Dmitrii
  Zagorodnov and Arnaud legrand
• Research Report