Is 99% Utilization of a Supercomputer a Good Thing? Scheduling in Context: User Utility Functions

1
Is 99 Utilization of a Supercomputer a Good
Thing?Scheduling in Context User Utility
Functions

• Cynthia Bailey Lee
• Department of Computer Science and Engineering
  and
  San Diego Supercomputer Center
• University of California, San Diego
• November 13, 2006

2
What is a Utility Function?
Outline What is a Utility Function?
Metrics Metric Pitfalls
Other factors coordinate with other grid sites
or sensors, paper deadlines, weather and
hurricane prediction,
3
Some Proposed Models
What is a Utility Function? Proposed
Models Real Functions Scheduling
My favorite
4
Real Functions
What is a Utility Function? Proposed
Models Real Functions Scheduling

• Randomly selected SDSC users provided these
  functions for jobs they were submitting
• Utility is in terms of the SDSC charge unit
  (SU)

5
More Real Functions
What is a Utility Function? Proposed
Models Real Functions Scheduling
6
Parallel Job Scheduling Explicitly by Utility
Function
What is a Utility Function? Proposed
Models Real Functions Scheduling
?
Finding the best solution is NP-hard

• Tennis Court Scheduling (Human-powered
  scheduling)
• Still practiced occasionally at most centers
  (officially and not) -- a phone call to sysadmins
  gets a job a reservation or to the front of the
  queue)
• Custom Heuristics
• Sort by current value, a combination of start
  value and slope Chun and Culler 02, Irwin,
  Grit, Chase 04
• Linear Solver / CPLEX

7
Comparing Schedulers
What is a Utility Function? Proposed
Models Real Functions Scheduling
Expansion Factor 25/50/75 ile Aggregate Utility rounded, x107 Time to Solution
EASY 0/0/3307 4.33 0.7 s
Conservative Backfilling 0/82/9071 4.24 1.9 s
Priority Queues EASY 0/2883/28124 4.20 0.8 s
Genetic Algorithm 0/289/6582 110. 886 s
Expansion Factor 25/50/75 ile Aggregate Utility rounded, x107
EASY 0/0/3307 4.33
Conservative Backfilling 0/82/9071 4.24
Priority Queues EASY 0/2883/28124 4.20
For 500 scheduling rounds (need to do a round
when a job arrives or ends). All timings done
with a (not particularly optimized) modular
scheduling simulator framework in Python (email
me for code).
8
Questions

• For more information, see paper
• Gathering real users utility functions
• Lee, Cynthia Bailey and Allan Snavely. "On the
  User-Scheduler Dialogue Studies of User-Provided
  Runtime Estimates and Utility Functions."
  International Journal of High Performance
  Computing Applications, vol. 20, pp.495-506,
  2006.
• Contact Cynthia Lee CL_at_SDSC.EDU