Influence of heavy-tailed distributions on load balancing

1
Dimensionality Reduction for Analyzing Cycle
Stealing, Priority Queueing, Threshold
Policies, and more
Mor Harchol-Balter
Computer Science Dept. Carnegie Mellon University
2
Many examples multiserver sharing
Common Problem 2D-infinite Markov chain
3
Dimensionality Reduction for Markov Chains

• Approximate technique
• Problem-dependent.
• Doesnt always apply.

• Provides first accurate performance
• numbers for many common problems.
• Very fast (less than 1 sec).
• lt 1 error typically, and can improve.
• Models any job sizes, and any loads.
• Adding thresholds and switching costs
• is often easy.

Taka Osogami Adam Wierman Alan
Scheller-Wolf Mark Squillante Li Zhang
Sigmetrics 03 SPAA 03 ICDCS 03 Tools 03a,
03b Allerton 04 Perf. Eval 05 QUESTA
05 Perf. Eval 06a Perf. Eval 06b Mgmt
Science 07
4
Cycle Stealing Problem
Sigmetrics 03, Perf.Eval. 05
lB jobs/sec
lD jobs/sec
Load rB
Load rD
5
Cycle Stealing Problem
lB jobs/sec
lD jobs/sec
Load rB
Load rD
6
Cycle Stealing Problem
lB jobs/sec
lD jobs/sec
Load rB
Load rD
switch back
When new donor job arrives, donor switches back
to donor queue.
7
Generalized Cycle Stealing
lD jobs/sec
Load rB
Load rD
8
Cycle Stealing Problem
lB jobs/sec
lD jobs/sec
Load rB
Load rD
What is Bettys/Dans mean response time?
9
Whats so hard?
Even simplest-case chain grows infinitely in 2D.
10
Prior work cycle stealing coupled-processor
Exponential job sizes (80s)
General Job sizes (80,90,00s)
Truncate the chain
Tail Asymp or Heavy traff.
Bob Foley McDonald Mike Harrison Boxma Borst van
Uitert Williams
Green Stanford Grassman Rao Posner
Convert to Riemann-Hilbert problem
Convert to Wiener-Hopf boundary problem
Very complex integrals.
Very complex integrals for WORKLOAD. No numbers.
Fayolle, Iasnogorodski, Konheim,
Meilijson, Melkman ...
Cohen, Boxma, Borst, Uitert, Jelenkovic ...
11
Dimensionality Reduction Key idea
12
Whats so hard?
Even simplest-case chain grows infinitely in 2D.
13
Solution
0B,0D
1B,0D
2B,0D
BD
BD
lD
lD
lD
BD
New type of transition. Donor Busy period BD
1D-infinite chain!
14
Solution
lD
lD
lD
Approximation, but can be made as close to
exact as desired. Tools03a,Tools03b

BD
15
Markov chain is easy to generalize

• Generalize service distribution
• Generalize to switching times
• Generalize to include thresholds

16
Some interesting questions
Q Exactly how does cycle stealing affect
beneficiary and donor?
17
Some interesting questions
Q How does donor job size variability affect
benef. resp. time?
A Hardly, for rB lt 1 !
ETB
C250
C28
rB
Exp
18
Some interesting questions
Q How should we set NBth?
Q How should we set NDth?
19
SPAA 03, ICDCS 03
Server farm application
Size-based task assignment
Shorts only.
Longs only.
Q Can we enhance this further with cycle
stealing?
20
Two cycle-stealing enhancements
21
Allerton 04, PER 05
N-sharing model
cycle-stealing Dan helps Betty with her work
when hes free.
But,can do better with more aggressive cycle
stealing, (if Dan processes Bettys jobs
faster than his own).
Studied by S. Bell, R. Williams, M. Harrison,
M. Lopez, M. Squillante, C. Xia, D.Yao, L. Zhang,
R. Schumsky, L. Green, S. Meyn, A. Ahn, D.
Stanford, W. Grassman,
22
Who gets control man or woman?
23
Who gets control man or woman?
Answer Mean response time ET lower when woman
controls!
24
Which policy is more robust?
Answer Donor control helps, but even better is
to let Benef. have 2 thresholds, where Donor
controls which threshold is used.
25
Results Adaptive Dual Threshold policy
TB6 (opt when rD 0.6)
TB20 (opt when rD0.8)
TB6 (opt)
Mean response time
ADT meets both goals.
Dans load
Similar idea suggested by Sean Meyn under finite
queues.
26
Priority Scheduling
QUESTA 05, Perf. Eval. 06
M/PH/k with priority classes
Goal Mean response time per job type.
27
Prior Work Multi-Server Priority Queues
Two job classes, exponential
Multi-class simple approx.
Generating Functions
Truncation w/Matrix Analytic or Aggregation
Scaling as Single-server Buzen and Bondi
Kao and Narayanan, Kao and Narayanan, Kapadia,
Kazumi, Mitchell, Nishida Ngo and Lee, Leemans,
Miller
Davis, Kella Yechiali, Feng, Kawada, Adachi,
Kao and Wilson, Gail, Hantler, Taylor
Aggregation into Two classes Mitrani and
King Nishida
Little known for gt 2 classes or non-exponential.
28
2 Priority classes is easy via Dimensionality
Reduction
29
What about 3 classes H,M,L?

• Can get H and M
• response times from
• this chain

• Obvious Idea
• Combine H and M
• into single class HM.
• Ls are second class.

• Need to be careful
• Duration of HM busy period depends on who
  started it
• HH, HM, or MM. Must differentiate.

30
Extensions

• Recursive Dimensionality Reduction
• extends to any number of classes and servers.
• Generalize service distribution.
• Simple representation allows us to derive not
• just mean response time, but all moments of
• response time.

31
Results 4 priority classes
M/M/2 with 4 classes
error in delay
Mean response time
32
Effect of Variability
33
Effect of more servers
M/G/k with 2 priority classes
Mean response time
C2H
34
Optimal number of servers (2 classes with same
mean)
C2H
r
35
Optimal number of servers
Smart Prioritization (Low priority jobs BIGGER)
Stupid Prioritization (Low priority jobs SMALLER)
C2H
C2H
r
r
36
Lessons learned on priorities in multiserver

• Multiserver priority queue performance is
• very different from single server.
• Performance depends on many factors
• Variability of job size
• Load
• Number of servers
• Stupid vs. Smart prioritization

37
There are many other examples of problems
analyzable via Dimensionality Reduction
Open problem Proving the approximation
38
THANK YOU!
39
BACKUP
40
MODIFIED 1D-chain cycle stealing with switching
costs and NBth 3