Job Fairness in Queue Scheduling: A Tutorial

1
Job Fairness in Queue Scheduling A Tutorial
Hanoch Levy School of Computer Science Tel Aviv
University
Prepared jointly with Benjamin Avi-Itzhak,
RUTGERS University David Raz, Tel-Aviv
University
SIGMETRICS, July 2005
2
Why Fairness in Queues?
Why (disciplined) Queues?
Not Fair!!!
To provide FAIRNESS in waiting/service
Queue A Fairness Management Facility
3
Why Fairness in Queues? (2)

• Fairness inherent/crucial part of queues

• Recent studies, Rafaeli et. al. 2003
  (experimental psychology)
• Experiments on humans in queue scenarios
• Fairness in queue is very important to people
• Perhaps even more than delay itself!

The issue
The audience Audience.ppt
4
Outline

• Queue Model
• Job-Based systems, Flow-Based systems
  applications
• Prior work
• Requirements
• The performance issue Delay vs. Service
• The granularity level How fine
• Dealing with stochastics
• The physical entities seniority, service
  requirement, resources
• The Fairness Measures - Overview properties
• Seniority based fairness
• Service time based fairness
• Resource allocation based fairness
• Mixture based
• Application perspective
• References

5
Queue Model (single server)
6
Job-Based vs. Flow-BasedApplications

• JOB BASED
• Customer Job
• Applications
• Networking Application level equipment Web
  server, file server, peer2peer
• Supermarket, Bank, public office alike
• Call center
• Computer systems, grid computing

• FLOW BASED
• Customer Flow
• Applications
• Networkingnetwork level equipment
• Routers, gateways, load-balancers

7
History Queueing Theory and Fairness

• Queueing theory Decades of research
• Delay of individual
• Practical Applications many diverse
• Fairness in queues
• Many importance statements
• Importance of fairness Larson (1988), Palm
  (1953), Mann (1969), Whitt (1984), Rothkopf
  Rech (1987)
• Analysis was Little, now growing(job fairness)
• Morris Wang (85)
• Gordon (87)
• Avi-Itzhak Levy (96)
• Bender, Chakrabarti . Muthukrishnan (98), Wierman
  Harchol-Balter (03), Harchol-Balter et. al.
  (03)
• Avi-Itzhak, Levy Raz (05), Raz, Avi-Itzhak
  Levy (04, 05a,b), Raz, Levy Avi-Itzhak (04),
  Brosh, Levy Avi-Itzhak (05)
• Sandmann (05) (more is coming)

Exception Flow Fairness
8
History Queueing Theory and Fairness (2)

• ? We know only little about queue (job) fairness!
  
• ? More complex than measuring individual delay!
• Need to measure customer-relations (interactions)

9
Designing a Metrics - Keep in mind

• To be used by
• Researchers
• Designers / operators
• Customers (appeal to)

• Doing research
• Keep application(s) in mindas many as possible
• Design/pick a metrics based on a sound
  intuitive basis
• Build confidence
• Consider simple cases
• Examine sanity
• Conduct analysis (new derivations)
• Match to intuition
• If it does ? GOOD!
• Now ready to use for complicated cases (and
  discover surprises).

• Desired properties (requirements)
• Fits application(s) as many as possible
• Based on a sound intuitive basis
• Fit widely accepted intuition in simple cases
• Yields to analysis

10
The performance issues Delay vs. Service

• Delay
• Job delay (waiting time, sojourn time)
• Traditional queueing-theory measure
• The major factor when service is guaranteed

• Job (service) Completion
• Have the job done
• Less popular in queueing theory
• Applies when service is not guaranteed
• Ticket line for scarce tickets

11
Granularity of Fairness Evaluation

• At what granularity level, should fairness be
  evaluated

• Individual
• Scenario
• System (steady state)

• All are important
• Individual, scenario build confidence (scale of
  reference) in metrics
• System to evaluate systems/policies
• Note All exist for individual waiting times

12
Dealing with StochasticsActual measures vs.
Expected values

• Actual measure
• Fairness evaluated for every scenario
• Expectation used to summarize scenarios

• Expected values
• Expected performance per customer class evaluated
• Classes compared to each other gt fairness

13
Dealing with StochasticsActual measures vs.
Expected values (2)

• Actual measure
• Harder to compute

• Expected values
• May be misleading
• Like accepting Sigmetrics papers randomly with
  equal probability.

14
Dealing with StochasticsThe Black Box question

• Customer does not see other customers

• Close approach dont see (arrivals) dont
  care.
• Only size matters Is my size class doing ok?

• Open approach Customer is still interested in
  Fairness even though he cannot check it directly

15
The Physical Factors (queues)
16
Size and Seniority preference principles
(requirements)

• Seniority
• Size

17
Size and Seniority preference principles
(requirements)

• Seniority principle
• Weak All jobs same service times ? if ailt aj
  then more fair to complete service of Ji before
  Jj
• Strong Ji and Jj same service times

• Service-requirement principle
• Weak All jobs same arrival times ? if silt sj
  then more fair to complete service of Ji before
  Jj
• Strong Ji and Jj same arrival times

18
How Scheduling policies meet the principles (are
fair by principle)
19
The Size vs. Seniority Dilemma

• Mr. Short vs. Mrs. Long
• Is it more fair to serve Short ahead of Long? By
  how much?

20
Review of Measures (job based)

• Seniority based

• Mixtures

21
Approach 1 Order (seniority) Fairness

• Gordon (87) Ph.D MIT
• Deals with skips and slips
• Performance measure of overtaking to quantify the
  level of social justice
• B skips A (A experiences a slip) A arrives
  before B, B leaves before A
• Considers several Markovian models
• Analyses distribution of slips, skips
• Result Eskips E slips
• Measure No explicit suggestion
• Question how to weigh skips and slips?

22
Approach 1 Order (seniority) Fairness results

• Systems analyzed
• 2 M/M/1 in parallel
• 2 M/M/1s with one SJQ smart customer
• Multi server system M/M/m
• Infinite server system M/G/Inf
• Analysis distribution of SKIP and SLIP
• Results
• E(skips) E (slips)
• Most systems Dist(skips) ! Dist (slips)
• Smart customer increases skips, decreases
  slips
• M/U/Inf with symmetric service time dist
  Dist(skips) Dist (slips) only system found

23
Approach 1 Order (seniority) Fairness How do I
use it?

• Compute Dist(skips), Dist (slips)
• Analysis or simulation
• How to use for fairness measure open issue.
• Perhaps Eskips
• ? FCFS most fair (Eskips0)

24
Measure 1 Order (seniority) Fairness

• Avi-Itzhak Levy (96, 04)
• Axioms (for G/D/1) what happens to unfairness
  measure when interchanging customers
• P1 Monotonicity in seniority difference of
  interchanged neighbors
• P2 Reversibility of neighbor interchange
• P3 Independence on position and time
• P4 Fairness change is not affected by customers
  not interchanged
• P4G interchange of non-neighbors

25
Order Fairness results

• ai - Arrival time of customer i
• Di - Waiting displacement of customer i
• C gt 0, arbitrary constant
• Expected fairness per customer

• FCFS most fair (LCFS least)

• Thm Let (W, W) be the steady state waiting
  time under (policy, FIFO), then

26
Order Fairness results (2)

• Adhering to principles
• Strong seniority service principle YES
• Service-time service principle NOT

27
Order Fairness Properties Applicability

• Good for
• S. times identical
• S. times dont matter
• Issue is Job completion
• Applications
• Scarce-ticket lines
• Some call-centers

• FCFS is most fair (LCFS least)
• Intuition concepts
• Peoples strong belief in order fairness

28
Measure 1 Order (seniority) Fairness How do I
use it?

• Compute

• Simulation
• Compute VarW for steady state
• Remark range of variance

• Research How relates to Eskips

29
Measure 2 service-time Fairness

• Bender, Chakrabarti . Muthukrishnan (1998),
  Harchol-Balter et. al. (2003)
• Wierman Harchol-Balter (2003)
• Propose a Fairness Criterion
• Emphasis on service requirement
• Slowdown for job of size x compute ET(x)/x
• If the slowdown is lower then 1/(1-?) for all x -
  FAIR
• Classification of a large variety of policies
• Always fair fair for all dist. loads
• Always unfair unfair for all dist. loads

30
Service-time Fairness Results

• Classification (not measure) of a large variety
  of policies
• Any preemptive size based policy is always unfair
  (all loads all service dists).
• All non-size based non-preemptive policies are
  always unfair for service time dist defined on
  neighborhood of zero (short jobs discriminated).
• Age based policies are always unfair
• FCFS is always UNFAIR
• LCFS (preemptive) PS are always FAIR

31
Service-time Fairness Results (2)

• Adhering to principles
• Seniority service principles NOT
• Service-time service principle Open question

32
S. time Fairness Properties Applicability

• Good for
• A. times identical
• A. times not known / dont see the queue
• Your size is always the same
• Issue is wash seniority by averaging.
• Advantage relatively simple analysis

• Applications Computer systems (?)

33
S. time Fairness How do I use it

• Compute ET(x) (use queueing theory or
  simulation), divide by x and check rules
• You end up with a criterion.
• Can create a measure.

• Research create measure / study for more
  systems
• Multiple queues
• Multiple servers

34
Measure 3 Resource Allocation Fairness

• Raz, Levy, Avi-Itzhak (04)
• Aim at the dilemma between size and seniority
• Focus on fair share of resources
• Ideal At t, each customer deserves 1/N(t) of
  system resources (N(t) customers(t))
• Compare warranted service with granted service

35
Resource allocation Fairness Results

• Adhering to principles
• STRONG Seniority service principles YES
• WEAK Service-time service principle YES
• STRONG Service-time service principle NOT

36
Resource allocation Fairness Results (2)

• PS most fair
• Reacts to both s.time and seniority
• FCFS gt LCFS (seniority dominant)
• FCFS lt PLCFS (s. time dominant)

37
Resource allocation Fairness Sample of Results
(3)

• Multiple server more fair than single server with
  same combined rate
• Single Queue more fair than Multi-Queue
• Jockeying-on-idle increases fairness
• Jockeying from head more fair than jockeying from
  tail
• Short job prioritization is more fair in most
  cases (exception if short jobs are almost as
  large as rest of population)
• Small variance FCFS lt LCFS-PR Large variance
  FCFSgt LCFS-PR
• Locality of reference Locality.ppt

38
Resource Allocation Fairness Applicability

• Good for
• S. times and A. times arbitrary
• Issue is Waiting times

• Applications
• Waiting lines where resources guaranteed
• Call centers (non-scarce resources)
• Web services
• Supermarkets
• Airport services

39
Resource Allocation Fairness How Do I use it

• Derive variance of discrimination at steady
  state
• Use Queueing Theory methodology developed
• Good for variety of Markovian systems
• Large systems need approximations (more research)
  Waiting lines where resources guaranteed
• Use simulation

40
Measure 4 Mixture

• Sandmann (05)
• Mixture seniority and size
• Accounts for 2 parameters
• Overtaking (slips) seniority
• Large jobs Size
• X is a large job for C if
• i) Upon Cs arrival X has no-less residual
  service than C.
• ii) X departs before C.

• Fairness measure

41
Measure 4 - Mixture results

• Adhering to principles
• STRONG Seniority service principles YES
• STRONG Service-time service principle YES
• Difficulties Asymmetry in sensitivity to size
  seniority
• E.g. X arrives with 100 sec s.time, and waits an
  hour. Y arrives 1 hour later with 99 sec s.time.
  It is as fair to serve in both orders.
• Same if 10 hours.

42
Measure 4 - Mixture How do I use it

• Analysis of skips see Gordon (87) Markovian
  can do.
• Analysis of Large not provided yet (perhaps in
  the making).

43
Job-Based vs. Flow-BasedApplications (repeat)

• FLOW BASED
• Customer Flow
• Applications
• Networkingnetwork level equipment
• Routers, gateways, load-balancers

• JOB BASED
• Customer Job
• Applications
• Networking Application level equipment Web
  server, file server
• Supermarket, Bank, public office alike
• Call center
• Computer system

44
A word on flow-based measures

• Deal with flows (of packets)
• Interested mainly in throughput
• Literature
• Fair bandwidth allocation (network)
• MinMax fairness (Jaffe (81))
• Proportional Fairness (Kelly (97))
• Fair Scheduling
• Weighted Fair Queueing (WFQ)
• Demers, Keshav and Shenker (1990), Greenberg and
  Madras (1992), Parekh (1992), Parekh and Gallager
  (1993), (1994), Golestani (1994), Rexford,
  Greenberg and Bonomi (1996), Bennet and Zhang
  (1996), others.

45
Measure 5 PS proximity WFQ/RFB literature

• Scheduling fairness measures
• Worst Case deviation from PS (extreme values)
• Relative Fairness Bound (Golestani (94))

46
Measure 5 PS proximity WFQ/RFB literature

• Absolute Fairness Bound (AFB) (Greenberg and
  Madras (1992) and Keshav (1997))

• Maximum (time) discrepancy between schedule and
  PS
• Applying to jobs
• Try PS completion discrepancy of job
• LCFS FCFS infinity!
• Most non-PS based (non-WFQ) infinity! (SJF, LJF,
  SRPT..)
• Good for very precise PS imitations

47
What fits? Go by the application
48
How People Perceive Queue Fairness

• Rafaeli et. al. (2005) Queue experiments on
  humans
• FCFS is FAIR (seniority)
• Violation of FCFS unfair
• Regardless if close to the person asked or away
  of it.
• Size-issue not addressed yet.
• Multi-server
• Multi-Queue (MQ) less fair than single queue.
• One queue in MQ is shorter ? less fair.
• If people PAY for the short Q (first-class) ?
  Fair.

49
Concluding remarks

• Fairness in Queues is important
• Measures must
• Fit applications
• Agree with ones intuition / be consistent
• Researcher, designer, customer
• Yield to analysis
• Research of subject in its early stages
• Much more to study (systems measure
  examination)
• Scheduling policies
• Weights
• Multiple queues /servers
• Complex structures
• Relations between measures
• Other measures

50
THANK YOU
51
Audience and Objectives

• System designers / (queueing tool)
• Use of methodology
• Queueing researchers
• Conduct research in this new area

52
Doing Research

• Doing research
• Keep application(s) in mindas many as possible
• Design/pick a metrics based on a sound
  intuitive basis
• Build confidence
• Consider simple cases
• Examine sanity
• Conduct analysis (new derivations)
• Match to intuition
• If it does ? GOOD!
• Now ready to use for complicated cases

53
Locality of Reference

• Fairness Comparing customers to each other
• Can use Variance
• E.g. Var W
• Question Over what population Take Variance??
• Natural approach Use the steady state variance
• However Want to compare customers of a busy
  period!
• Only they affect each other!! (compete for
  resources)
• ? Need to use Busy-period Local Variance
• Global Variance may be misleading!

54
Locality (2)
55
FAQ (1)

• Q Should measure be normalized?
• A1 Not necessarily. Unfairness in units of
  waiting time (reflecting customer suffering).
• A2 Can normalize by 1st-moment-squared of
  service time.
• Same as VarW ? Coefficient of Variation W

56
FAQ (2)

• Q Results are intuitive. Whats new?
• A
• Intuitive results are very important.
• They provide support/confidence in metrics.
• Crucial for using a new abstract (and
  complicated) measure in complicated cases.