Queuing Theory - PowerPoint PPT Presentation

1 / 63
About This Presentation
Title:

Queuing Theory

Description:

Arrival rate .99, mean 2583, max 5068. Develop queuing theoretic model. Run a simulation ... Defaults B= , K= SD=FCFS. M/M/1 = M/M/1/ / /FCFS. Multiserver queue Model ... – PowerPoint PPT presentation

Number of Views:280
Avg rating:3.0/5.0
Slides: 64
Provided by: markc195
Category:
Tags: queuing | theory

less

Transcript and Presenter's Notes

Title: Queuing Theory


1
Queuing Theory
2
Web Performance
  • We talked about the importance of performance in
    web applications
  • It is not easy to separate the different
    components of response time
  • It is important to be able to make predictions
    from information that is easy to gather.

3
Approaches to performance analysis
  • Try new configuration and observe results
  • Scale up from prior results
  • Arrival rate .5, mean 43, max 627
  • Arrival rate .95, mean 1859, max 4285
  • Arrival rate .99, mean 2583, max 5068
  • Develop queuing theoretic model
  • Run a simulation

4
Predicting response from past experience
5
Queue Model
6
Multiserver queue Model
Single web server forwards requests to cluster
7
Multiple Single-server queues
DNS rotation for cluster
8
A Quick Probability Refresher
  • A random variable, X, can take on a number of
    different possible values
  • Example the number of students who came to
    office hours is a random variable with values
    1,2,3,
  • Each time we observe (or sample) the random
    variable, it may take on a different value
  • A random variable takes on each of these values
    with a specified probability
  • Example X 0, 1, 2, 3, 4
  • PX0 .1, PX1 .2, PX2 .4, PX3
    .1, PX4 .2
  • The sum of the probabilities of all values equals
    1
  • ?all values PXvalue 1

9
A Quick Probability Refresher
  • Example
  • Suppose we throw two dice and the random
    variable, X, is the sum of the two dice
  • Possible values of X are 2, 3, 4, 5, 6, 7, 8, 9,
    10, 11, 12
  • PX2 PX12 1/36
  • PX3 PX11 2/36
  • PX4 PX10 3/36
  • PX5 PX9 4/36
  • PX6 PX8 5/36
  • PX7 6/36

10
A Quick Probability Refresher
  • A probability distribution function matches each
    possible value of a random variable with its
    associated probability

11
A Quick Probability Refresher
  • The cumulative distribution function of a random
    variable, X, is defined by
  • CDF PX ? x ?all yx Pxy

12
A Quick Probability Refresher
  • Expected Value
  • Can be thought of a long term average of
    observing the random variable a large number of
    times
  • Example dice
  • EX 21/36 32/36 43/36 54/36 65/36
    76/36 85/36 94/36 103/36 112/36
    121/36

13
A Quick Probability Refresher
  • Average vs. Expected Value
  • Short term average
  • Suppose a random variable X is sampled N times
  • Let ni of X i was observed
  • Average of samples
  • (n00 n11 n22 n33 )/N
  • n0/N0 n1/N1 n2/N2 n3/N3
  • As N ? ?, the ratio ni/N becomes pi
  • Thus, EX
  • limN?? n0/N0 n1/N1 n2/N2 n3/N3
  • p00 p11 p22 p33
  • ??i0 i pi

14
A Quick Probability Refresher
  • Continuous Random Variables
  • In many cases, a random variable takes a value
    drawn from a continuous interval
  • Ex processing time for a packet may be any real
    value 0, ?)
  • The distribution of possible values a continuous
    random variable can take is given by a
    probability density function, F(x)
  • P(a ? x ? b) ?ab F(x)dx ?bia P(x i)
  • Ex ?-?? xF(x)dx ? i P(x i)

15
Exponential Distribution
  • The cumulative distribution function F(x) and
    probability density function f(x) are

In queuing theory, we often assume the service
time follows an exponential distribution. The
service time corresponds to the packet
transmission time and is proportional to the
packet length. When standard deviation is equal
to mean, we estimate it to be exponential
16
Poisson Distribution
17
Poisson Distribution
  • The random variable X has a Poisson distribution
    with mean ?, if for non-negative integers i
  • PX i (?ie-?)/i!
  • Represents the number of arrivals in a time unit
    given an exponential arrival process
  • Facts
  • Discrete distribution
  • EX ?

18
Poisson Point Process
  • Definition
  • A Poisson point process with parameter ?
  • A point process with interpoint times that are
    independent and exponentially distributed with
    parameter ?.

19
Poisson Point Process
  • Equivalently
  • The number of points in disjoint intervals are
    independent, and the number of points in an
    interval of length t has a Poisson distribution
    with mean ?t

20
Poisson Point Process
  • Exercise
  • Given a Poisson point process with rate ? 0.4,
    what is the probability of NO arrivals in an
    interval of length 5?

Try to answer two ways, using two equivalent
descriptions of a Poisson process
21
Poisson Point Process
Solution 1 PX gt 5 e-5? 0.1353
Solution 2 PN 0 e-5? 0.1353 (remember
PN i (5?)i (e-5?) / i!, for i 0)
22
Basic Queueing Theory
  • Elementary notions
  • Things arrive at a queue according to some
    probability distribution
  • Things leave a queue according to a second
    probability distribution
  • Averaged over time
  • Things arriving and things leaving must be equal
  • Or the queue length will grow without bound
  • Convenient to express probability distributions
    as average rates

23
Littles Law
  • Goal
  • Estimate relevant values
  • Average number of customers in the system
  • The number of customers either waiting in queue
    or receiving service
  • Average delay per customer
  • The time a customer spends waiting plus the
    service time
  • In terms of known values
  • Customer arrival rate
  • The number of customers entering the system per
    unit time
  • Customer service rate
  • The number of customers the system serves per
    unit time

24
Littles Law
  • For any box with something steady flowing through
    it

Allows us to express the natural idea that
crowded systems (large N) are associated with
long customer delays
N ?T
25
Littles Law
  • Example
  • Suppose you arrive at a busy restaurant in a
    major city
  • Some people are waiting in line, while other are
    already seated (i.e., being served)
  • You want to estimate how long you will have to
    wait to be seated if you join the end of the line
  • Do you apply Littles Law? If so
  • What is the box?
  • What is N?
  • What is ??
  • What is T?

26
Littles Law
  • Box
  • Include the people seated (i.e., being served)
  • Do not include the people waiting in line
  • Let N the number of people seated (say 200)
  • Let T mean amount of time a person stays seated
    (say 90 min)
  • Conclusion
  • Throughput 200/90 2.22 persons per minute
  • Wait time
  • If 100 people are waiting, you could estimate
    that you will need to wait 100/2.22 45 min

27
Littles Law
  • Suppose ATM streams are multiplexed at an output
    link with speed 622 Mbps
  • Question
  • If 200 cells are queued on average, what is the
    average time in queue?
  • Answer
  • T N/?
  • T 200 53 8 / 622Mbps
  • T 0.136 ms

28
Queue Statistics
  • Arrival rate load - Poisson -
  • Packet length Distribution - exponential
  • E(m) expected value (mean) of random variable
  • TsMean service time, TqMean queue time
  • Standard Deviation of service time
  • Link utilization

29
Fixed Packets
  • Average transmission time
  • ? average packets serviced/sec capacity

n
Buffer
Arriving Packets
Departing Packets
30
Practical Example
Tr
Ts
31
Littles Law
  • Applies to systems where no jobs are lost or
    created
  • Arrival rate arrivals/total timeN/Tt
  • Mean time in system J/N
  • Mean number in system

32
J is the shaded areaExamine time up to D619
T13 T27 T310 T46 T56 T66 J38
Arrival Rate N/Tt6/19.31 Mean Time in system
J/N 38/66.3 Mean number in system
J/Tt38/192 (J/N)(N/Tt)6.3.312 ?Tq
33
Throughput
  • Throughpututilization/service time
  • Throughput ?/Ts
  • For ?.5 and Ts1ms
  • Throughput is 500 packets/sec

34
Littles Law Examples N rT
  • How long to answer email
  • 50 messages/day
  • 150 messages in Inbox
  • 3 days for answer
  • Hospital
  • 2.5 days stay
  • 5 new cases/day
  • 12.5 in the system
  • Average Time in Hospital
  • 102.5 average patients
  • Discharge 67.5 patients/day
  • 1.5 days in hospital
  • Customers in store
  • 10/hour arrival rate
  • Stay on average 0.5 hours
  • 5 in store

35
Queueing Theory
  • Mathematical study of queues
  • Enables analysis of several related processes
  • Can calculate several steady-state performance
    measures
  • average waiting time
  • expected number waiting
  • probability of a given number of items in the
    queue
  • etc.
  • Can model several different types of systems
  • Different queueing disciplines
  • Different arrival, service, and departure
    probability distributions
  • etc.

36
Kendall Notation
  • A/S/NS/B/K/SD
  • A,SInterarrival time, service time distribution
  • M Exponential
  • Ek Erlang
  • Hk Hyperexponential
  • D Deterministic
  • NSNumber of servers
  • BNumber of Buffers
  • KPopulation size
  • SDService Discipline
  • FCFS,FCLS

37
Kendall Notation
  • A/S/NS/B/K/SD
  • Defaults
  • B ? , number of buffers
  • K ? , population size
  • SDFCFS service discipline
  • M/M/1 M/M/1/ ? / ? /FCFS
  • exponential arrival rate and service time
  • 1 server
  • infinite buffers and infinite population
  • FCFS queue

38
M/M/1 queue

39
Markov Models - analyze state n(n buffers are
occupied)
  • n1
  • n
  • n-1

departure
Buffer Occupancy
  • n

arrival
0 1 ... n-1
n n1
40
Probability of being in state n
one arrived and one departed
None arrived or departed
None arrived, one departed
one arrived, none departed
41
Markov Chains
42
Substituting Utilization
Rate leaving n rate entering n
43
Substituting P1
  • Higher states have decreasing probability
  • Higher utilization causes higher probability
  • of higher states

44
What about P0
Queue determined by
45
(No Transcript)
46
Single Server
?Ts0
(?Ts)/Ts1
(?Ts)/Tsgtgt1
47
Example
  • Given things that you can easily measure
  • Arrival rate of 1000 requests/sec
  • Service capacity of 1100 requests/sec
  • Find
  • Utilization
  • Probability of having 4 buffers filled

48
Example
49
(No Transcript)
50
Examples
  • Web Site with 100 Clients and 1 server.
  • Service time .6sec and ?? (exponential)
  • What is average response time at 20
    queries/min1q/3sec?
  • Use M/M/1 model
  • ??Ts(1/3)(.6)0.2
  • TrTs/(1-?).6/.8.75sec
  • If 1.5 seconds is too long, what utilization is
    allowable (90 of responses are less than 1.5sec)
  • mTr(r)Trln(100/(100-r))
  • mTr(90)Trln(100/10)Ts/(1-?)2.31.5 sec
  • 2.3 Ts(1-?)1.5 sec, 1.5 ? 1.5-2.3.6 , ?.08
  • The utilization must actually be reduced

51
Multi-server System with separate queues (DNS
Rotation)
  • five processors (5 M/M/1), average service time
    0.1 sec.
  • Assume that the standard deviation of service
    time is observed to be 0.094 sec (exponential
    service time).
  • Web page hits 40 per second.
  • Separate Queue Approach
  • If processes are evenly distributed among the
    processors, then the load for each processor is
  • 40/5 8 requests per second. Thus,
  • r ? Ts 8 ? 0.1 0.8
  • The residence time is then easily calculated
  • tr Ts/(1- r).1/.2.5 seconds

52
Single Queue - Multiple server
  • (M/M/5) aggregate arrival rate of 40 processes
    per second. Utilization is still ?(? Ts /5)
    0.8
  • To calculate the residence time first calculate
    Erlang C function, table lookup, ? 0.8 for 5
    servers C 0.554
  • Tr (C/N)(Ts/(1- ?)) Ts ((0.544)(0.1))/(5(1-
    0.8)) (0.1) 0.1544
  • So the use of multiserver single queue has
    reduced average residence time from 0.5 sec
    (previous slide) down to 0.1544 sec,
  • which is greater than a factor of 3.
  • If we look at just the waiting time, the single
    queue case is 0.0544 seconds compared to 0.4
    seconds, for multiple queues
  • which is a factor of 7.

53
Another Example
  • Arrival rate
  • ? 125 requests/sec
  • Service rate
  • ?1/0.002 500 requests/sec
  • Gateway utilization
  • ? ?/ ?0.25
  • Probability of n packets in gateway
  • (1- ?) ?n 0.75(.25)n
  • Mean time spent in web server
  • Ts/(1- ?).002/(1-.25)2.66ms

54
Simulation
55
Workload Models
  • Identify Performance Problems
  • Problems may only occur under high load
  • Benchmark Web Components
  • Deployment decisions
  • Evaluate new features
  • Capacity Planning
  • Determine network, memory, disk and clustering
    needs

56
Web Workload Characterization
  • Based on the results of numerous studies
  • Key properties
  • HTTP Message Characteristics
  • Several request methods and response codes
  • Resource Characteristics
  • Diverse content-type, size, popularity, and
    modification frequency
  • User Behavior
  • User browsing habits significantly affect workload

Category Parameter
Protocol Request Method Response Code
Resource Content type Resource size Response size Popularity Modification freqency Temporal Locality embedded resources
Users Session interarrival times clicks per session Request interarrival times
57
Parameter Characterization
  • Associate each parameter with quantitative values
  • Statistics
  • Mean, median, mode
  • OK for parameters that dont vary much
  • Probability Distributions
  • Capture how a parameter varies over a wide range
    of values

58
Probability Distribution
  • Every random variable gives rise to a probability
    distribution
  • Probability Density Function
  • Assigns a probability to every interval of the
    real numbers
  • Cumulative Distribution Function
  • Describes the probability distribution of a
    real-valued random variable X
  • F(x) P(X lt x)
  • The probability that a random variable will be
    less than or equal to x
  • In the following slides, we will show the CDF of
    commonly used distributions

59
Poisson Distribution
  • F(x) (e-??k)/k!
  • Used to model the time between independent events
    that happen at a constant average rate
  • The number of times a web server is accessed per
    minute is a Poisson distribution
  • For instance, the number of edits per hour
    recorded on Wikipedia's Recent Changes page
    follows an approximately Poisson distribution.

60
Exponential Distribution
  • F(x) e-?x
  • Used to model the time until the next occurrence
    of an event in a Poisson process
  • Session interarrival times are exponential
  • Time between the start of one user session and
    the start of the next user session

61
Pareto Distribution
  • F(x) (x/a)-k
  • k is shape, a is minimum value for x
  • Power law
  • 80-20 rule
  • 20 of the sample is responsible for 80 of the
    results
  • Response sizes, Resource sizes, Number of
    embedded images, Request interarrival times
  • Often used to model self-similar patterns

62
Probability Distributions in Web Workload Models
Distribution Workload Parameter
Exponential Session interarrival times
Pareto Response Sizes Resource Sizes Number of embedded images Request interarrival times
Lognormal Response sizes Resource sizes Temporal Locality
Zipf-like Resource Popularity
63
Probability Distribution Conversion
  • Most languages have random number library
    functions
  • Uniform distribution
  • Must convert from uniform distribution to the
    chosen distribution
  • Given the cumulative distribution function, CDF,
    of the chosen distribution
  • 1. Generate a random number call this number p
  • 2. Compute x such that CDF(x) p
  • Determine the inverse of the CDF
  • 3. x is the random number you use

Inverse of the CDF For the exponential
distribution
Write a Comment
User Comments (0)
About PowerShow.com