Oblivious AQM and Nash Equilibria

1
Oblivious AQM and Nash Equilibria

• D. Dutta, A. Goel and J. Heidemann
• USC/ISI USC
  USC/ISI
• IEEE INFOCOM 2003 - The 22nd Annual Joint
  Conference of the IEEE Computer and
  Communications Societies
• Presented By Sharon Mendel
• Game Theory in Networks Seminar 25/01/2006

2
Before we Begin

• Todays Internet - Motivation

Oblivious AQM Nash Equilibria
3
Active Queue Management ??
Before we Begin

• A congestion control protocol (e.g. TCP) operates
  at the end-points and uses the drops or marks
  received from the Active Queue Management
  policies (e.g. Drop-tail, RED) at routers as
  feedback signals to adaptively modify the sending
  rate in order to maximize its own goodput.

4
Transport Control Protocol
Before we Begin

• TCP is the dominating transport layer protocol in
  the Internet and accounts for over 90 of the
  total traffic.
• The TCP Protocol is well defined, robust and
  congestion-reactive (thus stable).
• The end-to-end congestion control mechanisms of
  TCP have been a critical factor in the robustness
  of the Internet.
• It is widely believed that if all users deployed
  TCP, networks will rarely see congestion
  collapses and the overall utilization of the
  network will be high.

5
Todays Internet
Before we Begin

• There are indications that the amount of
  non-congestion-reactive traffic is on the rise.
• Most of this misbehaving traffic does not use
  TCP.
• e.g. Real media, network games, other real time
  multimedia applications.
• The unresponsive behavior can result in both
  unfairness and congestion collapse for the
  Internet.
• The network itself must now participate in
  controlling its own resource utilization.

Some of the previous slides Promoting the Use
of End-to-End Congestion Control in the
Internet, S. Floyd and K. Fall, IEEE/ACM
Transactions on Networking Vol. 7 1999.
6
The Papers Motivation
Before we Begin

• TCP (and in fact, any transport protocol) does
  not guarantee good performance in the face of
  aggressive, greedy users (who are willing to
  violate the protocol to obtain better
  performance).
• Protocol Equilibrium A protocol which leads to
  an efficient utilization and a somewhat fair
  distribution of network resources (like TCP
  does), and also ensure that no user can obtain
  better performance by deviating from the
  protocol.
• If protocol equilibrium is achievable, then it
  would be a useful tool in designing robust
  networks.

7
More Introduction

• Oblivious AQM and Nash Equilibria

Oblivious AQM Nash Equilibria
8
Oblivious AQM
More Introduction

• Oblivious (stateless) AQM scheme a router
  strategy that does not differentiate between
  packets belonging to different flows.
• Stateful schemes e.g. Fair-Queuing.
• Stateful Schemes offer good performance, but
  oblivious schemes are easier to implement.

9
Popular AQM Schemes (1)
More Introduction

• Congestion avoidance is achieved through packet
  dropping.
• Drop-Tail Buffers as many packets as it can and
  drops the ones it can't buffer
• Distributes buffer space unfairly among traffic
  flows.
• Can lead to global synchronization as all TCP
  connections "hold back" simultaneously, hence
  networks become under-utilized.

DT
10
Popular AQM Schemes (2)
More Introduction

• RED Random Early Dedication - Monitors the
  average queue size and drops packets based on
  statistical probabilities
• If the buffer is almost empty, all incoming
  packets are accepted As the queue grows, the
  probability for dropping an incoming packet
  grows When the buffer is full, the probability
  has reached 1 and all incoming packets are
  dropped.
• Considered more fair than tail drop - The more a
  host transmits, the more likely it is that its
  packets are dropped.
• Prevents global synchronization and achieves
  lower average buffer occupancies.

RED
11
Oblivious AQM and Nash Equilibria
More Introduction

• The paper studies the existence and quality of
  Nash equilibria imposed by oblivious AQM schemes
  on selfish agents
• Existence
• Efficiency
• Achievability

12
Content

• Introduction
• The Model
• Existence
• Efficiency
• Achievability
• Summary and Future Work

13
The Model

• The Markovian Internet Game

Oblivious AQM Nash Equilibria
14
The Internet Game
The Model

• Players The end-to-end selfish traffic agents.
• Each player has a strategy which is to control
  the average rate he tries to push through the
  network.
• Users Performance Metric goodput.
• Rules set by the AQM policies (AQM schemes in
  routers).
• Nash Equilibrium No selfish agent has any
  incentive to unilaterally deviate from its
  current state.
• Oblivious AQM scheme leads to a protocol
  equilibrium only if it imposes a Nash equilibrium
  on the selfish users.
• Papers focus AQM schemes that guarantee
  bounded average buffer occupancy regardless of
  the total arrival rate.

15
The Markovian Internet Game
The Model

• The agents generate Poisson traffic.
• Does not accurately model Internet traffic, yet a
  reasonable first step.
• Each user controls its own offered load the
  average Poisson traffic rate.
• The system is modeled as a M/M/1/K queue
• Average service time - Without loss of generality
  assumed to be unity.
• Buffers capacity - KB.
• No assumptions are made on the selfish protocol
  (i.e. TCP, AIMD etc).

16
The M/M/1/K Internet Game
The Model

• n Number of users players.
• ?i The Poisson average arrival rate of player
  i.
• Ui Utility function of player i.
• µi Goodput , Ui µi .
• p AQM router drop probability due an average
  aggregated load (offered load) of ? and an
  average service time of unity.
• A symmetric Nash equilibrium - Ensures that every
  agent has the same goodput at equilibrium.
• Unless mentioned otherwise, quantities such as
  the rates, goodput and throughput are averages
  (Poisson traffic sources).

17
Nash Equilibrium Conditions (1)
RED
DT
The Model

• No agent can increase their goodput, at Nash
  equilibrium, by either increasing or decreasing
  their throughput
• A symmetric Nash equilibrium
• Oblivious AQM scheme, hence functions of router
  state (drop probability, queue length) are
  independent in i

Utility Function in Nash Equilibrium
Nash Equilibrium Condition Nash Equilibrium
Satisfying Condition Necessary and sufficient
18
Nash Equilibrium Conditions (2)
The Model

• ?Nash , µNash , pNash - The aggregate throughput
  (offered load), goodput, drop probability
  respectively.
• The Nash equilibrium imposed by an AQM scheme is
  efficient if the goodput of any selfish agent is
  bounded below when the throughput (offered load)
  of the same agent is bounded above

Nash Equilibrium Efficiency Condition
c1? c2 some constants
pNash is bounded
19
Existence

• Are there oblivious AQM schemes that impose Nash
  equilibria on selfish users?

Oblivious AQM Nash Equilibria
20
Drop-Tail Queuing
Existence

• p - Drop Probability Probability to find a full
  system.
• From Queuing Theory
• Theorem 1 There is NO Nash Equilibrium for
  selfish agents and routes implementing Drop-Tail
  queuing.
• Proof applying the condition for Nash
  equilibrium

QED.
21
Random Early Detection
Existence

• Approximated steady state RED model
• From Queuing Theory, at steady state
• RED Router, at steady state
• Steady State

Faster network simulation with scenario
pre-filtering Tech. Rep., USC/ISI Tech Report
550, November 2001.
lq Queue length p Drop Probability ?
Aggregated Offered Load
lq ? maxth
22
RED and Nash Equilibria
Existence

• Theorem 2 RED Does NOT impose a Nash equilibrium
  on uncontrolled selfish agents.
• Proof applying the condition for Nash
  equilibrium
• Summary
• RED punishes all flows with the same drop
  probability.
• The nature of the drop function is considerably
  gentle.
• Misbehaving flows can push more traffic and get
  less hurt (marginally).
• There is no incentive for any source to stop
  pushing packets.

QED.
RED is oblivious
Nash Equlibria does not exist.
23
Virtual Load RED
Existence

• Virtual Load RED model
• Theorem 3 VLRED imposes a Nash Equilibrium on
  selfish agents if
• Proof
• Throughput at Nash equilibrium is independent of
  minth

lvq M/M/1 Queue length when facing the
same load
24
Efficiency

• If an Oblivious AQM scheme can impose a Nash
  equilibria, is that equilibria efficient, in
  terms of achieving high goodput and low drop
  probability?

Oblivious AQM Nash Equilibria
25
Example - Efficiency of VLRED
Efficiency

• Proof Applying Nashequilibrium satisfying
  andefficiency conditions

• Theorem 4 VLRED is not efficientimposing a
  Nash Equilibriumon selfish agents

a, ß - some constants
nµ ?(lvq2)
nµ/a
lvq2
The goodput falls to zero asymptotically. QED.
26
Efficient Nash AQM scheme
Efficiency

• Assume Totaldesirable offered loadat Nash
  equilibrium
• Oblivious mechanism can ensure an efficient Nash
  equilibria under selfish behavior of users.

ENAQM drop probability is bounded
27
Achievability

• How easy is it for players (users) to reach the
  equilibrium point? or How can we ensure that
  agents actually reach the Nash equilibrium state?

Oblivious AQM Nash Equilibria
28
Ensuring a Nash equilibrium by an Oblivious AQM
Achievability

• ?i Offered load at equilibria when the number
  of agents is i.
• ?i ?i - ?i-1 ia a some constant
• p ƒ(?i) non decreasing and convex.
• Applying Nash equilibrium satisfying and
  efficiency conditions
• From convexity of pi
• From efficiency
• The equilibrium imposed by any oblivious AQM
  strategy is (very) sensitive to the number of
  agents, thus making it impractical to deploy in
  the Internet.
• Agents need the help of the router to reach the
  equliibria.

c1, c2 - some constants
The sensitivity coefficient falls faster than the
inverse quadric.
c - constant
29
Summary and Future Work

• Overview
• Now what? Future Work
• Some further work

Oblivious AQM Nash Equilibria
30
Overview
Summary and Future Work

• Introduction Todays Internet.
• The proposed model Markovian (M/M/1/K) Game
• Existence
• Drop tail and RED cannot impose a Nash equilibra.
• VLRED imposes a Nash equilibra.But the
  equilibrium points do not have a very high
  utilization.
• Efficiency - ENAQM imposes an efficient Nash
  equilibra.
• Achievability - Equilibrium points in oblivious
  AQM strategies are very sensitive to the change
  in the number of users.
• It may be hard to deploy oblivious schemes that
  do have Nash equilibria without the explicit help
  of a protocol.

31
Now What ? - Future Work
Summary and Future Work

• VLRED Explore why the Nash equilibria do not
  result in good network utilization.
• Conjecture VLREDs drop function becomes very
  harsh as we reach equilibria.
• Study gentler versions of VLRED and determine
  whether such modification can still impose Nash
  equilibria.
• Design protocols which lead to efficient network
  operation, such that no user has any incentive to
  unilaterally deviate from the protocol can it
  be done ? The Protocol Equilibrium Question.

32
Some Further work (1)
Summary and Future Work

• Towards Protocol Equilibrium with Oblivious
  Routers D. Dutta, A. Goel and J. Heidemann, IEEE
  INFOCOM 2004.
• In this paper, we show that if routers used
  EWMA to measure the aggregate rate, then the best
  strategy for a selfish agent to minimize its
  losses is to arrive at a constant rate. Even
  though the protocol space is arbitrary, our
  scheme ensures that the best greedy strategy is
  simple, i.e. send with CBR. Then, we show how we
  can use the results of an earlier paper to
  enforce simple and efficient protocol equilibria
  on selfish traffic agents

33
Some Further work (2)
Summary and Future Work

• Pricing Differentiated Services A Game
  -Theoretic Approach, E. Altman, D. Barman, R. El
  Azouzi, D. Ros, B. Tuffin, (accepted for
  publication in Computer Networks, 2005).
• The goal of this paper is to study pricing of
  differentiated services and its impact on the
  choice of service priority at equilibrium. We
  consider both TCP connections as well as non
  controlled (real time) connections. We first
  study the performance of the system as a function
  of the connections parameters and their choice
  of service classes. We then study the decision
  problem of how to choose the service classes. We
  model the problem as a noncooperative game. We
  establish conditions for an equilibrium to exist
  and to be uniquely defined. We further provide
  conditions for convergence to equilibrium from
  non equilibria initial states. We finally study
  the pricing problem of how to choose prices so
  that the resulting equilibrium would maximize the
  network benefit
• The paper (Oblivious AQM and Nash Equilibria)
  restricted itself to symmetric users and
  symmetric equilibria and the pricing issue was
  not considered. In this framework, with a common
  RED buffer, it was shown that an equilibrium does
  not exist. An equilibrium was obtained and
  characterized for an alternative buffer
  management that was proposed, called VLRED. We
  note that in contrast to (Oblivious AQM and Nash
  Equilibria), since we also include in the
  utility of CBR traffic a penalty for losses we do
  obtain an equilibrium when using RED

34
Thank You !
Oblivious AQM Nash Equilibria
35
Appendixes

• Stateful Schemes Fair Queuing
• From Queuing Theory
• Nash Equilibrium Conditions
• Approximated Steady State RED Model
• VLRED Model, Theorem 3 - Nash Equilibrium
  Existence - Proof
• Efficient Nash AQM Drop Probability

Oblivious AQM Nash Equilibria
36
Stateful Schemes Fair Queuing - Nagles
Algorithm
Introduction - Appendix

• Gateways maintain separate queues for packets
  from each individual source.
• The queues are serviced in a round-robin manner.
• Nagles algorithm, by changing the way packets
  from different sources interact, does not reward,
  nor leave others vulnerable to, anti-social
  behavior.

Analysis and Simulation of a Fair Queuing
Algorithm, A.Demers, S. Keshav and S.J. Shenker,
ACM SIGCOMM, 1989.
37
From Queuing Theory

• M/M/1/K Queuing system - Poisson arrivals,
  Exponentially distributed service times, one
  server and finite capacity buffer
• PASTA - Poisson Arrivals See Time Averages For
  a queuing system, when the arrival process is
  Poisson and independent of the service
  processThe probability that an arriving
  customer finds i customers in the system Equals
  The probability that the system is at state i.
• pi Probability that the system is in state i,
  Using birth-death model
• Block Probability -

38
Nash Equilibrium Condition
The Model Appendix

• Substituting 2,3 in 5 we obtain

Nash Equilibrium Satisfying Condition
39
Approximated Steady State RED Model
Existence - Appendix
Faster network simulation with scenario
pre-filtering Tech. Rep., USC/ISI Tech Report
550, November 2001.
Example minth 10 maxth 20 n 1,..,50 Pmax
0.3
40
VLRED Model, Theorem 3 - Nash Equilibrium
Existence - Proof (1)
Existence - Appendix

• Virtual Load RED model
• Proof of Theorem 3 VLRED imposes a Nash
  Equilibrium on selfish agents if
• Assume the drop probability can be written as a
  continues function for all lvqltmaxth
• Nash equilibrium condition

lvq M/M/1 Queue length when facing the
same load
41
VLRED Model, Theorem 3 - Nash Equilibrium
Existence - Proof (2)
Existence - Appendix

• Substituting and simplifying we obtain
• This is true if (by Substituting n1)

42
Efficient Nash AQM Drop Probability
Efficiency - Appendix

• Total desirable offered load at Nash equilibrium
• Substituting y21-? and then integrating
• k is determined so when there is one user, the
  drop probability is zero as long as his offered
  load is less than unity
• The offered load at Nash equilibria is
  bounded drop probability at equilibria is bounded
  (proved by substitution).

k - arbitrary constant to be determined