Potential Functions and the Inefficiency of Equilibria - PowerPoint PPT Presentation

1 / 75
About This Presentation
Title:

Potential Functions and the Inefficiency of Equilibria

Description:

Potential Functions and the Inefficiency of Equilibria. Tim Roughgarden. Stanford University ... example: POA 4/3 if G has affine cost fns. 36. Proof of General Thm ... – PowerPoint PPT presentation

Number of Views:31
Avg rating:3.0/5.0
Slides: 76
Provided by: timr74
Category:

less

Transcript and Presenter's Notes

Title: Potential Functions and the Inefficiency of Equilibria


1
Potential Functions and the Inefficiency
of Equilibria
  • Tim Roughgarden
  • Stanford University

2
Pigou's Example
  • Example one unit of traffic wants to go from s
    to t
  • Question what will selfish network users do?
  • assume everyone wants smallest-possible cost
  • Pigou 1920

cost depends on congestion
c(x)x
s
t
c(x)1
no congestion effects
3
Motivating Example
  • Claim all traffic will take the top link.
  • Reason
  • ? gt 0 ? traffic on bottom is envious
  • ? 0 ? equilibrium
  • all traffic incurs one unit of cost

Flow 1-?
c(x)x
s
t
c(x)1
this flow is envious!
Flow ?
4
Can We Do Better?
  • Consider instead traffic split equally
  • Improvement
  • half of traffic has cost 1 (same as before)
  • half of traffic has cost ½ (much improved!)

Flow ½
c(x)x
s
t
c(x)1
Flow ½
5
Braesss Paradox
  • Initial Network

Cost 1.5
6
Braesss Paradox
  • Initial Network Augmented Network

½
½
x
1
0
s
t
½
½
x
1
Cost 1.5
Now what?
7
Braesss Paradox
  • Initial Network Augmented Network

x
1
0
s
t
x
1
Cost 1.5
Cost 2
8
Braesss Paradox
  • Initial Network Augmented Network
  • All traffic incurs more cost! Braess 68
  • also has physical analogs Cohen/Horowitz 91

x
1
0
s
t
x
1
Cost 1.5
Cost 2
9
High-Level Overview
  • Motivation equilibria of noncooperative network
    games typically inefficient
  • e.g., Pigou's example Braess's Paradox
  • don't optimize natural objective functions
  • Price of anarchy quantify inefficiency w.r.t
    some objective function
  • Our goal when is the price of anarchy small?
  • when does competition approximate cooperation?
  • benefit of centralized control is small

10
Selfish Routing Games
  • directed graph G (V,E)
  • source-destination pairs (s1,t1), , (sk,tk)
  • ri amount of traffic going from si to ti
  • for each edge e, a cost function ce()
  • assumed continuous and nondecreasing

Examples (r,k1)
c(x)x
c(x)1
½
c(x)x
½
c(x)0
s1
t1
s1
t1
c(x)1
½
½
c(x)x
c(x)1
11
Outcomes Network Flows
  • Possible outcomes of a selfish routing game
  • fP amount of traffic choosing si-ti path P
  • outcomes of game flow vectors f
  • flow vector nonnegative and total flow ? fP on
    si-ti paths equals traffic rate ri (for all i)

12
Outcomes Network Flows
  • Possible outcomes of a selfish routing game
  • fP amount of traffic choosing si-ti path P
  • outcomes of game flow vectors f
  • flow vector nonnegative and total flow ? fP on
    si-ti paths equals traffic rate ri (for all i)
  • Question What are the equilibria (natural
    selfish outcomes) of this game?

13
Nash Flows
  • Def Wardrop 52 A flow is at Nash equilibrium
    (or is a Nash flow) if no one can switch to a
    path of smaller cost. I.e., all flow is routed
    on min-cost paths. given current edge
    congestion

Examples
1
½
x
x
s
t
s
t
1
1
½
½
x
1
x
1
0
0
1
s
t
s
t
x
x
1
1
½
14
Our Objective Function
  • Definition of social cost total cost C(f)
    incurred by the traffic in a flow f.
  • Formally if cP(f) sum of costs of
    edges of P (w.r.t. flow f), then
  • C(f) ?P fP cP(f)

15
Our Objective Function
  • Definition of social cost total cost C(f)
    incurred by the traffic in a flow f.
  • Formally if cP(f) sum of costs of
    edges of P (w.r.t. flow f), then
  • C(f) ?P fP cP(f)
  • Example

x
½
s
t
Cost ½½ ½1 ¾
½
1
16
The Price of Anarchy
  • Defn
  • definition from Koutsoupias/Papadimitriou 99

price of anarchy of a game
obj fn value of selfish outcome

optimal obj fn value
Example POA 4/3 in Pigou's example
1
½
x
x
s
t
s
t
1
1
½
Cost 3/4
Cost 1
17
A Nonlinear Pigou Network
  • Bad Example (d large)
  • equilibrium has cost 1, min cost ? 0

18
A Nonlinear Pigou Network
  • Bad Example (d large)
  • equilibrium has cost 1, min cost ? 0
  • ? price of anarchy unbounded as d -gt infinity
  • Goal weakest-possible conditions under which
    P.O.A. is small.

19
When Is the Price of Anarchy Bounded?
  • Examples so far
  • Hope imposing additional structure on the cost
    functions helps
  • worry bad things happen in larger networks

xd
x
1
s
t
0
s
t
1
x
1
20
Polynomial Cost Functions
  • Def linear cost fn is of form ce(x)aexbe
  • Theorem Roughgarden/Tardos 00 for every
    network with linear cost functions
  • 4/3

cost of Nash flow
cost of opt flow
21
Polynomial Cost Functions
  • Def linear cost fn is of form ce(x)aexbe
  • Theorem Roughgarden/Tardos 00 for every
    network with linear cost functions
  • 4/3
  • Bounded-deg polys (w/nonneg coeffs) replace 4/3
    by T(d/log d)

cost of Nash flow
cost of opt flow
xd
tight example
s
t
1
22
A General Theorem
  • Thm Roughgarden 02, Correa/Schulz/Stier Moses
    03 fix any set of cost fns. Then, a Pigou-like
    example 2 nodes, 2 links, 1 link w/constant cost
    fn) achieves worst POA

23
Interpretation
  • Bad news inefficiency of selfish routing grows
    as cost functions become "more nonlinear".
  • think of "nonlinear" as "heavily congested"
  • recall nonlinear Pigou's example
  • Good news inefficiency does not grow with
    network size or of source-destination pairs.
  • in lightly loaded networks, no matter how large,
    selfish routing is nearly optimal

24
Benefit of Overprovisioning
  • Suppose network is overprovisioned by ß gt 0 (ß
    fraction of each edge unused).
  • Then Price of anarchy is
    at most ½(11/vß).
  • arbitrary network size/topology,
    traffic matrix
  • Moral Even modest (10) over-provisioning
    sufficient for near-optimal routing.

25
Potential Functions
  • potential games equilibria are actually optima
    of a related optimization problem
  • has immediate consequences for existence,
    uniqueness, and inefficiency of equilibria
  • see Beckmann/McGuire/Winsten 56, Rosenthal
    73, Monderer/Shapley 96, for original
    references
  • see Roughgarden ICM 06 for survey

26
The Potential Function
  • Key fact BMV 56 Nash flows
    minimize potential function
    ?e ?f ce(x)dx (over all flows).

ce(fe)
0
e
0
fe
0
27
The Potential Function
  • Key fact BMV 56 Nash flows
    minimize potential function
    ?e ?f ce(x)dx (over all flows).
  • Lemma 1 locally optimal solutions are precisely
    the Nash flows (derivative test).
  • Lemma 2 all locally optimal solutions are also
    globally optimal (convexity).
  • Corollary Nash flows exist, are unique.

ce(fe)
0
e
0
fe
0
28
Consequences for the Price of Anarchy
  • Example linear cost functions.
  • Compare cost potential function
  • C(f) ?e fe ce(fe) ?e ae fe be fe
  • PF(f) ?e ?f ce(x)dx ?e (ae fe)/2 be fe

2
2
e
0
29
Consequences for the Price of Anarchy
  • Example linear cost functions.
  • Compare cost potential function
  • C(f) ?e fe ce(fe) ?e ae fe be fe
  • PF(f) ?e ?f ce(x)dx ?e (ae fe)/2 be fe
  • cost, potential fn differ by factor of 2
  • gives upper bound of 2 on price on anarchy
  • C(f) 2PF(f) 2PF(f) 2C(f)

2
2
e
0
30
Better Bounds?
  • Similarly proves bound of d1 for degree-d
    polynomials (w/nonnegative coefficients).
  • not tight, but qualitatively accurate
  • e.g., price of anarchy goes to infinity with
    degree bound, but only linearly
  • to get tight bounds, need "variational
    inequalities"
  • see my ICM survey for details

31
Variational Inequality
  • Claim
  • if f is a Nash flow and f is feasible, then
  • ?e fe ce(fe) ?e f ce(fe)
  • proof use that Nash flow routes flow on shortest
    paths (w.r.t. costs ce(fe))

e
32
Pigou Bound
  • Recall goal want to show Pigou-like examples are
    always worst cases.
  • Pigou bound given set of cost functions (e.g.,
    degree-d polys), largest POA in a network
  • two nodes, two links
  • one function in given set
  • one constant function
  • constant cost of fully congested top edge

xd
s
t
1
33
Pigou Bound (Formally)
  • Let S a set of cost functions.
  • e.g., polynomials with degree at most d,
    nonnegative coefficients
  • Definition the Pigou bound a(S) for S is
  • max
  • max is over all choices of cost fns
    c in S, traffic rate r ? 0, flow y ? 0

r c(r)
xd
s
t
y c(y) (r-y) c(r)
1
34
Pigou Bound (Example)
  • Let S c c(x) ax b linear functions
  • Recall the Pigou bound a(S) for S is
  • max
  • max is over all choices of cost fns
    c in S, traffic rate r ? 0, flow y ?
    0
  • choose c(x) x r 1 y 1/2 ? get 4/3
  • calculus a(S) 4/3 d/ln d for deg-d
    polynomials

r c(r)
x
s
t
y c(y) (r-y) c(r)
1
35
Main Theorem (Formally)
  • Theorem Roughgarden 02, Correa/Schulz/Stier
    Moses 03 For every set S, for every selfish
    routing network G with cost functions in C, the
    POA in G is at most a(S).
  • POA always maximized by Pigou-like examples
  • That is, if f and f are Nash optimal flows in
    G, then C(f)/C(f) a(S).
  • example POA 4/3 if G has affine cost fns

36
Proof of General Thm
  • Let f and f are Nash optimal flows in G.

37
Proof of General Thm
  • Let f and f are Nash optimal flows in G.
  • Step 1 for each e, invoke Pigou bound with c
    ce, y f, r fe
  • a(S) ? fe ce(fe)/f ce(f) (fe -f )
    ce(fe)

e
e
e
e
38
Proof of General Thm
  • Let f and f are Nash optimal flows in G.
  • Step 1 for each e, invoke Pigou bound with c
    ce, y f, r fe
  • a(S) ? fe ce(fe)/f ce(f) (fe -f )
    ce(fe)
  • Step 2 rearrange and sum over e
  • C(f) ?e f ce(f)

e
e
e
e
e
e
39
Proof of General Thm
  • Let f and f are Nash optimal flows in G.
  • Step 1 for each e, invoke Pigou bound with c
    ce, y f, r fe
  • a(S) ? fe ce(fe)/f ce(f) (fe -f )
    ce(fe)
  • Step 2 rearrange and sum over e
  • C(f) ?e f ce(f) ? ?e fe ce(fe)/a(S)
    ?e (f - fe) ce(fe)

e
e
e
e
e
e
e
40
Proof of General Thm
  • Let f and f are Nash optimal flows in G.
  • Step 1 for each e, invoke Pigou bound with c
    ce, y f, r fe
  • a(S) ? fe ce(fe)/f ce(f) (fe -f )
    ce(fe)
  • Step 2 rearrange and sum over e
  • C(f) ?e f ce(f) ? ?e fe ce(fe)/a(S)
    ?e (f - fe) ce(fe)
  • Step 3 apply VI

e
e
e
e
e
e
e
? 0
41
Proof of General Thm
  • Let f and f are Nash optimal flows in G.
  • Step 1 for each e, invoke Pigou bound with c
    ce, y f, r fe
  • a(S) ? fe ce(fe)/f ce(f) (fe -f )
    ce(fe)
  • Step 2 rearrange and sum over e
  • C(f) ?e f ce(f) ? ?e fe
    ce(fe)/a(S)
  • Step 3 apply VI, done!

e
e
e
e
e
e
C(f)
42
Recap
  • selfish routing simple, basic routing game
  • inefficient equilibria Pigou Braess examples
  • price of anarchy ratio of objective fn values of
    selfish optimal outcomes
  • potential functions equilibria actually solving
    a related optimization problem
  • immediate consequence for existence, uniqueness,
    and inefficiency of equilibria

43
Recap
  • variational inequality inequality based on
    "first-order condition" satisfied by equilibria
  • Pigou bound given a set of cost functions,
    largest POA in a Pigou-like example
  • main result for every set of cost fns, Pigou
    bound is tight (all multicommodity networks)
  • POA depends only on complexity of cost functions,
    not on complexity of network structure

44
Outline
  • Part I The Price of Anarchy in Selfish Routing
    Games
  • Part II The Price of Stability in Network
    Connectivity Games

45
Selfish Network Design
  • Given G (V,E),
  • fixed costs ce for all e ? E,
  • k vertex pairs (si,ti)
  • Each player wants to build a network in which its
    nodes are connected.
  • Player strategy select a path connecting si to
    ti.
  • Anshelevich et al 04

46
Shapley Cost Sharing
  • How should multiple players
  • on a single edge split costs?
  • Natural choice is fair sharing,
  • or Shapley cost sharing
  • Players using e pay for it evenly
    ci(P) S ce/ke
  • Each player tries to minimize its cost.

e ? P
47
Comparison to Selfish Routing
  • Note like selfish routing, except
  • finite number of outcomes
  • in selfish routing, outcomes fractional flows
  • positive (not negative) externalities
  • cost function (per player) ce/ke
  • Objective C Si ci(Pi) S ce
  • where S union of Pi's

e ? S
48
What's the POA?
  • Example

t1, t2, tk
t
1?
k
s
s1, s2, sk
49
What's the POA?
  • Example

t1, t2, tk
t
t
1?
k
1?
k
s
s
s1, s2, sk
OPT (also Nash eq)
50
What's the POA?
  • Example

t1, t2, tk
t
t
t
1?
k
1?
k
1?
k
s
s
s
s1, s2, sk
OPT (also Nash eq)
another Nash eq
51
Multiple Equilibria
  • Moral in Shapley network design games, different
    Nash eq can have different costs.
  • Recall
  • Note not well defined if Nash eq not unique.
  • which one do we look at?

obj fn value of selfish outcome
POA of a game

optimal obj fn value
52
The Price of Stability
  • General definition of POA KP99
  • POA k in last example, uninteresting

cost(worst NE) cost(OPT)
Price of Anarchy
53
The Price of Stability
  • General definition of POA KP99
  • POA k in last example, uninteresting
  • Alternative
  • POS 1 in last example

cost(worst NE) cost(OPT)
Price of Anarchy
cost(best NE) cost(OPT)
Price of Stability
54
The Price of Stability
  • Note small price of stability only guarantees
    that some Nash eq has low cost.
  • much weaker guarantee than small POA
  • Interpretation best solution consistent with
    self-interested players
  • natural outcome for centralized planner to
    suggest e.g., network protocol designer

55
Example High Price of Stability
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
56
Example High Price of Stability
cost(OPT) 1e
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
57
Example High Price of Stability
cost(OPT) 1e but not a NE player k
pays (1e)/k, could pay 1/k
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
58
Example High Price of Stability
so player k would deviate
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
59
Example High Price of Stability
now player k-1 pays (1e)/(k-1),
could pay 1/(k-1)
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
60
Example High Price of Stability
so player k-1 deviates too
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
61
Example High Price of Stability
Continuing this process, all players defect.
This is a NE! (the only Nash) cost 1

t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
1 1
2 k
Price of Stability is Hk T(log k)!
62
The Price of Stability of Selfish Network Design
  • Thus the price of stability of selfish network
    design can be as high as ln k. k players
  • Our goals in all such games,
  • there is at least one pure-strategy Nash eq
  • one of them has cost ln k OPT
  • i.e. price of stability always ln k
  • Anshelevich et al 04
  • Technique potential function method.

63
Potential Functions
  • Recall potential function ? of a game function
    optimized by selfish players
  • not necessarily a natural objective function
  • Defn ? (fn from outcomes to reals) is a
    potential function if for all outcomes S, players
    i, and deviations by i from S
  • ?? ?ci

64
Potential Functions
  • So potential fn tracks deviations by players
  • Thus equilibria of game local optima of ?
  • so finite potential games have pure-strategy Nash
    equilibria (proof just do "best-response
    dynamics") Monderer/Shapley 96
  • precursors Rosenthal 73, Beckmann et al 56

65
Potential Functions
  • So potential fn tracks deviations by players
  • Thus equilibria of game local optima of ?
  • so finite potential games have pure-strategy Nash
    equilibria (proof just do "best-response
    dynamics") Monderer/Shapley 96
  • precursors Rosenthal 73, Beckmann et al 56
  • Claim every Shapley network design game has a
    potential function.

66
Proof of Potential Function
  • Define ?e(S) ce1 1/2 1/3 1/ke
  • where ke is players using e in S. Hk
  • Let ?(S) S ?e(S)
  • Consider some solution S (a path for each
    player).
  • Suppose player i is unhappy and decides to
    deviate.
  • What happens to ?(S)?

e
e ? S
67
Proof of Potential Function
  • ?e(S) ce1 1/2 1/3 1/ke
  • Suppose player is new path includes e.
  • i pays ce/(ke1) to use e.
  • ?e(S) increases by the same amount.
  • If player i leaves an edge e,
  • ?e(S) exactly reflects the change in is
    payment.

ce1 1/2 1/ke
e
i
e
ce1 1/2 1/ke
68
Proof of Potential Function
  • ?e(S) ce1 1/2 1/3 1/ke
  • Suppose player is new path includes e.
  • i pays ce/(ke1) to use e.
  • ?e(S) increases by the same amount.
  • If player i leaves an edge e,
  • ?e(S) exactly reflects the change in is
    payment.

ce1 1/2 1/kece/(ke1)
e
i
e
ce1 1/2 1/ke -ce/ke
69
Bound on Price of Stability
  • Compare cost potential function
  • C(S) ?e ce
  • PF(S) ?e ce1 1/2 1/3 1/ke
  • cost, potential fn differ by factor of Hk
  • gives upper bound of Hk on price on stability
  • let S min-potential soln note also a Nash
    eq
  • let S opt solution
  • C(S) PF(S) PF(S) Hk C(S)

70
Undirected Networks
  • Open Question what is the POS in undirected
    graphs?
  • best known lower bound 12/7
  • Fiat et al 06 O(log log k) for special case

71
Shapley Cost-Sharing
  • Summary with Shapley cost sharing,
  • POA k, even in undirected graphs
  • POS Hk in directed graphs
  • (unknown in undirected graphs)
  • Question 1 can we do better?
  • Question 2 subject to what?

72
In Defense of Shapley
  • Essential properties (non-negotiable)
  • "budget-balanced" (total cost shares cost)
  • "local" (cost shares computed edge-by-edge)
  • pure-strategy Nash equilibria exist
  • Bonus good properties (negotiable)
  • "uniform" (same definition for all networks)
  • "fair" (characterizes Shapley)

73
Other Cost Shares?
  • Theorem Chen/Roughgarden/Valiant 07 Shapley
    minimizes POS among all uniform protocols in
    directed graphs.
  • Shapley justified on efficiency grounds!
  • non-uniform schemes not well understood

74
Other Cost Shares?
  • Theorem Chen/Roughgarden/Valiant 07 Shapley
    minimizes POS among all uniform protocols in
    directed graphs.
  • Shapley justified on efficiency grounds!
  • non-uniform schemes not well understood
  • Theorem Chen/Roughgarden/Valiant 07 Can do
    much better in undirected graphs.
  • can get POA O(log2 k)
  • better for special cases or non-uniform protocols

75
Wrap-Up
  • network games arise in many CS applications
  • price of anarchy/stability/etc a flexible tool to
    measure inefficiency of selfish behavior
  • future direction inform protocol design
  • potential functions are an easy-to-use, versatile
    techniques to bound POA/POS
  • many open questions...
  • looking forward to future theorems from you!
Write a Comment
User Comments (0)
About PowerShow.com