Congestion Games with Player-Specific Payoff Functions

1
Congestion Games with Player-Specific Payoff
Functions

• Igal Milchtaich, Department of Mathematics, The
  Hebrew University of Jerusalem, 1993
• Presentation By Eran Werner
• Computational Issues in Game Theory Seminar
  (2002/3)

2
Congestion Games with Player Specific Payoff
Functions

• The paper describes a set of noncooperative games
  where players share a common set of strategies.
• The payoff a player receives for playing a
  particular strategy depends only on the number of
  players playing the same strategy.
• The payoff decreases as more players play the
  same strategy, but in a manner which is specific
  to every player.
• Such games have realizations in economics,
  traffic flow, and ecology.

3
Result Existence of Equilibrium

• It is shown that each game in this class
  possesses at least one Nash equilibrium in pure
  strategies.
• Best-reply paths, may be cyclic, but there is
  always at least one path that connects an
  arbitrary initial point to an equilibrium.
• In the case were individuals possess different
  competitive ability (weighted games) Nash
  equilibrium may not exist.

4
Results - Convergence to Equilibrium

• The players may reach an equilibrium by some sort
  of adaptation process. Is such a process bound to
  converge?
• The process will always converge for 2-strategy
  games or when players have equal payoff functions
• In the general case of unweighted congestion
  games counterexamples for convergence may be
  shown
• However, if the order of deviation is stochastic,
  convergence is almost surely to occur.

5
The Model

• There are n players sharing a set of r
  strategies. The strategy played by the player
  is noted .
• The payoff that player i receives for playing
  strategy j is a monotonically non decreasing
  function of the number of players
  playing the same strategy.
• The strategy-tuple is a Nash
  equilibrium iff each is a best-reply
  strategy.
• Is called the congestion vector
• corresponding to .

6
The Symmetric Case

• A congestion game is symmetric iff all players
  share the same set of payoff functions. These
  games have exact potential functions (Rosenthal
  1973).
• The existence of exact potential function implies
  the Finite Improvement Property (FIP) a sequence
  in which a single deviator strictly increases the
  payoff he receives.
• Obviously any maximal Finite Improvement Path
  ends with an equilibrium.

7
The Two-Strategy Case

• Theorem 1
• Congestion games involving only two strategies
  possess the finite improvement property.
• Proof
• Suppose on the contrary that there is an infinite
  improvement path then for
  some
• WLOG
• This implies that player i, the unique deviator
  in the first step, deviates from 1 to 2 hence
• By Monotonicity
• Hence player i, never deviates back to strategy
  1.
• Contradicting the assumption that

8
Games without the Finite Improvement Property

• The Finite improvement property is equivalent to
  the existence of an ordinal potential for the
  game.
• Eg. The potential function assigning each
  strategy-tuple with the number of
  strategy-tuples which are initial points of
  improvement paths leading to .
• If a game has no FIP thus no ordinal potential it
  still may have a Nash Equilibrium.

9
A two player congestion game with no finite
improvement path
3 strategies are involved (a minimal number by
Theorem 1)
The game does not admit even a generalized
ordinal potential, But pure strategy Nash
equilibria exists These are strategies (1,2)
and (2,1)
10
Best Reply Paths

• A path in which each deviator shifts to the best
  reply against the strategies played by other
  players is called Best Reply Paths.
• The Finite improvement property (FIP) implies the
  Finite Best Reply Property (FBRP) but not the
  converse.

11
Infinite Best Reply Improvement paths

• IBRP require at least 3 players.
• Assume by contrary that 2 players suffice.
• When player A shifts strategy, the second player
  B is negatively effected only if A plays the same
  strategy as B (congestion).
• It is this second player B which makes the next
  move, thus only possibly increasing the payoff of
  the player A (monotonicity), thus the strategy
  played by A remains a Best Reply strategy and
  Equilibrium is reached.

12
An infinite best-reply improvement path in a
3-player, 3-strategy unweighted congestion game
The Strategy-tuples (3,1,2) and (2,3,1) are
equilibria of this game
13
IBRP with Nash Equilibrium

• The Strategy-tuples (3,1,2) and (2,3,1) are
  equilibria of this game

14
The Existence of a Pure-Strategy Nash Equilibrium

• Theorem 2
• Every (unweighted) congestion game possesses a
  Nash equilibrium in pure strategies
• First we proof a Lemma (two parts).

15
Lemma Part 1

• The first part of the Lemma is concerned with
  paths where each deviator moves to the next
  deviators present position
• If is a sequence of
  strategies, is a
  best-reply improvement path and results
  from the deviation of one player from
  to
  then .

16
Lemma Part 1

• Proof
• Let be the congestion
  vector of and set
  .
• Then
  holds for all j and k.
• Hence by deviation to of the unique
  deviator in step k brings to its maximum
  and all other to their minimum.
• By monotonicity of payoff , j(k) remains the best
  reply for that player in all later steps, thus
  each player deviates at most once and
  .

17
Lemma Part 2

• The second part of the Lemma is concerned with
  paths were each deviator takes the last
  deviators previous position.
• If the deviation at step k is from j(k) to
  j(k-1) (k1,2.M) then

18
Lemma Part 2

• Proof
• Here too
• By deviating from j(k), the deviator at step k
  brings to its minimum, this implies that
  the payoff in is greater than when he
  deviated to j(k) ,if he did, or that he will get
  by deviating to j(k) at any later step.
• Therefore a player will not return to a strategy
  he deviated from each player deviates at most
  r-1 times.

19
Proof of Theorem 2

• By induction on the number of players n, by
  reducing an n player game to an n-1 player game.
• Proof omitted, we will see a more interesting
  result, using the same Lemma.

20
Convergence to an Equilibrium

• The proof of Theorem 2 is a by construction of an
  algorithm. Adding player by player in at most
  steps. But will we reach the equilibrium in
  the real?
• Theorem 3 Given an arbitrary strategy-tuple in a
  congestion game , there exists a BRIP such that
  is an equilibrium and

21
An Almost Equilibrium

• Initially .
  . Suppose that is a best reply for all
  but maybe not for
• Starting from we can find a
  sequence
• of strategies and a BRIP
  as in Lemma (A) such that M
  is maximal.
• The first deviator is obviously . if
  then starting from we
  can find a sequence
• and a BRIP
  connected to it as in Lemma (B) such that N
  is maximal. If then we set
  .

22
Convergence to an Equilibrium

• Claim is an
  equilibrium. Suppose it is not, then for some
  player i, is not a best reply for
  . Suppose the best reply is j. Then if
  then by construction
  is best reply against
• Then why is j and not a best reply for
  
• ?
• 1.
• 2. Or
  Both 1,2

23
Convergence to an Equilibrium

• can be true only if
  (construction) contradicting the maximality
  of .
• can hold only if
• which is impossible by construction
  (maximality of M)
• Therefore must be a best reply for

24
Convergence to an Equilibrium

• The theorem is true for one-player games. To
  complete the proof by induction on the number of
  player n, we reduce an initial n-player game
  to an n-1 player game by restricting the
  strategy played by player n.
• By the induction hypothesis there exists a BRIP
  in , where the terminal point
  is an equilibrium of . Back to ,
  is almost an equilibrium of . As shown,
  this can be extended to reach an equilibrium, and
  the extension requires at most steps.
• This gives the upper bound of the
  length of the shortest BRIP connecting an
  arbitrary point to an equilibrium.

25
Convergence to an Equilibrium

• Games in which every strategy-tuple is connected
  to some NE by a best reply path are called weakly
  acyclic (WA).
• If
• The number of strategies is finite
• The order of deviators is chosen randomly
• Deviators do not deviate simultaneously
• Then for WA games a best-reply path almost surely
  reaches an equilibrium.

26
Stochastic Convergence Process

• Treating the game as a stochastic process, each
  player not currently in the best reply strategy
  has a positive probability of at least of
  being the next deviator.
• If each strategy-tuple is connected to an
  equilibrium by a best reply path of length at
  most L then the probability that at least one of
  the strategy tuples
• given
  is an equilibrium is at least, for all k and
  all histories.
• Equilibrium is not reached within the first
  steps with probability

27
Coping with lack of Information

• If players occasionally make mistakes (play not
  the best reply strategies), then the concept of
  equilibrium strategy-tuple should be replaced
  with stationary distribution.
• Mistakes can be the result of lack of
  information, players starts with a priori
  estimates of associated payoff which are later
  modified to a posteriori knowledge of actual gain.

28
Weighted Congestion Games

• Up till now the players had similar influence
  upon the congestion. This model is generalized by
  introducing weights and modifying the
  congestion vector

29
Weighted Congestion Games

• Weighted congestion games involving only two
  players, involving two strategies or when players
  have equal payoff functions possess the finite
  improvement path or (at least) the finite best
  reply property.
• Therefore these games possess a Nash equilibrium
  in pure strategies, and the equilibrium can be
  reached by constructing a maximal best-reply
  improvement path

30
Weighted Congestion Games The General Case

• Weighted congestion games may not possess a pure
  strategy Nash equilibrium.
• Even a three-player, three-strategy weighted
  congestion game may not possess a pure-strategy
  Nash equilibrium.

31
A three-player, three-strategy congestion game
with no pure-strategy Nash Equilibrium
For each player there is effectively only two
strategies as the third one is always minimal. A
deviation is considered either as a right to left
or left to right move.
32
A three-player, three-strategy congestion game
with no pure-strategy Nash Equilibrium
It is always optimal (unique best reply) for the
deviator to play the opposite strategy played by
the player preceding him. As the number of
players is odd, a pure strategy Nash equilibrium
clearly does not exist.
33
Unweighted Vs. Weighed Congestion Games