On the interactions among self-interested users of network resources

1
Introduction to Game Theory
Game Theory Seminar (CS 791J) Lecture 3a
Replicator Dynamics Honggang Zhang Lecture 3b
Cooperative Game Theory Giovanni Neglia
2
Introduction to Game Theory
Game Theory Seminar (CS 791J) Lecture
3a Honggang Zhang Department of Computer
Science University of Massachusetts Amherst March
2006
3
Outline

• Review of normal form, symmetric game
• Replicator dynamics in evolutionary game theory

4
Review of two-player normal-form game
Prisoners Dilemma (Payoff Matrix) Prisoners Dilemma (Payoff Matrix) P2 P2
Prisoners Dilemma (Payoff Matrix) Prisoners Dilemma (Payoff Matrix) Cooperate Defect
P1 Cooperate 5, 5 -3, 8
P1 Defect 8, -3 0, 0
-3, 8
payoff to P1
payoff to P2
Payoff matrix of player 1 (row player)
Payoff matrix of player 2 (column player)
5 8
-3 0
5 -3
8 0
5
Symmetric and doubly symmetric normal-form games
Prisoners Dilemma Game ABT

• Symmetric game
• A players payoff matrix is the transpose of the
  other players payoff matrix.
• Players identity is not important.

Payoff matrix of player 1 (row player)
5 -3
8 0
A
5 8
-3 0
Payoff matrix of player 2 (column player)
B
Coordination Game AB

• Doubly symmetric game
• Both players payoff matrix are the same.

Payoff matrix of player 1 (row player)
2 0
0 1
A
2 0
0 1
Payoff matrix of player 2 (column player)
B
6
Review of strategies in normal-form game
Payoff matrix of player 1 (row player)
pure strategy 1 (cooperate)
pure strategy e1 -- pure strategy 1 e2 -- pure
strategy 2 mixed strategy x(x1, x2), with 0
x1 1, 0 x2 1, x1x2 1.
5 -3
8 0
pure strategy 2 (defect)
Payoff matrix of player 2 (column player)
5 8
-3 0
pure strategy 1 (cooperate)
pure strategy 2 (defect)
7
Review of payoff
Prisoners Dilemma Game (Payoff Matrix) Prisoners Dilemma Game (Payoff Matrix) P2 P2
Prisoners Dilemma Game (Payoff Matrix) Prisoners Dilemma Game (Payoff Matrix) C D
P1 C 5, 5 -3, 8
P1 D 8, -3 0, 0

• Since this is a symmetric game, we can talk about
  the payoff of a strategy without referring to a
  specific player.

• u(e1, e2) payoff of pure strategy 1 when played
  against pure strategy 2
• u(e1, e2) -3. (same for both P1 and P2.)
• u(e1, x) payoff of pure strategy 1 when played
  against mixed strategy x(x1,x2)
• u(e1, x)5x1 (-3)x2
• u(y, x) payoff of mixed strategy y(y1,y2) when
  played against mixed strategy x(x1,x2)
• u(y, x)5x1y1 (-3)x1y2 8x2y1 0x2y2
  y A x
• where A is payoff matrix.

8
Evolutionary Game Theory

• Maynard Smith, Game Theory and the Evolution of
  Fighting, Edinburgh University Press, 1972.
• Evolutionary process contains two basic elements
  
• mutation mechanism gives variety (static aspect)
  
• captured by Evolutionarily Stable Strategy (ESS)
• selection mechanism favors some varieties over
  others.
• captured by Replicator Dynamics (dynamic aspect)
• An entropy function provides the key link between
  static (ESS) and dynamic (replicator dynamics)
  evolutionary approaches.
• In this class, we focus on Replicator Dynamics
  for normal-form, two player symmetric games.

9
The Replicator Dynamics

• It highlights the role of selection.
• formalized as a set of ordinary differential
  equations.
• implicitly takes care of robustness against
  mutations.
• Some strategies are going to survive
• Some other strategies are going to die out

10
The Replicator Dynamics Model

• There are n pure strategies in the whole
  population. Individuals can only be programmed to
  play pure strategies.
• A mixed strategy x (x1,x2,,xn) is interpreted
  as a population state, each component xi is the
  population share of individuals who play pure
  strategy i.
• Payoff represents fitness (the number of
  offsprings), and each offspring inherits its
  single parents strategy.
• Replicators are pure strategies, which can be
  copied without error from parent to child.
• Reproduction takes place continuously over time.

11
Payoff in replicator dynamics
ei pure strategy i. xi population share of
pure strategy ei. (equivalent to a component of
mixed strategy x.) u(ei, x) expected payoff
(fitness) of strategy ei in a random match with a
random player when the population is in state
x(x1, , xn). (equivalent to payoff of strategy
ei against mixed strategy x.) u(x, x)
population expected payoff (fitness) is the
expected payoff to an individual drawn at random
from the population. (equivalent to the payoff of
mixed strategy x against mixed strategy x.)
12
Replicator Dynamics Model

• xi population share of pure strategy i.
• ei pure strategy i.
• u(ei, x) expected payoff (fitness) of strategy
  i at a random match when the population is in
  state x(x1, , xn).
• u(x, x) population expected payoff (fitness) is
  the expected payoff to an individual drawn at
  random from the population

13
Example replicator dynamics for a doubly
symmetric game
a1 0
0 a2
Fitness of strategy 1
A
Mixed strategy
Average population fitness
14
Stability Concepts in Nonlinear System
Nonlinear system with state variable
x(t)(x1(t),,xn(t))

• Lyapunov Stability a state x is stable or
  Lyapunov stable if no small perturbation of the
  state induces a movement away from x(x1,,xn).
• no push away from x
• Asymptotical Stability a state x is
  asymptotical stable if it is Lyapunov stable
  and all sufficiently small perturbations of the
  state induce a movement back toward x.

15
ESS and Replicator Dynamics

• ESS x asymptotical stability of population
  state x.
• proved by choosing a Lyapunov function, which is
  a relative-entropy function in this case.
• converse may be not true.

16
Example Rock-Scissors-Paper (RSP) Game

• Unique NE strategy x(1/3, 1/3, 1/3) is NOT ESS !
• How about the Replicator Dynamics?

RSP Game (Payoff Matrix) RSP Game (Payoff Matrix) P2 P2 P2
RSP Game (Payoff Matrix) RSP Game (Payoff Matrix) Rock Scissors Paper
P1 Rock 0, 0 1, -1 -1, 1
P1 Scissors -1, 1 0, 0 1, -1
P1 Paper 1, -1 -1, 1 0, 0
17
Example Rock-Scissors-Paper (RSP) Game
A is the payoff matrix of one player
RSP Game RSP Game P2 P2 P2
RSP Game RSP Game R S P
P1 R 0, 0 1, -1 -1, 1
P1 S -1, 1 0, 0 1, -1
P1 P 1, -1 -1, 1 0, 0
0 1 -1
-1 0 1
1 -1 0
Replicator Dynamics
18
Rock-Scissors-Paper (RSP) Game
NE strategy x(1/3, 1/3, 1/3), but not ESS NE
strategy is Lyapunov stable, but not
asymptotically stable
Replicator Dynamics
Start from any initial state, the system moves
forever along a closed curve!
Paper (x3)
Rock (x1)
Scissors (x2)
19
Summary

• Evolutionary Game Theory
• evolutionarily stable strategy (ESS)
• replicator dynamics
• Single population
• Multiple population
• Ideas in evolutionary game theory can borrowed to
  study computer networking

20
Evolutionary Game Theory and Computer Networking

• An evolutionary game perspective to ALOHA with
  power control.
• E. Altman, N. Bonneau, M. Debbah, and G. Caire.
  Proceedings of the 19th International Teletraffic
  Congress, 2005.
• An evolutionary game-theoretic approach to
  congestion control.
• D.S. Menasch a, D.R. Figueiredob, E. de Souza e
  Silvaa. Performance Evaluation 62 (2005)

21
Thank you !Any Questions ?