Evolutionary Game Theory

1
Evolutionary Game Theory
Amit Bahl CIS620
2
Outline

• EGT versus CGT
• Evolutionary Stable Strategies Concepts and
  Examples
• Replicator Dynamics Concepts and Examples
• Overview of 2 papers Selection methods,
  finite populations

3
EGT v. Conventional Game Theory

• Models used to study interactive decision making.
• Equilibrium is still at heart of the model.
• Key difference is in the notion of rationality of
  agents.

4
Agent Rationality

• In GT, one assumes that agents are perfectly
  rational.
• In EGT, trial and error process gives strategies
  that can be selected for by some force (evolution
  - biological, cultural, etc).
• This lack of rationality is the point of
  departure between EGT and GT.

5
Evolution

• When in biological sense, natural selection is
  mode of evolution.
• Strategies that increase Darwinian fitness are
  preferable.
• Frequency dependent selection.

6
Evolutionary Game Theory (EGT)

• Has origins in work of R.A. Fisher The Genetic
  Theory of Natural Selection (1930).
• Fisher studied why sex ratio is approximately
  equal in many species.
• Maynard Smith and Price introduce concept of an
  ESS The Logic of Animal Conflict (1973).
• Taylor, Zeeman, Jonker (1978-1979) provide
  continuous dynamics for EGT (replicator
  dynamics).

7
ESS Approach

• ESS Nash Equilibrium Stability Condition
• Notion of stability applies only to isolated
  bursts of mutations.
• Selection will tend to lead to an ESS, once at an
  ESS selection keeps us there.

8
ESS - Definition

• Consider a 2 player symmetric game with ESS given
  by I with payoff matrix E.
• Let p be a small percentage of population playing
  mutant strategy J?I.
• Fitness given by W(I) W0
  (1-p)E(I,I) pE(I,J) W(J) W0 (1-p)E(J,I)
  pE(J,J)
• Require that W(I) gt W(J)

9
ESS - Definition

• Standard Definition for ESS (Maynard Smith).
• I is an ESS if for all J ? I,
  E(I,I) ? E(J,I) and
  E(I,I) E(J,I) ? E(I,J) gt E(J,J)
  where E is the payoff function .

10
ESS - Definition
Assumptions 1) Pairwise, symmetric contests 2)
Asexual inheritance 3) Infinite population 4)
Complete mixing
11
ESS - Existence

• Let G be a two-payer symmetric game with 2 pure
  strategies such that
  
  E(s1,s1) ? E(s2,s1) AND E(s1,s2) ?
  E(s2,s2) then G has an ESS.

12
ESS Existence

• If a gt c, then s1 is ESS.
• If d gt b, then s2 is ESS.
• Otherwise, ESS given by playing s1 with
  probability equal to (b-d)/(b-d)(a-c).

13
ESS - Example 1

• Consider the Hawk-Dove game with payoff matrix
  
• Nash equilibrium given by (7/12,5/12).
• This is also an ESS.

14
ESS - Example 1

• Bishop-Cannings Theorem
  If I is a mixed ESS with support a,b,c,,
  then E(a,I) E(b,I) E(I,I).
• At a stable polymorphic state, the fitness of
  Hawks and Doves must be the same.
• W(H) W(D) gt The ESS given is a stable
  polymorphism.

15
Stable Polymorphic State
16
ESS - Example 2

• Consider the Rock-Scissors-Paper Game.
• Payoff matrix is given by R S
  P R -e 1 -1 S -1 -e 1
  P 1 -1 -e
• Then I (1/3,1/3,1/3) is an ESS but stable
  polymorphic population 1/3R,1/3P,1/3S is not
  stable.

17
ESS - Example 3

• Payoff matrix
• Then I (1/3,1/3,1/3) is the unique NE, but not
  an ESS since E(I,s1)E(s1,s1) 1.

18
Sex Ratios

• Recall Fishers analysis of the sex ratio.
• Why are there approximately equal numbers of
  males and females in a population?
• Greatest production of offspring would be
  achieved if there were many times more females
  than males.

19
Sex Ratios

• Let sex ratio be s males and (1-s) females.
• W(s,s) fitness of playing s in population
  of s
• Fitness is the number of grandchildren
• W(s,s) N2(1-s) s(1-s)/s W(s,s)
  2N2(1-s)
• Need s s.t. ?s W(s,s) ? W(s,s)

20
Dynamics Approach

• Aims to study actual evolutionary process.
• One Approach is Replicator Dynamics.
• Replicator dynamics are a set of deterministic
  difference or differential equations.

21
RD - Example 1

• Assumptions Discrete time model, non-overlapping
  generations.
• xi(t) proportion playing i at time t
• ?(i,x(t)) E(number of replacement for
  agent playing i at time t)
• ?j xj(t) ?(j,x(t)) v(x(t))
• xi(t1) xi(t) ?(i,x(t))/ v(x(t))

22
RD - Example 1

• Assumptions Discrete time model, non-overlapping
  generations.
• xi(t1) - xi(t) xi(t) ?(i,x(t)) -
  v(x(t)) v(x(t))

23
RD - Example 2

• Assumptions overlapping generations, discrete
  time model.
• In time period of length ?, let fraction ? give
  birth to agents also playing same strategy.
• ?j xj(t)1 ? ?(j,x(t)) v(x(t))
• xi(t?) xi(t)1 ? ?(i,x(t))
  v(x(t))

24
RD - Example 2

• Assumptions overlapping generations, discrete
  time model.
• xi(t?) - xi(t) xi(t)? ?(i,x(t)) - ? v(x(t))
  1 ? v(x(t))

25
RD - Example 3

• Assumptions Continuous time model, overlapping
  generations.
• Let ? ? 0, then dxi /dt
  xi(t)?(i,x(t)) - v(x(t))

26
Stability

• Let x(x0,t) ?S X R ??S be the unique solution to
  the replicator dynamic.
• A state x ? ?S is stationary if dx/dt 0.
• A state x is stable if it is stationary and for
  every neighborhood V of x, there exists a U ? V
  s.t. ?x0 ? U, ? t x(x0,t) ? V.

27
Propostions for RD

• If (x,x) is a NE, then x is a stationary state of
  the RD.
• dxi /dt xi(t)?(i,x(t)) - v(x(t))
• What about the converse?
• Consider population of all doves.

28
Propostions for RD

• If x is a stable state of the RD, then (x,x) is a
  NE .
• Consider any perturbation that introduces a
  better reply.
• What about the converse? Consider

29
Stronger notion of Stability

• A state x is asymptotically stable if it is
  stable and there exists a neighborhood V of x
  s.t. ?x0 ? V, limt?? x(x0,t) x.

30
ESS and RD

• In general, every ESS is asymptotically stable.
• What about the converse?

31
ESS and RD

• Consider the following game
• Unique NE given by x (1/3,1/3,1/3).
• If x (0,1/2,1/2), then E(x,x)E(x,x)2/3
  but E(x,x)5/4 gt 7/6E(x,x).

32
ESS and RD
In 2X2 games, x is an ESS if and only if x is
asymptotically stable.
33
A Game-Theoretic Investigation of Selection
Methods Used in Evolutionary Algorithms Ficici,
Melnik, Pollack
34
Selection Methods

• How do common selection methods used in
  evolutionary algorithms function in EGT?
• Dynamics and fixed points of the game.

35
Selection function

• xi(t1) S(F(xi(t)),xi(t)) where S is
  the selection function, F is the fitness
  function, and xi(t) is the proportion of
  population playing i at time t.

36
Fitness Dependent Selection
f (p X f)/(p f) x(x0,t) t ?R orbit
passing through x0.
37
Truncation Selection
1) Sort by fitness
2) Replace k of lowest by k of highest
38
Truncation Selection

• Consider the Hawk-Dove game with ESS given by
  (7/12 H, 5/12 D) If .5 lt xH(t)
  lt 7/12, then xH(t1) 1.

39
Truncation Selection
Map Diagram
40
(?, ?)-ES Selection

• Given a population of ? offspring, the best ? are
  chosen to parent the next generation.
• More extreme than truncation selection.

41
Linear Rank Selection

• Agents sorted according to fitness.
• Assigned new fitness values according to their
  rank.
• Causes fitness to change linearly with rank.
• Causes cycles around ESS.

42
Linear Rank Selection
Map Diagram
43
Boltzman Selection

• Inspired by simulated annealing.
• Selection pressure slowly increased over time to
  focus search.
• In some cases, BS is able to retain the dynamics
  and equilibria in EGT.

44
Boltzman Selection
Map Diagram
45
Effects of Finite Populations on Evolutionary
Stable Strategies. Ficici, Pollack
46
Finite Populations

• Effects of finite population on EGT.
• Begin at ESS (7/12,5/12) and test
  n60,120,300,600, and 900 for 2000 generations.
  
• 100 replicates of each experiment.

47
Finite Populations

• Results

48
Convergence

• For a n player name, consider the MC with states
  given by of hawks.
• Define transition matrix P.
• bt b0Pt
• E(xH(t)) (1/n) ?ni0 bHt i
• limt?? E(xH(t)) b?H
• Estimate E(xH(t)) - b?H

49
Convergence Simulation

• Results