Game Theory Sequential Games, Subgame perfection, backward induction paradox

1
Game Theory Sequential Games, Subgame
perfection, backward induction paradox

• Univ. Prof.dr. M.C.W. Janssen
• University of Vienna
• Winter semester 2009-10
• Week 44 (October 26, 27)

2
Three Examples of sequential games

• Take-it-or-leave it
• Centipede game
• Ultimatum Bargaining Game (UG)
• One player proposes how to divide a pie
• Other player can only accept or reject proposal
• Stackelberg duopoly
• Leader, follower both choose quantities
• What is a strategy in each of these cases?
• What is, are the Nash equilibria of these games?

3
Closer look at discrete version of UG I

• Consider following Nash equilibrium
• P chooses 0.5
• R accepts proposal iff it is equal split
• Why is this a Nash equilibrium?

0.9, 0.1
acc
R
rej
0, 0
0.9
0.5, 0.5
acc
R
0.5
rej
0, 0
P
0.1
0.1, 0.9
acc
R
rej
0, 0
4
Closer look at discrete version of UG II

• Red path is equilibrium path (this is what you
  see when equilibrium is played)
• Everything else is off the equilibrium path
• What is specified off the equilibrium path is
  important in determining whether something is an
  equilibrium
• If for example R would accept a proposal of 0.9,
  P would have an incentive to deviate
• Nash equilibrium does not impose any restrictions
  on what is played off-the-equilibrium path

0.9, 0.1
acc
R
rej
0, 0
0.9
0.5, 0.5
acc
R
0.5
rej
0, 0
P
0.1
0.1, 0.9
acc
R
rej
0, 0
5
More formally

• Define history of play as follows.
• Let a0 (a01 , a02 ,,a0n ) as the action
  profile that is played in stage 0, i.e., the
  actions played by all players
• History at the beginning of stage 1, h1 a0
• History at the beginning of stage k1, hk1
  (a0,,ak)
• The set Hk1 is the set of all possible histories
  hk1 and Ai(hk) is the set of actions that player
  i can choose after history hk and Ai(Hk) is the
  union of this set over all possible histories
• Strategy si of player i is a sequence of mappings
  ski where each ski maps Hk to the set of
  feasible actions

6
Multi-stage game with observable actions

• All players observe history hk
• Denote the game from stage k on with history hk
  before stage k as G(hk)
• If after history hk actions ak,,aK are chosen,
  then pay-offs in G(hk) are defined as the
  pay-offs that accrue in the whole game if history
  hK1(hk,ak,,aK) is played
• Strategies in G(hk) are simply those strategies
  that are consistent with the history of play
  being given by hk. Denote these strategies by
  sihk

7
Subgame perfect equilibrium (SPE)

• A strategy combination (profile) (si ,s-i ) of
  a multi-stage game with observed actions is a
  subgame perfect equilibrium if for all histories
  hk the strategy restrictions sihk form a Nash
  equilibrium of G(hk)
• Can also be in mixed strategies

8
Multi-stage games with perfect information

• Sequential game is a game where in each stage
  only one player can choose an action
• Also called game with perfect information
• In this type of game, subgame perfection reduces
  to Backward Induction
• BI is an algorithm according to which you can
  start the analysis by considering the optimal
  choices in the final stage K for each history hK
• The one can work back to stage stage K-1 and
  determine optimal choices given the choicesthat
  are fixed for period K
• Etc.

9
SPE in the three examples

• All three examples are sequential games with
  perfect information
• Thus, we can apply backwards induction
• Usually, this implies that there is a unique SPE
• Note the value of commitment (one reduces ones
  flexibility by choosing first), but because of
  the favourable response from opponent, this
  restriction of the freedom to choose may be
  beneficial

10
Subgames, more generally

• What is a subgame?
• Part of the game that can be analyzed on its own
• A subgame starts in a singleton information set
• A subgame includes that part of the whole game
  that follows opon this singleton information set
• A subgame cannot cut through an information set

11
How many subgames?

• There are just two subgames
• How many subgames are there in the two times
  repeated PD game?

up
acc
R
rej
S
down
0.9
acc
R
0.5
rej
P
0.1
acc
R
rej
12
Behavior Strategies I

• A behavior strategy bi for player i is an
  element of the Cartesian product Xhi?Hi ?(A(hi)),
  i.e., a probability distributions over actions at
  each history hi) and the probability
  distributions at different information sets are
  independent
• Nash equilibrium in behavior strategies is
  combinations of behavior strategies, one for each
  player, s.t. no player can benefit by
  individually choosing a different behavior
  strategy
• In a game of perfect recall, mixed and behaviour
  strategies are equivalent
• To see the potential difference, consider the
  following very simple game

13
Backward induction paradox I

• Consider the Take-it-or-Leave-it game. Start at
  the end, player 2 should take the 4 euros
• Going one step back it is also rational for
  player 1 to take the (then) 3 euros
• Otherwise he does not get anything
• But should player 2 at her first choice moment
  really go for the 2 euros?
• Yes, according to backwards induction and SPE
• But, he may also reason as follows
• If there is common knowledge of rationality and
  backward induction applies, I should not have to
  make a choice in the first place why did player
  1 not grab the 1 euro?
• His behaviour is inconsistent with CKR. I should
  now come up with a theory why he chose not to
  grab the euro.
• Game theory does not necessrily say anything
  about this. Maybe player 1 is not interested in
  the money? Maybe he always leaves the money?
  Maybe
• But if there is a reasonably chance that he
  always leaves the money on the table (and this
  probability is actually large than 1/32), then I
  should also not take the 2 euro now, but
  rationality decide to leave it and later grab 4
  euro
• But then player 1 can rationally choose not to
  grab the euro in the first round and hope that
  player 2 thinks.

14
Behavior Strategies II example

• This is a one-player game.
• A pure strategy specifies what to do in each
  information set
• A mixed strategy randomizes over pure strategies,
  e.g. (1/2Dd, 1/2Uu)
• A behaviour strategy that is equivalent specifies
  choose U and D both with probability half. After
  history U choose u
• Equivalence is then not difficult to see

u
U
d
D
15
Backward induction paradox II

• How can backward induction be repaired?
• One way is to say that players should believe
  that mistakes (i.e., actions off the equilibrium
  path) should be uncorrelated with each other.
• Thus, if player 2 makes his first move, he should
  belief that the fact that player 1 chose not to
  play according to the SPE is no indication of her
  not playing according the equilibrium path from
  that moment onwards
• Another way is to say that it is not CKR that is
  assumed, but only K1K2R1, K2R1. If 2 then
  observes player choosing L, he may make the most
  minimal assumption to justify this and he only
  needs to say that apparently K1K2R1 is violated.