StateSpace Approximations for Extensive Form Games

1
State-Space Approximations for Extensive Form
Games

• Avi Pfeffer
• Harvard University
• Daphne Koller, Ken T. Takusagawa
• Stanford University

2
Goal Solve LARGE Games

• Large lots of states in extensive form
• E.g. Texas Holdem Poker has states
• Solve obtain solutions that are in or close to
  Nash equilibrium

3
Sequence Form Algorithm

• Generate LP or LCP whose size is linear in number
  of states
• If you can generate the game tree, you can
  probably solve the game
• Experimentally, we can solve games with approx
  106 states
• What if you cant build the game tree?

4
Abstraction
G
5
Abstraction mapping ?
6
Abstraction mapping ?

• Ignorance preserving

7
Abstraction mapping ?

• Ignorance preserving

8
Abstraction mapping ?

• Ignorance preserving
• Action simulability

9
Abstraction mapping ?

• Ignorance preserving
• Action simulability

10
Abstraction mapping ?

• Ignorance preserving
• Action simulability

11
Example Poker
12
Hand Abstraction

• Describe a hand by its essential features
• e.g. 7?7?K?K?K? ?full house, K high

13
Hand Abstraction
14
Bet Abstraction
15
Identity Abstractions

• ? may be identity, with other aspects of the game
  simplified
• change chance probabilities
• allow players to hold the same card
• change outcomes
• treat straight flush as flush
• May be useful in combination with other
  abstractions

16
Good abstractions

• How close is the abstract game to the original
  game?
• How good are the strategies obtained from using
  the abstract game?

17
Naïve idea

• Accurate simulation
• if ? simulates , they should have similar
  outcomes
• Definition is an ?-approximation if
  whenever ? simulates ,

18
Loss of Information

• Not every ? simulates a
• Optimal strategy for G may not be achievable by
  simulating a strategy

19
A Thought Experiment

• Players play ? in G
• defines probability distribution over runs in G
• ? ?induces a distribution over runs in
• You observe induced play in
• at each information set of , defines a
  distribution over action taken
• determines a behavioral strategy
• note may define a different distribution
  over runs in !

20
Good abstractions

• Definition is an ?-approximation if
  for every pure ?,

21
Good abstractions

• Theorem If
• is an ?-approximation
• is a Nash equilibrium strategy vector for
• ? simulates
• Then for each player I
• 
  for all

22
A Simple Application

• G is N-card poker
• G is n-card poker
• ? sorts hands into n equal buckets
• where B is maximum showdown payoff
• using abstraction is within of optimal

23
Ongoing work

• Strengthening the theory
• How well does abstraction work in practice?
• Developing a near-optimal Texas Holdem player