Computational Issues in Game Theory Lecture 1: Matrix Games

1
Computational Issues in
Game Theory Lecture 1
Matrix Games

• Edith Elkind
• Intelligence, Agents, Multimedia group (IAM)
• School of Electronics and CS
• U. of Southampton

2
Games and Strategies

• Games strategic interactions between rational
  entities
• Solution concepts whats going to happen?
• dominant strategies
• Nash equilibrium
• .
• Can it be computed?
• if your computer cannot find it, the market
  probably cannot either

3
Matrix (Normal Form) Games

• finite set of players 1, , n
• each player has k actions
• pure strategies actions 1, , k
• mixed strategies probability dists over actions
  
• payoffs of the ith player Pi 1, , kn ? R

Row player
Column player
4
Prisoners Dilemma

• C collaborate, D defect
• for each player D is better than C
• no matter what the other player does
• D is a dominant strategy

Row player
Column player
5
Dominant Strategy Definition

• Notation
• si strategy of player i
• s-i (s1, , si-1, si1, , sn)
• s, t two strategies of player i
• s strictly dominates t if for any s-i Pi (s, s-i
  ) gt Pi (t, s-i )
• s weakly dominates t if for any s-i Pi (s, s-i )
  Pi (t, s-i ) and ? s-i s.t. Pi
  (s, s-i ) gt Pi (t, s-i )
• s is (weakly/strictly) dominant for i if
  it
  (weakly/strictly) dominates all t ? s

6
Dominant Strategy Discussion

• Very strong solution concept
• no assumptions about other players
• may not exist
• e.g., co-ordination game

Row player
Column player
7
Eliminating Dominated Strategies

• two players X and Y, strategies x and y
• 10 strategies per player 1, , 10
• X gets 1 if x is closer to 0.9(xy)/2 than y,
  0 otherwise
• 9 dominates 10 0.9(xy)/2 9
• if 10 is eliminated, 8 dominates 9 0.9(xy)/2
  .81
• if 10, 9 are eliminated, 7 dominates 8
• eventually, 1 is the only strategy not eliminated
• note 1 is not a dominant strategy!

8
Eliminating Dominated Strategies Discussion

• may end up with more than one strategy per player
• surviving strategy need not be dominant
• what if there is more than one dominated
  strategy?
• if strongly dominated, final outcome is
  path-independent
• if weakly dominated, may depend on choices

9
Nash Equilibrium

• Nash equilibrium a strategy profile such that
• noone wants to deviate given other players
  strategies, i.e., each players strategy is a
  best response to others strategies
• Battle of Sexes (0, 0) and (1, 1) are both NE

Row player
Column player
10
Pure vs. Mixed Strategies

• NE in pure strategies may not exist!
• matching pennies
• Mixed strategy a probability distribution over
  actions
• 50 tail, 50 head

Row player
Column player
11
Existence of NE

• Theorem (Nash 1951)
  any n-player k-action game
  in normal form has an equilibrium
  in mixed strategies
• can we find one in efficiently?

12
Existence of NE proof sketch

• Brouwers Theorem Any continuous mapping from
  the simplex to itself has a
  fixpoint.
• Nash ? Brouwer proof sketch
• set of all strategy profiles ? simplex
• mapping (s1, , sn) ? (s1d1, , sndn), where
  di is a shift in the direction of
  best response to (s1, , si-1, si1, , sn)
• NE is a point where noone wants to deviate, i.e.,
  a fixpoint

13
2 (rconst) players, n actions

• Input representation
• 2 players two n x n matrices
• r players r n x n x x n matrices
• poly-size for constant r
• Output representation
• for 2 players all NE are in Q
• but not for 3 and more players
• Checking for pure NE easy
• at most n2 strategy profiles

14
Warm-up 2-player 2-action games
Row player
Column player
BR(C)
Suppose R plays 1 w.p. r EP(C) from playing 0
(1-r)1 EP(C) from playing 1 r3 1-r gt 3r
iff r lt ¼
Suppose C plays 1 w.p. c EP(R) from playing 0
(1-c)2 EP(R) from playing 1 c1 (1-c)2 gt c
iff c lt 2/3
c
1
r
1
mixed NE r1/4, c2/3
15
Mixed strategies and payoffs

• Payoff matrices
• the row player plays a (a1, , an)
• the column player plays b (b1, , bn)
• expected payoff of R when playing i (Ri, , b)
• expected payoff of C when playing j (C, j, a)

R11 R12 R1n R21 R22 R2n Rn1
Rn2 Rnn
C11 C12 C1n C21 C22 C2n Cn1
Cn2 Cnn
R
C
16
Special case zero-sum games

• Definition a game is zero-sum if Rij -Cij
• one players gain is the others loss
• matching pennies
• Solvable in polynomial time via LP duality
• fix b row players goal
  max (a, Rb) subject to ai
  0, a1an 1
• dual LP
  min u
  subject to u (Rb)i for all i
• i.e., row player is guaranteed max Rbi
• column players goal
  min v subject to
  v (Rb)i for all i, bi 0, b1bn 1

17
General case support guessing

• if 1st players strategy a supported on I ? N
  ai ? 0 iff i ? I
• 2nd players strategy b supported on J ?
  N bj ? 0 iff j ? J
• then I ? BR(b) (b, Ri, ) (b, Rk, ) for all
  i? I, k? N
• J ? BR(a) (a, C, j) (a, C, k)
  for all j? J, k? N
• LP on variables a1, , an, b1, , bn
• solutions to LP ? Nash equilibria
• guess supports, solve LP 22npoly(n) steps

linear inequalities!
18
Support guessing remarks

• can eliminate dominated strategies first
• strictly dominated strategies cannot be
  in the support of NE
• for any weakly dominated strategy, there is a NE
  that does not have it in its support
• may be able to reduce the problem size
  considerably

19
Finding mixed NE
other approaches

• Naïve approaches exp(n)
• Simplex-like approach
  (Lemke-Howson algorithm)
• works well in practice
• exp(n) in the worst case (2004)
• Is it time to give up?
• maybe the problem is NP-hard?

20
Is Finding NE NP-hard?

• Reminder a problem P is NP-hard if you can
  reduce 3-SAT to it
• yes-instance 3-SAT ? yes-instance of P
• no-instance 3-SAT ? no-instance of P
• Problem each instance of NASH is
  a yes-instance!
• every game has a NE
• Formally if NASH is NP-hard then NP coNP
• Need complexity theory for
  total search problems

21
Reducibility Among Search Problems
S X Y
T X Y

• S associates x in X with a solution set S(x)
• Total search problem for any x, S(x) is not empty

If T is easy, so is S
22
END OF THE LINE

• Input Boolean circuits
  S (Successor), P
  (Predecessor)
• n inputs, n outputs
• S(0n) ? 0n, P(0n) 0n
• Output x ? 0n s.t.
• S(P(x)) ? x or P(S(x)) ? x
• Intuition G(V, E)
• V Sn
• E (x,y) yS(x), xP(y)

00000
11001
01011
01011
23
PPAD

• PPAD Polynomial Parity Argument, Directed
  version
• PPAD is the class of all search problems that are
  reducible to END OF THE LINE

search problem solution
g
f
circuits S, T end of the
line
24
Problems in PPAD

• In PPAD
• end of the line (by definition)
• finding a fixpoint (1991)
• finding NE for rconst players (1991)
• PPAD-complete
• end of the line (by definition)
• finding a fixpoint (1991)
• finding NE
• 4 players Aug 2005, 3 players Sep 2005,
  2 players Oct 2005

25
Approximate NE

• e-Nash equilibrium a strategy profile such that
  noone can gain gt e by deviating
• normalize game so that all payoffs are in 0, 1
• 2-player games
• PPAD-complete for eO(1/n)
• e 0.5
• Y starts with arbitrary y
• X sets x to be the best response to y
• Y sets z to be the best response to y and plays
  (zy)/2
• current best e 0.339 (Dec 2007)

26
Correlated equilibrium

• Suppose that there is a central authority that
  can tell each player what to do
• Suppose also central authority can toss a coin
• Battle of sexes couple can achieve a fair
  outcome
• (1, 1) w.p. ½, (0, 0) w.p. ½
• if you are told to play 1, it is in your best
  interest to do so
• practical implementation theater if rains,
  football if sunny

27
Correlated equilibrium definition

• CE dist X on the space of strategy profiles
• (NE for each player, dist on his strategies)
• s.t. conditioned on the ith component of a
  profile drawn from X being s, i prefers s to
  any other strategy
• need not see other players signals
• CE always exist (each NE is a CE)
• CE are poly-time computable (2005)

28
What is a good NE?
Row player
Column player

• Nash equilibria
• (0, 0) total payoff is 3
• (1, 1) total payoff is 4
• (1/4, 2/3) total payoff is 17/12
• not all NE are created equal

29
Finding good NE

• checking for NE with total payoff gt T
  NP-hard
  maximizing individual players
  payoff in a NE
  NP-hard
• deciding whether a particular strategy is played
  in a NE NP-hard
• checking if a NE is unique
  NP-hard
  
  (Gilboa, Zemel89, Conitzer, Sandholm03)

30
n players representation

• even with 2 strategies per player, need to
  represent payoffs to each player for every action
  profile (vector in 0, 1n)
  n2n numbers
• interesting special cases
• graphical games
• anonymous games

31
Graphical games

• players are vertices of a graph
• Vs payoff depends on
  actions of W in N(V) U V
• n players, max degree d gt
  n2d1 numbers

t0, u0, v0, w0 12 t1, u0, v0, w0 31
. t1, u1, v1, w1 -6
W
Ws payoffs (16 cases)
T
V
U
32
Graphical games algorithms

• Graphs of max deg2
  (collections of paths and cycles)
• poly-time algorithm (Elkind, Goldberg, Goldberg,
  ACM EC06)
• Bounded-degree trees
• Exp-time algorithm/poly-time approximation
  algorithm to find all NE (Kearns, Littman, Singh,
  UAI 2001)
• Heuristics for graphs with cycles

33
Graphical games hardness results

• NP-hard?
• no total search problem
• PPAD-hard?
• yes!
• in fact, this is how the hardness result for
  4-player games was obtained
  (Goldberg, Papadimitriou, Aug 2005)

34
Anonymous games

• Each players utility depends on how many other
  players chose each strategy
• other players are indistinguishable
• Special cases
• symmetric games in addition, all players have
  the same utility function
• congestion games players only care how many
  other players use the same strategy
• intuition strategy resource

35
Other concepts

• Sequential games
• players take turns to choose their moves
• solution concept subgame-perfect equilibrium
• Repeated games
• if the same game is played repeatedly, new
  equilibria arise
• tit-for-tat in prisoners dilemma

36
Summary

• Matrix games
• Solution concepts
• dominant strategies
• elimination of dominated strategies
• Nash equilibrium
• pure
• mixed
• e-Nash equilibrium
• correlated equilibrium
• Games with n players, sequential games