Learning and the problem of the emergence of convention

1
Learning and the problem of the emergence of
convention

• Pelle Guldborg Hansen
• Section for Philosophy and Science studies
• Roskilde University, Denmark

While game theorists are very far from a
complete theory of convention, dynamics like the
Dirichlet rule may go far in explaining the
emergence of convention without communication or
precedent. Vanderschraaf 199583
2
Three examples

• The 50-50 split in the Dividing a Dollar game.
• The rendevouz problem in (Schelling 1960).
• The Oberlin convention for reconnecting cut off
  phone calls.

3
The Problem of the Emergence of Social Convention

• The arbitrariness of given practice in solving a
  recurrent coordination problem suggests that the
  justification of the underlying expectations
  ensuring continuous conformity is given by their
  self-reinforcing character.
• The Problem of the Emergence of Social
  Convention
• The problem of how to account for the
  emergence of such systems of mutually reinforcing
  beliefs from a truly pre-conventional state
  without getting viciously entangled in the
  circular process by which the involved
  expectations are characterized.

4
Outline

• The Lewisian theory of convention and its
  failure.
• Vanderschraafs learning approach to the problem
  of emergence.
• How the original challenges re-appear in the
  learning approach.

• The stationarity assumption a vicious circle
  or a key to salvation?
• A threatening problem due to the stationarity
  assumption.
• Learning under the Schelling-hypothesis.
• Beating off the two challenges.
• Re-orienting game theory.

5
The Lewisian Theory of Convention

• The raison dêtre of conventions as providing
  salient solutions by precedent for solving
  coordination problems.
• Coordination problems
• Situations of interdependent decision by two or
  more agents in which coincidence of interests
  predominates and in which there are two or more
  proper coordination equilibria. (Lewis 196924)
• Convention
• A behavioral regularity continuous conformity to
  which instantiates one such equilibrium in a
  recurrent coordination problem made salient by
  precedent and operational by this being common
  knowledge, or

6
(No Transcript)
7
Two coordination problems
8
The spuriousness of a common description?

• "There seems to be a difference between
  equilibrium combinations in which every agent
  does the same action and equilibrium combinations
  in which agents do different actions. This
  difference is spurious, however. We say that the
  agents do the same action if they do actions of
  the same kind, particular actions falling under
  some common description. But actions can be
  described in any number of ways, of which none
  has any compelling claim to primacy. For any
  combination of actions, and a fortiori for any
  equilibrium combination of actions, there is some
  way of describing the agents' alternative actions
  so that exactly those alternative actions in the
  given combination fall under a common
  description... Whether it can be called a
  combination in which every agent does the same
  action depends merely on the naturalness of that
  classification... But in Telephone tag what
  makes the first pair of action-descriptions more
  natural than the second? And so what if it is?"
  (Lewis 1969)

9
Lewis on the origins of convention

• Conventions work by making salient by precedent a
  certain strict equilibrium in a recurrent
  coordination problem and this in such a way that
  all agents come to expect one another to conform
  to it because of its conspicuousness in this
  particular respect is common knowledge to those
  partaking in it.
• Salience, drawn from (Schelling 1960) a focal
  point of convergence for expectations that may
  not only result from considerations based on
  logic, but just as well from imagination,
  analogy, aesthetics, casuistic reasoning, or, as
  utilized by Lewis, precedent.
• Different from Lewis, Schelling puts a special
  emphasis on how such focal points find gives a
  reason for action as a means to coordinate
  expectations by providing agents with a clue or
  key to the a coordination problem, when this is
  perceived as representing a kind of riddle with
  a definite solution, cf. (Sugden Zamarrón 2006).

10
Lewis implicit learning account of the origin of
convention

• In short, certain conditions common
  knowledge of a general preference for using any
  sufficiently popular notation, plus common
  knowledge that logicians can tell how much the
  various notations are being used tend somewhat
  to amplify any fluctuation in the logicians
  expectations and propensities about their choice
  of notation. A convention is produced when a big
  enough fluctuation meets strong enough amplifying
  forces. The source of the fluctuation is
  unimportant, given its size. It does not matter
  whether it was created with the intention of
  starting a convention or whether it occurred in
  some or all of the population. (Lewis 196986)
• Conventions are like fires under
  favourable conditions, a sufficient concentration
  of heat spreads and perpetuates itself. The
  nature of the fire does not depend on the
  original source of heat. Matches i.e. agreement
  may be our best fire starters, but that is no
  reason to think of fires started otherwise as any
  less fires. (Lewis 196988)

11
Reception

• Early interpretations readily embraced the ideas
• that salience of a given action or action profile
  followed from common knowledge of a successful
  precedent along with suitable assumptions of
  agents reasoning abilities and inductive
  standards. (Grandy 1977)
• that given such salience of a precedented profile
  coordination would follow. (Heal 1978)
• We must remember that a pattern whether of
  the past or of the future is always arbitrary or
  partial in that there could always be a different
  one or a further elaboration of the same one.
• (Palliser 1989)

12
Two challenges

• that salience of a given action or action profile
  followed from common knowledge of a successful
  precedent along with suitable assumptions of
  agents reasoning abilities and inductive
  standards. (Grandy 1977)
• The challenges of rule interpretation and rule
  availability.
• Since any successful precedent in the sense of a
  series of successful combinations of actions may
  be elaborated or projected in infinitely many
  ways into the future, any action profile in an
  ensuing stage game will be salient under some
  description of the precedent. (Gilbert
  1989333-334)

• that given such salience of a precedented profile
  coordination would follow. (Heal 1978)
• The challenge of sufficient reason.
• Even given a successful precedent rational agents
  will not have sufficient reason to act in
  accordance though a rational agent would surely
  want to play his part in the salient profile if
  others were to play theirs, he would also know
  that they would only play their part if he was to
  play his but this is exactly what he himself is
  still trying to figure out.

13
Learning and theory of convention

• The equilibrium selection problem and the
  evolutionary turn in game theory.
• Reinterpreting games as played repeatedly.
• Substituting strategically rational agency
  leading to choice with strategies evolving over
  time in populations of agents pre-programmed or
  conditioned for playing such strategies and
  reproducing according to their success.
• Reimported into the social sciences as an
  aggregate level interpretation of individual
  learning and social imitation processes in need
  of an individual level interpretation in terms of
  learning.

14
Best reply models of learning

• Best reply models of learning are perhaps the
  most popular interpretation in social science.
• Agents adopt strategies that optimize their
  payoffs given what they expect others to do.
• The available models encompass a variety of
  learning rules that ascribe different degrees of
  rationality and sophistication in the agents
  abilities in forming these expectations.

15
Vanderschraafs theory of convention

• Social conventions are reconstructed as
  correlated equilibrium behavior in coordination
  problems resulting from the application by each
  agent of the Dirichlet rule in forming his
  expectations.
• The Dirichlet rule generalizes the method of
  Fictitious Play introduced by (Brown 1951) as a
  means to compute mixed equilibria.
• The stationarity assumption Each agent is
  assumed to apply the Dirichlet rule under the
  presupposition that others are playing some
  stationary, possibly mixed as well as correlated,
  strategy of the game.
• Each agent chooses his best reply at each round
  of a recurrent game to what he expects others to
  do given his expectations, where these are formed
  on the basis of the Dirichlet rule.

16
The Dirichlet rule
17
The emergence of convention

• Assuming that agents update their expectations
  following the Dirichlet rule systems of
  concordant mutual expectations emerge and stay
  forever at one out of the multiple available
  strict equilibria in the driving game (this
  generally holds for any 2x2, 2 player symmetric
  labelled game).

18
The emergence of convention

• In the recurrent game where agents have to
  perform different actions in equilibrium, like in
  Telephone tag, this is no longer true when the
  population of agents playing the game is larger
  than two. Instead expectations converge on the
  mixed strategy equilibrium of the game (when the
  number of players is equal or unequal and large).

19
The problem of asymmetric labelling games

• Mixed equilibria do not qualify as conventions in
  the Lewisian theory
• Contrary to what (Binmore 2006) claims mixed
  equilibria may not be regarded as conventions
  since in these the individual expectations do not
  cause an determine an action of conformity, why
  the interdependency of expectations held as a
  necessary condition for convention is not
  satisfied.
• The learning approach as specified so far cannot
  account for asymmetric conventions like that of
  oberserved by Lewis in Oberlin.

20
Solving the problem of asymmetric labelling games

• The problem may easily be solved if embracing
  Lewis point that no pair of action descriptions
  is more natural than any other.
• Re-describing the game in terms of the strategies
  Ak(call if original caller, wait if original
  receiver) (wait if original caller, call if
  original receiver), thereby transforming the
  game to the Driving game.
• A particular elegant way of doing this is by
  following (Vanderschraaf 2003) in letting agents
  condition their original strategies on possible
  states of the world. This results in conventions
  interpreted as correlated equilibria.
• Further, it has the advantage over Lewis
  definition of convention that it formally
  incorporates the notion of salience by
  formalizing the various pieces of information at
  the agents disposal by which they correlate
  their actions and expectations via a convention.

21
Challenging the learning approachrule
availability and interpretation

• It is important to notice, however, that a theory
  of convention like Vanderschraafs, still does
  not explain how agents initially come to
  correlate their expectations with particular
  states of the world,
• i.e. how they tackle the problems of rule
  availability and interpretation.
• Infinitely many states of the world will be
  available to the agents for pegging their
  original strategies on,
• i.e. infinitely many strategies are available in
  a way such that for any series of observations
  the Dirichlet rule will put equal weight on an
  agents expectation concerning the probability of
  every future play of his opponents occurring.
• Supplementing the learning approach with
  naturally or psychologically salient actions
  and patterns has been argued to be implausible
• since such actions and patterns are observed to
  be conventional of nature, too abundant and too
  inflexible, e.g. (Latsis 2005).

22
Challenging the learning approachthe role of
the stationarity assumption

• A learning approach to the problem of the
  emergence of convention like Vanderschraafs
  assumes that
• the strategies, (states) and the payoffs of the
  game are common knowledge.
• each agent applies the Dirichlet rule under the
  stationarity assumption.
• However,
• In applying the Dirichlet rule, each agent must
  tie this to a series of observed situations and
  outcomes in casu, a series of coordination
  problems considered as one and the same and their
  outcomes.
• For the stationarity assumption to make sense
  each agent must believe that other agents
  behavior is correlated with this recurrent game.
• But if he believes this, does it not imply that
  he must believe that they are already following
  what amounts to a convention?

23
The challenge exemplified for the Driving game

• In playing the driving game each agent will know
  the strategies and the payoffs of the game as
  well as that this is common knowledge.
• In order to have a reason to apply the learning
  rule he must believe other agents to be following
  a stationary, possibly mixed, strategy.
• But from what he knows he can infer that the only
  possible stationary states of the game are the
  equilibria of the game (in so far as others are
  not made of wood).
• If the learning agent believes that the other
  agents are playing the mixed strategy equilibrium
  of the game he will have no reason to apply the
  learning rule.
• Hence, the only belief consistent with the
  stationarity assumption is the belief that other
  agents must be playing one of the pure-strategy
  equilibria. But this will continuously be
  falsified.

24
The obvious counterexample

• In playing the driving game each agent will know
  the strategies left and right - and the payoffs
  of the game as well as that this is common
  knowledge.
• In order to have a reason to apply the learning
  rule he only needs to be uncertain or wanting to
  keep track of the relative frequencies by which
  the two strategies are observed with the aim of
  maximizing his average payoffs.
• If he believes that everyone are equally likely
  to follow either strategy he can do whatever he
  likes, but still keep track of frequencies to
  secure his average payoff.
• If he comes to believe that one strategy is more
  widespread than another he will follow this and
  still have a reason to apply the learning rule.
• Of course he will be able to infer that this is
  most likely not a stable state, but he may just
  decide to work on the stationarity assumption as
  a working hypothesis.

25
An objection to the counterexample

• Given the problems of rule availability and
  interpretation, an agent has no way of knowing
  what the relevant strategies of the game are,
  neither in a truly pre-conventional state or in
  current situations.
• Strategies like left and right, and the like
  necessary for convention, cannot be claimed to be
  naturally salient or psychologically salient
  in any sensible way. They are not natural
  responses like crouching, avoiding, or
  ducking, and such natural responses cannot give
  conventions as they arenatural.
• Assuming that agents believe that other agents
  are playing some stationary, possibly mixed,
  strategy simply begs the question by introducing
  concepts made fit and ready for a convention to
  emerge.
• An agent would already need a belief in what
  amounts to the existence, or possible existence
  of conventions in order to ascribe these
  strategies to their situation and hence also in
  order to believe that other agents are playing
  such a stationary, possibly mixed, strategy.

26
The stationarity assumption begging the
question or a key to salvation?

• Does the pre-conventional state exclude the
  possibility that an agent may consider using the
  stationarity assumption as a working hypothesis?
• Lewis spuriousness challenged For instance, if
  he recognizes a recurrent coordination problem
  and perceives the potential in coordination, may
  he not ask what a stationary strategy could be
  for solving this in general?... (the Schelling
  Hypothesis and our three initial examples).
• The role of learning Would this allow him at
  least to narrow down the potential stationary
  strategies to a sufficiently limited set that
  could be handled by learning?
• Sufficient reason But why should an agent expect
  other agents to recognize the same recurrent
  problem as well as expect them to consider
  working under the Schelling Hypothesis? (could
  this itself be salient?)
• Reorienting game theory The challenge of
  Schellings re-orientation of game theory
  modelling the emergence and solution of the
  riddle.